The Curse of Complexity

"Learning is any change in a system that produces a more or less permanent change in its capacity for adapting to its environment"

Herbert Simon, The Sciences of the Artificial



Now more than ever, business tastes better if you put technology in it - the tasks get done faster, more accurately, employees can delegate more left-brain activities to their computing devices and can focus more on right-brain activities that nurture their humanity more. Who knows, they might even start to like their jobs.

Now let's shake off this rosy picture because, just like with any deep change, adding technology to a business model introduces its own set of challenges, like security, vendor lock-in, task commoditization etc.

But the biggest challenge is the added complexity layer. 

As the world around us explodes into a myriad of perspectives and new voices that were, until the advent of ubiquitous connectivity, unknown to us, our quest for technological advance exposes us more and more to this new, hyper-aware, world.

Let me give you an easy example:
1/ you build a website for your company...
2/ then you decide to turn it into a portal that manages customer and vendor workflows (appointments, customer service, MDM)...
3/ then you want to integrate those workflows into your IT landscape...
4/ then you want to start proactively managing your external stakeholders...
5/ then you realize you need social media connectors...
6/ then you start analyzing social feeds in order to manage your social media risk...
7/ then you decide that your website should really be a web app...
8/ then you want that app to be managed in the cloud...
9/ then you want to outsource the app development lifecycle...
10/ then you want to mitigate the vendor and cloud risks...

And of course you need people to manage all this new complexity and they need to interact in new ways that need to be managed as well...

This brings me back to my early days (when artificial intelligence -AI- was a bit more than an exciting academic topic) when I was researching AI techniques for the forecasting of financial time series. We were back then using as main tool a custom-built back-propagation neural network (NN) and we were struggling to optimally size the number of layers in order to give the NN the maximum predictive capability.

For those new to this, NN's are software programs that mimic the way the human brain learns (by altering the chemical information in the synapses - the links between living brain cells), by non-linear error-correction algorithms that alter the weights of the connections between software constructs that imitate brain cells. 

Nowadays, NN's have gone mainstream and are at the core of any pattern recognition software: OCR, photo and video recognition, voice recognition (Siri, Google Now, Cortana etc).

The training procedure for a NN is that you train it on a dataset, then you test it on another similar (but different) dataset, then you validate it on another similar (but different) dataset. The performance in the validation phase measures the NN's predictive ability.

As we built the NN, we could witness what was called in the literature "the curse of complexity" - as we added more layers, the NN just became dumber and dumber - the validation performance was dropping abruptly. This seems counterintuitive - you'd expect that, as you add more layers with more neurons and more connections, the NN gets smarter. 

It turns out that, when you have too many layers, the NN overlearns the training dataset, to the point of learning the data itself rather than the underlying non-linear equation that generated that data.

To draw an analogy to business, when you have too much complexity embedded in your business, your organization tends to learn best how to manage the current set of activities, but without being able to anticipate new business opportunities and without being able to closely follow the business strategy as set forth by its leadership.

So, how does a business entity reconcile the unnatural need for it to grow (i.e. increase complexity) with the natural need that it remains within the computational limits (or bounded rationality, as Herbert Simon calls it) of its decision making layer (i.e. contain complexity)?

There are no easy answers to this question - large corporations have gone through great pains in the 80's and 90's to keep the business manageable by establishing strict and rigid business rules, by delegating decision making abilities, and by deeply embedding business logic into systems and procedures. This approach, however, brings its own set of complexity, leading ultimately to ossification, which is the enemy of adaptation and economic survival.

My personal belief is that the answer revolves around the enterprise's ability to upgrade and pivot its business model based on the changing environment.

Enterprise strategists need to develop an uncanny ability to always ask themselves the excruciating question: 

"If I add this to my business, what else should I remove in order to keep it sane?"

A Review of the "jBPM5 Developer Guide"

Those who know me, know that I am passionate about business processes and about open-source software.

I was asked some weeks ago to review a new book called "jBPM5 Developer Guide". I am no developer, but I consider jBPM as one of the key components of a lightweight, flexible and ultimately inexpensive technology stack that seriously addresses business processes.

So I was easily excited to talk about it, especially since the book, available at Packt Publishing here, is written by two of the biggest contributors to the jBPM project: 
Mauricio Salatino and Esteban Aliverti.

jBPM is THE open-source BPM suite. It provides a workflow engine, a visual modelling tool, a process repository, a visual forms editor and an integration with the Drools rules engine. I am sure it provides a bunch of other stuff, being an open architecture, but for a tech-minded business person like me, these are the must-have components. It is also compatible with the BPMN 2.0 standard, which is useful since BPMN 2.0 incorporates, most significantly, the process execution logic. A lot of commercial BPMS's still struggle to incorporate BPMN 2.0, just for perspective.



The books starts off with a smooth introduction into the world of BPM, just to familiarize new developers with the key concepts they ought to be manipulating by playing with the jBPM framework.

The following chapter describes the structure of the suite, with introduction to critical components: the core, the semantic module, the process engine, the rules engine, the API's, the WS-BPEL and other service orchestration concepts, SoA, ESB, the audit / logging facility, the identity management component etc. All very important, to anyone concerned with Business Architecture. 

The next few chapters go into a lot of details for developers, I'm sure all are highly useful but frankly a bit obscure to me :-)

But one key takeaway for business people from this book is how well jBPM tackles complex event processing and the proper design of business rules in a separate rules engine rather than by implementing them in the process logic. This is such a critical aspect and I have seen so many cases where business people struggle to embed an inherently changeable rules logic into a hopefully persistent process logic, leading to frequent redesigns followed by lots of debugging...

I think we are witnessing the dawn of highly intelligent, highly customizable business processes that are built for change, not built to last. 

By its modern design, jBPM is correctly addressing forthcoming development challenges related to this new BPM paradigm and for that the book is worth reading and gifting to your developers and IT managers.

On Waste

The most abundant thing in the world is waste.



When I was still junior and somehow struggling with deadlines and deliverables, my mentor gave me a stunningly simple advice: 

"If you don't have time, you make time"

First, this advice looks impossible, obviously. You even don't have time to make time!
Then, you let it sink and realize you can make time by prioritizing and letting unimportant things off the hook. 
Then, when it really grows on you, and you realize it's about carving your life around the activities, events, people that you most care about and make that part as meaningful as you can. Meaningful is here the limiting factor.

Today's world, in all its social-techy splendour, competes aggressively for our attention span and for our time. Because now it is seductively easy to consume irrelevant news and social feeds, we tend to ignore the decades-old Habit 1 of Stephen Covey (in his highly-regarded "The Seven Habits of Highly Effective People): Be Pro-active.

Always ask yourself if you should be concerned by, if you are able to influence or control events that come to your attention and based on this test decide what action is appropriate.

You would be surprised just in how many cases you should decide not to continue to read some news or to browse your Facebook stream. In those cases, make sure you curate your social and news feeds to those fields that will help you grow your influence and control and weed out the blabber.

You will easily get your wasted time and energy under control and you can focus on meaningful activities.