Microservices and organizational barriers

Off late more and more I hear about traditional IT organizations wanting to implement microservices architectural pattern. The underlying reasoning is that some of the tech leaders in these organization view “microservices” as a technical solution that will enable them to achieve velocity in the “market” (i.e deliver faster, better, cheaper). However I see that some of these folk fail to see that microservices is an architectural style like Service Oriented Architecture rather than an tool or a product or a solution that could be purchased, installed and configured. Martin fowler in this excellent article talked about technical & non technical aspects related to micro service architecture. In this post I would like to discuss some of the underlying organizational structure and implications.

There are some key considerations in any traditionally structured organization wanting to go a microservices centric model. Yes it true that microservices centric organizations are able to roll out products at amazing velocity (example. Amazon). But then there are two key considerations to building an effective microservices centric architecture and they are (1) organizational culture and (2) organizational architecture. In this post I will tackle the org architecture and not culture. A _typical_ IT organization is structured like this
Some organizations are structured along technology lines, others are structured along business process line or some times combinations of both. For instance, an organization structured around technology lines will have one leader incharge of ERP systems (like SAP) and other leader incharge of .NET based web applications and other leader incharge of all Java based applications. In a business process centric organization, for instance,as in a Mortgage Banking technology group, one leader will be heading up all mortgage originations related application, another leader would be incharge of all servicing applications etc. In such a set up, implementing a microservices centric application architecture will be extremely complicated or even impossible. In this set up, monolightic application is a natural way of getting things done. The monolithic application has targetted purpose and the applications are structured to along the lines of hiearcical organization.

One of the reasons the IT organizations are organized the way they are… is to achieve cost efficiencies! period. A typical IT organization is a supporting group (excludes software product firms) and delivers capabilities to _support_ the business. IT is a cost center and hence there is ALWAYs the constant pressure to reduce the OpEx. One of the ways to reduce expense is by establishing hiearchical centralization to help achieve scale Human Resource efficiencies (spread a developer across multiple projects). In the last couple of decades offshoring and outsourcing has been another tool to reduce expenses. While some organization claim they are in India or elsewhere for talent arbitrage, most of the traditional technology shop leverage offshore as cost arbitrage opportunity with imposed structure supporting monolithic legacy applications.

In constrast a service oriented organization is more decentralized. In that model, what we will have is a set of services that provides certain functionality. To make it concrete, lets take mortgage as an example. Some of the possible services are… Modification Service, Underwriting Service, Risk Service (default risk, liquidity risk etc), Customer Service etc. From organization perspective, the service is an atomic unit supported by a team that builds, maintains and runs the service. The team is accountable for the service and is on the hook to keep up with changes. In the case of Modification Service, if there is a HAMP program, the team enhance the service to incorporate HAMP related logic in the service offering as manadated by regulatory bodies. The app developers in the service oriented world will expose smart and rich end points (and not dumb ones) that could be consumed by any application (web, mobile, voice, gesture). In addition, with this set up, the service now can be scaled up and down independently. A real life use case – during the mortage crisis, with the number of defaults increasing rapidly many many mortgage companies struggled to keep up with the volumes both on the technology side as well as on the organizational side (ex. call center staffing). Even though firms were rapidly able to hire people to service the defaulting customers, technology pretty much struggled for a long time because of inability to scale during times of crisis. The companies had to scale multiple monolightic applications (there by increasing cost) just to keep the lights on. With service oriented approach, it would have been possible to scale up the critical default related services rather than a shot gun approach of scaling up everything.

In addition, with service oriented set up, companies need to fundamentally shift thinking on how projects are conceputalized and executed. For instance, today, many companies still think along the lines of creating a business case for a capitalizable capital projects with distinct start and end dates. One the business case gets approved, the capex projects builds upon existing application(s) by layering in more features and functions. While model works, it doesnt align with service oriented thinking. Companies needs to fundamentally shift to into a product centric thinking instead of project centric thinking. For instance, a mortage company could think of Modification Service, Under Writing service as a product. Features need to be built into the product so that it can support the business needs. It will require a forward thinking product strategist and a product owner who can define the lifecycle & roadmap for the product. Now if a company figures out a killer Under Writing service it could now think of monitizing by exposing the service to other morgage companies (possibly after fully vetting out any legal implications)

Once such services are in place, then creating composite application that leverage the services becomes _relatively_ simple. An user interface centric application (read mobile or web) or Non UI application (voice, gesture) can access the services and provide the desired outcomes. This model is extremely desirable especially now when the desire for achieving rapid velocity to rollout differentiated products and services is more than any other time. Traditional companies steeped in motholithic application are unable to respond fast enough. The business leaders perinneal gripe is that IT is not fast enough. However with focus on cost supported by rigid org structure & traditional mindset, CIO have miles and miles of ground to cover before they can be nimble like the way their business leaders wants them to be. The business leaders on the other hand, need to stop looking at technology as a cost center, and need to make strategic investments. However in the interim the chasm created by what is possible versus what is current is giving opportunities for the new, nimble service oriented upstart to come in and disrupt the established players (example Honest Dollar versus rest of the 401k industry).

Frobenius norm

A quck intro on how to calculate Frobenius norm or 2 norms

X = np.array([[3.,5.,8.],[4.,12.,15.]])
norms = np.linalg.norm(X, axis=0)
This results in
[  5.  13.  17.]

This the equation

To calculate this
SQRT ( 3**2 + 5**2 ), SQRT ( 5**2 + 13**2 ), SQRT ( 8**2 + 15**2 )
The output is 5, 13, 17