EA without Vision is dead

This is response to an interesting post from Richard Veryard of the Next Practice Research Initiative. Two people I have high regard Alex Buterman and Brenda Michelson commented on it, so I figured it was worth reading. Hopefully this post will be worth reading too. Warning, this was written @ 1AM after watching House, so it might be a bit….1AMish (as in like something written at 1 AM not the religious order who practice living simply in a complex world)

The problem is there isn't a good way of doing Enterprise Architecture (or anything structural thinking aid) without having someone with vision and power. EA is a structure designed to hold information about a business and make it accessible as well as the processes, policies, etc. Blah blah blah. As the Burton Group's Report points out, there is value in being results oriented and aligning yourself to the business and learning to communicate. (the Burton's Group Report was the subject of what the post by Richard Veryard which was commented by Alex and Brenda which lead me to post this! So, its kinda germane)

But what good is that?

I am becoming more and more convinced that EA, and most other IT initiatives fail not because they aren't understood or don't have the management backing or they weren't completed enough. No, I believe the reason is the people involved don't have vision. Most people are not innovative nor do they have vision. (Although they do have perspective!) They regurgitate what others have tried without understanding the physics behind why what the mimicked did worked. If you don't get that sentence. Stop reading!

What seems to be missing in our search for silver bullets and hyper-hyped philosophies is the fundamental principle that all of them (the hyped philosophies, methodologies, do-hickies) are just tools. Things! Things to help us go! But if they that wish to go are without a person who understand where to go inside of their gut, the tools are at best ok.

Take a simple example of SOA. We debate endlessly as to the correct form of SOA. how large, how generic, how encompassing should it be? Should processes be part of a service, consumed by a service or both? These questions are irrelevant, yet that is what people's minds are able to grab a hold. They are stuck in metaphor land. Thinking in terms of the metaphor rather than terms of reality. It's like all those episodes of Star Trek where they said "It's like taking a match to stick of dynamite laced with gasoline." The problem isn't the metaphor, it is the people who believe they understand warp theory now because they know what happens when you take a match to a stick of dynamite laced with gasoline.

Enterprise Architecture, in the hands of a virtuoso, is wonderful because order can be established inside of the processes, activities and stuff that a enterprise does. But the goal isn't order and establishment (well it is for GE and Six-Sigma places who want to take variation out of their processes but that is beside the point) or even to create flexibility and agility. To quasi-quote the "Good Book" (a.k.a. Bible (the Christian one, not the SQL one)) If I have the flexibility of ballerina and have not vision, then I am a nimble dancer in the wrong act of the play. If I have agility and can change the direction of my company on the slightest change of the market, but have not vision, my agility is for not because I will sit and spin with no direction like a child on a sit and spin and spinning until they puke! (Not sure how that last quote ended up there!)

All of our programs, architectures, frameworks, etc. bring order in the chaos. But unless you have someone who pushes you into the chaos your stuck with status-quo.

Brenda has been tweeting about "is a larger or smaller consulting firm better for helping a company 'do' EA (or fill in the blank)" I say, it doesn't matter. What does matter is to have a person of vision to lead us not just in the safe trusted areas, but into the place where Angels fear to tread for there is way to … (Add in your favorite hyper-hyped business term here)

Last thing, If you call yourself an Enterprise Architect, you should buy (that way he gets a royalty) and read Stephen Pinker's book "The Stuff of Thought: Language as a Window into Human Nature." It has a great chapter called "The Metaphor Metaphor." After you read that chapter, read "Down the rabbit hole." If you don't like either of those, he has a great chapter on explicative usage.

The last section of one of the chapters (I think it was "down the rabbit hole") he talks about Plato's cave. Where people are chained head and body made to look at the back wall. There is a fire behind them and people are using cut-outs to make shadows on the walls. The people think "This is reality!" for this is all they know. One of the people escape and find out that they are actually in a cave and real reality is much better. He comes back into the cave and tries to get his brothers out so that they can see the real reality. But they refuse believing instead that what he saw was captured in what they saw on the wall. If you call yourself an enterprise architect, (and actually read this nonsense this far) then you need to be that person who chains are off and can see outside the cave. Use the tools that you have in EA and others frameworks to pry your brothers (or sisters) eyes away from the back of the wall.

Other types of functionality... where does it go on the cloud

I have been curious of how will the cloud evolve.  We have picked basic functionality to start the cloud.  Queuing, Storage, instruction processing, network are the blocks that the rest of IT is built upon.  As an industry, those blocks have been/are being solidified.  What I would like to know is what is next to be built or perhaps next after that.  As usual, I have an opinion, but it will, also as usual, require a bit of context.  I hope the conclusion will be worth it!

My contention is this: "The cloud" is a mechanism to enable services of commodity functionality  [Which is another name for work] and as such, a well understood taxonomy or organization of these commodity functionality will need to be developed in order for higher level functionality to prosper in the cloud.  

To understand this, understand that functionality can be described as: Core (value adding), Unique (necessary, but no provider we trust), Commodity (necessary and providers we trust)  and Extraneous (not necessary).  Core competencies (functionality), as described by Jack Welch, is work that a company focuses on being the best at to set itself apart from its competitors.  Unique is work that is needed to be done but it doesn't set us apart directly and there aren't providers that we can trust to do the work.  Commodity is work that is needed to be done that doesn't provide differentiation but there are providers who can do the work.  Extraneous are waste in the lean six-sigma sense.

In real-world products, we have seen business outsource accounting, payroll, manufacturing, sales, IT.  Following Jack Welch's advice of focus on core competencies and allow others to offer a service which encapsulates the work that isn't core to your value proposition.  This makes sense. It leads a company to only, for the most part, focus on things that matter to its bottom line.  However, a company will still need to do unique functionality because no one else can.  Our goal should be to convert, as much as advisable in terms of security, functionality from unique to commodity.  Even potentially creating new industries to provide that functionality.  Stated differently all work that is necessary for a business to operate should be provided by an entity whose core competency is doing that particular work.

So applying this to information technology, there are algorithms that are core, unique, commodity and waste.  If we follow the same advice, our efforts should be focused on our IT's core competency.  Work that is unique we should plan how to make it commoditized.  Commodity should be being done by others and waste should be eliminated.  But how do we make this happen?  

With the advent of services and well-understood interfaces, it has become operationally easier to separate out the core from the commodity services.  The cloud starting from the ground up, with storage, database, computing power, networking.  Functionality that all systems share and because they have stable, mature, well-understood interfaces.  This is not really due to the efforts of the cloud community for they stood on the shoulder of giants. Rather it was the efforts of IT Vendors and standards groups that standardized the interface allowing the creation of their products.  Remember these standards allowed the syntactic ambiguity and also, to the extent possible, the semantic ambiguity to disappear.  However, this canonization effort is expensive, time-consuming and potentially a pre-ripe standard may stifled innovation to soon.

High level functionality such as business rules, calculation, decision logic may indeed be capable of being a commodity.  However, trust, the ecosystem of providers, the organization of functionality and the interfaces standards for the functionality are immature.  Thus it traps potential commodity functionality as unique.  Or it requires that business purchases a large package that stands in for the undeveloped ecosystem which can hinder your IT organization from being enabling business differentiation which should be its primary objective.

SAP and other large package vendors will tell you that they are opening up to allow highly customized processes and manipulation of data.  They will even allow you to call a web service from an external (to SAP) provider (probably will be you!) and use the response in your process.  So, in essence SAP (and all the other ones too) has turned into a platform that provides a rich array of functionality and process to accelerate your mapping of your emulated business to your real business.  This platform can become the focal point of your business.  Is this new?  no.  Is this bad? no.  But the ecosystem needs to extend out.  Potentially even replacing parts of the rich array of functionality or perhaps replacing the platform.  A business using a package should ask itself, what is the core competency of our large package software and map that answer to the previous goal: all work that is necessary for a business to operate should be provided by an entity whose core competency is doing that particular work.  No vendor can provide all functionality to all types of parties as part of their core competency.  At least not to the determent of some of the functionality.  So, how do we build an ecosystem that will allow best of breed augmentation using external functionality?  How will we know who has what?  And how difficult will it be to have multiple providers or to switch providers?  Asked another way, how do we abstract out the functionality from the provider?

These, I believe, are the core questions to prompting us to make high order functionality available on the cloud.  We tried UDDI, but there wasn't a good taxonomy developed to describe where the functionality fit.  As well, there wasn't a good semantic ontology developed to describe the interfaces.  Nothing helped us be agnostic to the physical interface.

So my solution:

Imagine in a folks-onomy way a provider builds a version of their functional taxonomy.  This would include the semantics of what the functionality does.  As well, the semantics of the interface.

This world would also includes a more generic taxonomy built with help of linguists, computer scientists that is semantically accurate.  These taxonomies will be built, probably, along industry segments lines.  The company accomplishing this intellectual property, will work with individual providers to help them map their own taxonomy to the more generic taxonomies potentially across industry boundaries.  Where holes appear, the company will modify the generic taxonomies to make it a more complete and living entity.

The beauty of this is when a company wants to consume commodity functionality.  They search the functional taxonomy to find what they need.  Their platform/package provider will jump start their own taxonomy and the individual consumer will modify it to match their particular implementation.  The individual company will a more specialized functionality for its business than its package provider provides.  Because providers publish what functionality they provide and a match can be found at run-time.  Also, because the interface is semantically tagged, the consumer can provide the correct information based on its data's association with the generic taxonomy at run-time and convert the response to its own format.  

Therefore, the physical interface doesn't need to be well-understood, just semantically understood.  Which has been a huge problem all along.  We don't semantically tag our information (as a general rule) to an authoritative source .  In the same vein, we could do the same for describing nouns, processes and events.  It is in organizing our functionality (work) and our information that will allow our trapped unique functionality to become commoditized and therefore meet our goal.  

I think a company that can build and maintain the taxonomy so that semantic accuracy can be a first order principle in software design would be an awesome place to work.  As the cloud grows and encapsulates higher order functionality, semantics will be foremost!  Well, until it itself is commoditized.  

Anyone want to help me build this company?

What should a Enterprise Architect look to find in a job?

I was chatting with a fellow SOA expert Tim Vibber also known as the SOA Chief.  We were talking about what a couple of guys who want to start an SOA consulting company should do.  This took the conversation to why SOA fails and to a post about SOA being dead.

After talking about the failure points, I wondered what makes a good/great architect.  Tim stats that we need to develop systems to be more nimble.  I couldn't agree more, but the question is why can't companies do that?  Why can't architects accomplish that simple statement of "make systems more nimble"?

Both Tim and I are looking for new opportunities so what are these attributes?  Do we have control of them?

To explain these attributes, I am going to borrow something from one of my favorite podcasts: Entrepreneurial Thought Leaders Lecture from Stanford University Technology Ventures Program.  Although I have learned so much from all of the various pod casts, one of my favorite is from Tina Seeliq entitled What I wish I knew when I was 20.  Tina described many important attributes such as Every problem is an opportunity, "the harder I work, the luckier I am" and "never miss an opportunity to be fabulous" and many others.

But the one I want to use here is "Find the intersection between your Interests, Skills and the market."  Tina uses this to describe where one should look to find a career.  She shows where one of those attributes are missing that it is less than ideal.

Interest + market but no skill then your a fan

Interest + Skills but no market then that's a hobby

skills + market but no interest then that's a "job"

Interest + skills + Market = career: A place where you can invest yourself with reward.

For an enterprise architect, I want to change this slightly.  I want to find the traits that are needed for an enterprise architect.

Thus far I've come up with Vision, Passion, Power and Need.

Vision + passion + need but no power = impotent and frustration

Vision + passion + power but no need = unneeded/wasteful projects

Passion + power + need but no vision = future integration opportunity or stuck in the way we've been doing it forever

Vision + power + need but no passion = no buy-in

I would love some thoughts on this.  I think this is very critical to finding good architects and empowering them but also how does an architect find a place s/he should be?

Intersection between Enterprise Architcture, ITIL and the SOA Movement

The past few weeks I have had an ephitheny about the relationship between ITIL and Enterprise Architecture. The relationships that the Enterprise Architecture bricks describe the information needed for a configuration item (CI) inside of the CMDB. That a CI has to have a type associated with it and that type should be a brick inside of EA.

How I came about this discovery is I was trying to figure out what are the CI for our application infrastructure. It seem to me that ITIL focused on the component aspect of things inside of the production environment. This usually consisted of things like physical devices and brought-in software. But the more I thought about that, it seemed as if something was missing. Bespoke (British for custom-made) software (stuff you build yourself) didn't seem to be as well covered.

So, it seemed as if the connection between the service provided and the hardware, software used were connected through some indirect way. However accurate, it seem something was missing.

Dr. Warner Vogel, CTO of Amazon, gave a speech on Enterprise Architecture where he indicated that Amazon was really a platform where buyers and sellers get together. The largest seller using the Amazon Platform was the Amazon Store. But when you think of Amazon, you have to think of the various domains that are interconnected. One domain is the platform which does things like provides services such as item management, buyer-centric human and systemic interfaces, seller-centric human and systemic interfaces, shopping cart management, promotions, advertiser-centric human and systemic interface, etc.

But there are other domains involved in the ecosystem. Front-end providers which sell through their interfaces items in the Amazon product set. Also, you have product providers (sellers) which offer their products and services as part of the Amazon product set. Another domain is the actual buyer who has their own process of purchasing a product that may connect to any of these sellers.

The point is that there are many intersections between these domains. That became significant to me in my quest for understanding. I asked myself, "What is really happening at these intersections." The answer I got was that control was being transferred.



In this diagram, you see that there is an interdependency between these domains. In particular, there are processes inside these domains that temporarily pass control (perhaps asynchronously) to the provider domain, which in turn can pass control, etc. When I thought about this harder, I came to the conclusion that there were two views into this intersection: the Interface and Interaction.

The interaction defines the reason for the intersection between the domain. Such as Request to purchase item or inquiry of books with "Amazon" in its title. An interface, however, is the actual connection between the domains. A "request to purchase item" can be connected to a consumer via multiple interface. There might be human or systemic. Said another way: the reason for the connection from the consumer process is the same regard of the method (interface) of connecting.



It was at this moment, that I realized that the domain boundary became critically important for sanity. So, that begged the question, "What is inside of the domain and what is outside?" and "What really is a domain?"

My conclusion is that a domain is an unit of authority that accomplishes work, orchestrates that work, provides and consumes services (collection of work) to other domains and manages exceptions, situations inside its sphere of control. Interactions specifically are what connect the processes (orchestrator) to other domains.