Events: The story of business

"There is no greater agony than bearing an untold story inside you. " - Dr. Maya Angelou

I read that quote and I think about Event Processing. Your probably thinking? "What are you talking about? What does a story have to do with event processing?"
It is this: The event is the captured story of business.

Event Processing is much more than the making clever observations

There are many people who focus on the mechanics of event processing. They think about mathematically how do we combine/aggregate/pattern recognize these event into something more meaningful. This is an incredibly important for without it, the "magic" is gone. But I feel there is a lack of people teaching about what is an event and how to we find them.

As said in many places, the event is the happening, the instant. Something happens to change the world as we know it. It might be something relatively small like a taking money out of an ATM or something with significant impact like a tsunami. But even small things change, in one context or another, the world as we know it. Event Processing has to do with detecting or deriving those changes in the world as we know it. The magic is not observing the happening, but capturing it in a way that we can react to it in our systems.

But how does one become a magician? Where to start? How do we bridge the gap between the physical domain and the virtual domain?

How do we change our mindset to see events

The main thing I would suggest is not to think about the individual events, but rather think of the story. We humans think in terms of stories. Of linear activities, of plot, of tension and inter-relationships. Is satisfying the customer order not a story? Or buying a stock at the right time? True, it might not even be interesting enough to make a "made for television" movie. But it is still a story. And in these stories are character and things happen to these characters. These characters do things and hand things off to other characters. Sure; you can be unromantic and call it a "business process" but still, it is a story. And in this story, as in all stories, there are natural times of interest where something of note happens. These times of interest are the events...

Think about your favorite movie or book. One of mine would be the Fellowship of the Ring by J.R.R. Tolkien. (perhaps you saw the movie) In this movie, some changes in the world were very obvious. Bilbo Left, Frodo got possession of the Ring, Nazgûl search for Ring, Frodo is stabbed with the Mordor Blade, Elrond heals Frodo to the best of his ability, etc. It is easy to see these are major changes in the state of the world. Of course there are more fine-grain events, but don't worry about them yet. Find the big ones in your story.
The way we find the big events in a movie is imagine someone says "Tell me what happened in the Fellowship of the Ring." You would tell all of, what the screenwriters call, the plot points. The major things. Again, there are different layers to the story. You can always go down a layer, later. But think of the large story plot points in your companies story.
"We get an order, we make a product, we bill the customer, we receive the check, we deposit the check" That, my friends, is a story. There are five events that occurred here. Then you start thinking of other end-to-end stories. "We discover a needed capability, we articulate that in a job description, we determine what we will pay for that capability, we advertise, we interview, we offer the position, we on-board the employee". Each of these plot points or events you should capture on a white board. Start to figure out when that event occurs, what do I know, who is involved, what products are being manipulated, when did it occur. These are the things that will make up the meta-data of your events.
Then ask, how can I capture that this occurred? Is this something that happens in a computer orchestrated process that I simply can publish that it happened? Is there a sensor that can sense the physical domain and determined it happened? Is there a way that I can determine it based on other events? Do I need to have a human put it into some UI because there are no other way, yet, to capture it? Does it come from customer, partner, information feed?
What is neat about these stories is that they are really the processes of our business. These events are the natural segmentation points in the process. So you can ask your question to go to the next layer: "What happens between receiving the check and depositing it?" This gives you more processes and more events, but more importantly, a natural hierarchy of events happens. It will also occur that you realize a story is part of a larger story. Of the ecosystems your company belongs; of the rivalry between you and your competitor. That things are occurring outside of your control that affects your story. These plot points are very important to capture.

How does this relate to Dr. Angelou's quote?
I was in a working group with Roy Schulte when an idea hit me. Events are happening in the bowls of the computer system. They generally happen inside of application code and are only observed by a small subset of applications. So, that program is the only one who can react to it. They are stuck there and are limited to expression by the program, which has been our practice, sending out command and control signals to order systems to do a particular part of the work. But that violates the principle of maximize cohesion and minimize coupling. Command and control couples the issuing system to the receiving system; increasing the complexity and ultimately the usefulness of the ecosystem. If the ecosystem had emotions, would it not be in agony?
With events, we decouple the applications. We give voice to the story outside of one application. We allow others to hear the events of the story and allow them to become another plot-line within the story without requiring the "main application" to become more complex. It allows other systems and capability to arise. To fill in gaps. To be built by others and work in tandem and be purposeful.

How does the story metaphor bring event processing to the masses?

We live and breath stories. Stories have as its foundation a state transition: An equilibrium, the equilibrium is lost because of some change (detect/derive), we trying to figure out how to get back to equilibrium (decide), we act (do), equilibrium is reestablished. When you think about your business processes as the stories of your business, the effort of noticing the large events and drilling down brings on a whole new way of thinking. The masses of business people and business analysts may not get what is happening under the covers to allow the derived events to be observed. But the story is a metaphor that all humans get!

So, tell your story well.