On My Watch

Friday, October 21, 2005

CEP >= Stream Processing?
A lot has been written recently about complex event processing (CEP) and the question I have is whether or at least how it will be used, in the real world, beyond stream processing. Stream processing - which is not really new to real time developers (of applications like trading systems and security monitors) - is essentially geared to processing data (called events) in real time - in other words, before it is put in database or file system and its context is lost and, critically, time passes (read latency). What is new is that a distinct product category is emerging that offers stream processing in product form with admittedly some new spins (such as injectable scripts) and the general *-abilities that productized software offers.

Advocates, such as David Luckham in particular, suggests that CEP represents a whole new way of analyzing data to infer context where it may have been lost and, by analyzing event sequences across time, can be recovered. Seems like the top usage scenario proferred by most vendors putting themselves in this CEP space (e.g.
Apama, Streambase and Aleri), however, is in low latency environments, particularly algorithmic trading. In this scenario, software processes make trading decisions automatically by interpreting price and other feeds within the context of trading strategies designed by the business (typically experienced traders). And latency is the enemy since each automated processes is competing with other automated processes - as well as human traders - for the same quotes. It is essentially a race to the order book.

While these products
offer a host of other features (Stream SQL from Streambase is particularly interesting), they are basically each addressing this latency issue by offering - albeit in different ways - better and faster ways of processing streaming data. A worthy endeavor, to be sure, but it is stream processing.

It will curious to see what other types of applications develop and whether they fit the CEP moniker. I haven't seen it yet. The problems of discovering context for an event or other data (whether histroical or real time) do exist but can this crop of technology adequately address it?


Labels:

Friday, October 14, 2005

Since this is first post in this blog, I'll use it to say this blog will be mostly about, for lack of a better term, business technology, the use of technology (particularly software) to improve business performance and business-IT alignment - you know work stuff. So topics (read: acronyms/buzzwords) like SOA, EII, BPM, CEP, RSS, Real Time Enterprise, etc. as well as specific applications within various verticals will certainly be inbounds. But I am sure other random topics (personal and otherwise) will creep in as well from time to time.

One topic I have been thinking a bit more than usual about lately has been RSS and blogging - mostly in how it relates - or at least could relate - to SOA and other application data flows. And mostly because a smart guy I know sort of led the horse to water. Some intriguing relationships certainly are possible. Treating RSS as at least a semi-structured feed can certainly help integrate and coordinate human activity with business applications and human-readable information flows with application-readable ones. And maybe even smart or at least interesting things could be done.

Obviously weblogs (recently acquired by Verisign) is helpful for any budding "real time enterprise". On a side note, I really love weblog, once again, like RSS, for its simplicity. Rest and XML-RPC - no complex subscription protocols. Thanks again, Dave.

And still watching next gen (Web 2.0) search engines like Sphere and others because it seems increasingly likely that the 1st gen search providers (we know who they are) will need to start tweaking their algorithms to account for freshness/timeliness AND content quality/relevance as blogs and other content are ranked - or else open the door for new entrants.

Building this out has interesting possibilities ... And a likely topic for future posts.

One unrelated note to close on: Adam Bosworth's piece on intelligent reaction is worth a gander, particularly for those interested in user-driven innovation but really for all since it is applicable to building a demand-driven and reactive business, in general. And maybe not so unrelated to RSS after all....