CEP >= Stream Processing?
A lot has been written recently about complex event processing (CEP) and the question I have is whether or at least how it will be used, in the real world, beyond stream processing. Stream processing - which is not really new to real time developers (of applications like trading systems and security monitors) - is essentially geared to processing data (called events) in real time - in other words, before it is put in database or file system and its context is lost and, critically, time passes (read latency). What is new is that a distinct product category is emerging that offers stream processing in product form with admittedly some new spins (such as injectable scripts) and the general *-abilities that productized software offers.

Advocates, such as David Luckham in particular, suggests that CEP represents a whole new way of analyzing data to infer context where it may have been lost and, by analyzing event sequences across time, can be recovered. Seems like the top usage scenario proferred by most vendors putting themselves in this CEP space (e.g. Apama, Streambase and Aleri), however, is in low latency environments, particularly algorithmic trading. In this scenario, software processes make trading decisions automatically by interpreting price and other feeds within the context of trading strategies designed by the business (typically experienced traders). And latency is the enemy since each automated processes is competing with other automated processes - as well as human traders - for the same quotes. It is essentially a race to the order book.

While these products offer a host of other features (Stream SQL from Streambase is particularly interesting), they are basically each addressing this latency issue by offering - albeit in different ways - better and faster ways of processing streaming data. A worthy endeavor, to be sure, but it is stream processing.

It will curious to see what other types of applications develop and whether they fit the CEP moniker. I haven't seen it yet. The problems of discovering context for an event or other data (whether histroical or real time) do exist but can this crop of technology adequately address it?