Real Time Web Architecture

Kind of an eye chart, but here is an architecture for real time web apps, a possible reference architecture.

Note the inclusion of CEP (like Apama, Streambase or Esker) and Data streaming (like Cometd) along with traditional web and mashup servers. I think that is where the future resides - being able to handle both client pulls (dynamic and static) as well as real time data feeds. Real time data feeds traditionally have been feeds (like securities prices) that are limited to certain markets but increasingly they are broader, mass market feeds - such as Feedburner, Facebook, Twitter and phone/sms - integrating both system-generated and human-generated messages.

Programming languages will be all over the map (from compiled C#, Java, etc) to open web scripting (Python, Javascript etc.) to messaging and proprietary optimized event processing languages like StreamSQL and Apama's monitor script.

Depending on feedback, I hope to describe these components in more detail in future posts and also keep this updated to reflect technology and market developments.

Using Apama Trading Strategies Platform

I have recently had the occasion to pound on the algorithmics trading platform from Apama - both for use at a major Wall Street sell side shop and as a platform for an enterprise risk management system my company is exploring.

The Apama platform offers several components, including:

• Event Modeller - an IDE for developing applications. (See screenshot below)
• Event Manager - the event processing engine. They call it a correlator since it "correlates" events.
• Smart Blocks builder - for building reusable components
• Dashboard generator - to generate the UI's
  
• Some out of the box trading analytics(VWAP, EWMA, etc) , utilities (e.g. timer) and calculators (P&L, position, etc)
• Some sample trading strategies, such as Statistical arbitrage (StatArb)

Event Manager

The Event Manager is the guts of the system. It processes events guided by a proprietary event definition and scripting language (monitorscript). Events are defined using a simple text-based scheme. For example, here is the definition for a stock quote provided in one of the samples:

event StockTick {
string stockCode;
wildcard integer bidQuant;
wildcard float bidPrice;
wildcard integer askQuant;
wildcard float askPrice;
wildcard integer lastQuant;
wildcard float lastPrice;
}


One of the nice nice things about defining events or objects in this way is the ease in which they are defined. There is also no run time casting like a compiled language - which has both pros and cons. I tend like simplicity - must be simple minded.

Monitor script is more complex. It defines relationships between events. It is designed for speed, to enable processing of large volumes of events looking for potentially complex time-based relationships at high speed. All in flight events are kept in memory to speed access and processing. There are several key constructs but I'll call out 2 in particular:

• event handlers. This is fundamental to Apama and where it is optimized. The language sets filters and actions on events. The filters can be very complex.
• Routers and Emitters - are used to generate new derivative events
  

There is a Java interface to monitorscript that most Java developers use but run time processing uses a proprietary binary data format.

Event Modeller

The Event Modeller is the IDE. While they claim it is for business users, I believe it is to be used by application support working with the trading strategist. While not a full featured IDE one has come to expect from Visual Studio or Eclipse, it is indispensable, particularly in its autogeneration of proprietary MonitorScript. And there is no other IDE thjat generates Apama's Monitorscript.

Here is a screen shot taken from a StatArb example.

Like to See...

Beyond performance metrics which I have not had time to analyze yet, biggest thing I'd like to see are ability to process XML data using XML expressions (such as XPath) and some day to process unstructured text in real time.

Overall does its job of processing complex rules at high speed - which is what high frequency quant traders need.