The Dawn of High Speed Event Processing Software

The High-Speed Data Avalanche
In the highly competitive financial services market, IT organizations are tapping greater numbers of high-speed data sources to uncover new opportunities, identify risk and track increasingly complicated instruments and business scenarios. The data comes from many sources both outside a financial institution (exchanges, customers, news feeds) and inside (networks, servers, applications), driving two distinct groups of applications:

• Business Applications –
  o Algorithmic trading and execution
  o Real-time pricing and offers
  o Customer problem escalation and resolution
  o Risk and compliance management
  o Fraud detection
• IT and Operational Applications –
  o Security monitoring and management
  o Load balancing
  o Infrastructure monitoring
  o Application Grid Monitoring

The problem facing IT organizations lies not just in the volume of data, but also the speed at which the data flows and the response times demanded by the users. Users want this information analyzed in real-time so they can see and act on opportunities and risks as they occur. This can provide a financial services firm a major competitive edge. But this also requires an advanced architecture that can analyze tens to hundreds of thousands of messages per second.

Today’s data infrastructure is ill-equipped to meet this combined requirement of volume and speed. Databases have long been able to manage large datasets, but not at real-time speeds. Enterprise Service Buses can handle the speeds but cannot analyze the messages in any significant way. This leaves financial services IT departments with the task of writing custom applications which are expensive to build, maintain, and expand.

To support the increasing demand for high speed data applications, Financial Services firms are seeking a new breed of packaged software. This software blends technology to meet the speed, volume and latency requirements with the best features of existing data management platforms (language, scalability and reliability) into a single package. A common platform will allow IT team to rollout new high-speed data applications faster and easier, while lowering the total cost of ownership.

Next Generation High-Speed Event Processing Infrastructure
A new breed of high-speed event processing software is emerging to meet the demands of high-speed data applications for financial services and other industries. These infrastructure products meet the high speed and volume requirements of these next generation applications, while providing the development, maintenance and management benefits associated with traditional infrastructure software.

But not all high-speed event processing engines are alike. Financial services applications are deployed in the most demanding IT environments. To be successful in this environment, high-speed event processing software products must address the following rigorous requirements:

1. Enable continuous real-time processing.
   Unlike a database, which executes queries in response to explicit requests, continuous processing enables queries to constantly execute on real-time data to find incoming events or the absence of certain events. This processing may involve filtering, aggregation, complex computations, correlation across multiple sources, and event pattern analysis.
2. Provide a familiar programming model.
   The “learning curve” is the greatest factor slowing the adoption of new software infrastructure, with the programming model being the largest component of this learning curve. For data infrastructure, SQL is the established language, with millions of knowledgeable programmers. Extending SQL to high-speed event processing allows IT organizations to leverage this large knowledge-base to produce applications faster.
3. Manage business events across high-speed and stored data.
   High-speed data can provide basic event information, while stored data provides the remaining, business-level context of the event. High-speed event processing infrastructure must work with real-time, reference, current state, and historical data to provide a 360o context to the business user. And it must work with stored data with the same response times as real-time data to maintain the overall performance of the event application.
4. Support enterprise-class features for performance, availability and manageability.
   High-speed event processing applications are mission-critical, requiring the highest levels of performance and availability, and seamless management. Traditional enterprise-class features such as clustering, fail-over, persistence and distributed management must be supported in high-speed event processing software to support the mission-critical nature of these applications.
5. Offer easy integration with existing infrastructure.
   No application is an island – it must work in concert with existing applications, data and tools. To ease deployment and management, high-speed event processing infrastructure must work with all data sources (enterprise service buses, databases and applications), plug into existing application frameworks (application servers, web services and client tools ) and support open platforms and development tools.

A common platform that supports these requirements can be deployed as a common infrastructure in demanding environments such as financial services, allowing IT organizations to gain the economies of scale that shared infrastructure software provides. It greatly reduces time to deployment of new applications and offers a more cost-effective application development, maintenance and management model than custom coding.

Coral8: The Enterprise-ready Event Processing Engine
Coral8 has created the industry’s first enterprise-class high-speed event processing engine which enables financial services firms to reliably and cost-effectively build, deploy and manage high-speed data applications across the entire organization. The Coral8 engine combines performance, availability and management features found in enterprise-class data management software with a next-generation data engine designed for real-time, continuous processing. Coral8 is based on the industry-standard SQL language to ensure the highest levels of enterprise compatibility and programmer productivity.

The Coral8 engine is designed specifically for real-time performance. The throughput of the Coral8 Engine has been measured at over a millions messages per second, with the latency typically measured in milliseconds. It uses advanced caching facilities to maintain real-time performance levels with stored data. Clustering features allow applications to scale across CPUs, systems and computing GRIDs.

The Coral8 clustering features can also be used to ensure the 24x7 availability of high-speed data applications. State persistence and guaranteed message delivery options are also available to ensure continuous availability. In addition, seamless deployment is supported by allowing new streams, queries and applications to be distributed on the shared infrastructure without restarting the servers.

By extending SQL with continuous processing operators, windows, time series operations, pattern matching, and other operators needed for high-speed data processing, the Coral8 engine has a much lower learning curve than similar products. This enables high levels of programmer productivity. The Coral8 Continuous Computation Language (CCL) does for high-speed data processing what SQL did for database processing while maintaining compatibility with this widely used language.

The Coral8 engine easily plugs into existing IT environments, allowing customers to leverage existing assets. Many common data adapters are built-into the Coral8 engine, including JMS, Tibco Rendezvous, databases and flat-files. Customers can easily create input and output adapters and user-defined functions using an extensive SDK which supports C, C++, Java, .NET, Perl, and Python. The Coral8 studio contains advanced stream application debugging tools, further speeding the development and deployment process.

These enterprise-class features and top-level performance make the Coral8 Engine a good foundation for the high-speed data applications for any financial IT organization. But, the best way to explore high-speed event processing software is to evaluate it yourself. You can download the Coral8 engine and access free developer resources at www.coral8.com. Build your application to see the how easy it is to use the Coral8 engine and take advantage of the powerful, enterprise-class functionality.

Coral8 is a Silicon Valley based software startup focused on the market for high-speed event processing software. The Coral8 Engine is available for an immediate download and evaluation at www.coral8.com. For more information, please contact Coral8 at info@coral8.com.