StreamInsight is relatively new but already Synthesis Consulting is working with Microsoft and RapidAddition to help build an adapter for the FIX protocol. This puts us in a unique position to offer our consultants who have working relationships with key product teams and a deep understanding of StreamInsight long before most consulting companies even read the wikipedia page. Which by the way doesn't exist (as at 4/1/2011) there's only a small mention on SQL Server's page:

SQL Server 2008 R2 includes a number of new services, including PowerPivot for Excel and SharePoint, Master Data Services, StreamInsight, ReportBuilder 3.0, Reporting Services Add-in for SharePoint, a Data-tier function in Visual Studio that enables packaging of tiered databases as part of an application, and a SQL Server Utility named UC (Utility Control Point), part of AMSM (Application and Multi-Server Management) that is used to manage multiple SQL Servers.

Not to worry, Microsoft comes through:

Microsoft StreamInsight™ is a powerful platform that you can use to develop and deploy complex event processing (CEP) applications. Its high-throughput stream processing architecture and the Microsoft .NET Framework-based development platform enable you to quickly implement robust and highly efficient event processing applications. Event stream sources typically include data from manufacturing applications, financial trading applications, Web analytics, and operational analytics. By using StreamInsight, you can develop CEP applications that derive immediate business value from this raw data by reducing the cost of extracting, analyzing, and correlating the data; and by allowing you to monitor, manage, and mine the data for conditions, opportunities, and defects almost instantly.


CEP stands for Complex Event Processing and according to Wikipedia that means:

Complex event processing (CEP) consists of processing many events happening across all the layers of an organization, identifying the most meaningful events within the event cloud, analyzing their impact, and taking subsequent action in real time.

Complex event processing refers to process states, the changes of state exceeding a defined threshold of level, time, or value increment or just of a count as the event. It requires the respective event monitoring, event reporting, event recording and event filtering. An event may be observed as a change of state with any physical or logical or otherwise discriminated condition of and in a technical or economical system, each state information with an attached time stamp defining the order of occurrence and a topology mark defining the location of occurrence.

What all that means is StreamInsight as an add-on to Microsoft SQL Server that allows you process a stream of events at or near enough to real time. Leveraging proven frameworks like .NET and LINQ, StreamInsight offers organisations an approach to complex event processing on a platform that you're already comfortable with and probably have running.

With RFID sensors becoming more cost effective and even mandatory, algorithmic trading is placing and increasing number of trades and the need to analyize thousands of clicks a second. The IT industry has been looking more and more at CEP offerings and finally Microsoft has entered this domain. The world of CEP is a complex one and StreamInsight is a step towards making that world more accessible.


With complex event processing on the Microsoft stack being relatively new, we're still learning. Having said that we're learning by working closely with the Microsoft product team to build adapters for the finance industry.

This experience can help your organisation implement and development adapters and processing engines that leverage StreamInsight. There aren't many industries as demanding in terms of latency, performance and complex transactions or events as financial markets. What we learn is easily transferable to any vertical.