Application Example: Market Data Handling

[Terry_ALE]

Market Data Handling is the most popular application of FPGAs in the low latency, high throughput domain; see article below. Market Data (http://en.wikipedia.org/wiki/Market_data) comes from various exchanges, ECNs, ATS, etc. typically in the form of UDP multicast.  This contains ALL information on thousands of financial instruments (equities/stocks, options, currency, etc.).  The datarate is in the 1 to 10 Gigabit/sec ranage.  The data is not alway coming in at the full rate but there are burst periods when trading is very active that it becomes much more than even the best computers can handle.  These computers are also running algorithms, models, etc. so the less data they have to deal with, the better.  For example if a trader or algorithm is only looking at a particular sector or group of say 100 stocks, why not find filter out the symbols (stocks) that are not of interest?  It should be noted that during peak trading periods are when the most money is be made and lost so it is not acceptable for one to say "we'll just turn off our machines during the busy periods."

The basic approach is an FPGA placed directly after a PHY chip so the system can process Ethernet Frames as they come in.  The data formats are public and available so one knows exactly how the data is going to look.  FPGAs are preferred over ASICs because the formats do change from time to time and the field programmability of FPGAs are the best approach for this.  The determinism of hardware increases the determinism of the entire trading system.  BATS (http://batstrading.com/) is one of the newest (since about 2005) exchanges and one their data format is called PITCH.  See this link http://batstrading.com/resources/membership/BATS_MC_PITCH_Specification.pdf for US Equities/Options Multicast PITCH specification; see section 4.0 for the actual trade message formats.  Note the date and version of this document (Version 2.9.6 February 2, 2011).  Data formats can change many times in a single year making it important to be able to update the FPGA quickly (let alone programming it).  Each exchange has their own format, NYSE, Nasdaq, London Stock Exchange, etc.

See the following article http://www.a-teamgroup.com/article/fpgas-established-for-market-data-now-being-leveraged-for-transac... A-Team Group provides analysis and news on business of financial information technology.

FPGAs, Established for Market Data, Now Being Leveraged for Transactions, Pre-Trade Risk

07 Dec 2010 

Field Programmable Gate Arrays (FPGAs) - providing hardware-based acceleration for processing typically performed in software - are being increasingly adopted for financial markets applications, with signs that their use is moving beyond boosting market data delivery, to address transactional performance as well. Fixnetix and Wall Street FPGA are providing two such examples, as hardware acceleration veterans like Activ Financial Systems roll out the latest versions of its ticker plant, doubling performance.

Activ Financial - which first introduced FPGA technology into its market data processing units in 2007, last month unveiled its Activ MPU Gen2 ticker plant, providing higher performance with more flexibility, and a smaller deployment footprint.

According to Shawn Kaplan, director of business development at Activ, the company’s first foray into leveraging FPGAs convinced them of the performance benefits to be attained, but noted that actual implementation of them - using a third-party FPGA card - tied the company to an “appliance approach,” which was not the original intent.

With Gen2, the company has designed its own PCIe card, which can be installed in just about any PC host. The card includes an Altera Stratix IV FPGA - twice the power of the Stratix II originally used - to provide performance that equates to the processing of five million market data messages per second.

Activ is selling Gen2 either as a PCIe card for others to integrate into PCs, or along with a PC host, which can support multiple cards. Activ sells its ticker plants along with both consolidated and direct data feeds from financial markets sources worldwide.

Kaplan says that Activ will benefit from having design control over the PCIe card, and suggests that future developments will see cards with Ethernet ports, so that entire direct feed handlers can be implemented on a single card.

Activ isn’t the only vendor of FPGA-accelerated market data solutions. Rival Exegy is continuing to develop its ticker plant offerings, most recently rolling out Data Tone, a bundled ticker plant and data service offering, in conjunction with broker/dealer Essex Radez.

Also, Paris-based NovaSparks - founded in 2007 as HPC Platform - offers its own FPGA ticker plant product, with support for feeds formats, such as Fast and Nasdaq’s Itch, and incorporating order book and index computation. The company has gained momentum in recent months through partnerships with the likes Solace Systems, which sells hardware-accelerated messaging middleware.

But just as FPGAs are becoming widely accepted for market data processing, so their use to accelerate transactional applications is now gaining momentum.

Seattle-based In-FPGA Trading Systems recently emerged with an FPGA-based “reference design,” which demonstrates Itch data feed handling, and order entry via the Ouch feed specification, in under two microseconds. The company is providing a developer platform (also using the Altera Stratix IV FPGA), and consulting services and education specifically to financial markets participants.

Meanwhile, New York City-based Wall Street FPGA recently announced a prototype of an FPGA-accelerated open-source QuickFIX gateway, allowing the FPGA logic to very rapidly cancel a large number of open orders, as is typically required in high frequency trading.

According to Wall Street FPGA executive director Terry Stratoudakis, the company plans to prototype an entire order matching engine, by leveraging FPGAs to process a number of FIX message types, as well as introduce network monitoring tools, such as a network taps and sniffers, incorporating IEEE 1588 timestamping.

Perhaps the most interesting and substantive development in deploying FPGAs in transactional environments has come from market data and connectivity specialist Fixnetix, which recently announced its iX-ecute initiative aimed at massively speeding up pre-trade risk processing. Fixnetix plans to offer iX-ecute along with its array of data, co-location and trade execution services.

With the recent introduction of regulations to ban so-called naked access, responsibility is falling on those offering sponsored direct market access (DMA) to provide a set of real-time controls and checks on incoming orders before routing them to markets for execution. With such processing working against the continuing push to lower latency, an area of significant competition among DMA providers, Fixnetix reckons its hardware-accelerated approach can be implemented with negligible overhead.

Says Fixnetix’s director of trading solutions Matt Dangerfield, iX-ecute has been designed as a pure FPGA solution, leveraging Xilinx Vertix devices on custom-designed PCIe cards, of which up to 16 can be fitted into a rack-mountable 4U chassis. The only requirement for a PC is for command and control purposes, while high availability features allow for the workload on one PCIe card to be transferred seamlessly to another when potential fail conditions are detected.

Dangerfield says that standard functionality for iX-ecute will be support for a range of message protocols, such as FIX and Ouch, and a number of commonly required pre-execution checks, including order quantity, quantity * price, % away from last tick or open, and matching against a restricted list of counter parties. New message protocols and risk checks can be added with about eight weeks notice.

Prototype tests against the London Stock Exchange have shown client-to-market latency of around eight microseconds, with 1 gigabit Ethernet. Dangerfield expects this to drop to less than 2.5 microseconds, with 10 gigabit Ethernet, early in 2011 when the service is expected to go live for the first customers. As well as risk checks, the FPGAs also perform TCP/IP processing and FIX message validation.

According to Henry Young, director at TS-Associates, which leverages FPGAs in its latency monitoring TipOff appliance, the focus for hardware acceleration in transactions is not just about reducing latency, but also on improving deterministic performance - a reduction of jitter - which allows for better tuning of trading algorithms. Young suggests, though, that to be more successful in that endeavor, exchanges and other execution venues need to look to deploy FPGAs in their trading gateways.

Despite the uptick in FPGA activity, Activ’s Kaplan cautions that creating software for FPGA environments is still challenging, and that hiring developers with specialist skills - in particular the low-level VHDL required to map business functionality to the FPGA logic - is required to reap real performance benefits.

That said, the emergence in the financial markets space of technology vendors, such as Impulse Accelerated Technology, which offers the Impulse C optimising compiler specifically for hardware co-processors, and Stone Ridge Technology, which offers development platforms and expertise for implementing software on FPGAs, is making the adoption of FPGAs more viable, for vendors and trading firms alike.

[Terry_ALE]

Video interview of Activ Financial, one of the providers of hardware accelerated (FPGA) based market data handling:

http://www.youtube.com/user/TabbGroup#p/a/u/0/BrdCAcTg3rc

[Terry_ALE]

Market Data using LabVIEW FPGA

http://fpganow.com/index.php/2017/01/19/filter-market-data-inside-an-fpga/

[JeffinCA]

---

@Terry_ALE wrote:

Market Data using LabVIEW FPGA

http://fpganow.com/index.php/2017/01/19/filter-market-data-inside-an-fpga/

---

  This is very useful information.  I wish I had noticed the notification sooner, but I'm happy to have it.

Thanks,

Jeff

[AAlZibdeh]

Hi Jeff, 

I'm interested in learning more about your application. Is it possible to share more information?

Thanks, 

Abdullah 

Account Manager

National Instruments

[JeffinCA]

Hello, Abdullah:

I haven't written an application, yet.  My goal is to have a utility that will track my investments to automatically update a spreadsheet and analyze the trends.

To do that, I will need to be able to acquire data from remote sites (something I haven't done, yet).

Regards,

Jeff

[Gyc]

I realize this is an old post, but I am interested in getting/accessing trading data using labview.

While I found a number of python libraries to connect to trading providers, I found nothing with LabView.

Did you make any progress on this subject that you can share?