We understand that market data, like many other fields, has its own set of jargon. So we thought it might be helpful for you to see what we mean.
The Feed Handler (FH) application is responsible for reliably receiving the particular Exchange feed (or other data source) and passing the contents to the next application tier (the Translator). The Translator (T) application receives the raw data from the Feed Handler, translates it into an internal common format and will also typically enrich the data (by calculating value-added fields). The Translator application is usually deployed on the same physical server as the Feed Handler, and the 2 applications together are referred to as a Feed Handler / Translator.
The Upstream Content Server (UCS) applications are only ever present at an ACTIV data center. They receive multiple enriched, translated message streams from the FH/T applications or Contribution Gateways and generate a smaller number of compressed, higher-bandwidth, consolidated ActivFeed streams. Each ActivFeed stream includes a forward error correcting (FEC) code and diverse routing capability which together ensure reliable WAN delivery. At the client site, the ActivFeed stream is received and processed by the Downstream Content Server. The UCS also cycles through every record in each UCS database table, filling any spare output bandwidth with full refresh record images. This enables the last-value cache in the Downstream Content Server applications to be seeded as well as enabling them to recover from any feed loss (e.g. due to loss of primary and backup connectivity).
The Downstream Content Server (DCS) applications can receive both ActivFeed data (delivered from an ACTIV data center) and ActivFeed Direct data (delivered from locally-sited FH/T applications). The DCS builds a complete, persisted, last-value cache of all received records. The DCS also contains a rules engine that is used for generating additional derived fields (e.g. highs, lows, volumes etc) when a record is received. This helps minimize bandwidth usage on ActivFeed as well as helping to simplify Translator applications. The DCS cache can be queried by the Content Gateway, and updates from the DCS are distributed out to the Content Gateway and several other optional server applications that can be connected to the DCS.
The Content Gateway is the application that acts as the server for CG API-based applications. It accepts logons from the CG API, authenticates them, and accepts requests for data. If the CG doesn’t have a data element cached, then it will query the last-value cache in the DCS for the item. The CG enforces a data permissioning model and will only pass on data that an API is permissioned for. It also maintains watch lists for CG API-based applications and, as updates are received by the CG from the DCS, it will forward on as appropriate.
Time Series Server (TSS) processes the output stream from the DCS and the CS.
Unlike the DCS, the TSS databases quote and trade data tick-by-tick and also supports bar building. It also contains a historical database of daily values that can go back in time up to 15 years. The CG can connect back to this server and satisfy time-series requests from ActivContentGateway API-based applications.
New Server processes the output stream from the DCS and the CS. The NS can store and index millions of news stories that are received. The NS database supports fast searching (30 millisecond response time average) for stories by date, symbol, category code, headlines and story body content. The CG can connect back to the NS and satisfy news story requests from ActivContentGateway API-based applications.
ActivContentGateway API is a programming interface used for querying and subscribing to any data held in the ACP. It connects to the Content Gateway. Its clean, modern, high-performance design enables 30,000 or more records to be snapshot per second. It has been specifically engineered to work well in a WAN (as well as a LAN) environment:
- It operates over TCP and is Network Address Translation (NAT) friendly.
- The wire-protocol minimizes bandwidth by using bandwidth efficient binary formats.
- Advanced server-side filtering (field-level and event-type) further reduces bandwidth.
- Server-side streaming software and hardware compression facilities can be enabled from the client-end, reducing bandwidth further.
- The wire protocol supports compound requests and is not “chatty”, enabling large numbers of records to be retrieved with minimal round-trips.
The Contribution Server (CS) application has the same functionality as a DCS but receives content from client applications rather than from ACTIV FH/T or UCS applications. Data can be published into the CS using either the ActivContributionServer API or the ActivContribution Gateway API.
The Contribution Gateway accepts connections from ActivContributionGateway API-based applications. It passes data from these applications through to the CS and enforces a permissioning model that can restrict:
The CG is usually deployed at a client site but can be hosted at an ACTIV data center supporting remoteapplications which contribute data into hosted DCS applications or into the consolidated ActivFeed streams via the UCS.