29. Sample Use Cases

To further your understanding of AMPS, we provide some sample use cases that highlight how multiple AMPS features can be leveraged in larger messaging solutions. For example, AMPS is often used as a back-end persistent data store for client desktop applications.

The provided use case shows how a client application can use the AMPS command sow_and_subscribe to populate an order table that is continually kept up-to-date. To limit redundant data from being sent to the GUI, we show how you can use a delta subscription command. You will also see how to improve performance and protect the GUI from over-subscription by using the TopN query limiter along with a stats acknowledgment.

View Server Use Case

Many AMPS deployments are used as the back-end persistent store for desktop GUI applications. Many of the features covered in previous chapters are unique to AMPS and make it well suited for this task. In this example AMPS will be act as a data store for an application with the following requirements:

  • allow users to query current order-state (SOW query)

  • continually keep the returned data up to date by applying incremental changes (subscribe)

    For purposes of highlighting the functionality unique to AMPS, we’ll skip most of the details and challenges of GUI development.

Setup

For this example, let’s configure AMPS to persist FIX messages to the topic ORDERS. We use a separate application to acquire the FIX messages from the market (or other data source) and publish them into AMPS. AMPS accumulates all of the orders in its SOW persistence, making the data available for the GUI clients to consume.
AMPS View Server Deployment Configuration

Figure 28.1: AMPS View Server Deployment Configuration

SOW Query and Subscription

The GUI will enable a user to enter a query and submit it to AMPS. If the query filter is valid, then the GUI displays the results in a table or “grid” and continually applies changes as they are published from AMPS to the GUI. For example, if the user wants to display active orders for Client-A, then they may use a query similar to this:

/11 = 'Client-A' AND /39 IN (0, 'A')

This filter matches all orders for Client-A that have FIX tag 39 (the FIX order status field) as 0 (’New’) or ’A’ (’Pending New’).

From a GUI client, we want to first issue a query to pull back all current orders and, at the same time, place a subscription to get future updates and new orders. AMPS provides the sow_and_subscribe command for this purpose.

tip A more realistic scenario may involve a GUI Client with multiple tables, each subscribing with a different AMPS filter, and all of these subscriptions being managed in a single GUI Client. A single connection to AMPS can be used to service many active subscriptions if the subscription identifiers are chosen such that they can be demultiplexed during consumption.

The GUI issues the sow_and_subscribe command, specifying a topic of ORDERS and possibly other filter criteria to further narrow down the query results. Once the sow_and_subscribe command has been received by AMPS, the query returns to the GUI all messages in the SOW that, at the moment, match the topic and content filter. Simultaneously, a subscription is placed to guarantee that any messages not included in the initial query result will be sent after the query result.

The GUI client then receives a group_begin message from AMPS, signaling the beginning of a set of records returned as a result of the query. Upon receiving the initial SOW query result, this GUI inserts the returned records into the table, as shown in Figure 28.2. Every record in the query will have assigned to it a unique SowKey that can be used for future updates.

The receipt of the group_end message serves as a notification to the GUI that AMPS has reached the end of the initial query results and going forward all messages from the subscription will be live updates.

AMPS GUI Instance With sow_and_subscribe

Figure 28.2: AMPS GUI Instance With sow_and_subscribe

Once the initial SOW query has completed, each publish message received by the GUI will be either a new record or an update to an existing record. The SowKey sent as part of each publish message is used to determine if the newly published record is an update or a new record. If the SowKey matches an existing record in the GUI’s order table, then it is considered an update and should replace the existing value. Otherwise, the record is considered to be a new record and can be inserted directly into the order table.

For example, assume there is an update to order C that changes the order status (tag 39) of the client’s ORCL order from ’A’ to 0. This is shown below in Figure 28.3

AMPS Mesage Publish Update

Figure 28.3: AMPS Mesage Publish Update

Out-of-Focus (OOF) Processing

Let’s take another look at the original filter used to subscribe to the ORDERS SOW topic. A unique case exists if an update occurs in which an ORDER record status gets changed to a value other than 0 or ’A’. One of the key features of AMPS is OOF processing, which ensures that client data is continually kept up-to-date. OOF processing is the AMPS method of notifying a client that a new message has caused a SOW record’s state to change, thus informing the client that a message which previously matched their filter criteria no longer matches or was deleted. For more information about OOF processing, see 9. Out-of-Focus Messages (OOF).

When such a scenario occurs, AMPS won’t send the update over a normal subscription. If OOF processing is enabled within AMPS by specifying the oof option for this subscription, then updates will occur when previously matching records no longer match due to an update, expiration, or deletion.

For example, let’s say the order for MSFT has been filled in the market and the update comes into AMPS. AMPS won’t send the published message to the GUI because the order no longer matches the subscription filter; AMPS instead sends it as part of an OOF message. This happens because AMPS knows that the previous matching record was sent to the GUI client prior to the update. Once an OOF message is received, the GUI can remove the corresponding order from the orders table to ensure that users see only the up-to-date state of the orders which match their filter.

AMPS OOF Processing

Figure 28.4: AMPS OOF Processing

Conclusion and Next Steps

In summary, we have shown how a GUI application can use the sow_and_subscribe command to populate an order table, which is then continually kept up-to-date. AMPS can create further enhancements, such as those described below, that improve performance and add greater value to a GUI client implementation.

sow_and_delta subscribe

The first improvement that we can make is to limit redundant data being sent to the GUI, by placing a sow_and_delta_subscribe command instead of a sow_and_subscribe command. The sow_and_delta_subscribe command, which works with most AMPS message types message types, can greatly reduce network congestion as well as decrease parsing time on the GUI client, yielding a more responsive end-user experience.

With a delta subscription, AMPS Figure 28.3 sends to the subscriber only the values that have changed: C:39=0 instead of all of the fields that were already sent to the client during the initial SOW query result. This may seem to make little difference in a single GUI deployment; but it can make a significant difference in an AMPS deployment with hundreds of connected GUI clients that may be running on a congested network or WAN.

TopN and Stats

We can also improve client-side data utilization and performance by using a TopN query limiter with a stats acknowledgment, which protects the GUI from over-subscription.

For example, we may want to put a 10,000 record limit on the initial query response, given that users rarely want to view the real-time order state for such a large set. If a TopN value of 10000 and an AckType of stats is used when placing the initial sow_and_subscribe command, then the GUI client would expect to receive up to 10,000 records in the query result, followed by a stats acknowledgment.

The stats acknowledgment is useful for tracking how many records matched and how many were sent. The GUI client can leverage the stats acknowledgment metrics to provide a helpful error to the user. For example, in a scenario where a query matched 130,000 messages, the GUI client can notify the user that they may want to refine their content filter to be more selective.

In the AMPS clients, a stats acknowledgment is returned to the client after the group_end for the query with an acknowledgment command type. The message contains statistics about the query. See the AMPS Command Reference for details on the stats acknowledgment message. See the API documentation for your development language of choice for information on processing messages.