Subscriber Concerns¶
Once a client places a subscription with AMPS, the AMPS instance ensures that it sends every matching message to that client. If the subscriber loses its connection to AMPS, the subscriber will never see any of the messages that AMPS processed while the subscriber was away. To allow clients to resubscribe at the point where they originally lost connectivity, AMPS provides a bookmark mechanism whereby it stamps messages with a string (the bookmark) that uniquely identifies the message and its order relative to other messages in the transaction log for this instance of AMPS.
By default, AMPS saves network bandwidth by omitting the bookmark on
messages that it sends to a subscriber. Subscribers that are interested
in a bookmark must supply a bookmark when issuing their subscribe
or
command, supplying one of the following as the bookmark:
- A valid bookmark string. Valid bookmark strings identify a message and publisher id. AMPS will begin the subscription at the point in the transaction log immediately following this bookmark.
- The value ``0``. This special value represents the epoch. AMPS
will begin the subscription with the first message in its transaction
log. The AMPS clients include a constant value for this, typically
named
EPOCH
. - The value ``0|1|``. AMPS interprets this value as a request to
begin the subscription with the next message; that is, now. AMPS will
not perform a replay of past messages. The AMPS clients include a
constant value for this, typically named
NOW
. - A specific timestamp in
YYYYmmddTHHMMSS
format. AMPS will begin the subscription immediately after this point in the transaction log.
To implement a client that does not lose messages upon reconnect, the
client must retain the recovery bookmarks for each subscription
and, when re-subscribing, supply these bookmark in the header field of
the subscribe
command. The AMPS HAClient
can track this for
the application using a BookmarkStore
.
If an application requires resilience to
subscriber failure, the application should keep the set of bookmarks
needed for recovery on disk, using a disk-backed BookmarkStore
.
If performance is the greatest concern of the application and the ability
to ensure no lost messages on subscriber failure is not required,
then the set of bookmarks needed for recovery can simply be kept in
memory.
Notice that when using a BookmarkStore
, the application must
discard()
each message after processing it so that the
BookmarkStore
can update the recovery point (since it is no
longer necessary to recover that particular message).
Resubscribing after Disconnect¶
If disconnection occurs, the AMPS server does not automatically reinstate the subscriptions when your client reconnects. Each subscriber is responsible for reissuing subscriptions that it wishes to continue; and for bookmark subscriptions, these must include the bookmark at which the subscription should resume.
The AMPS Client Library provides a facility for remembering the
subscriptions your application has made and reissuing them on reconnect
in the SubscriptionManager
interface. AMPS provides a default
implementation in the MemorySubscriptionManager
class, which the
client automatically creates for you when you use the HAClient
:
HAClient myClient("myClient"); // client name
DefaultServerChooser chooser = new DefaultServerChooser();
chooser.add("tcp://localhost:9004/fix");
myClient.setServerChooser(chooser);
myClient.connectAndLogon();
myClient.subscribe(handler, "someTopic", 10000); // handler defined elsewhere
// someTopic will be resubscribed to, even
// if we disconnect (and reconnect).
In this example, after the subscribe()
call returns successfully,
the HAClient
will resubscribe to someTopic
whenever a
disconnection and subsequent reconnection occurs.
If you would like to implement an application that is capable of
automatically recreating an arbitrary set of subscriptions on start-up,
then you should implement the SubscriptionManager
interface, writing
subscriptions to disk. On recovery of the file in the new subscriber
process, your new SubscriptionManager
must determine how to bind
previous subscriptions to new MessageHandler
instances, and then
re-create the desired subscriptions on the new Client.
Message Receive Order versus Processing Order¶
Some subscribers need to process messages in a different order from which AMPS sends them. Imagine a subscriber that calculates statistics on a moving window of trades. At any given time, it needs to wait for the current window to fill up before calculating and issuing the new output; once it has computed and delivered the output, it can free up the messages and resources associated with that window. This hypothetical subscriber might need to have multiple windows open at any given time as well, because messages do not necessarily arrive in the order that real events occur.
If this hypothetical subscriber failed and restarted, it would need to re-subscribe to AMPS to get messages corresponding to all of the windows that were inprogress at the time it went down. It would be important not to lose any messages that occurred while the subscriber was down. It would also be important not to receive messages again for any windows that are already closed even though those messages might have originally been delivered after messages for windows that are open. In short, the application should not reprocess all messages after the re-subscribe point; only those messages that the application has not yet discarded should be reprocessed.
In this complex (but common) scenario, the order of message processing is different from the order of message reception. Clients seeking to adequately support this scenario must store both a bookmark to resubscribe to ”the earliest message that the application has not discarded” along with the bookmarks associated with the discarded messages after that point. After a resubscribe, the application should filter out incoming messages that have bookmarks matching those already discarded.
The BookmarkStore
implementations provide with the AMPS Client
Library include the ability to handle this situation with filtering
of already delivered and already discarded messages through the
BookmarkStore
interface. The AMPS Client Library provides three
implementations of the BookmarkStore
:
MemoryBookmarkStore
implements an in-memory store that allows you to automatically re-subscribe where you left off, in case of a connection failure or server outage;LoggedBookmarkStore
implements a disk-based store that logs a bookmark for each incoming message and each discarded message, in case of network, server, or subscriber failure;RingBookmarkStore
implements a much faster disk-based store that does not filter discarded messages after the resubscribe point, thus potentially delivering duplicates if your application processes messages in a different order than it receives them. For applications that will always discard messages exactly in the order in which they are received, this could be a lighter-weight choice than theLoggedBookmarkStore
.
When using a BookmarkStore
, it is incumbent upon your application to
call the store’s discard()
method when it is logically finished with
a message, so that AMPS does not redeliver the message upon resubscribe:
public void invoke(Message message)
{
...
myClient.getBookmarkStore().discard(
message.getSubIdRaw(), // The subscription ID
message.getBookmarkSeqNo() // The sequence number of this message.
);
...
}
A subscriber must call discard() as soon as possible
after processing a message. If a subscriber uses a
bookmark store but fails to call discard() on
messages as they are processed, recovery will always
result in duplicate messages. |
Publisher Failures and Duplicate Detection¶
It is possible for a subscriber to receive duplicate messages from AMPS, even when it properly re-subscribes to the bookmark it last saw before disconnection. The following conditions are necessary for this situation to occur:
- The subscriber must have specified the
live
option when placing the subscription, indicating that AMPS should deliver matching messages before they are stored in the transaction log; - There must be a failure of connectivity or an AMPS server that causes publishers and subscribers to fail over to a secondary.
If this scenario occurs, it is possible for publishers to replay a message that the new AMPS instance does not recognize as a duplicate, but that subscribers have already seen. It is therefore possible that the subscriber will, upon connection to the secondary and re-subscription, observe messages arriving from the new AMPS instance that were already delivered by the old instance, thus resulting in apparent duplicates.
To prevent this scenario, subscribers need to determine if an incoming bookmark is an unexpected duplicate of one already seen. Subscribers must parse the bookmark into the publisher ID and sequence number components, which are separated by the | (pipe) character. Subscribers should filter out messages with out-of-order sequence numbers from a given publisher ID. For example, if a subscriber receives messages from the same publisher with sequence numbers 1000 and 1002 followed by 999, it must discard 999.
If the subscriber builds on the AMPS Client Library, then nothing special is required to implement this logic. The following implementations allow you to track the latest message seen per publisher and to discard messages that have already been sent by a previous AMPS instance:
LoggedBookmarkStore
MemoryBookmarkStore