11. Transaction LogΒΆ
AMPS includes the ability to record and replay messages. This capability
can be used by applications for durable subscriptions, reliable publish,
and historical replay. The AMPS transaction log is also the foundation
of the high availability features in AMPS. To enable message recording
and replay, configure a TransactionLog
to keep a journal of messages
published to an AMPS instance. The Transactional Messaging and Bookmark Subscriptions chapter in the AMPS User Guide covers how to use the
transaction log for historical replay, durable publish, and durable
subscriptions. The Replication and High Availability chapter in the AMPS User Guide covers the use cases where
a TransactionLog
can be used to maximize the up-time of your
AMPS instance.
Element | Description |
---|---|
JournalDirectory
|
Filesystem location where journal files will be stored. This is the directory where AMPS will create new journal files as messages are recorded to the transaction log. A journal directory should be dedicated to a single instance of AMPS. |
JournalArchiveDirectory |
Filesystem location where journal files are archived. The archive directory is intended to allow older files to be stored on a higher-capacity (but potentially slower) device. Journal files in this directory are part of the transaction log. AMPS can replicate from these files, provide bookmark replay from these files, distribute queue messages from these files, and so on. A journal archive directory should be dedicated to a single instance of AMPS. Use AMPS actions to move files from the |
PreallocatedJournalFiles |
The number of journal files AMPS will create as part of the server startup. Default: 2. Minimum: 1 |
JournalSize |
Sets the target size for AMPS to use when calculating the size of journal files. AMPS allocates journal files based on the size of an internal buffer. This
option sets the target size for the journal file: AMPS will use the smallest
file size that is an even multiple of the internal buffer
without going under the specified AMPS accepts Default: 1GB. Minimum: 10M |
Topic |
The topic to include in the transaction log. When no Multiple There is no default for this element. If no |
FlushInterval |
AMPS batches writes to the transaction log to optimize for maximum sustained throughput. If a batch is not full within a certain period of time, AMPS will write the partially-filled batch to the transaction log so that the messages can be replicated, delivered to subscribers, and so on. The interval at which messages will be flushed the journal file during periods of slow activity is the FlushInterval. Setting this explicitly has the potential to reduce latency during periods of low traffic, at the risk of somewhat lower performance during periods of higher traffic. Reducing this interval below the default may produce larger journal files during periods of low traffic, since AMPS may write more partial batches to the transaction log. 60East recommends leaving this option at the default unless the application is intended to optimize for low data rates, and testing at production volumes demonstrates a performance advantage from reducing the interval. Default: 100ms Maximum: 1000ms Minimum: 1us |
O_DIRECT |
Where supported, 60East does not recommend disabling Default: enabled. |
InactiveClientAckExpiration |
Sets the amount of time to retain records for an inactive publisher. On
recovery, AMPS will remove the This option can help reduce Default: retain records indefinitely |
CompressedJournalCacheMemoryLimit |
Sets the maximum amount of server memory to use for caching compressed journal files when one or more bookmark replays is using a compressed journal file. Increasing this setting can improve performance when replaying bookmark subscriptions from compressed journals, in exchange for temporarily consuming more memory to service those replays. Default: 10% of server memory or 10GB, whichever is lower |
DEPRECATED
BatchSize |
This element is no longer necessary in releases of AMPS 4.0 and greater. If this element is present in the configuration, AMPS emits a deprecation warning and ignores the configured value. |
TransactionLog Configuration Parameters
The Topic
element in a TransactionLog
can contain the following elements:
Element | Description |
---|---|
Name
|
The name of the topic to record. This element can be a literal name or a regular expression. This element is required. There is no default for this element. |
MessageType
|
The message type of the topic. This must be one of the message types loaded by default or a message type declared in the configuration file. This element is required. There is no default for this element. |
TransactionLog Topic Configuration Parameters
The following example demonstrates a
transaction log where the journal file will be written to ./amps/journal
. When AMPS starts, a single
journal file will be pre-allocated as noted by the
PreallocatedJournalFiles
setting; and when the first journal file is
completely full, a new journal file will be created. This journal is
going to contain only those messages which match the topic orders
and also have a message type of fix
. If, at any time, there is 40us
of inactivity while there is data to be flushed to the journal file,
AMPS will proactively flush the data to the file.
<AMPSConfig>
...
<TransactionLog>
<JournalDirectory>./amps/journal/</JournalDirectory>
<PreallocatedJournalFiles>1</PreallocatedJournalFiles>
<MinJournalSize>10MB</MinJournalSize>
<Topic>
<Name>orders</Name>
<MessageType>nvfix</MessageType>
</Topic>
<Topic>
<Name>LOGGED_.*</Name>
<MessageType>json</MessageType>
</Topic>
</TransactionLog>
...
</AMPSConfig>
Transaction Log Configuration Example