3. amps_journal_dump¶

The AMPS journal dump utility is used in examining the contents of an AMPS journal file for debugging and program tuning. The amps_journal_dump utility is most commonly used as a tool to debug the forensic lifespan of messages that have previously been published to AMPS. The amps_journal_dump tool is used to show that messages exist in a journaled topic, and to show the order the message was received in, and the timestamp associated with the message.

Command Line Options¶

The amps_journal_dump program has the following options available. These can also be printed to the screen by typing amps_journal_dump -help.

Option	Description
filename	Filename of the AMPS journal file.
-h, –help	Show the program help message and quit.
-l LIMIT	Limit range of output to entries N:M where N is the first entry and M is the last entry. Passing in a single value, M, will return the first M results.
–localtime	Display ISO 8601 timestamp in localtime.
–extents	Add local and replication extents information at the end of the journal dump.
–no-data	Do not display data values in the journal dump output.

Table 3.1: Parameters for amps_journal_dump

Looking at the Output¶

In this section will examine some sample output from running amps_journal_dump. We will then go over what each of the entries emitted by the program means.

File Name       : AMPS-Sample.0000000000.journal
File Size       : 10485760
Version         : amps::txlog/v8
Extents         : [1623783047000000005:1623783047000002563]
First Timestamp : 20210615T185233.126619Z
Last Timestamp  : 20210615T185234.032520Z
_____________________________________________________
Entry               :                              0
CRC                 :                      175497491
type                :                        publish
flags               :                           none
offset              :                           4096
entry size          :                            256
msg len             :                             10
msg type            :                           json
localTxId           :            1623783047000000005
previousLocalTxId   :                              0
sourceTxId          :             436334058474393276
source              :                              0
client              :           16094765439783135993
clientSeq           :            1623783047000000005
topicHash           :            9914047442222214141
SOW expiration      :                              0
iso8601 timestamp   :        20210615T185233.126619Z
timestamp           :             212490586353126619
previous size       :                              0
topic len           :                              5
topic               : [abcde]
data                : [{"test":1}]
correlation id len  :                              0
correlation id      : []
auth ID len         :                              0
auth ID             : []
rep path len        :                              0
rep path            : []
_____________________________________________________

...

Total Entries    : 2559
Total Bytes      : 10485760
Remaining Bytes  : 0

Example 3.1: Example of amps_journal_dump output

As is apparent in Example 3.1, the output from amps_journal_dump is split into three sections, a header, a listing of the contents of the journal file and a footer.

The header contains general information about the journal file as it is represented in the filesystem, and state data about the journal file.

Option	Description
File Name	The name of the file as it appears on the local filesystem.
File Size	Number of total bytes allocated by the journal file.
Version	This is the version of the formatting used to write the data in the journal file.
Extents	A pair of numbers where the first is the number of extents used by the journal file, and the second is the number of blocks allocated for the journal file.

Table 3.2: amps_journal_dump listing header.

The second section of the amps_journal_dump lists each of the entries contained in the journal file, along with all of the meta-data used to track and describe the entry. For the sake of simplicity, Example 3.1 only shows a single listing, but it is more likely that a journal will contain multiple entries.

The exact options present depend on the version of the journal file.

Option		Description
Entry		A monotonically increasing value representing the order in which the record was inserted into the transaction log file.
CRC		The cyclic redundancy check used for error checking the message.
type		The AMPS command used in the original message or the type of entry in the journal file.
offset		Offset for this entry.
flags		Flags for this entry.
entry size		The number of bytes allocated to the transaction log record.
msg len		The number of bytes consumed by the `data` segment of the record.
msg type		The message type used to format the data segment in the record.
localTxId		The monotonically increasing identifier used across all records local transaction log journal files.
previousLocalTxId		The monotonically increasing identifier used across all records local transaction log journal files.
sourceTxId		The `localTxId` as it appears on the upstream replication source.
source		A unique identifier used to represent the upstream source of the record. If the source is `0` then the transaction originated from the current host.
client		The unique identifier associated with the client that published the message recorded in the transaction log.
clientSeq		The monotonically increasing sequence identifier from the client.
topicHash		The unique identifier for the topic the record was published to.
iso 8601 timestamp		The ISO-8601 formatted timestamp representing the time the record was published to the transaction log. Notice that to keep timestamps consistent across instances that may be geographically dispersed, AMPS always timestamps in the UTC timezone.
timestamp \| The raw timestamp stored in the AMPS transaction log. This is a microsecond-precision timestamp.
previous size \| Size of the previous entry in the txlog. .
topic len		The number of characters in the topic field.
topic		The topic for this entry in the transaction log.
data		The data for this entry in the transaction log.
auth ID len		The number of characters in the authId field.
authID		The identifier associated with the authentication token used to enter this command.
correlation id len		The length of the correlation ID associated with this command.
correlation id		The correlation ID associated with this command.
auth ID len		The length of the authentication ID associated with this command.
auth ID		The authentication ID associated with this command.
rep path len		The length of the replication path associated with this command.
rep path		The replication path associated with this command.

Table 3.3: amps_journal_dump sample listing

As seen in Example 3.1, the final section contains general usage information about the data contained in the journal file.

Option	Description
Total Entries	Total number of journal entries entered into the journal file.
Total bytes	The number of reserved bytes consumed by the journal file.
Remaining Bytes	The number of unused bytes available out of the total reserved file size.

Table 3.4: amps_journal_dump listing header.

Timestamp formatting¶

The timestamp format used in amps_journal_dump is formatted by default using the system timezone for its location. To display the time in another timezone, the TZ environment variable can be configured to modify the output.

%> TZ='America/New_York' ./amps_journal_dump A.000000000.journal

%> TZ='Asia/Tokyo' ./amps_journal_dump A.000000000.journal

%> TZ='Europe/London' ./amps_journal_dump A.000000000.journal

NOTE: This will not work on dates prior to 1970.

Table Of Contents

Related Topics

3. amps_journal_dump¶

Command Line Options¶

Looking at the Output¶

Timestamp formatting¶