3. amps_journal_dump

The AMPS journal dump utility is used in examining the contents of an AMPS journal file for debugging and program tuning. The amps_journal_dump utility is most commonly used as a tool to debug the forensic lifespan of messages that have previously been published to AMPS. The amps_journal_dump tool is used to show that messages exist in a journaled topic, and to show the order the message was received in, and the timestamp associated with the message.

Command Line Options

The amps_journal_dump program has the following options available. These can also be printed to the screen by typing amps_journal_dump -help.

Option Description
Filename of the AMPS journal file.
-h, –help Show the program help message and quit.
-l LIMIT Limit range of output to entris N:M where N is the first entry and M is the last entry. Passing in a single value, M, will return the first M results.
–localtime Display ISO 8601 timestamp in localtime.
–extents Add local and replication extents information at the end of the journal dump.
–no-data Do not display data values in the journal dump output.

Table 3.1: Parameters for amps_journal_dump

Looking at the Output

In this section will examine some sample output from running amps_journal_dump. We will then go over what each of the entries emitted by the program means.

File Name  : ./AMPS.0000000000.journal
File Size  : 10485760
Version    :       amps::txlog/vx
Extents    :               [1:10]
Entry               :                        5644
CRC                 :                  3483899014
type                :                     publish
entry size          :                         512
msg len             :                          11
msg type            :                         fix
localTxId           :                        5645
sourceTxId          :                           0
source              :                           0
client              :        13683435528643874114
clientSeq           :                        1353
topicHash           :        10864674256041732524
SOW     Key         :        18446744073709551615
iso8601 timestamp   :      20130612T151247.410553
flags               :                           0
topic len           :                          40
auth ID len         :                           0
topic               : [test_topic_name]
auth ID             : []
data                : [1=1353 2=a ]

Total Entries    : 1
Total Bytes      : 5632
Remaining Bytes  : 10480128

Example 3.1: Example of amps_journal_dump output

As is apparent in Example 3.1, the output from amps_journal_dump is split into three sections, a header, a listing of the contents of the journal file and a footer.

The header contains general information about the journal file as it is represented in the filesystem, and state data about the journal file.

Option Description
File Name The name fo the file as it appears on the local filesystem.
File Size Number of total bytes allocated by the journal file.
Version This is the version of the formatting used to write the data in the journal file.
Extents A pair of numbers where the first is the number of extents used by the journal file, and the second is the number of blocks allocated for the journal file.

Table 3.2: amps_journal_dump listing header.

The second section of the amps_journal_dump lists each of the entries contained in the journal file, along with all of the meta-data used to track and describe the entry. For the sake of simplicity, Example 3.1 only shows a single listing, but it is more likely that a journal will contain multiple entries.

Option Description
Entry A monotonically increasing value representing the order in which the record was inserted into the transaction log file.
CRC The cyclic redundancy check used for error checking the message.
type The AMPS command used in the original message.
entry size The number of bytes allocated to the transaction log record.
msg len The number of bytes consumed by the data segment of the record.
msg type The message type used to format the data segment in the record.
localTxId The monotonically increasing identifier used across all records local transaction log journal files.
sourceTxId The localTxId as it appears on the upstream replication source.
source A unique identifier used to represent the upstream source of the record. If the source is 0 then the transaction originated from the current host.
client The unique identifier associated with the client that published the message recorded in the transaction log.
clientSeq The monotonically increasing sequence identifier from the client.
topicHash The unique identifier for the topic the record was published to.
SOW Key The unique identifier for the record in the SOW Topic.
iso 8601 timestamp The ISO-8601 formatted timestamp representing the time the record was published to the transaction log. Notice that to keep timestamps consistent across instances that may be geographically dispersed, AMPS always timestamps in the UTC timezone. You can, however, ask AMPS to convert timestamps to a different timezone as described in Timestamp section.
timestamp The raw timestamp stored in the AMPS transaction log. This is a microsecond-precision timestamp.
flags A bitmask used to represent any set flags on the transaction log record.
topic len The number of characters in the topic field.
auth ID len The number of characters in the authId field.
topic The plaintext name of the topic the record was published to.
authID The identifier associated with the authentication token used with the publish message.
data The raw data contained in the message.

Table 3.3: amps_journal_dump sample listing

As seen in Example 3.1, the final section contains general usage information about the data contained in the journal file.

Option Description
Total Entries Total number of journal entries entered into the journal file.
Total bytes The number of reserved bytes consumed by the journal file.
Remaining Bytes The number of unused bytes available out of the total reserved file size.

Table 3.4: amps_journal_dump listing header.

Timestamp formatting

The timestamp format used in amps_journal_dump is formatted by default using the system timezone for its location. To display the time in another timezone, the TZ environment variable can be configured to modify the output.

%> TZ='America/New_York' ./amps_journal_dump A.000000000.journal
%> TZ='Asia/Tokyo' ./amps_journal_dump A.000000000.journal
%> TZ='Europe/London' ./amps_journal_dump A.000000000.journal

NOTE: This will not work on dates prior to 1970.