3. amps_journal_dump¶
The AMPS journal dump utility is used in examining the contents of an
AMPS journal file for debugging and program tuning. The
amps_journal_dump
utility is most commonly used as a tool to debug
the forensic lifespan of messages that have previously been published to
AMPS. The amps_journal_dump
tool is used to show that messages exist
in a journaled topic, and to show the order the message was received in,
and the timestamp associated with the message.
Command Line Options¶
The amps_journal_dump
program has the following options available.
These can also be printed to the screen by typing
amps_journal_dump -help
.
Option | Description |
---|---|
filename
|
Filename of the AMPS journal file. |
-h, –help | Show the program help message and quit. |
-l LIMIT | Limit range of output to entris N:M where N is the first entry and M is the last entry. Passing in a single value, M, will return the first M results. |
–localtime | Display ISO 8601 timestamp in localtime. |
–extents | Add local and replication extents information at the end of the journal dump. |
–no-data | Do not display data values in the journal dump output. |
Table 3.1: Parameters for amps_journal_dump
Looking at the Output¶
In this section will examine some sample output from running
amps_journal_dump
. We will then go over what each of the entries
emitted by the program means.
File Name : ./AMPS.0000000000.journal
File Size : 10485760
Version : amps::txlog/vx
Extents : [1:10]
_________________________________________________
Entry : 5644
CRC : 3483899014
type : publish
entry size : 512
msg len : 11
msg type : fix
localTxId : 5645
sourceTxId : 0
source : 0
client : 13683435528643874114
clientSeq : 1353
topicHash : 10864674256041732524
SOW Key : 18446744073709551615
iso8601 timestamp : 20130612T151247.410553
flags : 0
topic len : 40
auth ID len : 0
topic : [test_topic_name]
auth ID : []
data : [1=1353 2=a ]
_____________________________________________
Total Entries : 1
Total Bytes : 5632
Remaining Bytes : 10480128
Example 3.1: Example of amps_journal_dump output
As is apparent in
Example 3.1,
the output from amps_journal_dump
is split into three sections, a header, a listing
of the contents of the journal file and a footer.
The header contains general information about the journal file as it is represented in the filesystem, and state data about the journal file.
Option | Description |
---|---|
File Name | The name fo the file as it appears on the local filesystem. |
File Size | Number of total bytes allocated by the journal file. |
Version | This is the version of the formatting used to write the data in the journal file. |
Extents | A pair of numbers where the first is the number of extents used by the journal file, and the second is the number of blocks allocated for the journal file. |
Table 3.2: amps_journal_dump listing header.
The second section of the amps_journal_dump
lists each of the
entries contained in the journal file, along with all of the meta-data
used to track and describe the entry. For the sake of simplicity,
Example 3.1
only shows a single listing, but it is more likely that a journal will contain multiple entries.
Option | Description |
---|---|
Entry | A monotonically increasing value representing the order in which the record was inserted into the transaction log file. |
CRC | The cyclic redundancy check used for error checking the message. |
type | The AMPS command used in the original message. |
entry size | The number of bytes allocated to the transaction log record. |
msg len | The number of bytes consumed by the data segment of
the record. |
msg type | The message type used to format the data segment in the record. |
localTxId | The monotonically increasing identifier used across all records local transaction log journal files. |
sourceTxId | The localTxId as it appears on the upstream
replication source. |
source | A unique identifier used to represent the upstream
source of the record. If the source is 0 then the
transaction originated from the current host. |
client | The unique identifier associated with the client that published the message recorded in the transaction log. |
clientSeq | The monotonically increasing sequence identifier from the client. |
topicHash | The unique identifier for the topic the record was published to. |
SOW Key | The unique identifier for the record in the SOW Topic. |
iso 8601 timestamp | The ISO-8601 formatted timestamp representing the time the record was published to the transaction log. Notice that to keep timestamps consistent across instances that may be geographically dispersed, AMPS always timestamps in the UTC timezone. You can, however, ask AMPS to convert timestamps to a different timezone as described in Timestamp section. |
timestamp | The raw timestamp stored in the AMPS transaction log. This is a microsecond-precision timestamp. |
flags | A bitmask used to represent any set flags on the transaction log record. |
topic len | The number of characters in the topic field. |
auth ID len | The number of characters in the authId field. |
topic | The plaintext name of the topic the record was published to. |
authID | The identifier associated with the authentication token used with the publish message. |
data | The raw data contained in the message. |
Table 3.3: amps_journal_dump sample listing
As seen in Example 3.1, the final section contains general usage information about the data contained in the journal file.
Option | Description |
---|---|
Total Entries | Total number of journal entries entered into the journal file. |
Total bytes | The number of reserved bytes consumed by the journal file. |
Remaining Bytes | The number of unused bytes available out of the total reserved file size. |
Table 3.4: amps_journal_dump listing header.
Timestamp formatting¶
The timestamp format used in amps_journal_dump
is formatted by
default using the system timezone for its location. To display the time
in another timezone, the TZ
environment variable can be configured
to modify the output.
%> TZ='America/New_York' ./amps_journal_dump A.000000000.journal
%> TZ='Asia/Tokyo' ./amps_journal_dump A.000000000.journal
%> TZ='Europe/London' ./amps_journal_dump A.000000000.journal
NOTE: This will not work on dates prior to 1970.