14. amps_bio_perf_test¶
AMPS contains a utility, amps_bio_perf_test, that measures the
raw performance of storage devices for the type of sequential writes (and,
optionally, reads) that are used by the AMPS transaction log.
The utility can be used to get a sense of the maximum throughput that a given device offers, and to compare one device to another.
Options and Parameters¶
| Option | Description |
|---|---|
-b,--batch_size |
The number of messages to be batched (1 to 256) |
-m,--message_size |
The message size in bytes (256 to 128K, must be a multiple of 256) |
-f,--file_size |
Size of the data file to write to, in MB (max 8192) |
-r,--read_mix |
Read mix specified as a percentage (0 - 99) |
-f,--file |
Name of the data file that is created (this should be on the device to be tested). The file will be allocated before the test begins running, and will contain nonsense data. |
Table 14.1: Parameters for amps_bio_perf_test
60East recommends running the amps_bio_perf_test tool on a system
that is otherwise quiet. The --file specified is the file that
the tool creates for writing data, so that file must be on the
device to be measured.
The amps_bio_perf_test uses the O_DIRECT flag for writes
(as does the AMPS transaction log writer), bypassing caching
at the operating system level.
Warning
The amps_bio_perf_test tool attempts to fully use
the I/O capacity of the target device. Do not run this
tool on a system that is being used for another purpose,
since the tool may impact the performance of any other
process using the device being tested.
Output¶
The amps_bio_perf_test tool produces the following metrics:
| Metric | Explanation |
|---|---|
| elapsed time | Time during which the tool was actively writing/ reading data |
| writer thread count | Number of threads writing data (always 1) |
| batch size | Number of messages (submissions) in a batch |
| submit size | Size of each message (submission), in bytes |
| write size | Total size of each batch written, in bytes |
| submit count | Number of messages (submissions) during the test |
| read count | Number of read operations during the test |
| write count | Number of write operations during the test |
| submit rate | Average number of messages (submissions) per second while the tool was actively writing/reading data |
| write rate | Average number of writes (batches written) per second while the tool was actively writing/reading data |
| actual read mix | Proportion of reads to writes during the test (this may differ from the target due to thread scheduling, etc.) |
| mean write latency | Mean latency for a write, in microseconds |
Table 14.2: Output fields for amps_bio_perf_test
The tool also writes csv files with the raw data used to calculate these metrics, which can be helpful for more in-depth analysis (for example, calculating the 99th percentile latency rather than the mean). One file contains the full set of latency samples collected, the other file contains the full set of throughput samples collected. If the files do not exist when the tool is run, it will create them. Otherwise, the tool will append to the files so that results can be tracked over multiple runs.
Tips on Evaluating Overall Performance¶
The amps_bio_perf_test tool tries to simulate the AMPS transaction
log workload under ideal conditions for maximum throughput. The tool
preallocates the file that it writes to, and also pregenerates all of
the data it will write and the data structures that it will use to
record results. During the active part of the test, the tool itself
does not allocate memory, but instead measures the time taken for writes
and records that data into the preallocated records for the results.
The submit rate is the absolute maximum number of messages that could be written per second by that device. To achieve these rates, the payload and metadata size for every message would have to exactly equal the submit size, and the ingress rate from the network would need to exactly equal the rate at which the device can write messages (that is, no gaps in message flow or pushback from the device).
The mean write latency (measured in microseconds) is the average amount of time that each write takes to complete. This affects the overall latency of the system, and is strongly correlated with performance (as reflected in the write rate).
For example, consider a result that shows a “submit rate” of 2500000 for 512 byte messages. This means that this device could (in principle, under absolutely ideal conditions), support a maximum write throughput of:
| Message + Metadata Size | Write Throughput |
|---|---|
| 512 bytes | 2,500,000 per second |
| 1024 bytes (1KB) | 1,250,000 per second |
| 10KB | 125,000 per second |
| 100KB | 12,500 per second |
These are, of course, approximations of the expected maximum throughput. In practice, message sizes rarely fit perfectly into a batch, application activity sees bursts and lulls, the number of threads reading from the transaction log varies based on activity (AMPS tries to consolidate replays from the transaction log into the minimum number of threads to save CPU and I/O bandwidth), and the transaction log device may be used for other traffic (for example, SOW topic storage, statistics, or error and event logging) that may share I/O bandwidth. These factors (and others) can affect the actual performance in deployment.
Usage¶
60East typically runs the amps_bio_perf_test tool with a command line
similar to the following:
$ amps_bio_perf_test -b 128 -m 512 -s 8192 -r 0 -f /mnt/fastdrive/ampsdir/bio.data
This provides a good baseline for the maximum write performance of the system.