1. AMPS Configuration Basics¶
If you have not become familiar with the AMPS User Guide, in particular the Getting Started chapter of the AMPS User Guide, please start there before reading this guide.
The easiest way to create a custom XML configuration file for AMPS is to
start with the sample configuration file produced by the
--sample-config
flag to AMPS.
Example 1.1 shows a simplified
sample configuration file.
<!-- Sample AMPS configuration
This file defines an AMPS instance that provides publish and
subscribe, topic filtering, and content filtering for JSON messages.
The instance provides messaging services on port 9007 of the server.
This configuration also provides an adminstrative interface on
port 8085, and logs serious messages (error and higher severity) to
stdout.
This sample file does not configure State of the World (SOW) Topics,
Transaction Logs, Aggregation and Views, Historical Query, Replication,
Authentication and Entitlement, Conflating Topic Replicas, or other
features of AMPS.
More details for the features available and how to configure them are
provided in the AMPS User Guide and the AMPS Configuration Reference.
Both are available at http://crankuptheamps.com/documentation/
-->
<AMPSConfig>
<!-- Name of the AMPS instance -->
<Name>AMPS-Sample</Name>
<!-- Configure the administrative HTTP server on port 8085
This HTTP server provides admin functions and statistics
for the instance -->
<Admin>
<InetAddr>localhost:8085</InetAddr>
</Admin>
<!-- Configure a transport for JSON messages over TCP on port 9007 -->
<Transports>
<Transport>
<Name>json-tcp</Name>
<Type>tcp</Type>
<InetAddr>9007</InetAddr>
<MessageType>json</MessageType>
<Protocol>amps</Protocol>
</Transport>
</Transports>
<!-- Log messages of severity 'error' and higher to stdout -->
<Logging>
<Target>
<Protocol>stdout</Protocol>
<Level>error</Level>
</Target>
</Logging>
</AMPSConfig>
Example 1.1: Simple AMPS configuration file
The AMPS configuration XML file is defined first by wrapping the config
file with an AMPSConfig
tag to identify it as a configuration file.
Next, the instance is given a name using the <Name>
tag.
Once our instance has a name, it is good to define the connection target
for the administration port. By default, the administration port can be
found by pointing a browser to http://localhost:8085
, but if a
different port or host name is desired, then that is defined in the
Admin
and InetAddr
tags. The Admin port is discussed more in
Table 4.1.
Next we describe how to get messages into AMPS. There are several
different transport types which can be parsed by AMPS, all of which are
discussed in greater detail in the Transports chapter, but for this
sample, we keep things simple by focusing on JSON messages over tcp
.
In AMPS, each key to defining each transport is to give them a unique
InetAddr
port and specify the type of message AMPS will process on
that port using MessageType
tag. The MessageType
tells AMPS how
to parse the incoming messages on a specific port. In the above example,
messages are coming in on port 9007, and AMPS uses the JSON parser to
parse the body of the message. AMPS also requires a Protocol
tag for
the Transport
, which specifies the format of the commands to AMPS.
In this case, we use the standard amps
protocol. (Older AMPS
applications and AMPS installations may require a different protocol
format, such as fix
or xml
. There’s no functional difference
between these protocols, but the AMPS server and the clients need to use
the same protocol format to successfully exchange messages.)
The last portion of the configuration is Logging. In the above example,
the Logging
tag defines only one log target, but it’s quite common
to have one or more Logging
targets. Again referring to the example,
all logging messages that are at error
level and above will be
logged to the logging Protocol
of stdout
. In other words, these
messages will be logged to the terminal, and not to a file. AMPS
supports a robust set of logging features and configurations, all of
which are covered in more detail in the Logging chapter of the AMPS User Guide and in Chapter 8 of this reference.
AMPS Configuration File Special Characters¶
In AMPS there are a few special characters that you should be aware of when creating your configuration file. These characters can provide some handy short cuts and make configuration creation easier, but you should also be aware of them so as not to introduce errors.
State of the World File Name¶
When specifying the file for a State of the World database, using the
%n
string in the file name specifies that the AMPS server will use
the message type and topic name in that position to create a unique
filename. Example 1.2
shows how to use this in the AMPS configuration file.
<SOW>
<Topic>
<Topic>Customers</Topic>
<FileName>./sow/%n.sow</FileName>
<MessageType>json</MessageType>
<Key>/customerId</Key>
</Topic>
</SOW>
Example 1.2: SOW file name tokens used in configuration file
Log Rotation Name¶
When specifying an AMPS log file which has RotationThreshold
specified, using the %n
string in the log file name is a useful
mechanism for ensuring the name of the log file is unique and
sequential. Example 1.3 shows a file name
token replacement in the AMPS configuration file.
<Logging>
<Target>
<Protocol>file</Protocol>
<Level>info</Level>
<FileName>log/log-%n.log</FileName>
<RotationThreshold>2G</RotationThreshold>
</Target>
</Logging>
Example 1.3: Log file name used in configuration file
In the above example, a log file will be created in the AMPSDIR/log/
directory. The first time this file is created, it will be named
log-1.log
. Once the log file reaches the RotationThreshold
limit
of 2G, the previous log file will be saved, and the new log file name
will be incremented by one. Thus, the next log file will be named
AMPSDIR/log/log-2.log
.
Dates¶
AMPS allows administrators to use date-based file names when specifying the file name in the configuration, as demonstrated in Example 1.4.
<Logging>
<Target>
<Protocol>file</Protocol>
<Level>info</Level>
<FileName>
log/log-%Y-%m-%dT%H%M%S.log
</FileName>
<RotationThreshold>2G</RotationThreshold>
</Target>
</Logging>
Example 1.4: Date tokens used in configuration file
In the above example, a log file will be created in the $AMPSDIR/log
named 2011-01-01-120000.log
if the log was created at noon on
January 1, 2011.
AMPS provides full support for the date tokens provided by the standard
strftime function, with the exception of %n
, as described above. The
following table shows some of the most commonly used tokens:
Token | Provides | Example |
---|---|---|
%a | Short weekday name | Fri |
%A | Full weekday name | Friday |
%b | Short month name | Feb |
%B | Full month name | February |
%c | Simple date and time | Fri Feb 14 17:25:00 2014 |
%C | Century | 20 |
%d | Day of the month (leading zero if necessary) | 05 |
%D | Short date format (MM/DD/YY) | 02/20/14 |
%e | Day of the month (leading space if necessary) | 5 |
%F | Short date format (YYYY-MM-DD) | 2014-02-20 |
%H | Hour (00-23) | 17 |
%I | Hour (00-12) | 05 |
%j | Day of the year (001-366) | 051 |
%m | Month (01-12) | 02 |
%p | AM or PM | PM |
%r | Current time, 12 hour format | 05:25:00 pm |
%R | Current time, 24 hour format | 17:25 |
%T | ISO 8601 Time format | 17:25:00 |
%u | ISO 8601 day of the week (1-7, Monday = 1) | 5 |
%V | ISO 8601 week number (00-53) | 07 |
%y | Year, last two digits | 14 |
%Y | Year, four digits | 2014 |
%Z | Timezone name or abbreviation (blank if undetermined) | PST |
Table 1.1: Commonly Used Date and Time Tokens
Using Units in the Configuration¶
To make configuration easy, AMPS permits the use of units to expand
values. For example, if a time interval is measured in seconds, then the
letter s
can be appended to the value. For example, the following
SOW topic definition used the Expiration
tag to set the record
expiration to 86400 seconds (one day).
<SOW>
<Topic>
...
<Expiration>86400s </Expiration>
...
</Topic>
</SOW>
Example 1.5: Expiration using seconds
An even easier way to specify an expiration of one day is to use the
following Expiration
:
<SOW>
<Topic>
...
<Expiration>1d</Expiration>
...
</Topic>
</SOW>
Example 1.6: Expiration using days
Table 1.2 shows a listing of the time units AMPS supports in the configuration file.
Units | Description |
---|---|
ns |
nanoseconds |
us |
microseconds |
ms |
milliseconds |
s |
seconds |
m |
minutes |
h |
hours |
d |
days |
w |
weeks |
Table 1.2: AMPS Configuration - Time Units
AMPS configuration supports a similar mechanism for byte-based units when specifying sizes in the configuration file. Table 1.3 shows a listing of the byte units AMPS supports in the configuration file.
Units | Description |
---|---|
kb |
kilobytes |
mb |
megabytes |
gb |
gigabytes |
tb |
terabytes |
Table 1.3: AMPS Configuration - Byte Units
Dealing with large numbers in AMPS configuration can also be simplified
by using common exponent values to handle raw values. This means that
instead of having to input 10000000
to represent ten million, a user
can input 10M
. Table 1.4 contains a list of the exponents
supported.
Units | Description |
---|---|
k | 103 - thousand |
M | 106 - million |
Table 1.4: AMPS Configuration - Numeric Units
To make it easier for users to remember the units, AMPS interval and byte units are not case sensitive. |
Environment Variables in AMPS Configuration¶
AMPS configuration also allows for environment variables to be used as
part of the data when specifying a configuration file. These variables
can be set in the environment when AMPS starts, or passed to AMPS using
the -D
option on the command line.
If a global system variable is commonly used in an organization, then it
may be useful to define this in one location and re-use it across
multiple AMPS installations or applications. AMPS will replace any token
wrapped in ${}
with the environment variable defined in the current
user operating system environment. Example 1.7
demonstrates how the environment variable ENV_LOG
is used to define a global
environment variable for the location of the host logging.
<Logging>
<Target>
<Protocol>file</Protocol>
<FileName>${ENV_LOG}</FileName>
<Level>info</Level>
<RotationThreshold>2G</RotationThreshold>
</Target>
</Logging>
Example 1.7: Environment variables in AMPS configuration
Internal Environment Variables¶
In addition to supporting custom environment variables, AMPS includes a set of environment variables automatically populated by the server.
These variables are listed below:
Variable Name | Contains |
---|---|
AMPS_CONFIG_DIRECTORY |
Directory in which the configuration file used to start AMPS is located. |
AMPS_CONFIG_PATH |
Full path to the configuration file used to start AMPS, including the file name. |
AMPS_VERSION |
Full version number of the AMPS server. |
When AMPS processes the configuration file, AMPS expands these variables just as though they were set in the environment. For example, assume that AMPS was started with the following command at the command prompt:
%>./ampServer ../amps/config/config.xml
Given this command, the log file configuration option shown in
Example 1.8 can be used to instruct AMPS to create the log files
in the same parent directory as the configuration file — in this case
../amps/config/logs/infoLog.log
.
<Logging>
<Target>
<Protocol>file</Protocol>
<FileName>${AMPS_CONFIG_DIRECTORY}/logs/infoLog.log</FileName>
<Level>info</Level>
<RotationThreshold>2G</RotationThreshold>
</Target>
</Logging>
Example 1.8: AMPS_CONFIG_DIRECTORY environment variable example
Including External Files¶
For production applications, AMPS configuration files can become large and complicated. In many cases, different instances of an AMPS server need to reuse the same definitions. For example, both servers in a High-Availability pair may need to use the same queue and SOW definitions.
To help you manage complicated configurations and more easily keep
configuration consistent on different servers, AMPS allows you to
include external files in the configuration file by using the
Include
directive.
For example, you could use this for a High-Availability pair to include a file that defines the queue, transaction log, and topic definitions. Both instances could include exactly the same file for those definitions, while having different instance names and port numbers.
When AMPS loads a configuration file that contains an Include directive, AMPS follows this process:
- Load and parse the configuration file
- If the file contains any
Include
directives, load and parse the files specified by those directives. If the included files contain Include directives, load and parse the files specified by those directives (and so forth until all Include directives have been processed). - Once all files have been loaded and parsed, replace the Include directives in the original files with the parsed files.
AMPS does not process the configuration file until all of the
Include
directives have been resolved and the files have been
parsed.
A file may not be included by any file that it includes, or it is impossible for AMPS to complete the parsing process.
Because each file is individually parsed, XML entities defined in a file
are not defined for the files that are Include
d by that file.
To make it easier to identify which elements of the complete AMPS
configuration file have been inserted through the Include
mechanism,
AMPS can include comments in the assembled file that indicate the source
file for configuration elements. By default, this feature is off, and
XML content is included verbatim. To change the default for the
instance, use the ConfigIncludeCommentDefault
configuration element
to enable comments, by default, for every Include
in the instance.
To override commenting behavior for an individual Include
, use the
comment
attribute.
Example¶
Consider a configuration file with the following Logging
element
defined:
<AMPSConfig>
...
<Logging>
<Include comment="true">filetarget.xml</Include>
</Logging>
</AMPSConfig>
After parsing the configuration file, AMPS loads and parses the
filetarget.xml
file and replaces the Include element with the
contents of that file.
Suppose filetarget.xml
contains the following Target
directive:
<Target>
<Protocol>file</Protocol>
<FileName>/var/log/amps-log-%n.log</FileName>
<Level>info</Level>
</Target>
The configuration that AMPS uses will be effectively the same as if the configuration file contained the following XML:
<AMPSConfig>
...
<Logging>
<!-- Start <Include>filetarget.xml</Include> -->
<Target>
<Protocol>file</Protocol>
<FileName>/var/log/amps-log-%n.log</FileName>
<Level>info</Level>
</Target>
<!-- End <Include>filetarget.xml</Include> -->
</Logging>
...
</AMPSConfig>