1. AMPS Configuration Basics

If you have not become familiar with the AMPS User Guide, in particular the Getting Started chapter of the AMPS User Guide, please start there before reading this guide.

The easiest way to create a custom XML configuration file for AMPS is to start with the sample configuration file produced by the --sample-config flag to AMPS. Example 1.1 shows a simplified sample configuration file.

<!-- Sample AMPS configuration

     This file defines an AMPS instance that provides publish and
     subscribe, topic filtering, and content filtering for JSON messages.
     The instance provides messaging services on port 9007 of the server.
     This configuration also provides an adminstrative interface on
     port 8085, and logs serious messages (error and higher severity) to
     stdout.

     This sample file does not configure State of the World (SOW) Topics,
     Transaction Logs, Aggregation and Views, Historical Query, Replication,
     Authentication and Entitlement, Conflating Topic Replicas, or other
     features of AMPS.

     More details for the features available and how to configure them are
     provided in the AMPS User Guide and the AMPS Configuration Reference.
     Both are available at http://crankuptheamps.com/documentation/

  -->

<AMPSConfig>

    <!-- Name of the AMPS instance -->
    <Name>AMPS-Sample</Name>

    <!-- Configure the administrative HTTP server on port 8085
       This HTTP server provides admin functions and statistics
       for the instance -->

    <Admin>
        <InetAddr>localhost:8085</InetAddr>
    </Admin>

    <!-- Configure a transport for JSON messages over TCP on port 9007 -->

    <Transports>
        <Transport>
            <Name>json-tcp</Name>
            <Type>tcp</Type>
            <InetAddr>9007</InetAddr>
            <MessageType>json</MessageType>
            <Protocol>amps</Protocol>
        </Transport>
    </Transports>

    <!-- Log messages of severity 'error' and higher to stdout -->

    <Logging>
        <Target>
            <Protocol>stdout</Protocol>
            <Level>error</Level>
        </Target>
    </Logging>

</AMPSConfig>

Example 1.1: Simple AMPS configuration file

The AMPS configuration XML file is defined first by wrapping the config file with an AMPSConfig tag to identify it as a configuration file. Next, the instance is given a name using the <Name> tag.

Once our instance has a name, it is good to define the connection target for the administration port. By default, the administration port can be found by pointing a browser to http://localhost:8085, but if a different port or host name is desired, then that is defined in the Admin and InetAddr tags. The Admin port is discussed more in Table 4.1.

Next we describe how to get messages into AMPS. There are several different transport types which can be parsed by AMPS, all of which are discussed in greater detail in the Transports chapter, but for this sample, we keep things simple by focusing on JSON messages over tcp. In AMPS, each key to defining each transport is to give them a unique InetAddr port and specify the type of message AMPS will process on that port using MessageType tag. The MessageType tells AMPS how to parse the incoming messages on a specific port. In the above example, messages are coming in on port 9007, and AMPS uses the JSON parser to parse the body of the message. AMPS also requires a Protocol tag for the Transport, which specifies the format of the commands to AMPS. In this case, we use the standard amps protocol. (Older AMPS applications and AMPS installations may require a different protocol format, such as fix or xml. There’s no functional difference between these protocols, but the AMPS server and the clients need to use the same protocol format to successfully exchange messages.)

The last portion of the configuration is Logging. In the above example, the Logging tag defines only one log target, but it’s quite common to have one or more Logging targets. Again referring to the example, all logging messages that are at error level and above will be logged to the logging Protocol of stdout. In other words, these messages will be logged to the terminal, and not to a file. AMPS supports a robust set of logging features and configurations, all of which are covered in more detail in the Logging chapter of the AMPS User Guide and in Chapter 8 of this reference.

AMPS Configuration File Special Characters

In AMPS there are a few special characters that you should be aware of when creating your configuration file. These characters can provide some handy short cuts and make configuration creation easier, but you should also be aware of them so as not to introduce errors.

State of the World File Name

When specifying the file for a State of the World database, using the %n string in the file name specifies that the AMPS server will use the message type and topic name in that position to create a unique filename. Example 1.2 shows how to use this in the AMPS configuration file.

 <SOW>
    <Topic>
        <Topic>Customers</Topic>
        <FileName>./sow/%n.sow</FileName>
        <MessageType>json</MessageType>
        <Key>/customerId</Key>
    </Topic>
</SOW>

Example 1.2: SOW file name tokens used in configuration file

Log Rotation Name

When specifying an AMPS log file which has RotationThreshold specified, using the %n string in the log file name is a useful mechanism for ensuring the name of the log file is unique and sequential. Example 1.3 shows a file name token replacement in the AMPS configuration file.

<Logging>
    <Target>
        <Protocol>file</Protocol>
        <Level>info</Level>
        <FileName>log/log-%n.log</FileName>
        <RotationThreshold>2G</RotationThreshold>
    </Target>
</Logging>

Example 1.3: Log file name used in configuration file

In the above example, a log file will be created in the AMPSDIR/log/ directory. The first time this file is created, it will be named log-1.log. Once the log file reaches the RotationThreshold limit of 2G, the previous log file will be saved, and the new log file name will be incremented by one. Thus, the next log file will be named AMPSDIR/log/log-2.log.

Dates

AMPS allows administrators to use date-based file names when specifying the file name in the configuration, as demonstrated in Example 1.4.

<Logging>
    <Target>
        <Protocol>file</Protocol>
        <Level>info</Level>
        <FileName>
            log/log-%Y-%m-%dT%H%M%S.log
        </FileName>
        <RotationThreshold>2G</RotationThreshold>
    </Target>
</Logging>

Example 1.4: Date tokens used in configuration file

In the above example, a log file will be created in the $AMPSDIR/log named 2011-01-01-120000.log if the log was created at noon on January 1, 2011.

AMPS provides full support for the date tokens provided by the standard strftime function, with the exception of %n, as described above. The following table shows some of the most commonly used tokens:

Token Provides Example
%a Short weekday name Fri
%A Full weekday name Friday
%b Short month name Feb
%B Full month name February
%c Simple date and time Fri Feb 14 17:25:00 2014
%C Century 20
%d Day of the month (leading zero if necessary) 05
%D Short date format (MM/DD/YY) 02/20/14
%e Day of the month (leading space if necessary) 5
%F Short date format (YYYY-MM-DD) 2014-02-20
%H Hour (00-23) 17
%I Hour (00-12) 05
%j Day of the year (001-366) 051
%m Month (01-12) 02
%p AM or PM PM
%r Current time, 12 hour format 05:25:00 pm
%R Current time, 24 hour format 17:25
%T ISO 8601 Time format 17:25:00
%u ISO 8601 day of the week (1-7, Monday = 1) 5
%V ISO 8601 week number (00-53) 07
%y Year, last two digits 14
%Y Year, four digits 2014
%Z Timezone name or abbreviation (blank if undetermined) PST

Table 1.1: Commonly Used Date and Time Tokens

Using Units in the Configuration

To make configuration easy, AMPS permits the use of units to expand values. For example, if a time interval is measured in seconds, then the letter s can be appended to the value. For example, the following SOW topic definition used the Expiration tag to set the record expiration to 86400 seconds (one day).

<SOW>
    <Topic>
        ...

        <Expiration>86400s </Expiration>

        ...
    </Topic>
</SOW>

Example 1.5: Expiration using seconds

An even easier way to specify an expiration of one day is to use the following Expiration:

<SOW>
    <Topic>
        ...

        <Expiration>1d</Expiration>

        ...
    </Topic>
</SOW>

Example 1.6: Expiration using days

Table 1.2 shows a listing of the time units AMPS supports in the configuration file.

Units Description
ns nanoseconds
us microseconds
ms milliseconds
s seconds
m minutes
h hours
d days
w weeks

Table 1.2: AMPS Configuration - Time Units

AMPS configuration supports a similar mechanism for byte-based units when specifying sizes in the configuration file. Table 1.3 shows a listing of the byte units AMPS supports in the configuration file.

Units Description
kb kilobytes
mb megabytes
gb gigabytes
tb terabytes

Table 1.3: AMPS Configuration - Byte Units

Dealing with large numbers in AMPS configuration can also be simplified by using common exponent values to handle raw values. This means that instead of having to input 10000000 to represent ten million, a user can input 10M. Table 1.4 contains a list of the exponents supported.

Units Description
k 103 - thousand
M 106 - million

Table 1.4: AMPS Configuration - Numeric Units

tip To make it easier for users to remember the units, AMPS interval and byte units are not case sensitive.

Environment Variables in AMPS Configuration

AMPS configuration also allows for environment variables to be used as part of the data when specifying a configuration file. These variables can be set in the environment when AMPS starts, or passed to AMPS using the -D option on the command line.

If a global system variable is commonly used in an organization, then it may be useful to define this in one location and re-use it across multiple AMPS installations or applications. AMPS will replace any token wrapped in ${} with the environment variable defined in the current user operating system environment. Example 1.7 demonstrates how the environment variable ENV_LOG is used to define a global environment variable for the location of the host logging.

<Logging>
    <Target>
        <Protocol>file</Protocol>
        <FileName>${ENV_LOG}</FileName>
        <Level>info</Level>
        <RotationThreshold>2G</RotationThreshold>
    </Target>
</Logging>

Example 1.7: Environment variables in AMPS configuration

Internal Environment Variables

In addition to supporting custom environment variables, AMPS includes a configuration variable, AMPS_CONFIG_DIRECTORY, which can be used to reference the directory in which the configuration file used to start AMPS is located. For example, assume that AMPS was started with the following command at the command prompt:

%>./ampServer ../amps/config/config.xml

Given this command, the log file configuration option shown in Example 1.8 can be used to instruct AMPS to create the log files in the same parent directory as the configuration file — in this case ../amps/config/logs/infoLog.log.

<Logging>
    <Target>
        <Protocol>file</Protocol>
        <FileName>${AMPS_CONFIG_DIRECTORY}/logs/infoLog.log</FileName>
        <Level>info</Level>
        <RotationThreshold>2G</RotationThreshold>
    </Target>
</Logging>

Example 1.8: AMPS_CONFIG_DIRECTORY environment variable example

In addition to the AMPS_CONFIG_DIRECTORY environment variable, AMPS also supports the AMPS_CONFIG_PATH, which is an absolute path to the configuration file used to start AMPS.

Including External Files

For production applications, AMPS configuration files can become large and complicated. In many cases, different instances of an AMPS server need to reuse the same definitions. For example, both servers in a High-Availability pair may need to use the same queue and SOW definitions.

To help you manage complicated configurations and more easily keep configuration consistent on different servers, AMPS allows you to include external files in the configuration file by using the Include directive.

For example, you could use this for a High-Availability pair to include a file that defines the queue, transaction log, and topic definitions. Both instances could include exactly the same file for those definitions, while having different instance names and port numbers.

When AMPS loads a configuration file that contains an Include directive, AMPS follows this process:

  • Load and parse the configuration file
  • If the file contains any Include directives, load and parse the files specified by those directives. If the included files contain Include directives, load and parse the files specified by those directives (and so forth until all Include directives have been processed).
  • Once all files have been loaded and parsed, replace the Include directives in the original files with the parsed files.

AMPS does not process the configuration file until all of the Include directives have been resolved and the files have been parsed.

A file may not be included by any file that it includes, or it is impossible for AMPS to complete the parsing process.

Because each file is individually parsed, XML entities defined in a file are not defined for the files that are Included by that file.

To make it easier to identify which elements of the complete AMPS configuration file have been inserted through the Include mechanism, AMPS can include comments in the assembled file that indicate the source file for configuration elements. By default, this feature is off, and XML content is included verbatim. To change the default for the instance, use the ConfigIncludeCommentDefault configuration element to enable comments, by default, for every Include in the instance. To override commenting behavior for an individual Include, use the comment attribute.

Example

Consider a configuration file with the following Logging element defined:

<AMPSConfig>
    ...

    <Logging>
        <Include comment="true">filetarget.xml</Include>
    </Logging>
</AMPSConfig>

After parsing the configuration file, AMPS loads and parses the filetarget.xml file and replaces the Include element with the contents of that file.

Suppose filetarget.xml contains the following Target directive:

<Target>
    <Protocol>file</Protocol>
    <FileName>/var/log/amps-log-%n.log</FileName>
    <Level>info</Level>
</Target>

The configuration that AMPS uses will be effectively the same as if the configuration file contained the following XML:

<AMPSConfig>
    ...

    <Logging>
        <!-- Start <Include>filetarget.xml</Include> -->
        <Target>
            <Protocol>file</Protocol>
            <FileName>/var/log/amps-log-%n.log</FileName>
            <Level>info</Level>
        </Target>
        <!-- End <Include>filetarget.xml</Include> -->
    </Logging>

    ...
</AMPSConfig>