22. Monitoring Interface¶
The AMPS monitoring interface has two distinct components:
- A basic monitoring interface that provides statistics for the AMPS instance in common machine-readable formats. This interface also provides administrative functions, such as enabling and disabling transports, disconnecting clients, and upgrading and downgrading replication links.
- The AMPS Galvanometer, a browser-based monitoring tool that shows a graphical representation of the statistics for AMPS. The Galvanometer includes information about replication flow across the set of connected instances. It includes the ability to enter subscriptions and queries, and display the results in a grid.
Configuration¶
The AMPS monitoring interface is defined in the configuration file used
on AMPS start up. Below is an example configuration of the Admin
tag.
<!-- Configure the monitoring interface: this
starts an http server in the AMPS process. -->
<Admin>
<FileName>stats.db</FileName>
<InetAddr>localhost:8085</InetAddr>
<Interval>10s</Interval>
</Admin>
In this example localhost
is the hostname and 8085
is the port
assigned to the monitoring interface. With this configuration:
http://localhost:8085/ | Root URI for Galvanometer. |
http://localhost:8085/amps | Root URI for simple monitoring interface. |
The Interval
tag is used to set the update interval for the AMPS
monitoring interface. In this example, statistics will be updated every
10 seconds.
It is important to note that by default AMPS will store the
monitoring interface database information in system memory. If
the AMPS instance is going to be up for a long time, or the
monitoring interface statistics interval will be updated
frequently, it is strongly recommended that the FileName
setting be specified to allow persistence of the data to a local
file. See the AMPS Configuration Reference Guide for more
information. |
The basic monitoring interface is accessible through a web browser, but also follows a Representational State Transfer (RESTful) URI style for programmatic traversal of the directory structure of the monitoring interface.
Basic Monitoring Interface¶
The basic monitoring interface is useful for examining many important aspects about an AMPS instance. This includes health and monitoring information for the AMPS engine as well as the host AMPS is running on. All of this information is designed to be easily accessible to make gathering performance and availability information from AMPS easy. The monitoring interface also provides easy access to perform administrative actions.
The root of the AMPS Monitoring interface URI contains the following child resources:
- The
host
resource provides information about the current operating system state - The
instance
resource provides information about the instance of AMPS - The
administration
resource provides access to functions that modify the state of the instance (such as disconnecting a client)
The information in the monitoring database is taken from the statistics database for the AMPS instance. AMPS provides actions for managing the statistics database, as described in the section on Actions, under Manage the Statistics Database.
The fields provided through the basic monitoring interface (and the statistics database) are described in the AMPS Monitoring Reference, available on the Documentation page on the 60 East web site.
Time Range Selection¶
AMPS keeps a history of the monitoring interface statistics, and allows that data to be queried. By selecting a leaf node of the monitoring interface resources, a time-based query can be constructed to view a historical report of the information. For example, if an administrator wanted to see the number of messages per second consumed by all processors from midnight UTC on October 12, 2011 until 23:25:00 UTC on October 10, 2011, then pointing a browser to
http://localhost:8085/amps/instance/processors/all/messages_received per_sec?t0=20111129T0&t1=20111129T232500
will generate the report and output it in the following plain text format (note: entire dataset is not presented, but is truncated).
20111130T033400,0
20111130T033410,0
20111130T033420,0
20111130T033430,94244
20111130T033440.000992,304661
20111130T033450.000992,301078
20111130T033500,302755
20111130T033510,308922
20111130T033520.000992,306177
20111130T033530.000992,302140
20111130T033540.000992,302390
20111130T033550,307637
20111130T033600.000992,310109
20111130T033610,309888
20111130T033620,299993
20111130T033630,310002
20111130T033640.000992,300612
20111130T033650,299387
All times used for the report generation and presentation are
ISO- 8601 formatted. ISO-8601 formatting is of the following
form: YYYYMMDDThhmmss , where YYYY is the year, MM is
the month, DD is the year, T is a separator between the
date and time, hh is the hours, mm is the minutes and
ss is the seconds. Decimals are permitted after the ss
units. |
As discussed in the following sections, the date-time range can be used with plain text (html), comma-separated values (csv), and XML formats. |
Output Formatting¶
The AMPS monitoring interface offers several possible output formats to ease the consumption of monitoring reporting data. The possible options are XML, CSV and RNC output formats, each of which is discussed in more detail below.
XML Document¶
All monitoring interface resources can have the current node, along with
all child nodes list its output as an XML document by appending the
.xml
file extension to the end of the resource name. For example, if
an administrator would like to have an XML document of all of the
currently running processors—including all the relevant statistics about
those processors—then the following URI will generate that information:
http://localhost:8085/amps/instance/processors/all.xml
The document that is returned will be similar to the following:
<amps>
<instance>
<processors>
<processor id='all'>
<denied_reads>0</denied_reads>
<denied_writes>0</denied_writes>
<description>AMPS Aggregate Processor Stats</description>
<last_active>1855</last_active>
<matches_found>0</matches_found>
<matches_found_per_sec>0</matches_found_per_sec>
<messages_received>0</messages_received>
<messages_received_per_sec>0</messages_received_per_sec>
<throttle_count>0</throttle_count>
</processor>
</processors>
</instance>
</amps>
Appending the .xml
file extension to any AMPS monitoring interface
resource will generate the corresponding XML document.
CSV Document Output¶
Similar to the XML document output discussed above, the .csv
file
extension can be appended to any of the leaf node resources to have a
CSV file generated to examine those values. This can also be coupled
with the time range selection to generate reports. See
Monitoring Timerange for more details on time range selection.
Below is a sample of the .csv
output from the monitoring interface
from the following URL:
http://localhost:8085/amps/instance/processors/all/matches_found_per_sec.csv?t0=20111129T0
This resource will create a file with the following contents:
20111130T033400,0
20111130T033410,0
20111130T033420,0
20111130T033430,94244
20111130T033440.000992,304661
20111130T033450.000992,301078
20111130T033500,302755
20111130T033510,308922
20111130T033520.000992,306177
20111130T033530.000992,302140
20111130T033540.000992,302390
20111130T033550,307637
20111130T033600.000992,310109
20111130T033610,309888
20111130T033620,299993
20111130T033630,310002
20111130T033640.000992,300612
20111130T033650,299387
20111130T033700.000992,304548
JSON Document Output¶
All monitoring interface resources can have the current node, along with
all child nodes list its output as an JSON document by appending the
.json
file extension to the end of the resource name. For example,
if an administrator would like to have an JSON document of all of the
CPUs on the server—including all the relevant statistics about those
CPUs—then the following URI will generate that information:
http://localhost:8085/amps/host/cpus.json
The document that is returned will be similar to the following:
{
"amps": {
"host": {
"cpus": [
{
"id":"all",
"idle_percent":"62.452316076294",
"iowait_percent":"0.490463215259",
"system_percent":"10.681198910082",
"user_percent":"26.376021798365"
},
{
"id":"cpu0",
"idle_percent":"75.417130144605",
"iowait_percent":"0.333704115684",
"system_percent":"7.563959955506",
"user_percent":"16.685205784205"
},
{
"id":"cpu1",
"idle_percent":"50.000000000000",
"iowait_percent":"0.642398286938",
"system_percent":"13.597430406852",
"user_percent":"35.760171306210"
}
]
}
}
}
Appending the .json
file extension to any AMPS monitoring interface
resource will generate the corresponding JSON document.
RNC Document Output¶
AMPS supports generation of an XML schema via the Relax NG Compact (RNC)
specification language. To generate an RNC file, enter the following URL
in a browser http://localhost:port/amps.rnc
and AMPS will display
the RNC schema.
To convert the RNC schema into an XML schema, first save the RNC output to a file:
%> wget http://localhost:9090/amps.rnc
The output can then be converted to an xml schema using Trang (available at http://code.google.com/p/jing-trang/) with
trang -I rnc -O xsd amps.rnc amps.xsd
Galvanometer¶
The AMPS Galvanometer provides an extensive set of visualizations of the state of the instance. Galvanometer also provides the ability to query the instance and display the results.
Authorization and Entitlement in Galvanometer¶
In order to enable Galvanometer to provide credentials to the AMPS
instance (in case it is required to access AMPS monitoring information),
the special WWWAuthenticate
option is supported.
The option can have the following values:
- Negotiate (Kerberos)
- NTLM
- Basic realm=”<SECURITY_DOMAIN>” (Basic Auth)
When using Negotiate
or NTLM
, Galvanometer will automatically
supply corresponding authorization tokens to AMPS. If Basic Auth
is used for authorization, the Login/Password dialog will require a
user to enter credentials.
<Admin>
...
<WWWAuthenticate>Basic realm="AMPS Admin"</WWWAuthenticate>
...
</Admin>
Enabling Queries and Subscriptions in Galvanometer¶
Much of the functionality available in Galvanometer uses the basic monitoring interface.
Galvanometer submits queries and subscriptions to AMPS using the websocket
protocol. To use these functions in Galvanometer, you must provide the name
of a Transport
of type websocket
for Galvanometer to use.
For example, the following directive specifies that Galvanometer will use
the Transport
with the Name
of websocket-any
to submit commands
to AMPS.
<Admin>
<!-- ... existing configuration ... -->
<!-- look up the transport named websocket-any in
this config file, and make connections to
that Transport for sending commands to AMPS -->
<SQLTransport>websocket-any</SQLTransport>
</Admin>
The configuration block above requires that the AMPSConfig file contains a
Transport
with the Name
of websocket-any
of Type
websocket
.
When this configuration item is specified, Galvanometer will enable the query
and subscription capabilities, and submit commands to AMPS over the specified
Transport
.
For example, the websocket-any
transport referenced in the snippet
above might be defined as follows:
<Transports>
<!-- ...
existing transports remain -->
<Transport>
<Name>websocket-any</Name>
<Protocol>websocket</Protocol>
<Type>tcp</Type>
<InetAddr>9008</InetAddr>
</Transport>
</Transports>
Notice that Galvanometer connects as a client using this Transport
. There
is no special transport or protocol for Galvanometer, and the security
configured for the instance (or the Transport
) applies to Galvanometer.
Queries and Subscriptions with Basic Auth in Galvanometer¶
When Basic Auth is used for authorization and entitlement, an additional
option TrustedAdmin
allows Galvanometer to use a valid session
cookie created after successful authorization for queries and subscriptions.
This option forces AMPS to reuse credentials supplied by Galvanometer
for websocket connections created by Galvanometer:
<Protocols>
...
<Protocol>
<Name>websocket-portal</Name>
<Module>websocket</Module>
<!-- disabled by default -->
<TrustedAdmin>enabled</TrustedAdmin>
</Protocol>
...
</Protocols>
TrustedAdmin
is only supported by the websocket-based protocols
and is disabled by default.
Disabling Galvanometer¶
Galvanometer is enabled in the monitoring interface by default. To disable
Galvanometer, add the following directive to the Admin
configuration
block:
<Admin>
<!-- ... existing configuration ... -->
<Galvanometer>disabled</Galvanometer>
</Admin>
Disabling Galvanometer with this configuration item has no effect on the basic monitoring interface.