3. AMPS Basics: Topics, Publish and Subscribe¶

The simplest way to use AMPS is as a low-latency publish and subscribe messaging system. Publish and subscribe messaging is at the heart of AMPS, and all of the other features of AMPS build on this foundation.

In a publish and subscribe messaging system, publishers send messages without necessarily knowing which subscribers will receive the message. This decouples publishers from subscribers for maximum flexibility.

While publishers do not need specific knowledge about the subscribers, publishers are responsible for adding information to the message so that subscribers know which messages are of interest. In publish and subscribe messaging systems, including AMPS, publishers send messages to a specific topic. The topic most often indicates the type of message, and is a way for the subscriber to locate the messages of interest. For example, in an order processing system, a publisher might publish messages to an ORDERS topic. Subscribers that need to receive orders then subscribe to the ORDERS topic, and receive messages that are sent to that topic.

Each message in AMPS is published to a specific topic. The publisher chooses the topic when the message is published, and subscribers can receive messages from that topic.

Unlike many messaging systems, AMPS provides an additional layer of selectivity for subscribers. Rather than receiving every message from a given topic, an AMPS subscriber can use content filtering to receive only the messages that the subscriber needs to process. Content filtering provides several advantages. First, being more selective about the messages delivered to the subscriber makes better use of bandwidth between AMPS and the subscriber, since the subscriber only receives messages that are of interest. Subscriber code is easier to write and more efficient because the subscriber is guaranteed that the messages received have the values requested. Further, because the subscriber chooses which messages to receive, content filtering makes publishers and subscribers less tightly-coupled. A publisher does not need to know what fields are important to a subscriber, or whether a field that was previously unused is now important.

The diagram below shows the basic concept of publish and subscribe messaging:

Publish and Subscribe

In the diagram above, there is a Publisher sending AMPS a message to the ORDERS topic. The message being sent contains information on Ticker IBM with a Price of 125. Both of these fields are contained within the message payload itself (i.e., the message content). AMPS routes the message to Subscriber 1 because it is subscribing to all messages on the ORDERS topic. Similarly, AMPS routes the message to Subscriber 2 because it is subscribed to any messages having the Ticker equal to IBM. Subscriber 3 is looking for a different Ticker value and is not sent the message.

Topic Configuration for Basic Pub/Sub¶

Unlike many messaging systems, AMPS does not require any configuration for simple publish and subscribe. For this basic functionality, there is no need to declare topics in advance. Since there is no need to declare these topics, topics that provide basic publish and subscribe are often referred to as ad hoc topics in AMPS.

It is valid for a publisher to publish to any topic, whether or not that topic has been previously configured. Likewise, it is valid for a subscriber to subscribe to any topic, whether or not a message has been previously published to that topic or whether the topic appears in a configuration file. Features that rely on persisted messages, however, are not available without configuration for a topic.

Every topic in AMPS has a specific message type. Publishers and subscribers don’t need to explicitly set the message type when publishing to or subscribing to a topic. Each connection to AMPS specifies the message type to be used for that connection – either implicitly, by connecting to a port that provides a specific message type, or explicitly (when connecting to a port that can provide multiple message types).

AMPS allows topics that use different message types to have the same name, but considers them to be different topics. Messages published to an XML topic named quotes will not be delivered to subscriptions to a JSON topic named quotes.

Regular Expression Subscriptions¶

AMPS allows subscribers to provide a regular expression that defines a set of topics rather than a literal topic name. This further decouples publishers and subscribers.

It’s important to remember that each message is published to a specific topic. Regular expressions are only applicable to subscriptions: a publisher should not use regular expressions in the topic when publishing messages.

Spark: Basic Publish and Subscribe Example¶

Here’s how to use spark to send and receive a message from AMPS. The example assumes that you’re using the sample configuration file produced by the AMPS server, and that you are running spark on the same system that AMPS is running on.

First, start a subscriber:

Open a terminal in your Linux environment.
Use the following command (with AMPS_DIR set to the directory where you installed AMPS) to start a subscription:
```
$ $AMPS_DIR/bin/spark subscribe -server localhost:9007 \
                                 -type json -topic test
```
This command starts a subscription to the JSON topic test.
spark will connect to AMPS, logon using default credentials (the current username and an empty password) and enter the subscription. Unless there are errors, the command will produce no output until a message arrives.
Leave this terminal running. When you publish a message to the test topic, spark will print the message in this terminal.

Next, publish a message to the same topic:

Open a new terminal in your Linux environment.
Use the following command (with AMPS_DIR set to the directory where you installed AMPS) to send a single message to AMPS:
```
$ echo '{"note":"Crank it up!"}' | \
  $AMPS_DIR/bin/spark publish -server localhost:9007 \
  -type json -topic test
```
As with the subscriber sample, spark automatically connects to AMPS and sends a logon command with the default credentials (the current username and an empty password). With the publish command, spark reads the message from standard input and publishes the message to the JSON topic test. The command produces output similar to the following line (the rate calculation will likely be different):
```
total messages published: 1 (333.33/s)
```
When the publisher sends the message, the subscriber should receive the message and produce the following output:
```
{"note":"Crank it up!"}
```

Congratulations! You’ve just sent your first message through AMPS.

Although this example is simple and relies on default behavior, the sample demonstrates some important AMPS concepts:

As mentioned earlier, there is no need to preconfigure simple publish/subscribe topics. Since the default configuration file doesn’t specify any settings for the JSON topic test, AMPS knows to treat the topic as a simple pub/sub topic. Also, since the spark commands specify the JSON message type when connecting to the server, the topic is a JSON topic.
For simple publish and subscribe topics, AMPS delivers the message verbatim to the subscriber. AMPS doesn’t interpret or normalize the message. In fact, AMPS doesn’t even parse the message unless there’s a need to. With this configuration and this subscription, there’s no need for AMPS to parse the message, so no parsing happens.
The spark program connects to AMPS and logs on to AMPS before sending any commands. All AMPS installations include authentication and entitlement. By default, AMPS loads an authentication and entitlement policy (implemented as an AMPS module) that requires a logon, but accepts any username and password as credentials. This policy is intended for evaluating, testing and development purposes. More information on securing an AMPS instance is available in the User Guide. For this example, the important point is to be aware that an AMPS instance always has a security policy and that policy was at work even in this simple example. The default behavior for spark works with the default policy for AMPS.

Content Filters¶

In the Basic Pub-Sub Example, the subscriber requested all messages on the JSON test topic. AMPS includes expressive, flexible and extensible content filtering that allows subscribers to specify exactly the messages that they want to receive. Content filtering is one of the most useful features of AMPS. When subscribers use content filtering, publishers can be completely independent of subscribers. The publisher does not need to know which parts of a message are important to subscribers. Subscribers can precisely declare the content that they are interested in, so they only receive relevant messages. Publishers do not need to be updated when subscribers add additional criteria or when new subscribers come online.

You can think of content filtering as adding a WHERE clause to the subscription. Like a WHERE clause, AMPS returns only matching messages.

AMPS content filters use a combination of XPath identifiers to locate a value within a message and SQL-92 operators for comparing those values. For example, given a JSON message like:

{"note":"Hello, World!"}

The following content filters would match the message:

/note = 'Hello, World!'

This filter uses the equality operator, =, to compare the /note field in the message with an exact match for the string.

/note LIKE '(?i)world'

The LIKE operator uses Perl Compatible Regular Expressions to match a field. In this case, the regular expression matches any string that contains world, using case-insensitive matching.

/note BEGINS WITH 'Hello'

The AMPS BEGINS WITH operator matches any string that begins with the exact sequence of characters provided.

The AMPS User Guide has full details on the expression language used in AMPS.

Spark: Subscription with Content Filter¶

Here’s how to use spark to subscribe using a content filter. The example assumes that you’re using the sample configuration file produced by the AMPS server and that you are running spark on the same system that AMPS is running on.

First, start a subscriber:

Open a terminal in your Linux environment.
Use the following command (with AMPS_DIR set to the directory where you installed AMPS) to start a subscription:
```
$ $AMPS_DIR/bin/spark subscribe -server localhost:9007  \
               -type json -topic test                   \
               -filter "/note LIKE '(?i)sample'"
```
This command starts a subscription to the JSON topic test. This subscription will only return messages where the filter matches.
spark will connect to AMPS, logon using default credentials (the current username and an empty password) and enter the subscription. Unless there are errors, the command will produce no output until a message arrives.
Leave this terminal running. When you publish a message to the test topic that matches the filter, spark will print the message in this terminal.

Next, publish messages to the subscriber:

Open a new terminal in your Linux environment.
Use the following command (with AMPS_DIR set to the directory where you installed AMPS) to publish a message to AMPS. This message matches the filter:
```
$ echo '{"note":"Filter sample!"}' | \
  $AMPS_DIR/bin/spark publish -server localhost:9007 \
  -type json -topic test
```
Use the following command (with AMPS_DIR set to the directory where you installed AMPS) to publish a message to AMPS. This message does not match the filter:
```
$ echo '{"note":"Not a match. Sorry."}' | \
  $AMPS_DIR/bin/spark publish -server localhost:9007 \
  -type json -topic test
```
Each time you run spark, it automatically connects to AMPS and sends a logon command with the default credentials (the current username and an empty password). With each publish command, spark reads the message from standard input and publishes the message to the JSON topic test. Each of the commands above produces output similar to the following line (the rate calculation will likely be different):
```
total messages published: 1 (333.33/s)
```
When the publisher sends a message that matches the filter, the subscriber should receive the message and produce the following output:
```
{"note":"Filter sample!"}
```

Table Of Contents

Related Topics