Welcome to AMPS!¶
AMPS, the Advanced Message Processing System, is a reliable high-performance publish and subscribe engine designed for real-world messaging applications that demand the highest performance and lowest latency possible. AMPS solves the hardest problems for modern pub/sub and queuing applications in a way that maximizes performance and reliability without compromising ease of use.
What makes AMPS different?¶
AMPS is designed for extremely demanding, real-world messaging applications. This means that AMPS has:
- Uncompromising focus on real-world needs
- Hardware-level optimization for performance
- Easy-to-use programming interface
- Detailed monitoring and diagnostics
- Complete solution for high-volume low-latency applications
- Durability and transactional consistency
- Both pub/sub and queuing delivery models
- Replication for high availability
- Persistent state-of-the-world database
- Content filtering with SQL92 WHERE clause semantics
- Topic views, projection and JOIN
- Message replay and historical point-in-time query
- Real-time computation and analysis using standard SQL semantics
- Delta publish and subscribe for lightweight updates
- Built-in handling for FIX, NVFIX, JSON, BSON, Google Protocol Buffer, MessagePack and XML messages
AMPS allows you to configure each server, topic, or view to use exactly the features that you need. This keeps AMPS programming and administration simple.
Get the Equipment¶
To get the most out of your evaluation, you will need:
- An installation of AMPS on a Linux host
- An Linux installation and a local installation of AMPS.
- If you do not already have a Linux installation, you can use a virtual machine, for example, VirtualBox available at http://virtualbox.org
- Make sure that Virtualization Technology is enabled on your development machine, as described at http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003944
- Create a Virtual Machine with enough memory and disk space to hold the data you intend to use with your AMPS evaluation
- Install the Linux distribution of your choice
- Download the AMPS Linux distribution at http://crankuptheamps.com/evaluation
- An Linux installation and a local installation of AMPS.
- The AMPS Python Client (which this file is part of). Visit crankuptheamps.com to download the latest kits and documents.
Turn It On¶
To install the Linux distribution of AMPS and start the evaluation server:
Extract the Linux distribution of AMPS and the AMPS client.
Make a config directory in the AMPS distribution directory, for example:
$ mkdir ~/amps_dir/config
Copy the
sample.xml
file from the samples directory into the config directory you just created, for example:$ cp sample.xml ~/amps_dir/config/sample.xml
Change directories to the AMPS distribution directory and start AMPS with the sample config file. For example:
$ cd ~/amps_dir/ $ ./bin/ampServer ./config/sample.xml
Building the Client¶
The AMPS Python client ships as both source and pre-built binaries. The full client distribution, which includes this guide, provides the source to the Python client. You build the client with your Python distribution before using it. On most Linux systems, you can build the client and install it into your Python distribution by navigating to the client directory in the evaluation kit and typing:
$ sudo python setup.py build install
If you do not want to (or are not allowed to) install the client into your Python distribution, or if you are building the client on Windows, see the release page for the AMPS clients at http://devnull.crankuptheamps.com/releases/amps/clients.
Crank It Up!¶
In this section, we describe using the AMPS samples from the command line. You can also use the PyDev extension to Eclipse or another IDE if you prefer.
To get started with AMPS, create a simple subscriber and publisher:
- Open a command shell
- Navigate to the directory where you unzipped the evaluation kit
- Navigate to
Python Evaluation/CrankItUp
Update the connection strings to connect to AMPS
FROM THE LINUX COMMAND LINE:
From the CrankItUp directory, type the following command to replace the IP address:
$sed -ri 's/127.0.0.1/ip address here/' *.py
FROM THE WINDOWS COMMAND LINE:
From the CrankItUp directory, type the following command to replace the IP address:
powershell "gci | foreach-object { (Get-Content $_) –replace '127.0.0.1' , 'ip address here' | Set-Content $_ }"
Run the sample subscriber
- From the CrankItUp directory, you can run the example by typing:
$ python AMPSConsoleSubscriber.py &
This command line runs the subscriber, and then sends it to the background. Messages from the subscriber will be printed to the console.
NOTE: To run this command line, python must be in your PATH.
- Run the sample publisher
- From the CrankItUp directory, you can run the example by typing:
$ python AMPSConsolePublisher.py &
- The program runs, and subscribers receive the Hello, World message. The publisher itself produces no output if it is successful. The subscriber will receive the message.
NOTE: To run this command line, python must be in your PATH.
Using a State of the World database¶
One of the most useful parts of AMPS is being able to keep track of the current value of a message, using the state of the world (SOW) database:
Add messages to a state of the world database
- From the CrankItUp directory, run the publisher by typing:
$ python AMPSSOWConsolePublisher.py &
- The program runs. This program sends 100 messages to AMPS. In this case, the messages are sent on a topic, messages-sow, that maintains a state-of-the-world database. AMPS saves the most recent version of each unique message. Unique messages are identified by the messageNumber in the message. The program sends messages 0-99, then updates message 5.
NOTE: To run this command line, python must be in your PATH.
Retrieve the state of the world
- From the CrankItUp directory, run the subscriber by typing:
$ python AMPSSOWConsoleSubscriber.py &
- The program runs. This program queries the state of the world for the
messages-sow
topic, filtered to messages with amessageNumber
less than 10. The console output shows the results of the query. Notice that for message 5, the database shows the latest value – the current state of the world.
NOTE: To run this command line, python must be in your PATH.
Retrieve the state of the world and subscribe to updates.
- From the CrankItUp directory, run the subscriber by typing:
$ python AMPSSOWandSubscribeConsoleSubscriber.py &
- The program runs. This program queries the state of the world for the
messages-sow
topic, filtered to messages with amessageNumber
less than10
. The console output shows the results of the query. Notice that for message5
, the database shows the latest value – the current state of the world. Rather than ending, the program continues to run. AMPS will send the program any new messages that match the filter. - From the CrankItUp directory, run the SOW publisher by typing:
$ python AMPSSOWConsolePublisher.py &
- The publisher sends another 100 messages to AMPS, updating the value of each message, then updates message 5 again. Notice that AMPS sends each update to the subscriber, meaning that the subscriber receives two updates to message 5. When subscribed, the subscriber always receives the most current information.
NOTE: To run these command lines, python must be in your PATH.
Detecting when messages go out of focus¶
One of the difficulties in conventional messaging systems and databases is being notified when an item no longer matches your query. For example, you may want to keep a list of all of the orders which are not yet fulfilled, or all of the taxis that are available to pick up passengers. In a traditional system, you detect change by running the query again and comparing the new results to the previous results.
With AMPS, you can ask to be notified when a message no longer matches the query. AMPS delivers the notification as a special kind of message – an Out of Focus message.
Retrieve the state of the world and subscribe to updates. Request out of focus notifications.
- From the CrankItUp directory, run the subscriber by typing:
$ python AMPSSOWandSubscribeWithOOF.py &
- The program runs. This program queries the state of the world for the messages-sow topic, filtered to messages where the messageNumber is a multiple of 10, and where an optional field is either not present or set to ‘ignore_me’.
Update messages so the subscriber receives Out of Focus messages.
- From the CrankItUp directory, run the publisher by typing:
$ python AMPSUpdateForOOF.py &
The program runs. This program does the following work:
- Sends a message with an expiration time. The subscriber receives the message.
- Sends a message that it will later delete. The subscriber receives the message.
- Sends a set of messages that match the filter. The subscriber receives these messages.
- Sends updates to a set of messages that cause them to no longer match the filter. The subscriber receives Out Of Focus messages. Notice that these messages include the information that the filter no longer matches.
- Deletes the message that it sent earlier. The subscriber receives an Out of Focus message saying that the message was deleted.
The program waits to help ensure that all messages have been published, then exits. The message with the expiration time set will expire, and the subscriber will receive an Out of Focus message saying that the message expired.
Faster. Louder. Better.¶
From here, the world of AMPS is yours to explore. The Evaluation Kit gives you a set of guides to help you understand how AMPS works and what you need to do to take advantage of AMPS in your applications.
The AMPS User Guide describes the features of AMPS in detail, with a focus on helping you understand the concepts behind AMPS. The AMPS C++ Developer Guide describes the C++ interface for AMPS.
You know what your application needs. We’ve provided some suggestions for simple programs that will help you get familiar with AMPS and help you understand how AMPS can help you.
Suggested learning projects:
- Understanding the state of the world. Fill the SOW database with messages. Verify this using the amps_sow_dump tool, as you did for AMPSSOWPublisher. Update a few messages in the database. Verify the updates using the amps_sow_dump tool. Then write a program that uses SOW and delta subscriptions to get the same information.
- Working with multiple AMPS instances. Create an application that forwards selected messages from one AMPS instance to another.
- Working with views. Create a view in AMPS that does simple calculation on values within incoming messages. Create an application that uses a filtered subscription on the view to view messages where the calculated value is within a certain range. Modify the application to use an out-of-focus message to be notified when the calculated value is outside of that range.
- Try a message queue. The sample configuration has a queue named
sample-queue
defined for the JSON message type. Try sending a few messages to the queue, then starting a few subscribers to receive and acknowledge the messages. - Make a remix. Add
Enrichment
to one of theTopic
definitions in the state of the world and see how AMPS can enrich and annotate messages as they’re published to the server.
Keep It Going¶
Want to know more about how AMPS can help you build great applications? Still having trouble getting AMPS to play your tune? For help or questions, send us a note at support@crankuptheamps.com. And if AMPS ever overloads (produces a minidump), send the dump file to crash@crankuptheamps.com.