9. State-of-the-World (SOW) Features

The SOW section of the configuration file holds the configuration for the AMPS State of the World.

Element Description
Topic Specifies that AMPS will record distinct messages for this topic in the State of the World. SOW topic definitions are used directly as a last-value cache, and are required for many of the advanced messaging features in AMPS such as out-of-focus notifications and delta messaging. SOW topic definitions can also be used as the underlying topics for views, aggregates, and conflated topics. SOW Topic configuration is described in SOW/Topic.
Queue Defines a message queue. Rather than delivering each message to all matching subscriptions, message queues provide features to help ensure that each message is delivered to and processed by a single subscriber. See the User Guide for a full description of the functionality of message queues. Queue configuration is described in SOW/Queue.
View Defines a View over one or more SOW topics, conflated topics, or other views. A view can perform aggregation, can JOIN multiple topics together. A view can be based on a SOW topic of one message type and project results of a different message type. View configuration is described in SOW/View.
ConflatedTopic Defines a copy of a SOW topic or view that receives current value updates at a specified interval, conflating any changes to values that occur between the scheduled updates. ConflatedTopic configuration is described in SOW/ConflatedTopic.

Table 9.1: SOW section configuration elements

The elements within the SOW section are described in detail in the following sections.

SOW/Topic

Creating a Topic element in the the SOW configuration section declares that AMPS will maintain state of the world for that topic. When a topic is recorded in the State of the World (SOW), AMPS maintains the most recent value of each distinct message in that topic. In this respect, the SOW for a topic is similar to a table in a relational database.

Notice that AMPS does not require topics to be predeclared. Creating this configuration item is declaring that the State of the World will be maintained for this topic: the configuration does not define a topic. Creating this configuration item is not required to publish messages to the topic.

There are several sets of options required when a topic is added to the State of the World. General options define the topic name and the overall behavior of the State of the World for this topic. Key specification options define how AMPS identifies unique messages within the topic (much like declaring a primary key for a table in a relational database). Growth specification options define how AMPS grows storage for the State of the World for this topic as messages are added.

Example 9.1 contains a listing of the parameters for a Topic section in the SOW section of an AMPS configuration file. AMPS accepts TopicDefinition as a synonym for Topic, since this was the element name used in versions before 5.0.

This section presents brief descriptions of the options for recording the State of the World for a topic. For full information on the State of the World, see State of the World in the AMPS User Guide.

Element Description
FileName

The file where the State of the World (SOW) data will be stored.

This element is required for SOW topics with a Durability of persistent (the default) because those topics are persisted to the filesystem. This is not required for SOW topics with a durability of transient.

MessageType

Type of messages to be stored. To use AMPS generated SOW keys, the message type specified must support content filtering so that AMPS can determine the SOW key for the message. All of the default message types, except binary, support content filtering. Since the binary message type does not support content filtering, that type can only be used for a SOW when publishers use explicit keys.

See the Message Types chapter in the AMPS User Guide for a discussion of the message types that AMPS loads by default. Some message types (such as Google Protocol Buffers) require additional configuration, and must be configured before using the message type in a SOW topic.

Name

The name of the SOW topic - all unique messages on this topic will be stored in a topic-specific SOW database.

Every SOW requires a method of determining which messages are unique. Several methods are provided within AMPS. See the AMPS User Guide for a discussion on SOW keys, and Key Table for relevant configuration items.

If no Name is provided, AMPS accepts Topic as a synonym for Name to provide compatibility with versions of AMPS previous to 5.0.

HashIndex

AMPS provides the ability to do fast lookup for SOW records based on specific fields.

When one or more HashIndex elements are provided, AMPS creates a hash index for the fields specified in the element. These indexes are created on startup, and are kept up to date as records are added, removed, and updated.

The HashIndex element contains a Key element for each field in the hash index.

AMPS uses a hash index when a query uses a exact matching for all of the fields in the index. AMPS does not use hash indexes for range queries or regular expressions.

AMPS automatically creates a hash index for the set of fields specified in the set of Key fields for the SOW, if those fields are specified.

Index

AMPS supports the ability to precreate memo indexes for specific fields using the Index configuration option.

When one or more Index elements are provided, AMPS creates memo indexes for any field specified in an Index element on startup, before a query that uses that field runs. Otherwise, AMPS indexes each field the first time a query uses the field. Adding one or more Index configurations to a SOW/Topic can improve retrieval performance the first time a query that contains the indexed fields runs for large SOW topics.

RecoveryPoint

For SOW topics that are covered by the transaction log, the point from which to recover the SOW if the SOW file is removed, or if the SOW topic has transient duration.

This configuration item allows two values:

  • epoch recovers the SOW from the beginning of the transaction log
  • now recovers the SOW from the current point in the transaction log

Defaults to epoch.

Expiration

Time for how long a record should live in the SOW database for this topic. The expiration time is stored on each message, so changing the expiration time in the configuration file will not affect the expiration of messages currently in the SOW.

AMPS accepts interval values for the Expiration, using the interval format described in the AMPS Configuration Guide section on units, or one of the following special values:

  • A value of disabled specifies that AMPS will not process SOW expiration for this topic, regardless of any expiration value set on the message. In this case, AMPS saves the expiration for the message, but does not process it. The value must be set to disabled (the default) if History is enabled for this topic.
  • A value of enabled specifies that AMPS will process SOW expiration for this topic, with no expiration set by default. Instead, AMPS uses the value set on the individual messages (with no expiration set for messages that do not contain an expiration value).

Default: disabled (never expire)

Durability

Defines the data durability of a SOW topic. SOW databases listed as persistent are stored to the file system, and retain their data across instance restarts. Those listed as transient are not persisted to the file system, and are reset each time the AMPS instance restarts.

Notice that when the durability is transient, and the topic is recorded in the transaction log, each time AMPS starts, AMPS will recover the state of the topic from the transaction log. The recovery begins at the RecoveryPoint specified for the topic, which defaults to epoch, the beginning of the transaction log. (For persistent topics, AMPS recovers from the last message written to the SOW topic, or from the RecoveryPoint if the SOW file is removed.)

Default: persistent

Valid values: persistent or transient

Synonyms: Duration is also accepted for this parameter for backward compatibility with configuration prior to 4.0.0.1

History

Enable historical query for this SOW. This element contains a Window and Granularity element. When the History element is present, historical query is enabled for this sow. Otherwise, AMPS does not enable historical query and does not store the historical state of the SOW.

Expiration must be disabled when History is enabled.

Window

For a historical SOW, the length of time to store history. For example, when the value is 1w, AMPS will store one week of history for this SOW.

Used within the History element.

Default: By default, AMPS does not expire historical SOW data.

Granularity

For a historical SOW, the granularity of the history to store. For many applications, it is not necessary for AMPS to store all of the updates to the SOW. This parameter sets the resolution at which AMPS will save the state of a message. A value of 0s or equivalent specifies that AMPS will preserve every update within the Window.

For example, when you set a granularity of 1m, AMPS will save the state of the message no more frequently than once per minute, even when the state of the message is updated several times a minute.

Used within the History element.

Preprocessing

When present, specifies the message enrichment to be performed before AMPS determines the SOW key for the message.

The Preprocessing element must contain one or more Field elements that specify the enrichment to perform.

Enrichment

When present, specifies the message enrichment to be performed after AMPS determines the SOW key for the message.

The Enrichment element must contain one or more Field elements that specify the enrichment to perform.

Table 9.2: SOW/Topic General Options

Each SOW topic must define how AMPS will determine which messages are unique. An application can either have AMPS determine the key by specifying one or more Key fields, provide a SOW key with the publish command each time a message is published to AMPS. AMPS also provides the ability to provide a custom SowKey generator with a plugin module.

See the Sow Key Generation section in the AMPS User Guide for for a full discussion. The following table lists the available configuration items for specifying how AMPS determines the SowKey for a message:

Element Description
Key

Specifies an XPath within each message that AMPS will use to generate a SOW key, which determines whether a message is unique. This element can be specified multiple times to create a composite key from the combined value of the specified Key elements.

When one or more Key elements is specified for the SOW, AMPS generates the SOW key for each message. When no Key fields are specified and no KeyGenerator is specified, publishers must explicitly provide the SOW key for each message when the message is published.

60East recommends configuring a Key and having AMPS generate the SOW key for a message unless your application has specific needs that make this impractical.

AMPS automatically creates a hash index for the set of fields specified in the Key elements.

There is no default for this element.

KeyDomain

The seed value for SowKeys used within the topic when AMPS generates the SOW key. The default is the topic name, but it can be changed to a string value to unify SowKey values between different topics.

For example, if your application has a ShippingAddress SOW and a CreditRating SOW that both use /customerID as the SOW key, you can use a KeyDomain to ensure that the generated SowKey for a given /customerId is identical for both SOW topics. This does not affect how AMPS processes the SOW topics, but can make correlating information from different SOW topics easier in your application.

This option can only be specified when one or more Key fields are specified. When a SOW key generator module is used, or the publisher must send a SOW key, this option is not valid.

Default: the name of the SOW topic.

KeyGenerator

Specifies the SOW key generator module to use for this topic. When this configuration element is present, AMPS calls the specified module to generate a SOW key for each incoming message.

Default: no SOW key generator module. When there is no SOW key generator module specified, AMPS uses the specified Key fields if the Key fields are provided. If no generator is specified and no Key fields are specified, AMPS requires publishers to set a SOW key on each message published.

A KeyGenerator element contains the following elements:

Element Description
Module
The name of the module. This module must be loaded elsewhere in the configuration file.
Options Contains one or more XML elements. These elements are provided to the key generator module as options. The options provided depend on the key generator. The creator of the key generator module must document the options for that module.

Table 9.3: SOW/Topic Key Specification Options

The SOW topic configuration also specifies how the SOW file is allowed to grow. See the AMPS User Guide for details on SOW growth. The configuration options for controlling how the file is allocated and how the file grows are listed below:

Element Description
SlabSize

The size of each allocation for the SOW file, as a number of bytes. When AMPS needs more space for the SOW, it requests this amount of space from the operating system. This effectively sets the maximum message size that AMPS guarantees can be stored in the SOW. This size includes headers set by AMPS on the message.

60East recommends setting this value only if you will be storing messages larger than the default SlabSize or if performance or capacity testing indicates a need to tune SOW performance. If you plan to store messages larger than the default setting, 60East recommends a starting value of several times the maximum message size. For example, if your maximum message size is 2MB, a good starting point for SlabSize would be 8MB.

If it becomes necessary to tune the SlabSize, see the Best Practices section of the AMPS User Guide and Capacity Planning section of the AMPS User Guide for a full discussion about tuning the SlabSize.

Default: 5MB

InitialSlabCount

The number of SOW slabs that AMPS will allocate on startup.

Default: 1

Maximum: 1024

DEPRECATED:

RecordSize

This parameter is deprecated beginning in AMPS 5.0. Use SlabSize parameter instead. Size (in bytes) of a SOW record for this topic.

Default: 512

DEPRECATED:

InitialSize

This parameter is deprecated beginning in AMPS 5.0. Use the InitialSlabCount parameter instead. Initial size (in records) of the SOW database file for this topic.

Default: 2048

DEPRECATED:

IncrementSize

This parameter is deprecated beginning in AMPS 5.0. Use the SlabSize parameter instead. Number of records to expand the SOW database (for this topic) by when more space is required.

Default: 1000

Table 9.4: SOW/Topic Growth Specification Options

An example of a SOW configuration looks like the following:

<SOW>
    <!-- Simple SOW topic definition -->
    <Topic>
        <Name>orders</Name>
        <Key>/orderId</Key>
        <MessageType>nvfix</MessageType>
        <FileName>./sow/%n.sow</FileName>
    </Topic>

    <!-- SOW with hash indexes -->
    <Topic>
        <Name>customers</Name>
        <Key>/customerId</Key>
        <MessageType>json</MessageType>
        <FileName>./sow/%n.sow</FileName>
        <HashIndex>
            <Key>/customerName</Key>
        </HashIndex>
        <HashIndex>
            <Key>/zipCode</Key>
            <Key>/customerType</Key>
        </HashIndex>
    </Topic>

    <!-- Historical SOW -->
    <Topic>
        <Name>catalog</Name>
        <Key>/sku</Key>
        <MessageType>json</MessageType>
        <FileName>./sow/%n.sow</FileName>
        <History>
            <Window>7d</Window>
            <Granularity>15m</Granularity>
        </History>
    </Topic>

    <!-- SOW with enrichment Add a /fullName field constructed
        from /firstName and /lastName. -->
    <Topic>
        <Name>sales-reps</Name>
        <Key>/employeeId</Key>
        <MessageType>bflat</MessageType>
        <Enrichment>
            <Field>CONCAT(/firstName, " ", /lastName) AS /fullName</Field>
        </Enrichment>
    </Topic>
</SOW>

Example 9.1: SOW Topic Configuration

SOW/Queue and SOW/LocalQueue

This section lists configuration parameters for queues.

The Queue tag and the LocalQueue tag are used to configure message queues.

When an AMPS queue is defined with the Queue tag, the queue will be a distributed queue. To make a queue that is limited to the local instance, use the LocalQueue configuration element

AMPS accepts QueueDefinition as a synonym for Queue.

Element Description
Name

The name of the queue topic. This name is the name that consumers subscribe to.

If no Name is provided, AMPS accepts Topic as a synonym for Name in the Queue definition.

MessageType
The message type of the queue.
UnderlyingTopic

A topic name or regular expression for the topic that contains the messages to capture in the queue. These topics must be recorded in a transaction log, and all must be of the same message type as the queue.

If an UnderlyingTopic is not provided, the UnderlyingTopic defaults to the Name of the queue.

DefaultPublishTarget

The topic to publish to when an application publishes a message to the queue. For simplicity, AMPS allows applications to publish messages to the queue, and for those messages to be routed to one of the underlying topics.

This element is required if the UnderlyingTopic contains regular expression characters. Otherwise, the UnderlyingTopic is a single topic and this element is optional and defaults to the UnderlyingTopic.

LeasePeriod

The amount of time that a subscriber has ownership of the message before the message is returned to the queue. For at-least-once delivery semantics, the consumer must process and acknowledge the message within this lease period, or the message may be provided to another subscriber.

The LeasePeriod is measured from the time that AMPS sends the message to the subscriber. Set the LeasePeriod to account for round trip network latency as well as the expected processing time for the subscribers.

Default: infinite (no expiration)

Semantics

The delivery semantics to use for this queue. Regardless of the delivery semantics, AMPS queues deliver a given message to a single subscriber at a time. When a subscriber fails to process a message, the delivery semantics specify how that failure should be handled.

  • at-least-once With these semantics, you can guarantee that a message has been processed by at least one subscriber, as described in the introduction to Queues in the AMPS User Guide. With this value, a subscriber must explicitly remove the message from the queue once the message is processed. If the subscriber fails before acknowledging the message, returns the message to the queue, or the lease expires, the AMPS server can deliver the message again to allow another attempt to process the message.
  • at-most-once With these semantics, AMPS removes the message from the queue immediately when AMPS sends the message. This allows you to guarantee that no more than one subscriber will process the message, even if the subscriber that receives the message fails without acknowledging the message.

Default: at-least-once

MaxBacklog

The maximum number of outstanding, unacknowledged messages in the queue at any one time. This parameter allows you to set limits on the number of pending messages from the queue overall. When the queue reaches the MaxBacklog, no incoming messages are delivered from the queue until a message is removed from the queue (either by expiring, or being acknowledged by a client). This parameter allows you to avoid overwhelming clients during periods of heavy activity.

Notice that this does not set a limit of any sort on the capacity of the queue. This parameter allows you to limit the number of messages that the queue will make available to subscribers at a given time, but does not restrict the capacity of the queue to track messages.

Default: infinite

MaxPerSubscriptionBacklog

The maximum number of outstanding, unacknowledged messages in the queue for an individual subscription. This parameter allows you to avoid overwhelming a single subscriber during a period of heavy activity.

Subscribers can declare the maximum number of messages that the subscription is prepared to lease at a given time. This maximum defaults to 1 when there is no maximum explicitly specified for a subscription. AMPS will lease the number specified in the subscription or the maximum set for the queue, whichever is lower.

Notice that this does not set a limit of any sort on the capacity of the queue. This parameter allows you to limit the number of messages that the queue will make available for a single subscription at a given time, but does not restrict the capacity of the queue to track messages.

Default: 1

Expiration

The length of time a message can remain in the queue before AMPS considers the message undeliverable.

Messages may expire while a subscriber has a lease on the message. AMPS does not send an additional notification in this case.

A publisher can override this setting by providing an expiration value on an incoming message. If an individual message in the transaction log has an expiration value, AMPS will use that expiration value for the message rather than the default for the queue.

Default: infinite

Filter

An AMPS Filter that is applied to the UnderlyingTopic. When a Filter is specified, only messages matching the Filter appear in the queue.

By default, there is no filter and all messages from the UnderlyingTopic are presented in the queue.

RecoveryPoint

This option allows you to specify the point at which AMPS begins reviewing the transaction log to recover the state of the queue when AMPS restarts. By default, AMPS reviews the full log to determine the contents and state of the queue.

The RecoveryPoint can be one of the following:

  • epoch - Recovery begins at the beginning of the transaction log
  • now - Recovery begins at the time AMPS starts queue recovery, so only new messages are added to the queue
  • creation - Recovery begins at the time the queue was created
  • AMPS bookmark - When an AMPS bookmark is provided, AMPS starts recovery at the specified bookmark.
  • ISO-8601 timestamp - When a timestamp is provided, AMPS starts recovery at the specified timestamp. The timestamp must be provided in the format AMPS uses for timestamp bookmarks.

Default: epoch

FairnessModel

AMPS provides different methods to distribute messages across active subscriptions:

  • fast - AMPS delivers to the first subscription found that can process the message
  • round-robin - AMPS distributes to the next subscription found that can process the message
  • proportional - AMPS delivers to the subscription with the lowest ratio of active messages to available backlog

Default: proportional for at-least-once queues, round-robin for at-most-once queues

Leasing

Ownership model for leased messages. AMPS supports the following models:

  • strict - AMPS allows a client to acknowledge (sow_delete) only messages that are leased to the client or currently unleased. If a client acknowledges a message leased to another client, there is no effect.
  • sublet - AMPS allows any client to acknowledge any message, regardless of whether another client has a lease on the message.

Default: sublet

MaxDeliveries

Specifies an upper bound to the number of times AMPS may deliver a queue message before automatically expiring it. For example if AMPS delivers a message twice and MaxDeliveries is set to 2, the message will be expired if the subscriber disconnects or unsubscribes before acknowledging it.

This counter is reset if the server restarts, and the counter is not replicated to other instances.

Default: no maximum (0).

MaxCancels

Specifies a limit to the number of times a subscriber may cancel a lease on a message before it is expired. For example if a message is canceled for the second time and MaxCancels is set to 2, AMPS automatically expires the message instead of returning it to the message queue.

This counter is reset if the server restarts, and the counter is not replicated to other instances.

Default: no maximum (0).

Priority

Specifies the order in which messages will be distributed from the queue. When present, this element constructs a value that specifies the priority for messages in the queue. Higher priority messages are delivered first, regardless of the order in which messages have been published to the queue.

The contents of the element can be either the name of a field in the message or an AMPS expression. Either way, the result is treated as an unsigned long value.

For example, to order message delivery based on the /priority field of the message, specify the following element:

<Priority>/priority</Priority>

To order message delivery based on the product of the /price field and the /quantity field in the message, specify the following element.

<Priority>/price * /qty</Priority>

Default: there is no default for this value. When not provided, the delivery order is the order in which messages were processed by this instance of AMPS

DeferredAckExpiration

Specifies the amount of time for AMPS to retain information about an acknowledgement (sow_delete) message received when the corresponding message is not in the queue. This can occur during failover, when messages are received over replication, or in cases where an application that uses a publish store has been offline for an extended period of time.

This element is configured as an interval, for example, 15m or 2h.

Default: 1d

Table 9.5: Queue configuration elements

<!-- Notice that the topics to use for the queue (ORDERS_.*) must be
    recorded in a transaction log. -->
<SOW>
    <Queue>
        <Name>MQ</Name>
        <MessageType>json</MessageType>
        <UnderlyingTopic>ORDERS_.*</UnderlyingTopic>
        <DefaultPublishTarget>ORDERS_DIRECT</DefaultPublishTarget>
        <LeasePeriod>60s</LeasePeriod>
        <Expiration>1d</Expiration>
        <MaxBacklog>3</MaxBacklog>
    </Queue>
</SOW>

SOW/View

Table 9.6 contains a listing of the parameters for a View section in the SOW section of an AMPS configuration file. For backward compatibility, AMPS accepts ViewDefinition as a synonym for View.

Element Definition
MessageType

The message type of the view. This does not need to be the same type as any of the topics in the aggregation, but does need to be a message type that supports views.

The message type must be one of the message types configured for the instance. AMPS includes fix, xml, nvfix, json, bson and bflat message types with full support for views. You can also use any custom message type defined for the configuration file, provided that the message type supports views.

Notice that the binary message type does not specify a fixed format for the message contents, so that message type cannot be used in a view.

Other message types provided with AMPS may have limitations in their support for views. see the Message Types chapter in the AMPS User Guide for a discussion of the limitations.

Name

Defines the topic name for this view.

If no Name is provided, AMPS accepts Topic as a synonym for Name to provide compatibility with versions of AMPS previous to 5.0.

UnderlyingTopic
Defines the SOW topic or topics on which this view is based. This element can contain a single topic name, or any number of Join elements.
Projection/Field
Defines what the view will contain. This element can be specified multiple times to compose a complex view. Complex expressions that use aggregation functions and conditional branching can also be used.
Grouping/Field
Defines how the records in the underlying topic will be grouped. This is analogous to a SQL GROUP BY clause.
KeyDomain The seed value for SowKeys used within this topic. The default is the topic name, but it can be changed to a string value to unify SowKey values between different topics.
Join

Within an UnderlyingTopic, each Join specifies two topics to join together to create the view, as well as the relationship between those topics.

An UnderlyingTopic can have any number of Join specifications. For more information on Join specifications, see the Join expression in the AMPS User Guide.

Conflation

Defines whether AMPS will attempt to conflate updates to the view.

This item accepts two values:

  • none - No conflation. AMPS fully processes every update to the view, in publication order.
  • inline - Conflation is active. AMPS conflates updates to the same underlying record where possible. See Inline Conflation in the AMPS User Guide for details.

Default: none

Filter

Defines a filter for the view. When provided, only messages from the underlying topic that match this filter will be included in the view.

This option is only valid when the view uses a single UnderlyingTopic. When the view contains a Join specification, this option may not be used.

Default: No filter, which includes all messages from the UnderlyingTopic.

FileName File location to store view data. Unused in this version of AMPS.

Table 9.6: SOW/View

<SOW>
    <!-- Single topic aggregation -->
    <Topic>
        <Topic>/ett/order</Topic>
        <MessageType>fix</MessageType>
        <Key>/orderId</Key>
    </Topic>
    <View>
        <FileName>./sow/%n.view.sow</FileName>
        <MessageType>nvfix</MessageType>
        <Topic>TOTAL_VALUE</Topic>
        <UnderlyingTopic>/ett/order</UnderlyingTopic>
        <Projection>
            <Field>/109</Field>
            <Field>SUM(/14 * /6) AS /71406</Field>
        </Projection>
        <Grouping>
            <Field>/109</Field>
        </Grouping>
    </View>

    <!-- Single topic aggregation with filter -->
    <Topic>
        <Name>orders</Name>
        <MessageType>json</MessageType>
        <Key>/orderId</Key>
        <FileName>./sow/%n.sow</FileName>
    </Topic>
    <View>
        <Name>CompleteByRegion</Name>
        <UnderlyingTopic>orders</UnderlyingTopic>
        <MessageType>json</MessageType>
        <Projection>
            <Field>COUNT(/orderId) AS /completedOrders</Field>
            <Field>/region AS /region</Field>
        </Projection>
        <Grouping>
            <Field>/region</Field>
        </Grouping>
        <Filter>/status = 'complete'</Filter>
    </View>

    <!-- Project from one message type to another -->
    <Topic>
        <Name>example</Name>
        <MessageType>json</MessageType>
        <Key>/id</Key>
        <FileName>./sow/%n.sow</FileName>
    </Topic>
    <View>
        <!-- notice that nvfix topic named 'example' is not the same topic as the json
            topic named 'example' -->
        <Name>example</Name>
        <MessageType>nvfix</MessageType>
        <UnderlyingTopic>[json].[example]</UnderlyingTopic>
        <Projection>
            <Field>[json].[example]./id AS /id</Field>
        </Projection>
        <Grouping>
            <Field>[json].[example]./id</Field>
        </Grouping>
    </View>

   <!-- JOIN topics -->
   <Topic>
        <Name>ORDERS</Name>
        <MessageType>nvfix</MessageType>
        <Key>/OrderID</Key>
        <FileName>./sow/%n.sow</FileName>
    </Topic>
    <Topic>
        <Name>COMPANIES</Name>
        <MessageType>nvfix</MessageType>
        <Key>/CompanyId</Key>
        <FileName>./sow/%n.sow</FileName>
    </Topic>
    <View>
        <Name>TOTAL_COMPANY_VOLUME</Name>
        <UnderlyingTopic>
            <Join>[ORDERS]./Tick = [COMPANIES]./Tick</Join>
        </UnderlyingTopic>
        <FileName>./views/totalVolume.view</FileName>
        <MessageType>nvfix</MessageType>
        <Projection>
            <Field>[COMPANIES]./CompanyId</Field>
            <Field>[COMPANIES]./Tick</Field>
            <Field>[COMPANIES]./Name</Field>
            <Field>SUM([ORDERS]./Shares) AS /TotalVolume</Field>
        </Projection>
        <Grouping>
            <Field>[ORDERS]./Tick</Field>
        </Grouping>
    </View>
</SOW>

Example 9.2: View Example

SOW/ConflatedTopic

AMPS provides the ability to create ongoing snapshots of a SOW topic, called conflated topics (also called topic replicas in previous releases of AMPS). Topic replicas are updated on an interval, and store a snapshot of the current state of the world at each interval. This helps to manage bandwidth to clients that do not act on each update, such as a client UI that refreshes every second rather than with every update.

For compatibility with previous versions of AMPS, AMPS allows you to use ReplicaDefinition as a synonym for ConflatedTopic.

Element Description
Name

String used to define the name of the conflated topic. While AMPS doesn’t enforce naming conventions, it can be convenient to name the conflated topic based on the underlying topic name. For example, if the underlying topic is orders, it can be convenient to name the conflated topic orders-C.

If no Name is provided, AMPS accepts Topic as a synonym for Name to provide compatibility with versions of AMPS previous to 5.0.

UnderlyingTopic
String used to define the SOW topic which provides updates to the conflated topic. This must exactly match the name of a SOW topic.
MessageType
The message format of the underlying topic. This MessageType must be the MessageType of the provided UnderlyingTopic.
Interval

The frequency at which AMPS updates the data in the conflated topic.

Default: 5s

Minimum: 100ms

Filter Content filter that is applied to the underlying topic. Only messages that match the content filter are stored in the conflated topic.
HashIndex

AMPS provides the ability to do fast lookup for SOW records based on specific fields.

When one or more HashIndex elements are provided, AMPS creates a hash index for the fields specified in the element. These indexes are created on startup, and are kept up to date as records are added, removed, and updated.

The HashIndex element contains a Key element for each field in the hash index.

AMPS uses a hash index when a query uses a exact matching for all of the fields in the index. AMPS does not use hash indexes for range queries or regular expressions.

AMPS automatically creates a hash index for the set of fields specified in the set of Key fields for the UnderlyingTopic, if that topic provides Key fields.

Table 9.6: SOW/ConflatedTopic Parameters

<ConflatedTopic>
    <Topic>FastPublishTopic-C</Topic>
    <MessageType>nvfix</MessageType>
    <UnderlyingTopic>FastPublishTopic</UnderlyingTopic>
    <Interval>5s</Interval>
    <Filter>/region = 'A'</Filter>
</ConflatedTopic>
<ConflatedTopic>
    <Topic>LongIntervalTopic-C</Topic>
    <MessageType>json</MessageType>
    <UnderlyingTopic>FastPublishTopic</UnderlyingTopic>
    <Interval>120s</Interval>
    <HashIndex>
       <Key>/order/status</Key>
    </HashIndex>
</ConflatedTopic>