Ongoing discussions within the SWIM Community of Interest
AMQP Message metadata
The following material describes different initiatives, at different maturity levels, that may help determine if (and, in case, which kind of) guidelines and best practices for AMQP 1.0 message properties definition may be needed in the ATM SWIM environment. They refer to different “domains” such as network management, meteorological data, and data link services but they all introduce AMQP metadata (i.e. properties) for multiple purposes (e.g. facilitate filtering, enable routing, tracing and troubleshooting etc..).
As they have been developed by different groups, they present some differences both regarding naming “style” and, in some cases, approaches on how to use the same properties. Therefore, this may suggest the need for guidelines providing some level of harmonization in the area of:
Consistent definition and usage of AMQP message properties (standard and application-specific)
Efficient filtering, routing, and processing of messages
Traceability with regulatory and operational requirement
NM B2B API Design Conventions 3.0
Status: Mature Draft
Context: Developed as part of the evolution of the Network Manager (NM) B2B services, aligning with the SWIM Technical Infrastructure (TI) Yellow Profile and the CP1 regulation. It aims to harmonize and future-proof NM B2B APIs, ensuring compliance, usability, and interoperability for both synchronous (HTTP/REST) and asynchronous (AMQP 1.0) exchanges.
The approach taken in the NM B2B API Design Conventions is based on the need for harmonization, future-proofing, and regulatory compliance for Network Manager (NM) B2B services. The document addresses both synchronous (HTTP/REST) and asynchronous (AMQP) patterns and introduces a well-defined approach for the development of future B2B services in this context.
The conventions defined in the document establish a clear separation between different types of message properties— header, message, and application properties— each serving a distinct purpose. They expect the use of machine-processable definitions (OpenAPI for HTTP, AsyncAPI for AMQP) ensuring that both human and machine consumers can reliably interpret service contracts.
A key decision is to use structured and semantically meaningful values for properties like subject and application-specific fields, enabling efficient filtering, routing, and traceability. The conventions also provide for extensibility, allowing new business needs and technologies to be integrated without disrupting existing clients.
For AMQP 1.0, the conventions specify a layered property model:
Header properties (e.g.,
durable,ttl,delivery-count) are set by the messaging infrastructure and control delivery semantics.Message properties (e.g.,
subject,content-type,absolute-expiry-time,correlation-id) are explicitly defined for each message type. Thesubjectproperty, in particular, is used to encode structured routing and business context, such asflight-management/FLIGHT_DATA/FSA, enabling both topic-based routing and fine-grained filtering.Application properties are reserved for business-level metadata and operational context. These include standardized fields such as
NM_BUSINESS_ID,NM_TYPE, andNM_SERVICE_VERSION, as well as topic-specific fields (e.g.,NM_CONCERNED_UNITS_ANU). These properties are important for supporting operational filtering, traceability, and integration with external systems.
EUROCONTROL Specification for Data Link Ground Distribution SWIM Services
Status: Published Specification
Context: This is a formal EUROCONTROL specification for SWIM services supporting the ground distribution of ADS-C and Logon Information, for initial trajectory information sharing in Europe. It defines requirements for the ground distribution of data link information, specifically ADS-C and DLIC (Datalink Initiation Capability) data, within a System Wide Information Management (SWIM) context. It enables the sharing of air-ground data through SWIM services, optimizing the use of air-ground and airborne resources. It details requirements for the establishment and operational deployment of these services by ATM stakeholders.
The EUROCONTROL Specification for Data Link Ground Distribution SWIM Services provides (among other aspects) a prescriptive, requirements-driven framework for the use of AMQP 1.0 message properties in the context of ground distribution of ADS-C and Logon Information. The specification mandates AMQP 1.0 for all asynchronous messaging and defines, for each interface, the exact set of required and optional message properties.
Header and message properties such as durable, priority, message-id, subject, content-type, and creation-time are specified with operational semantics. For example, the priority property is used to differentiate between routine and urgent messages, while durable is set to ensure message persistence as required by the operational context.
Application properties are central to the specification’s approach to filtering and routing. Properties such as aircraftAddress, flightId, departureAirport, destinationAirport, eobd, eobt, and adsVersion are defined as mandatory for certain message types, enabling consumers to use AMQP selectors to subscribe to messages matching specific operational criteria. The specification also defines the use of ISO 8601 formats for temporal properties.
The subject property is used to distinguish between business and technical messages, with values such as flightStatus, AdsDemandCnf, or serviceStatus. This enables both operational and administrative workflows to be handled in a unified, machine-interpretable way.
Payloads are strictly schema-driven, with all business messages validated against ASN.1-derived XML schemas. Error handling is standardized, with explicit conventions for status codes, reason phrases, and error details, ensuring interoperability and ease of integration with client systems.
MET SWIM AMQP guidance material
Status: Early Draft
Context: It is a draft guidance document from the EUROCONTROL MET3SG Task Team, focused on standardizing AMQP 1.0 message properties for meteorological (MET) data exchange in the context of MET-SWIM Services (CP1).
Source: https://github.com/iblsoft/swimdemo/blob/main/MET-SWIM-AMQP-Guidance.md
The MET-SWIM-AMQP-Guidance is a detailed, community-driven (early) draft that aims to standardize the use of AMQP 1.0 message properties for meteorological data exchange in the European MET-SWIM context. The guidance defines a comprehensive message structure, including addressing, header, standard, and application properties, with a focus on supporting both operational efficiency and interoperability.
Message addressing uses a simplified three-level hierarchy (e.g., weather.aviation.metar), inspired by the WIS 2 Topic Hierarchy but tailored for aviation MET data. This structure supports both precise and wildcard-based subscriptions, with an on-going discussion regarding the homogenous behavior of wildcards among different broker implementations (e.g., ActiveMQ Artemis, RabbitMQ, Qpid Proton).
Header and standard properties are mandated with operationally meaningful values. For example, priority is proposed to be set according to the operational importance of the message type (e.g., SIGMET and SPECI at priority 7, METAR at 4), and absolute-expiry-time is set based on the expected operational lifetime of the data (e.g., METAR/SPECI: +3h, TAF: +12h, SIGMET: +24h).
Application properties are categorized as mandatory, conditional, or optional:
Mandatory:
topic,properties.icao_location_identifier,properties.icao_location_type,properties.pubtime.Conditional: Temporal properties (
properties.datetime,properties.start_datetime,properties.end_datetime), external links (links.count,links[0].href,links[0].rel,links[0].type), integrity checks (properties.integrity.method,properties.integrity.value), and geometry for spatial filtering (geometry.type,geometry.coordinates.latitude, etc.).Optional: Additional links, payload integrity, geometry properties for advanced spatial queries.
The guidance provides detailed mapping and extraction rules for converting IWXXM XML data into AMQP properties, including XPath expressions for each property and handling of XML namespaces. It also discusses the use of cryptographic hashes for payload integrity, with recommendations for supported algorithms (e.g., SHA-512, SHA-256).
Filtering is specific concern: application properties are designed for server-side filtering using AMQP filter expressions, supporting operational use cases such as filtering by location, time, or report type. The guidance aligns with international standards (WMO WIS2, ICAO Annex 3). Nevertheless, discussions are on-going on property naming, optionality, and alignment with other SWIM documentation.
Summary Table: Key Features Across Initiatives
Initiative / Document | Scope | AMQP Property Focus | Filtering | Compliance |
|---|---|---|---|---|
NM B2B API Design Conventions | Future NM B2B Services | Full property set (header, message, application); structured subject; topic-specific application properties | Yes (but expected to be requested through the “subscription” service) | SWIM-TI Yellow Profile, CP1 |
EUROCONTROL Data Link Ground Distribution | ADS-C, Logon SWIM Services | Strict property requirements per interface; business/technical message distinction | Yes (selectors, application properties) | SWIM-TI Yellow Profile, CP1 |
MET-SWIM-AMQP-Guidance | MET-SWIM (CP1) | Detailed mapping of standard and application properties; spatial/temporal properties; integrity | Yes (selectors, spatial/temporal) | SWIM-TI Yellow Profile, CP1, ICAO Annex 3 |
Use of Standard AMQP 1.0 Properties across the documents
Commonalities
All three initiatives require the use of core AMQP 1.0 standard properties such as
subject,content-type,content-encoding,absolute-expiry-time, andpriorityfor operational messages.All use application properties for business/operational metadata and filtering.
Differences in Approach and Semantics
subjectproperty:NM B2B: Used as a structured string encoding business context (e.g.,
flight-management/FLIGHT_DATA/FSA). The structure is flexible and can be adapted per API or topicData Link Spec: Used to distinguish message types (e.g.,
flightStatus,AdsDemandCnf). The value is more type-oriented and less hierarchicalMET-SWIM: Mandates a specific, pattern-based structure for
subject(e.g.,DATA_TAF_LOWS_NORMAL_20250401051500), encoding report type, location, status, and issue time. This is more rigid and operationally focused. Note: This approach is currently under revision (it will likely rather host the “topic” name as foreseen in NM B2B but under the formweather.aviation.metar).
content-typeproperty:The approach and semantic is very well aligned. As it is normal, the expected values may differ depending on the expected content (e.g.
application/xmlfor XML payloads,application/jsonfor JSON payloads or other types for certain messages, depending on the payload and service).
priorityproperty:MET-SWIM: Explicitly assigns operational priorities (e.g., SIGMET/SPECI = 7, METAR = 4) and provides rationale for each
Data Link Spec: Uses default priorities, but less operational differentiation.
NM B2B: Priority is available but not as operationally prescribed as in MET-SWIM.
absolute-expiry-timeproperty:MET-SWIM: Specifies expiry times based on message type (e.g., METAR/SPECI: +3h, TAF: +12h, SIGMET: +24h)
Others: May set expiry, but not as operationally differentiated.
In general, they all link the value of this property to the value of
ttl
Application properties:
All use application properties for filtering, but the naming, structure, and required fields differ (see below).
Naming Conventions for Properties
Commonalities
All use lowerCamelCase or snake_case for application property names, but with domain-specific variations
Differences
NM B2B: Uses uppercase with underscores for NM-specific properties (e.g.,
NM_BUSINESS_ID,NM_TYPE).Data Link Spec: Uses lowerCamelCase for most application properties (e.g.,
aircraftAddress,flightId,departureAirport), but also includes some properties starting with uppercase (e.g.Delivery,Timeout)MET-SWIM: Uses a mix of dot notation and snake_case for hierarchical properties (e.g.,
properties.icao_location_identifier,properties.pubtime,geometry.type), and sometimes prefixes withproperties.orlinks.for grouping.
Topic/Addressing Structure and Symbols
Commonalities
All initiatives use some form of topic or address structure to organize message routing and filtering
Differences
NM B2B: Encodes topic and event type in the
subjectproperty using slashes (e.g.,flight-management/FLIGHT_DATA/FSA). Even if not explicitly mentioned, data publication over AMQP is linked to a previous subscription obtained through a dedicated Subscription interface (SOAP or REST based). It dynamically returns an AMQP endpoint address to the specific consumer where it can connect to consume the messages.Data Link Spec: Uses subscription management interfaces REST endpoint-based addressing (e.g.,
/flightStatusSubscriptions/{subscriptionId}) dynamically returning an AMQP endpoint address (e.g.:/<address>) to the specific consumer. There is no reference to “topic” concept in the Specification as it is implicit in the endpoint/subscription context.subjectproperty can be used to discern the specific message (e.g.AdsDemandCnf,AdsEventCnf,AdsPeriodicCnf…) that is received through the AMQP Endpoint. Filtering may be done via application properties, not topic hierarchy.MET-SWIM: Uses a dot-separated, hierarchical topic structure (e.g.,
weather.aviation.metar). Wildcards (*,#) are supported for subscriptions, but their interpretation may depend on the broker (e.g., ActiveMQ Artemis, RabbitMQ, Qpid Proton - Note: this is under verification). The topic structure is explicitly mapped to operational domains and report types.
Note: AMQP 1.0 Standard does not include references to the concept of “topic”. <So, it might be useful to clarify if/how it is possible to use AMQP - in interoperable ways - in relation to “topic” based subscriptions>
Semantics and Usage Patterns
Commonalities
All initiatives use properties for both routing and filtering, and to encode operational context.
Differences
NM B2B: The semantics of properties are flexible and can be adapted per API or topic. The focus is on extensibility and model-driven design.
Data Link Spec: Semantics are tightly coupled to operational requirements, with strict validation and error handling. The meaning of each property is fixed and documented per interface.
MET-SWIM: Semantics are operationally driven and standardized across the MET domain, with detailed mapping from IWXXM XML to AMQP properties and explicit rules for each property’s meaning and usage.
Potential subjects for guidance material and/or best practices
Naming style of Application Properties (e.g. camelCase, sneak_case …)
Approach for Properties Grouping
Guidance/Best Practices for usage of standard AMQP properties
Best practices for realizing a subscription mechanism supporting topic hierarchies which may be “portable” across different Broker implementations (final aim being that perceived behavior - consumers side - is not dependent by the Broker implementation being used by the particular provider)
Best practices/approaches to support different Authorization scenarios while avoiding dependencies on Broker/product specific features. E.g.
Consumers are basically authorized to consume any kind of message once authenticated
Authenticated Consumers are authorized to consume only a given kind (topic?) of messages flowing over a queue
Authenticated Consumers are authorized to consume messages depending on values of some metadata attached to them (e.g. value of “subject” property, value of metadata related to geographical coverage etc..).
The following is a example for a possible generic guidance on AMQP Properties (Standard and Application)
Guidelines for AMQP v1.0 Message Properties Definition
1. Introduction
In the context of fostering interoperability among different stakeholders, a consistent and clear definition of message properties is very important. In the following, we are providing guidelines for effective use of both standard AMQP properties and custom application-specific properties. These are meant to enhance message clarity, facilitate efficient routing and processing, and ultimately contribute to a more resilient and interoperable messaging ecosystem.
2. AMQP 1.0 Message Structure Overview
An AMQP 1.0 message is composed of several sections, each serving a distinct purpose. Below is a graphical representation as found in the AMQP 1.0 Standard:
While we will focus specifically on message properties, it's useful to briefly understand their context within the overall AMQP 1.0 message structure:
Properties Section: Contains a set of standard, well-defined message properties (e.g.,
message-id,content-type,correlation-id).Application Properties Section: A map of application-defined properties. These are custom key-value pairs that carry application-specific metadata.
Data Section(s): Contains the actual message payload, which can be binary, string, or a sequence of AMQP data types.
Delivery Annotations Section: A map of delivery-specific annotations.
Message Annotations Section: A map of message-specific annotations.
Header Section: Contains durable, priority, and time-to-live information.
Footer Section: Contains information about the message's outcome.
Our primary focus will be on the Properties and Application Properties sections.
3. Standard AMQP Message Properties
The AMQP v1.0 specification defines a set of standard properties within the properties section of a message. These properties provide common metadata that can be understood and processed by generic AMQP intermediaries and applications.
Here's an explanation of key standard properties, their aim, and best practices for their usage:
3.1 message-id
Aim: A unique identifier for the message. It allows for duplicate detection, message tracking, and idempotent processing.
Best Use:
Uniqueness: Ensure global uniqueness, typically by using UUIDs (Universally Unique Identifiers).
Tracking: Used by monitoring systems to trace the message's journey through the system.
Idempotency: Consumers can use the
message-idto ensure that processing a message multiple times (e.g., due to retries) has the same effect as processing it once.
Example:
message-id: "urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6"Scenario: An order placement system sends an
OrderCreatedevent. Themessage-idensures that if the event is redelivered, the order is not processed twice.
correlation-id
Aim: An identifier that links a response message to a request message, or relates messages within a conversation or workflow.
Best Use:
Request-Reply Patterns: The sender of a request message includes a
correlation-id. The receiver of the request copies thiscorrelation-idinto the response message, allowing the original sender to match the response to its request.Workflow Tracing: Link related messages in a complex business process (e.g., an
OrderCreatedmessage and a subsequentPaymentProcessedmessage that are part of the same order fulfillment workflow).
Example:
Request Message:
message-id: "req-12345" reply-to: "queue://my.reply.queue"Response Message:
message-id: "res-67890" correlation-id: "req-12345" to: "queue://my.reply.queue"Scenario: A client sends a request to a microservice. The microservice processes the request and sends a response. The
correlation-idallows the client to identify which request the response belongs to.
3.3 content-type
Aim: Describes the type of the message body (payload), typically using a MIME type.
Best Use:
Payload Interpretation: Crucial for consumers to correctly deserialize and interpret the message content.
Standardization: Use well-known MIME types (e.g.,
application/json,application/xml,text/plain,application/octet-stream).Character Encoding: Often combined with
content-encodingfor text-based payloads.
Example:
content-type: "application/json" content-encoding: "utf-8"Scenario: A service sends a message containing customer data in JSON format. The
content-typetells the receiving service how to parse the message body.
3.4 content-encoding
Aim: Describes the encoding of the message body, typically used for compression or character encoding.
Best Use:
Compression: Indicate if the payload is compressed (e.g.,
gzip,deflate).Character Sets: For text-based content, specify the character encoding (e.g.,
utf-8,iso-8859-1).
Example:
content-type: "application/json" content-encoding: "gzip"Scenario: A large JSON payload is compressed using Gzip before being sent. The
content-encodingproperty informs the receiver to decompress it.
3.5 to
Aim: The address of the intended destination for the message.
Best Use:
Direct Routing: Used by intermediaries to route the message to a specific queue or topic.
Recipient Identification: Helps the recipient identify if the message is intended for them, especially in scenarios where a single consumer might listen to multiple addresses.
Example:
to: "queue://orders.processing"Scenario: An order service sends a message directly to the
orders.processingqueue.
3.6 reply-to
Aim: The address to which a reply message should be sent.
Best Use:
Request-Reply Patterns: Specifies the return address for responses in an asynchronous request-reply interaction.
Dynamic Reply Queues: Often used with temporary or dynamically created queues for replies.
Example:
reply-to: "queue://temp.reply.queue.user123"Scenario: A client sends a request and specifies a temporary queue where it expects the response.
3.7 subject
Aim: A descriptive subject for the message.
Best Use:
Human Readability: Useful for logging, debugging, and monitoring systems to quickly understand the message's purpose without inspecting the payload.
Filtering (Limited): Can sometimes be used for basic filtering, though
application-propertiesare generally preferred for programmatic filtering.
Example:
subject: "New Customer Registration"Scenario: An event indicating a new customer has registered. The subject provides a high-level summary.
3.8 creation-time
Aim: The time at which the message was created.
Best Use:
Auditing: Essential for auditing and compliance, providing a timestamp of when the event occurred.
Ordering (Informal): Can be used as a hint for message ordering, though guaranteed ordering requires specific messaging patterns.
Latency Measurement: Helps in calculating message latency.
Example:
creation-time: 1678886400000 // Unix timestamp in milliseconds (e.g., March 15, 2023 12:00:00 PM UTC)Scenario: A system generates a sensor reading. The
creation-timerecords when the reading was taken.
3.9 absolute-expiry-time
Aim: The time at which the message is considered expired and should be discarded.
Best Use:
Time-Sensitive Messages: For messages that are only relevant for a limited duration (e.g., real-time stock quotes, temporary alerts).
Resource Management: Prevents stale messages from accumulating in queues.
Alternative to TTL: Provides an absolute timestamp for expiry, rather than a relative time-to-live.
Example:
absolute-expiry-time: 1678886460000 // 60 seconds after creation-time exampleScenario: A message containing a one-time password (OTP) that is valid for 60 seconds.
3.10 group-id
Aim: An identifier for a group of messages that are related and should be processed together.
Best Use: