RabbitMQ

Page last updated:

Offered Protocols and Plugins

RabbitMQ provides the following protocols which are all exposed in VCAP_SERVICES:

• AMQP
• MQTT
• STOMP
• HTTP (API and management interface)

The application should only use the uri or hostname properties of the desired protocol within VCAP_SERVICES to achieve proper loadbalancing. These properties contain rabbitmq.service.consul so the connection is established to a HAProxy that automatically distributes requests to all available RabbitMQ nodes.

The alternative properties uris and hostnames should only be used if single nodes need to be targeted explicitly. This can be used to achieve slightly better performance by connecting publishers and subscribers to the queue leader node. However, this comes with a serious disadvantage, since the connections fail in case the single node goes down.

The complete list of enabled plugins:

• rabbitmq_consistent_hash_exchange
• rabbitmq_event_exchange
• rabbitmq_management
• rabbitmq_management_agent
• rabbitmq_mqtt
• rabbitmq_prometheus
• rabbitmq_shovel
• rabbitmq_shovel_management
• rabbitmq_stomp
• rabbitmq_web_dispatch

The complete list of enabled feature flags:

• classic_mirrored_queue_version
• classic_queue_type_delivery_support
• delete_ra_cluster_mqtt_node
• detailed_queues_endpoint
• direct_exchange_routing_v2
• drop_unroutable_metric
• empty_basic_get_metric
• feature_flags_v2
• implicit_default_bindings
• listener_records_in_ets
• maintenance_mode_status
• message_containers
• message_containers_deaths_v2
• mqtt_v5
• quorum_queue
• quorum_queue_non_voters
• rabbit_exchange_type_local_random
• rabbit_mqtt_qos0_queue
• rabbitmq_4.0.0
• rabbitmq_4.1.0
• restart_streams
• stream_filtering
• stream_queue
• stream_sac_coordinator_unblock_group
• stream_single_active_consumer
• stream_update_config_command
• tracking_records_in_ets
• user_limits
• virtual_host_metadata

Integrating the Service With Your App

After the creation of the service and the binding of the service to the application, the environment variable VCAP_SERVICES is created.

Managing Your RabbitMQ Instances

To get access to the RabbitMQ management console, you can use the cf ssh command. Here’s how it works:

1. You’ll need to use an app as a host for SSH-ing. Any running app will do. If you don’t have one yet, push a default app through the web console
2. Create a service key for your RabbitMQ instance with cf create-service-key <service-name> manage (replace the term in <> with your service’s name)
3. Look at the service key’s credentials with cf service-key <service-name> manage
4. SSH into your app with cf ssh -L 13000:<credentials.hostname>:<credentials.management_port> <app-name> (replace the values in <> with the actual values from the service key and with your app’s name)
5. Open your browser and visit http://localhost:13000 and use <credentials.username> and <credentials.password> from your service key as the credentials
6. When you’re done, simply close the terminal window with the open SSH connection

Standard Configuration

Default Queue Policy

Since July 2024, Quorum Queues are the default queue type for newly created service instances. Older service instances still default to classic queues. Classic mirrored queues have been removed in RabbitMQ 4.0 and are no longer available.

Every service instance is provisioned with a default-policy user policy:

When a queue is full, new messages are rejected rather than silently dropped. Publishers using publisher confirms will receive a nack and can react accordingly.

The service instance owner is responsible for any degradation of the service or any data loss if the default-policy user policy is modified or removed!

Limitations

The RabbitMQ service enforces some limitations to ensure the cluster’s performance:

Per-Queue Message TTL of 14 Days

To protect the cluster’s disk from filling up with old messages that were never picked up, the per-queue message TTL is set to 14 days. Messages that are in the queue for longer than 14 days are automatically discarded!

Operator Policies

The RabbitMQ cluster implements operator policies to set certain limits on queues and messages. The following operator policies are enforced on all queues:

• max-length: 100'000
• max-length-bytes: 100MB

When either limit is reached, the overflow behavior of the queue determines what happens. With the default user policy (overflow: reject-publish), new messages are rejected. Only if a user explicitly sets overflow: drop-head in a custom policy will the oldest messages be silently discarded instead.

Connection Limits

To avoid high load caused by too many open connections, a limit on service instance level is implemented. A single service instance can hold a maximum of 500 connections. Once this limit is reached, all further connections will be terminated.

Advanced Configuration

In most cases, connecting your applications to the load-balanced service is completely sufficient. For some high-end setups, getting the last drop of performance and scalability from the cluster is an advanced task and might include strategies that need more logic on the client side:

Classic Queues

If your service instance was created after July 2024, queues are created as quorum queues by default. If high availability is not required and lower overhead is preferred, create the queue with type classic instead. Classic queues in RabbitMQ 4.x are not replicated and exist on a single node only.

Quorum Queues

Quorum queues were introduced in RabbitMQ 3.8 version. This type of queue implements a replicated FIFO queue based on Raft consensus algorithm. It came to replace classic mirrored queues as a more reliable solution for high availability with still good throughput.

To use quorum queues, just create the queue of type quorum.

Consistent Hash Exchange

For a different approach to scaling your queues on the cluster, the Consistent hash exchange plugin is enabled.

Best Practices

 Queues

Keep Queues Short

RabbitMQ queues work fast and reliably when they’re empty / have only a few messages queued up. RabbitMQ is not designed to hold messages over a long period of time and it should always be the goal to keep the queues empty. Not only does this minimise the risk of message loss, it also increases the overall stability of the cluster and its performance.

Lazy Queues (Deprecated)

The lazy queue mode is deprecated in RabbitMQ 4.x. Classic queues now always store messages on disk by default, and quorum queues have always operated this way. There is no need to declare queues as lazy anymore.

Auto-Delete Queues

Queues will remain forever on the cluster by default (even when they’re not active). This will put unnecessary strain on the cluster. To free up resources and don’t run into limits, queues should be declared with the auto-delete flag set to true. With this, the queues will be deleted as soon as there are no more active connections using them.

 Connections

General

Connections in RabbitMQ use up a lot of memory. In order to keep the cluster healthy and fast, it is important that there are as few open connections as possible. This can be achieved by keeping connections open for client applications. A good practice is to keep the connection open throughout the lifetime of the client application.

Auto reconnect

Connections between client applications and the RabbitMQ cluster are stable for the most part. However, it’s possible that a network outage can occur or the cluster has an unexpected downtime. In such a case, the client application usually needs to reconnect to the cluster and reopen connections and channels. To automate this process, most RabbitMQ Client APIs support the auto-reconnect feature (see the Java Client API for an example). It is advised to enable the auto-reconnect feature on the client applications.

Channels

General

Channels share the same restrictions as connections - they can use up quite a lot of memory if opened and closed frequently. Although less memory is used when creating a channel, it’s still recommended that channels are kept open for a long time.

Threads

Channels are not thread-safe and should not be shared between different threads.

Prefetch

RabbitMQ allows control over how many messages are sent to the consumer at the same time. Per default, the prefetch value is set to unlimited, meaning the server tries to send as many messages as possible at the same time. Setting a proper prefetch value can help to achieve the desired performance of the service. If the prefetch count is set too low, the server will idle and wait for the client to send more messages. On the other hand, if the prefetch count is too high the client might take a long time to consume the messages.

Setting the correct prefetch value depends on the setup of the client application. As such, there is not one prefetch value that is best for all cases. As a general rule, the prefetch count should be higher if there is only one / few consumers that consume messages quickly. If the client application features many consumers, a lower prefetch value should be set. More information can be found here: https://www.rabbitmq.com/docs/4.1/consumer-prefetch.

Messages

Message Size

Large messages can take up a lot of resources and should generally be avoided. RabbitMQ is designed for high message throughput and as such it is better to keep the messages sent over RabbitMQ small. Splitting up large payloads into smaller messages should be considered when designing the client application.

Message Ack’s and Confirms

RabbitMQ features acknowledgments (ack’s) for receiving messages and publisher confirms (confirms) for sending messages. These mechanisms can be used to ensure that a message has been sent/received correctly. Ack’s and confirms are generally a good idea to implement as they increase the reliability and consistency of the message flow.

Note: Ack’s and confirms have a small performance impact.

Backups and Restore

RabbitMQ contains two types of data: definitions (metadata, schema/topology) and messages. Our backup system takes care of saving only the definitions, but not the messages. The reason for this is that messages are short-lived, transient in nature, and RabbitMQ is not designed as a data storage solution. The automated backups for the definitions are stored in an encrypted, environment-specific S3 bucket for 30 days. This is done by the backman component every 12 hours.

The restoration process is manual and requires intervention from the RabbitMQ operations team; therefore it will only be performed for disaster recovery cases. All nodes in the cluster must be restored together and all customers will be affected, which means that it is not possible to restore a backup for a single cluster node or a single customer.

Migration from Classic to Quorum Queues

Since RabbitMQ 4.0, classic mirrored queues have been removed. Only classic (non-replicated) and quorum queues can be created. If your service instance still has classic queues, you can migrate them to quorum queues to benefit from replication and high availability.

Why Migrate

Quorum queues are replicated across multiple nodes using the Raft consensus algorithm, providing high availability and data safety. Classic queues exist on a single node only and are not replicated.

Reasons to migrate from classic to quorum queues:

• Data safety: messages are replicated across multiple nodes
• High availability: the queue remains available if a node goes down
• Better performance for most workloads

How to Migrate

Directly changing a queue’s type is not possible. Instead, there are two options:

Option 1: Create a new quorum queue and move the publisher, then the consumer.

• Declare a Quorum Queue: Set the x-queue-type queue argument to quorum.
• Move the publisher and drain the old queue: Point the publisher to the new queue and let the consumer finish consuming from the old queue until it’s empty.
• Move the consumer: Switch the consumer to the new queue.

Option 2: Move messages using a shovel.

• Declare the new quorum queue: Set the x-queue-type queue argument to quorum.
• Set up a shovel: In the RabbitMQ management interface, go to the Admin tab and select Shovel Management.
• Move messages: Configure the shovel to move messages from the old classic queue to the new quorum queue.

Compatibility Considerations

Not all classic queue features are supported by quorum queues. Key differences to be aware of:

Non-durable queues

Quorum queues are always durable. Non-durable classic queues cannot be migrated to quorum queues. Non-durable queues are being phased out in future RabbitMQ releases — use exclusive queues for ephemeral use cases instead.

Exclusive queues

Exclusive queues cannot be declared as quorum queues. They must remain classic. This is expected — exclusive queues are tied to a single connection and don’t need replication.

Overflow mode reject-publish-dlx

The overflow mode reject-publish-dlx is not supported by quorum queues. Standard dead letter exchanges work fine — only this specific overflow mode is unavailable.

Consumer exclusivity

Consumer exclusivity is not supported by quorum queues. Use Single Active Consumer instead to achieve similar behavior.

Global QoS for consumers

Global QoS is not supported for quorum queues. Use per-consumer QoS with an appropriate prefetch value instead.

For a full comparison, see the feature comparison in the RabbitMQ documentation.