Message Types
The following table describes types of messages in a message queue classified based on the features and use cases.
|
| | | High throughput, and no requirement for the production and consumption sequence |
Locally sequential message | All messages in a partition follow the First-In-First-Out (FIFO) rule. | | High throughput, and sequential messages within the same partition, but non-sequential messages across different partitions |
Globally sequential message | All messages under the same topic follow the FIFO rule. | | Moderate throughput, globally sequential, and single partition |
| | | Messages that cannot be consumed normally |
| Messages are consumed at a delayed time point after being sent to the server. | Deviation of around 1 second | Messages that are consumed at a specified time point |
| Messages are consumed after a delayed period of time after being sent to the server. | Deviation of around 1 second | Messages that are consumed after a specified time interval |
General Messages
General messages are a basic type of message. After the producer delivers them to a specific topic, they are consumed by consumers who subscribe to the topic. In a topic with general messages, there is no concept of sequence. Multiple partitions can be used to improve the message production and consumption efficiency, ensuring the best performance upon high throughput.
Locally Sequential Messages
Local sequential messages are locally sequential when compared with general messages. That is, consumers should strictly consume messages in a partition based on the sequence they are delivered by the producer to the partition. Locally sequential messages ensure a certain sequentiality while retaining the partition mechanism to improve performance. However, locally sequential messages cannot guarantee the sequence between different partitions.
Globally Sequential Messages
Globally sequential messages strictly ensure that messages are consumed in the sequence they are delivered by the producer. However, a single partition is used to handle messages, and users cannot customize the number of partitions. Compared with the previous two message types, this type of message has poorer performance.
Dead Letter Messages
Dead letter messages are messages that cannot be consumed normally. TDMQ for Apache Pulsar automatically creates a dead letter queue when a subscription is created (when consumers confirm their subscription relationships with a topic) to handle such messages.
Scheduled Messages
After a message is sent to the server, the business does not want the consumer to receive this message immediately, but rather at a later specified time. Messages of this type are collectively called scheduled messages. Scheduled messages can be considered as a special usage of delayed messages, and their ultimate implementation effects are the same.
Delayed Messages
After a message is sent to the server, the business does not want the consumer to receive this message immediately, but rather after a specific period of time. Messages of this type are collectively called delayed messages.
Retry Queues
A retry queue is a queue designed to ensure that messages are consumed normally. If no normal response is received after a message is consumed by a consumer for the first time, it enters the retry queue. When the number of retries reaches a threshold, no retry is performed, and the message is delivered to the dead letter queue.
In actual scenarios, messages may not be processed promptly due to temporary issues that will recover after a short period of time, such as network jitter and service restarts. In such scenarios, the retry mechanism of the retry queue can be a good solution.
Dead Letter Queues
A dead letter queue is a special message queue designed for centralized processing of messages that cannot be consumed normally. If a message cannot be consumed after a specified number of retries, TDMQ for Apache Pulsar determines that the message cannot be consumed under the current situation and delivers it to the dead letter queue.
In actual scenarios, messages may fail to be consumed due to prolonged service downtime or network disconnections. In such cases, messages are not discarded immediately. The dead letter queue persists such messages for a long period. After users obtain a solution, they can create consumers and subscribe to the dead letter queue.