Skip to content

Commit 9177ed1

Browse files
cristiano-pachecocristiano-pacheco
committed
Adds the module documentation
1 parent 48e9985 commit 9177ed1

File tree

7 files changed

+199
-1
lines changed

7 files changed

+199
-1
lines changed

‎LICENCE

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2023 run-as-root
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NON INFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎README.md

+178-1
Original file line numberDiff line numberDiff line change
@@ -1 +1,178 @@
1-
Magento 2 message queue retry
1+
# run-as-root/magento2-message-queue-retry
2+
3+
It gives the possibility to process the same queue message more than once,
4+
utilizing The RabbitMQ's [dead letter exchange](https://www.rabbitmq.com/dlx.html) feature.
5+
6+
## Table of Contents
7+
- [Prerequisites](#prerequisites)
8+
- [Installation](#installation)
9+
- [Features](#features)
10+
- [How it works](#how-it-works)
11+
- [Configuration](#configuration)
12+
- [License](#licence)
13+
14+
## Prerequisites
15+
16+
- Magento 2.4.5 or higher
17+
- PHP 8.1 or higher
18+
- RabbitMQ
19+
20+
To be able to use this module, you have to manually configure the dead letter exchange(s) for the queue(s) you want to enable the retry mechanismm through the `queue_topology.xml` file.
21+
An example will be given in the Setup section.
22+
23+
Other requisite is that your exchanges have to have a relation from one exchange to only one topic and queue,
24+
25+
For example:
26+
![topology](docs/queue-requirement.png)
27+
28+
## Installation
29+
30+
To install the module via composer:
31+
```bash
32+
composer require run-as-root/magento2-message-queue-retry
33+
```
34+
35+
To enable the module:
36+
```bash
37+
bin/magento module:enable RunAsRoot_MessageQueueRetry
38+
```
39+
40+
## Features
41+
42+
- Toggle activation
43+
- Configure the retry limit for a queue
44+
- Admin grid to manage the messages with the retry limit reached
45+
- Requeue the failed messages to their origin queue
46+
- Delete the message
47+
- Download the message body
48+
49+
## How it works
50+
51+
The default Magento's consumer behavior is to reject the message when an exception is thrown during the consumer's execution.
52+
If you use a standard configuration for the queue (without a dead-letter exchange), the message will be discarded and not processed again.
53+
54+
This behavior will change a bit with this module. It will introduce an extra step that will check if the message has reached your retry limit,
55+
if so, it will be discarded from RabbitMQ and sent to the `run_as_root_message` Mysql table and stay there until manual management through the admin panel.
56+
57+
If it has not reached the retry limit, the message will be rejected, and RabbitMQ will be sent to the dead letter exchange and routed automatically to the "delay" queue and stay there until de TTL time is reached.
58+
After the TTL time is reached, the message will be sent to the original queue and processed by the consumer.
59+
60+
The diagram below illustrates both approaches:
61+
![img.png](docs/flow.png)
62+
63+
In the admin panel a new grid will be available to manage the messages that have reached the retry limit:
64+
65+
Path: RunAsRoot > Manage Messages
66+
67+
![img.png](docs/messages-grid.png)
68+
69+
The grid colums:
70+
71+
- **Topic name**: The name of the topic that the message belongs to
72+
- **Total retries**: The total number of times the message has been processed
73+
- **Failure Description**: The exception message that was thrown during the processing of the message
74+
- **Message Body**: The original message body, it can be downloaded as a file and it is in JSON format.
75+
76+
The grid actions:
77+
78+
- **Requeue**: It will send the message to the original queue
79+
- **Download**: It will download the message body content as a JSON file
80+
81+
The grid also has a mass action to delete or requeue the selected messages.
82+
83+
Is possible to configure the ACL for each action in the grid and the module configuration:
84+
85+
![img.png](docs/acl.png)
86+
87+
### Configuration
88+
89+
Two steps are necessary to configure the retry for a queue:
90+
1. Configure the dead letter exchange
91+
1. Enable the message queue retry and delclare the retry limit configuration
92+
93+
Let's imagine a scenario that the `erp_order_export` queue already exists in your project and to simplify the example the topic name, exchange name and queue name are the same: `erp_order_export`.
94+
95+
We need to change these two files in order to declare and configure the delay queue:
96+
- `communication.xml`
97+
- `queue_topology.xml`
98+
99+
**The current queue configuration are like this**:
100+
101+
`etc/communication.xml`:
102+
103+
```xml
104+
<?xml version="1.0"?>
105+
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
106+
xsi:noNamespaceSchemaLocation="urn:magento:framework:Communication/etc/communication.xsd">
107+
<topic name="erp_order_export" request="string"/>
108+
</config>
109+
```
110+
111+
`etc/queue_topology.xml`:
112+
113+
```xml
114+
<?xml version="1.0"?>
115+
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
116+
xsi:noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/topology.xsd">
117+
<exchange name="erp_order_export" connection="amqp" type="topic">
118+
<binding id="erp_order_export" topic="erp_order_export" destinationType="queue" destination="erp_order_export"/>
119+
</exchange>
120+
</config>
121+
```
122+
123+
**You have to change it to**:
124+
125+
`etc/communication.xml`:
126+
127+
```xml
128+
<?xml version="1.0"?>
129+
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
130+
xsi:noNamespaceSchemaLocation="urn:magento:framework:Communication/etc/communication.xsd">
131+
<topic name="erp_order_export" request="string"/>
132+
<topic name="erp_order_export_delay" request="string"/>
133+
</config>
134+
```
135+
136+
`etc/queue_topology.xml`:
137+
```xml
138+
<?xml version="1.0"?>
139+
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
140+
xsi:noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/topology.xsd">
141+
<exchange name="erp_order_export" connection="amqp" type="topic">
142+
<binding id="erp_order_export" topic="erp_order_export" destinationType="queue" destination="erp_order_export">
143+
<arguments>
144+
<argument name="x-dead-letter-exchange" xsi:type="string">erp_order_export_delay</argument>
145+
<argument name="x-dead-letter-routing-key" xsi:type="string">erp_order_export_delay</argument>
146+
</arguments>
147+
</binding>
148+
</exchange>
149+
<exchange name="erp_order_export_delay" connection="amqp" type="topic">
150+
<binding id="erp_order_export_delay" topic="erp_order_export_delay" destinationType="queue" destination="erp_order_export_delay">
151+
<arguments>
152+
<argument name="x-dead-letter-exchange" xsi:type="string">erp_order_export</argument>
153+
<argument name="x-dead-letter-routing-key" xsi:type="string">erp_order_export</argument>
154+
<argument name="x-message-ttl" xsi:type="number">300000</argument>
155+
</arguments>
156+
</binding>
157+
</exchange>
158+
</config>
159+
```
160+
161+
In the `erp_order_export` we added the `x-dead-letter-exchange` and `x-dead-letter-routing-key` arguments to the binding, this will route the message to the `erp_order_export_delay` exchange when the message is rejected.
162+
163+
In the `erp_order_export_delay` we added the same arguments, however it points to the original exchange `erp_order_export` and the TTL time is 300000ms (5 minutes).
164+
165+
The `erp_order_export_delay` queue does not have a consumer, it will be used only to hold(delay) messages according with the period defined in the `x-message-ttl` argument. When the lifetime expires (TTL), RabbitMQ will send the message to `erp_order_export` automatically.
166+
167+
Now you have to define toggle the activation for the retry queue module and declare the retry limit for the queue:
168+
169+
System > Configuration > RUN-AS-ROOT > Message Queue Retry
170+
171+
![img.png](docs/configuration.png)
172+
173+
Note that if the queue is not declared in the configuration it will the default Magento consumer behavior.
174+
175+
For more information of how to configure message queues in Magento 2, you can take a look [here](https://developer.adobe.com/commerce/php/development/components/message-queues/configuration/).
176+
177+
## License
178+
[MIT](https://opensource.org/licenses/MIT)

‎docs/acl.png

22.3 KB
Loading

‎docs/configuration.png

117 KB
Loading

‎docs/flow.png

57.1 KB
Loading

‎docs/messages-grid.png

72.7 KB
Loading

‎docs/img.png renamed to ‎docs/queue-requirement.png

File renamed without changes.

0 commit comments

Comments
 (0)