Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/docker-compose.cluster.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
x-rabbit-common: &rabbit-common
image: ${RABBITMQ_IMAGE:-rabbitmq:4.3.1-management-alpine}
image: ${RABBITMQ_IMAGE:-rabbitmq:4.4.0-management-alpine}
environment: &rabbit-env
RABBITMQ_DEFAULT_USER: guest
RABBITMQ_DEFAULT_PASS: guest
Expand Down
2 changes: 1 addition & 1 deletion .github/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
services:
rabbitmq:
image: rabbitmq:4.3.1-management-alpine
image: rabbitmq:4.4.0-management-alpine
container_name: django-rmq-test
ports:
- "5672:5672" # AMQP — RMQ_PORT
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/testing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ jobs:
strategy:
matrix:
python-version: [ "3.10", "3.11", "3.12", "3.13", "3.14" ]
rabbitmq-version: [ "3.13", "4.0", "4.1", "4.2", "4.3" ]
rabbitmq-version: [ "3.13", "4.0", "4.1", "4.2", "4.3", "4.4" ]
fail-fast: true

services:
Expand Down Expand Up @@ -161,7 +161,7 @@ jobs:
fail-fast: true

env:
RABBITMQ_IMAGE: rabbitmq:4.3-management-alpine
RABBITMQ_IMAGE: rabbitmq:4.4-management-alpine
RMQ_HOST: localhost
RMQ_PORT: 5672
RMQ_MGMT_PORT: 15672
Expand Down
3 changes: 0 additions & 3 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,6 @@ ci:
autoupdate_schedule: "quarterly"
skip: [pyrefly]

default_language_version:
python: python3.10

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v6.0.0
Expand Down
42 changes: 39 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,46 @@ All notable changes to **Django-RMQ** are documented in this file.

## [Unreleased]

## [1.1.6] — 2026-11-04

### Added

- **Connection timeouts** — optional `SOCKET_TIMEOUT` and `STACK_TIMEOUT` keys in `RABBITMQ_CONNECTIONS` (defaults
match pika: `10` / `15` seconds).
- **`RABBITMQ_HEALTHCHECK` setting** — optional `DEADLINE`, `SOCKET_TIMEOUT` and `STACK_TIMEOUT` for
`check_rabbitmq_connections` (defaults `15` / `5` / `10` seconds).
- **`check_rabbitmq_connections --timeout`** — overrides `DEADLINE` for a single run.
- **`check_rabbitmq_connections --format {text,json}`** — output format, `text` by default.
- `HealthcheckConfig` and `RabbitMQConnectionManager.build_probe_parameters()`.
- **RabbitMQ 4.4 support** — supported range is now 3.13–4.4.

### Changed

- Documentation site migrated from VuePress 2 (`vuepress-theme-hope`) to Docusaurus 3 and lives entirely under `docs/`.
Content, English/Russian locales, and versions 1.0.5 / 1.0.4 stay in place.
- **`check_rabbitmq_connections`** checks all nodes and aliases in parallel; an alias is healthy as soon as any of its
nodes answers. An unresponsive node no longer causes intermittent probe timeouts on a healthy cluster.
- **Breaking (output only):** `check_rabbitmq_connections` output reworked — `startup`, per-alias `check` and
`summary` lines, no color. The old `OK: <alias>` lines are gone. Exit codes are unchanged.
- `check_rabbitmq_connections` no longer runs Django system checks; `--skip-checks` is still accepted.
- New timeout settings are validated at startup and raise `ImproperlyConfigured` on invalid values.

### Documentation

- Documentation site migrated from VuePress 2 to Docusaurus 3 (`docs/`).
- Management commands, Configuration, Clusters guides and API reference (EN/RU) updated for the healthcheck and the
new timeout settings.
- Current docs are labeled 1.1.6; a 1.0.5 snapshot (EN/RU) was added.

### Tests

- Unit tests for the new healthcheck output, parallel probing, deadline and timeout settings; an integration test for
an unanswering first node.
- RabbitMQ 4.4 added to the integration-test CI matrix; the cluster failover tests and the local
`.github/docker-compose.yml` / `.github/docker-compose.cluster.yml` now run on 4.4.
- New integration test `test_passive_declare_with_write_only_permission` (`tests/integration/test_mandatory.py`).

[1.1.6]: https://github.com/RDDLab/Django-RMQ/releases/tag/1.1.6

## [1.0.5] — 27.07.2026
## [1.0.5] — 2026-07-27

### Added

Expand Down Expand Up @@ -54,6 +88,8 @@ All notable changes to **Django-RMQ** are documented in this file.
"Version" dropdown in the navbar switches between them. The 1.0.4 sidebar omits the Clusters guide, which did not
exist in that release.

[1.0.5]: https://github.com/RDDLab/Django-RMQ/releases/tag/1.0.5

## [1.0.4] — 2026-07-11

### Fixed
Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
[![PyPI - Types](https://img.shields.io/pypi/types/django-rmq.svg?style=for-the-badge)](https://pypi.python.org/pypi/django-rmq)
---

[![RabbitMQ Support](https://img.shields.io/static/v1?label=RabbitMQ%20Support&message=v3.13%20%7C%20v4.0%20%7C%20v4.1%20%7C%20v4.2%20%7C%20v4.3&color=ff6600&labelColor=555&style=for-the-badge)](https://www.rabbitmq.com/)
[![RabbitMQ Support](https://img.shields.io/static/v1?label=RabbitMQ%20Support&message=v3.13%20%7C%20v4.0%20%7C%20v4.1%20%7C%20v4.2%20%7C%20v4.3%20%7C%20v4.4&color=ff6600&labelColor=555&style=for-the-badge)](https://www.rabbitmq.com/)

---
[![Tests](https://img.shields.io/github/actions/workflow/status/RDDLab/Django-RMQ/testing.yml?branch=main&label=Tests)](https://github.com/RDDLab/Django-RMQ/actions/workflows/testing.yml)
Expand All @@ -31,7 +31,7 @@

# Django-RMQ

Django-RMQ provides RabbitMQ wrappers and tools for Django projects, built on top of [Pika](https://pika.readthedocs.io/). It is a lightweight integration layer — not a task queue or a Celery replacement — for projects that need to publish and consume messages while keeping broker infrastructure code predictable and close to the Django configuration. Supports RabbitMQ 3.13–4.3.
Django-RMQ provides RabbitMQ wrappers and tools for Django projects, built on top of [Pika](https://pika.readthedocs.io/). It is a lightweight integration layer — not a task queue or a Celery replacement — for projects that need to publish and consume messages while keeping broker infrastructure code predictable and close to the Django configuration. Supports RabbitMQ 3.13–4.4.

## Features

Expand All @@ -41,9 +41,9 @@ Django-RMQ provides RabbitMQ wrappers and tools for Django projects, built on to
- **Reliable delivery** — publisher confirms, `mandatory=True`, and `delivery_mode=2` (persistent) are enforced on every message.
- **Auto-reconnect with backoff** — producers retry once on transient channel errors; consumers reconnect with exponential backoff capped at a configurable maximum.
- **Dead-letter routing** — declare queues with `QueueConfig(dead_letter_exchange=...)` so unhandled messages are nacked without requeue and routed to a DLX.
- **Management commands** — `setup_rabbitmq_topology` (idempotent exchange/queue/binding setup) and `start_consumers` (threaded runner with graceful SIGTERM/SIGINT shutdown).
- **Management commands** — `setup_rabbitmq_topology` (idempotent exchange/queue/binding setup), `start_consumers` (threaded runner with graceful SIGTERM/SIGINT shutdown), and `check_rabbitmq_connections` (healthcheck for container probes).
- **Multiple connections** — configure several broker aliases and select per producer/consumer via `using=`.
- **RabbitMQ clusters** — point one alias at several nodes via NODES for client-side failover; pika tries each node until one connects. Optional SHUFFLE_NODES spreads clients across the cluster.
- **RabbitMQ clusters** — point one alias at several nodes via NODES for client-side failover; pika tries each node until one connects. Optional SHUFFLE_NODES spreads clients across the cluster; per-alias `SOCKET_TIMEOUT`/`STACK_TIMEOUT` give faster failover.
- **Fully typed** — ships `py.typed`; compatible with pyrefly and standard type checkers.

## Installation
Expand Down
2 changes: 2 additions & 0 deletions django_rmq/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@

if TYPE_CHECKING:
from django_rmq.connections import RabbitMQConnectionManager
from django_rmq.dto.healthcheck_config import HealthcheckConfig
from django_rmq.registries.registry import ConsumersRegistry
from django_rmq.registries.setup_registry import SetupRegistry

connection_managers: dict[str, 'RabbitMQConnectionManager'] | None = None
setup_registries: dict[str, 'SetupRegistry'] | None = None
consumers_registries: dict[str, 'ConsumersRegistry'] | None = None
healthcheck_config: 'HealthcheckConfig | None' = None
72 changes: 71 additions & 1 deletion django_rmq/apps.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,13 @@
from django.apps import AppConfig
from django.core.exceptions import ImproperlyConfigured

from django_rmq.dto.healthcheck_config import HealthcheckConfig
from django_rmq.dto.rabbitmq_config import NodeConfig, RabbitMQConfig

_DEFAULT_SOCKET_TIMEOUT: float = 10.0
_DEFAULT_STACK_TIMEOUT: float = 15.0
_HEALTHCHECK_KEYS: frozenset[str] = frozenset({'DEADLINE', 'SOCKET_TIMEOUT', 'STACK_TIMEOUT'})


class RabbitMQAppConfig(AppConfig):
"""
Expand All @@ -25,7 +30,8 @@ def ready(self) -> None:
per configured alias and stores them on the `django_rmq` module so the
rest of the library can resolve them by alias.

:raises ImproperlyConfigured: If `RABBITMQ_CONNECTIONS` is missing or empty.
:raises ImproperlyConfigured: If `RABBITMQ_CONNECTIONS` is missing or empty,
or any setting value is invalid.
"""
from django.conf import settings

Expand Down Expand Up @@ -55,6 +61,9 @@ def ready(self) -> None:
django_rmq.connection_managers = connection_managers
django_rmq.setup_registries = setup_registries
django_rmq.consumers_registries = consumers_registries
django_rmq.healthcheck_config = self._build_healthcheck_config(
params=getattr(settings, 'RABBITMQ_HEALTHCHECK', None),
)

@classmethod
def _build_config(cls, alias: str, params: dict[str, Any]) -> RabbitMQConfig:
Expand Down Expand Up @@ -82,8 +91,69 @@ def _build_config(cls, alias: str, params: dict[str, Any]) -> RabbitMQConfig:
reconnect_initial_backoff=params['RECONNECT_INITIAL_BACKOFF'],
reconnect_max_backoff=params['RECONNECT_MAX_BACKOFF'],
shuffle_nodes=params.get('SHUFFLE_NODES', False),
socket_timeout=cls._resolve_timeout(
owner=f'RabbitMQ alias {alias!r}',
key='SOCKET_TIMEOUT',
value=params.get('SOCKET_TIMEOUT', _DEFAULT_SOCKET_TIMEOUT),
),
stack_timeout=cls._resolve_timeout(
owner=f'RabbitMQ alias {alias!r}',
key='STACK_TIMEOUT',
value=params.get('STACK_TIMEOUT', _DEFAULT_STACK_TIMEOUT),
),
)

@classmethod
def _build_healthcheck_config(cls, params: Any) -> HealthcheckConfig:
"""
Builds the healthcheck configuration from the `RABBITMQ_HEALTHCHECK` setting.

Every key is optional; missing keys keep the HealthcheckConfig defaults.

:param params: The `RABBITMQ_HEALTHCHECK` dict, or None when it is not set.
:return: The resolved, immutable HealthcheckConfig.
:raises ImproperlyConfigured: If the setting is not a dict, has unknown
keys, or holds a non-positive number.
"""
if params is None:
return HealthcheckConfig()
if not isinstance(params, dict):
raise ImproperlyConfigured('`RABBITMQ_HEALTHCHECK` must be a dict.')

unknown: list[str] = sorted(set(params) - _HEALTHCHECK_KEYS)
if unknown:
raise ImproperlyConfigured(
f'`RABBITMQ_HEALTHCHECK`: unknown key(s) {", ".join(unknown)}; '
f'allowed: {", ".join(sorted(_HEALTHCHECK_KEYS))}.'
)

defaults: HealthcheckConfig = HealthcheckConfig()
owner: str = '`RABBITMQ_HEALTHCHECK`'
return HealthcheckConfig(
deadline=cls._resolve_timeout(owner=owner, key='DEADLINE', value=params.get('DEADLINE', defaults.deadline)),
socket_timeout=cls._resolve_timeout(
owner=owner, key='SOCKET_TIMEOUT', value=params.get('SOCKET_TIMEOUT', defaults.socket_timeout)
),
stack_timeout=cls._resolve_timeout(
owner=owner, key='STACK_TIMEOUT', value=params.get('STACK_TIMEOUT', defaults.stack_timeout)
),
)

@classmethod
def _resolve_timeout(cls, owner: str, key: str, value: Any) -> float:
"""
Validates a timeout setting and converts it to float.

:param owner: Human-readable owner of the setting, used in the error message.
:param key: Setting key, used in the error message.
:param value: The raw setting value.
:return: The timeout in seconds.
:raises ImproperlyConfigured: If the value is not a positive int or float.
"""
if isinstance(value, bool) or not isinstance(value, (int, float)) or value <= 0:
raise ImproperlyConfigured(f"{owner}: '{key}' must be a positive number of seconds, got {value!r}.")
return float(value)

@classmethod
def _resolve_nodes(cls, alias: str, params: dict[str, Any]) -> tuple[NodeConfig, ...]:
"""
Expand Down
57 changes: 48 additions & 9 deletions django_rmq/connections.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
from pika.exceptions import AMQPError

import django_rmq
from django_rmq.dto.rabbitmq_config import RabbitMQConfig
from django_rmq.dto.rabbitmq_config import NodeConfig, RabbitMQConfig
from django_rmq.utils import resolve_alias

logger = logging.getLogger('rabbitmq')
Expand Down Expand Up @@ -39,24 +39,63 @@ class RabbitMQConnectionManager:

def __init__(self, config: RabbitMQConfig) -> None:
self.config: RabbitMQConfig = config
credentials: PlainCredentials = PlainCredentials(
self._credentials: PlainCredentials = PlainCredentials(
username=config.user,
password=config.password,
)
self._node_parameters: list[ConnectionParameters] = [
ConnectionParameters(
host=node.host,
port=node.port,
virtual_host=config.virtual_host,
credentials=credentials,
heartbeat=config.heartbeat,
blocked_connection_timeout=config.blocked_connection_timeout,
self._make_node_parameters(
node=node,
socket_timeout=config.socket_timeout,
stack_timeout=config.stack_timeout,
)
for node in config.nodes
]
self._shuffle_nodes: bool = config.shuffle_nodes
self._local: threading.local = threading.local()

def _make_node_parameters(
self,
node: NodeConfig,
socket_timeout: float,
stack_timeout: float,
) -> ConnectionParameters:
"""
Builds pika connection parameters for a single node of this alias.

:param node: The node address.
:param socket_timeout: Seconds to wait for the TCP connection to the node.
:param stack_timeout: Seconds to wait for the full TCP + AMQP bring-up.
:return: ConnectionParameters sharing the alias credentials and settings.
"""
return ConnectionParameters(
host=node.host,
port=node.port,
virtual_host=self.config.virtual_host,
credentials=self._credentials,
heartbeat=self.config.heartbeat,
blocked_connection_timeout=self.config.blocked_connection_timeout,
socket_timeout=socket_timeout,
stack_timeout=stack_timeout,
)

def build_probe_parameters(self, socket_timeout: float, stack_timeout: float) -> list[ConnectionParameters]:
"""
Returns one set of connection parameters per node, for probing nodes independently.

Used by the healthcheck, which connects to every node in parallel with its
own timeouts. The order follows `NODES` and is never shuffled; nothing is
cached on the manager.

:param socket_timeout: Seconds to wait for the TCP connection to a node.
:param stack_timeout: Seconds to wait for the full TCP + AMQP bring-up of a node.
:return: Per-node ConnectionParameters, in configuration order.
"""
return [
self._make_node_parameters(node=node, socket_timeout=socket_timeout, stack_timeout=stack_timeout)
for node in self.config.nodes
]

def _build_connection_sequence(self) -> list[ConnectionParameters]:
"""
Returns the node parameters to hand to a new BlockingConnection.
Expand Down
23 changes: 23 additions & 0 deletions django_rmq/dto/healthcheck_config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
from dataclasses import dataclass


@dataclass(frozen=True)
class HealthcheckConfig:
"""
Immutable configuration of the `check_rabbitmq_connections` healthcheck.

Built from the optional `RABBITMQ_HEALTHCHECK` setting. The probe uses its
own, shorter per-node timeouts instead of the alias ones, so a slow or dead
node cannot push the probe past the orchestrator's timeout.

:param deadline: Overall budget in seconds for checking every alias. When
it expires, aliases without a live node are reported as
failed and the command exits.
:param socket_timeout: Seconds to wait for the TCP connection to a single node.
:param stack_timeout: Seconds to wait for the full TCP + AMQP bring-up of a
single node.
"""

deadline: float = 15.0
socket_timeout: float = 5.0
stack_timeout: float = 10.0
8 changes: 8 additions & 0 deletions django_rmq/dto/rabbitmq_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,12 @@ class RabbitMQConfig:
:param shuffle_nodes: When True, the node order is reshuffled on every
connection attempt so clients spread themselves across
the cluster instead of all preferring the first node.
:param socket_timeout: Seconds to wait for the TCP connection to a single node
(pika `socket_timeout`). Nodes are tried one after
another, so each unreachable node before a live one
adds up to this much to a connection attempt.
:param stack_timeout: Seconds to wait for the full TCP + AMQP bring-up of a
single node (pika `stack_timeout`).
"""

nodes: tuple[NodeConfig, ...]
Expand All @@ -52,3 +58,5 @@ class RabbitMQConfig:
reconnect_initial_backoff: float
reconnect_max_backoff: float
shuffle_nodes: bool = False
socket_timeout: float = 10.0
stack_timeout: float = 15.0
Loading
Loading