feat!: replace Pyro4/ZMQ transport with gRPC

Replace the three-port Pyro4+ZMQ architecture with a single-port gRPC
service (default port 5556):

- Add mtda/grpc/ package: mtda.proto defines all RPCs including a
  server-streaming Subscribe RPC (replaces ZMQ PUB/SUB console/event
  stream) and a client-streaming StorageWrite RPC (replaces ZMQ
  PUSH/PULL storage data path).
- Add QueueDataStream in mtda/storage/datastream.py; remove dead
  NetworkDataStream and its zmq dependency.
- Rewrite mtda/console/remote.py: RemoteConsole and RemoteMonitor now
  subscribe via gRPC instead of ZMQ SUB sockets; topic bytes
  normalisation ensures correct dispatch.
- Fix ConsoleLogger._print(): always call publish() regardless of
  socket state so console data reaches subscriber queues.
- Rewrite Client in mtda/client.py with _GrpcImpl and storage socket
  adapters; extract session-name generation into a static method
  (ASCII-only word list prevents invalid gRPC metadata); remove zmq.
- Fix mtda-www WebConsole: normalise topic bytes, add reconnect loop,
  create reader tasks after mtda_start().
- Remove python3-zmq from mtda-service in debian/control; grpcio
  already present in mtda-common covers all packages transitively.
- Update docs: remove obsolete console/data port entries from
  [remote] config docs; correct default www port (9080 -> 5000).

BREAKING CHANGE: the console (5557) and data (5558) ports no longer
exist; all traffic flows over the single control port (default 5556).

Closes: #524
Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>

Author: chombourger

.github/wordlist.txt

@@ -45,6 +45,7 @@ gnutls
 googlegroups
 gpg
 GPIO
+gRPC
 GStreamer
 hombourger
 homekit

.github/workflows/proto.yml

@@ -0,0 +1,36 @@
+# ---------------------------------------------------------------------------
+# Verify mtda.proto is valid and the build step can generate stubs from it
+# ---------------------------------------------------------------------------
+#
+# This software is a part of MTDA.
+# Copyright (C) 2026 Siemens AG
+#
+# ---------------------------------------------------------------------------
+# SPDX-License-Identifier: MIT
+# ---------------------------------------------------------------------------
+
+name: Proto stubs
+
+on:
+  pull_request:
+    paths:
+      - 'mtda/grpc/mtda.proto'
+
+jobs:
+  check-stubs:
+    name: Verify stubs can be generated from mtda.proto
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Install grpcio-tools
+        run: pip3 install grpcio-tools
+
+      - name: Generate stubs
+        run: |
+          python3 -m grpc_tools.protoc \
+            -I mtda/grpc \
+            --python_out=mtda/grpc \
+            --grpc_python_out=mtda/grpc \
+            mtda/grpc/mtda.proto

.gitignore

@@ -5,6 +5,8 @@
 *.changes
 *.img
 *.pyc
+*_pb2.py
+*_pb2_grpc.py
 *.swp
 build
 cip-core

CONTRIBUTING.md

@@ -13,6 +13,12 @@ Contribution Checklist
 
 - follow the existing coding style (run `pycodestyle` on changed files) [**required**]
 
+- if `mtda/grpc/mtda.proto` is modified, the Python stubs
+  (`mtda_pb2.py`, `mtda_pb2_grpc.py`) are **not** committed — they are
+  generated automatically during `python setup.py build` / `pip install`.
+  The CI will verify that `grpc_tools.protoc` can still consume the updated
+  `.proto` without errors.
+
 - add the required copyright header to each new file introduced, see
   [licensing information](COPYING) [**required**]
 

configs/docker.ini

@@ -18,9 +18,9 @@ debug = 2
 # ---------------------------------------------------------------------------
 # Remote settings
 # ---------------------------------------------------------------------------
+# Set "control" to the TCP/IP port number for the gRPC control interface
 [remote]
 control = 5556
-console = 5557
 host    = localhost
 
 # ---------------------------------------------------------------------------

configs/qemu.ini

@@ -24,6 +24,7 @@ variant=qemu
 # ---------------------------------------------------------------------------
 # Remote settings
 # ---------------------------------------------------------------------------
+# Set "control" to the TCP/IP port number for the gRPC control interface
 [remote]
 control = 5556
 console = 5557

debian/control

@@ -32,7 +32,6 @@ Depends: mtda-common,
          python3-systemd,
          python3-usb,
          python3-zeroconf,
-         python3-zmq,
          python3-zstandard,
          usbrelay,
          ${misc:Depends},
@@ -82,7 +81,7 @@ Package: mtda-common
 Architecture: all
 Multi-Arch: foreign
 Depends: python3:any (>= 3.7~),
-         python3-pyro5 | python3-pyro4
+         python3-grpcio
 Description: common modules for Multi-Tenant Device Access
  Modules shared between the service and the client.
 

docs/config.rst

@@ -83,19 +83,13 @@ General settings
       ``pduclient``, ``qemu``, ``shellcmd`` and ``usbrelay``.
 
 * ``remote``: section [optional]
-    Specify the host and ports to connect to when using a MTDA client (such as
+    Specify the host and port to connect to when using a MTDA client (such as
     ``mtda-cli``).
 
   * ``control``: integer [optional]
-      Remote port listening for control commands (defaults to ``5556``).
-
-  * ``console``: integer [optional]
-      Remote port to connect to in order to get console messages (defaults to
-      ``5557``).
-
-  * ``data``: integer [optional]
-      Remote port for data transfers between the client and agent (defaults to
-      ``0`` for a dynamic port assignment).
+      Remote port listening for gRPC requests (defaults to ``5556``). All
+      traffic — control commands, console streaming and data transfers — flows
+      over this single port.
 
   * ``host``: string [optional]
       Remote host name or ip to connect to as a client to interact with the

docs/usage.rst

@@ -279,7 +279,7 @@ Usage
      ``http://<MTDA-AGENT-IPC>:<PORT>``
 
    Where <PORT> is as configured in the [www] section of the configuration settings.
-   Default is 9080.
+   Default is 5000.
  * A web UI with MTDA control options and video stream console will be loaded as shown:
 
  .. image:: www-1.png

mtda-service

@@ -12,6 +12,7 @@
 
 # System imports
 import argparse
+import concurrent.futures
 import lockfile
 import netifaces
 import os
@@ -22,16 +23,15 @@ import socket
 import zeroconf
 from systemd import daemon as sd
 
+# gRPC
+import grpc
+
 # Local imports
 from mtda.main import MultiTenantDeviceAccess
+from mtda.grpc import mtda_pb2_grpc
+from mtda.grpc.servicer import MtdaServicer
 import mtda.constants as CONSTS
 
-# Pyro
-try:
-    from Pyro5.compatibility import Pyro4
-except ImportError:
-    import Pyro4
-
 
 class Application:
 
@@ -79,11 +79,13 @@ class Application:
         if status is False:
             return False
 
-        # Start our RPC server
-        Pyro4.config.HOST = "0.0.0.0"
-        Pyro4.config.SERIALIZER = "marshal"
-        daemon = Pyro4.Daemon(port=self.agent.ctrlport)
-        daemon.register(self.agent, objectId="mtda.main")
+        # Start our gRPC server
+        self._grpc_server = grpc.server(
+            concurrent.futures.ThreadPoolExecutor(max_workers=10))
+        mtda_pb2_grpc.add_MtdaServiceServicer_to_server(
+            MtdaServicer(self.agent), self._grpc_server)
+        self._grpc_server.add_insecure_port(f'[::]:{self.agent.ctrlport}')
+        self._grpc_server.start()
 
         # Initialize ZeroConf
         interfaces = zeroconf.InterfaceChoice.All
@@ -111,14 +113,16 @@ class Application:
 
         try:
             sd.notify('READY=1')
-            daemon.requestLoop()
+            self._grpc_server.wait_for_termination()
         except KeyboardInterrupt:
             self.stop()
         return True
 
     def stop(self, signum=0, frame=None):
         if self.zerosrv is not None:
             self.zeroconf.unregister_service(self.zerosrv)
+        if hasattr(self, '_grpc_server'):
+            self._grpc_server.stop(grace=5)
         if self.agent is not None:
             self.agent.stop()
         sys.exit(signum)