feat: OPA authorizer for Airflow 3 (#1127)

* patches I

* patches I

* wip: patch hbase to use listener endpoints

* changed rendering

* rework patch for UI

* wip

* removed wip deadwood

* removed wip deadwood II

* move extras to argument

* re-org constraints

* changelog

* wip

* wip

* fix merge errors

* split opa auth manager in two implementations

* replace poetry with uv

* tests: airflow 3 opa auth manager (and some reformatting)

* cleanup print() call

* update readme

* update changelog

---------

Co-authored-by: Andrew Kenworthy <[email protected]>

Author: razvan

CHANGELOG.md

@@ -47,6 +47,7 @@ All notable changes to this project will be documented in this file.
 - hbase: add 2.6.2 and upgrade dependencies ([#1101])
 - kafka: Add `4.0.0` ([#1117])
 - Include `.tar.gz` snapshots of the product source code in container images ([#1126])
+- airflow: OPA authorizer for Airflow 3.x ([#1127])
 
 ### Changed
 
@@ -147,6 +148,7 @@ All notable changes to this project will be documented in this file.
 [#1124]: https://github.com/stackabletech/docker-images/pull/1124
 [#1125]: https://github.com/stackabletech/docker-images/pull/1125
 [#1126]: https://github.com/stackabletech/docker-images/pull/1126
+[#1127]: https://github.com/stackabletech/docker-images/pull/1127
 [#1128]: https://github.com/stackabletech/docker-images/pull/1128
 
 ## [25.3.0] - 2025-03-21

airflow/Dockerfile

@@ -1,5 +1,7 @@
 # syntax=docker/dockerfile:1.15.1@sha256:9857836c9ee4268391bb5b09f9f157f3c91bb15821bb77969642813b0d00518d
-# check=error=true;skip=InvalidDefaultArgInFrom
+# Disabled error checks:
+# - SecretsUsedInArgOrEnv : OPA_AUTH_MANAGER is a false positive and breaks the build.
+# check=error=true;skip=InvalidDefaultArgInFrom,SecretsUsedInArgOrEnv
 
 ARG GIT_SYNC
 
@@ -9,17 +11,30 @@ FROM oci.stackable.tech/sdp/git-sync/git-sync:${GIT_SYNC} AS gitsync-image
 
 FROM stackable/image/shared/statsd-exporter AS statsd_exporter-builder
 
-FROM python:3.12-bookworm AS opa-auth-manager-builder
+FROM stackable/image/vector AS opa-auth-manager-builder
 
-COPY airflow/opa-auth-manager/ /tmp/opa-auth-manager
+ARG OPA_AUTH_MANAGER
+ARG PYTHON
+ARG UV
+
+COPY airflow/opa-auth-manager/${OPA_AUTH_MANAGER} /tmp/opa-auth-manager
 
 WORKDIR /tmp/opa-auth-manager
 
 RUN <<EOF
-pip install --no-cache-dir poetry
-poetry build
-poetry install
-poetry run pytest
+microdnf update
+microdnf install python${PYTHON}-pip
+microdnf clean all
+
+pip${PYTHON} install --no-cache-dir uv==${UV}
+
+# This folder is required by the tests to set up an sqlite database
+mkdir /root/airflow
+
+# Warnings are disabled because they come from various third party testing libraries
+# that we have no control over.
+uv run pytest --disable-warnings
+uv build
 EOF
 
 FROM stackable/image/vector AS airflow-build-image

airflow/opa-auth-manager/README.md

@@ -0,0 +1,12 @@
+# Airflow 2 OPA auth manager
+
+Auth manager for Airflow 2 which delegates the authorization to an Open Policy
+Agent
+
+[uv](https://docs.astral.sh/uv/) is used to build the project:
+
+    uv build
+
+The unit tests can be run as follows:
+
+    uv run pytest

airflow/opa-auth-manager/.gitignore renamed to airflow/opa-auth-manager/airflow-2/.gitignore

@@ -0,0 +1,27 @@
+[project]
+name = "opa-auth-manager"
+version = "0.1.0"
+description = "Auth manager for Airflow which delegates the authorization to an Open Policy Agent"
+authors = [
+    { name = "Siegfried Weber", email="[email protected]"},
+    { name = "Razvan Daniel Mihai", email="[email protected]"}
+]
+readme = "README.md"
+requires-python = ">=3.9,<3.13"
+
+dependencies = [
+    "requests~=2.32.3",
+    "cachetools~=5.5.0",
+    "overrides~=7.7.0"
+]
+
+[dependency-groups]
+dev = [
+    "apache-airflow~=2.9.3",
+    "pylint~=3.3.1",
+    "pytest~=8.3.3"
+]
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"

airflow/opa-auth-manager/airflow-2/README.md

@@ -0,0 +1,2 @@
+*.pytest_cache/
+dist/

airflow/opa-auth-manager/opa_auth_manager/__init__.py renamed to airflow/opa-auth-manager/airflow-2/opa_auth_manager/__init__.py

@@ -0,0 +1,15 @@
+# Airflow 3 OPA auth manager
+
+Auth manager for Airflow 3 which delegates the authorization to an Open Policy
+Agent
+
+[uv](https://docs.astral.sh/uv/) is used to build the project:
+
+    uv build
+
+The unit tests can be run as follows:
+
+    # Create directory for an SQLite database used by the test suite
+    mkdir ~/airflow
+
+    uv run pytest