feat: add Apple Silicon support via Rosetta emulation

apple-silicon/README.md

@@ -0,0 +1,186 @@
+# Apple Silicon Support
+
+Run instant-bosh on Apple Silicon Macs using Colima with Rosetta x86_64 emulation.
+
+**Goal:** Get these fixes upstream and remove this directory.
+
+## Requirements
+
+**Colima with VZ + Rosetta is REQUIRED.** Podman Machine is NOT supported.
+
+### Why Podman Machine doesn't work
+
+The BPM fix detects Rosetta emulation by checking for `/proc/sys/fs/binfmt_misc/rosetta`. Podman Machine uses **QEMU user-mode emulation** (`qemu-x86_64-static`) instead of Rosetta, even when `"Rosetta": true` is set in the machine config.
+
+The `Rosetta: true` setting in Podman only affects the VM hypervisor layer, not container emulation inside the VM. This means:
+- No `/proc/sys/fs/binfmt_misc/rosetta` exists (only `qemu-x86_64`)
+- BPM cannot detect emulation and fails with seccomp errors
+
+See [containers/podman#28181](https://github.com/containers/podman/issues/28181) for details.
+
+### Supported Configuration
+
+```bash
+# Start Colima with Rosetta (REQUIRED)
+colima start --arch aarch64 --vm-type vz --vz-rosetta --cpu 4 --memory 8
+```
+
+## Overview
+
+Two types of containers need patching:
+
+1. **Director container** - BPM seccomp filters fail on arm64 kernel
+2. **Stemcell VMs** - systemd services crash due to `MemoryDenyWriteExecute=yes` conflicting with Rosetta JIT
+
+## Upstream Issues
+
+| Issue | Workaround | Upstream PR |
+|-------|------------|-------------|
+| Seccomp filters fail (x86_64 on arm64) | `director/Dockerfile` builds BPM with Rosetta detection | [cloudfoundry/bpm-release#201](https://github.com/cloudfoundry/bpm-release/pull/201) |
+| systemd services crash in stemcell VMs | `stemcell/Dockerfile` adds drop-in overrides | Add `ConditionVirtualization=!container` or detect Rosetta |
+
+## Directory Structure
+
+```
+apple-silicon/
+├── director/
+│   ├── Dockerfile                # Multi-stage build: compiles BPM from PR branch
+│   ├── build.sh                  # Build script
+│   └── wait-for-postgres-role.sh # Helper script for postgres startup
+├── stemcell/
+│   ├── Dockerfile                # Patches CloudFoundry stemcell images
+│   └── build.sh                  # Build script
+├── start-colima-rosetta.sh       # Start Colima with Rosetta emulation
+└── README.md
+```
+
+## Quick Start
+
+### 1. Start Colima with Rosetta
+
+```bash
+./apple-silicon/start-colima-rosetta.sh
+```
+
+Or manually:
+```bash
+colima start --arch aarch64 --vm-type vz --vz-rosetta --cpu 4 --memory 8
+```
+
+### 2. Build Patched Images
+
+Build the patched director image (compiles BPM from source):
+```bash
+./apple-silicon/director/build.sh ghcr.io/rkoster/instant-bosh:sha-6de5b3c
+```
+
+Build the patched stemcell image:
+```bash
+./apple-silicon/stemcell/build.sh ubuntu-noble latest
+```
+
+### 3. Start instant-bosh with Patched Director
+
+```bash
+export IBOSH_IMAGE=ghcr.io/rkoster/instant-bosh:sha-6de5b3c-apple-silicon
+ibosh start
+```
+
+### 4. Upload Patched Stemcell and Deploy
+
+```bash
+# Upload the patched stemcell
+ibosh upload-stemcell ghcr.io/rkoster/ubuntu-noble-stemcell:latest-apple-silicon
+
+# Deploy (example with zookeeper)
+eval "$(ibosh print-env)"
+bosh -d zookeeper deploy test/manifest/zookeeper.yml
+```
+
+## How the Patches Work
+
+### Director Patches (`director/Dockerfile`)
+
+The Dockerfile uses a **multi-stage build** to compile BPM from a branch that includes Rosetta emulation detection:
+
+1. **Stage 1 (bpm-builder)**: Clones [bpm-release PR #201](https://github.com/cloudfoundry/bpm-release/pull/201) and compiles the `bpm` binary
+2. **Stage 2 (final)**: Copies the patched BPM binary into the instant-bosh image
+
+Note: The existing `runc` binary from the base image is kept as-is because the Rosetta detection logic is in the `bpm` binary's `sysfeat` package, not in runc.
+
+**How the BPM fix works:**
+- BPM now includes a `sysfeat` package that detects Rosetta emulation by checking:
+  - `/proc/sys/fs/binfmt_misc/rosetta` (Rosetta binfmt registration)
+  - `/proc/cpuinfo` for `VirtualApple` vendor string
+- When emulation is detected, BPM sets `SeccompSupported = false`
+- This causes BPM to skip loading seccomp filters (which would fail with "invalid argument")
+- Unlike the previous privileged mode workaround, this **only** disables seccomp without granting additional privileges
+
+**Environment variables for build.sh:**
+- `BPM_BRANCH` - Override the BPM branch (default: `disable-seccomp-for-docker-cpi-on-apple-silicon`)
+- `BPM_REPO` - Override the BPM repository URL
+- `OUTPUT_IMAGE` - Override the output image name/tag
+
+### Stemcell Patches (`stemcell/Dockerfile`)
+
+The Dockerfile creates systemd drop-in overrides that disable security features incompatible with Rosetta JIT:
+
+```ini
+[Service]
+MemoryDenyWriteExecute=no
+SystemCallArchitectures=
+SystemCallFilter=
+LockPersonality=no
+NoNewPrivileges=no
+```
+
+Services patched:
+- `systemd-journald`
+- `systemd-resolved`
+- `systemd-networkd`
+- `systemd-logind`
+- `systemd-timesyncd`
+- `auditd`
+
+Also masks `systemd-binfmt.service` to prevent clearing host binfmt_misc registrations.
+
+## Pre-built Images
+
+If available, you can use pre-built images instead of building locally:
+
+```bash
+# Director
+export IBOSH_IMAGE=ghcr.io/rkoster/instant-bosh:sha-6de5b3c-apple-silicon
+
+# Stemcell
+ibosh upload-stemcell ghcr.io/rkoster/ubuntu-noble-stemcell:latest-apple-silicon
+```
+
+## Troubleshooting
+
+### Check if director processes are running
+```bash
+docker exec -it $(docker ps -qf name=instant-bosh) bash
+monit summary
+```
+
+### Verify BPM version (should show `-apple-silicon` suffix)
+```bash
+docker exec -it $(docker ps -qf name=instant-bosh) /var/vcap/packages/bpm/bin/bpm version
+```
+
+### Check stemcell VM systemd status
+```bash
+docker exec -it <stemcell-container-id> systemctl status
+```
+
+### View BPM logs
+```bash
+docker exec -it $(docker ps -qf name=instant-bosh) bash
+tail -f /var/vcap/sys/log/*/bpm.log
+```
+
+### Build with a different BPM branch
+```bash
+BPM_BRANCH=my-feature-branch ./apple-silicon/director/build.sh
+```

apple-silicon/director/Dockerfile

@@ -0,0 +1,76 @@
+# Dockerfile for patching instant-bosh director for Apple Silicon
+#
+# This uses a multi-stage build to compile BPM from a branch that includes
+# Rosetta emulation detection. When BPM detects it's running under Rosetta
+# (x86_64 on ARM64 kernel), it automatically disables seccomp filters which
+# would otherwise fail with "error loading seccomp filter: invalid argument".
+#
+# This approach is cleaner than the previous privileged mode workaround because
+# it only disables seccomp (the actual problem) without granting additional
+# privileges or requiring privilege dropping scripts.
+#
+# Usage:
+#   docker build --platform linux/amd64 \
+#                --build-arg BASE_IMAGE=ghcr.io/rkoster/instant-bosh:sha-xxx \
+#                -t my-patched-instant-bosh .
+#
+# Or use the build.sh script:
+#   ./build.sh ghcr.io/rkoster/instant-bosh:sha-xxx
+
+# Global ARGs - must be declared before first FROM to be used in FROM lines
+ARG BASE_IMAGE=ghcr.io/rkoster/instant-bosh:latest
+ARG BPM_BRANCH=disable-seccomp-for-docker-cpi-on-apple-silicon
+ARG BPM_REPO=https://github.com/cloudfoundry/bpm-release.git
+
+# ============================================================================
+# Stage 1: Build BPM binary from PR branch with Rosetta detection
+# ============================================================================
+FROM golang:1.25-bookworm AS bpm-builder
+
+# Re-declare ARGs needed in this stage (they don't persist across FROM)
+ARG BPM_BRANCH
+ARG BPM_REPO
+
+# Install git and clone the bpm-release repo
+RUN apt-get update && apt-get install -y --no-install-recommends git ca-certificates \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /build
+RUN git clone --depth 1 --branch ${BPM_BRANCH} ${BPM_REPO} bpm-release
+
+# Build the bpm binary
+# Note: The Rosetta detection logic is in src/bpm/sysfeat/sysfeat.go
+WORKDIR /build/bpm-release/src/bpm
+RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \
+    go build -mod=vendor \
+    -ldflags "-X bpm/commands.Version=$(cat /build/bpm-release/src/version)-apple-silicon" \
+    -o /build/bpm \
+    ./cmd/bpm
+
+# ============================================================================
+# Stage 2: Final image with patched BPM binary
+# ============================================================================
+# Note: We only replace the bpm binary. The existing runc binary from the base
+# image is kept as-is because the Rosetta detection is in BPM, not runc.
+# BPM decides whether to apply seccomp filters before invoking runc.
+FROM ${BASE_IMAGE}
+
+LABEL maintainer="instant-bosh"
+LABEL description="instant-bosh patched for Apple Silicon (Rosetta x86_64 emulation)"
+
+# Copy the patched BPM binary that includes Rosetta emulation detection
+# This binary automatically disables seccomp when it detects architecture mismatch
+COPY --from=bpm-builder /build/bpm /var/vcap/packages/bpm/bin/bpm
+
+# ============================================================================
+# Postgres startup fix: wait for postgres role before migrations
+# ============================================================================
+# The create-database script runs in the background and creates the postgres role.
+# We need to wait for it before running migrations.
+COPY wait-for-postgres-role.sh /usr/local/bin/wait-for-postgres-role.sh
+RUN chmod +x /usr/local/bin/wait-for-postgres-role.sh && \
+    sed -i 's|/var/vcap/packages/director/bin/bosh-director-migrate|/usr/local/bin/wait-for-postgres-role.sh\n\n/var/vcap/packages/director/bin/bosh-director-migrate|' \
+        /var/vcap/jobs/director/templates/director
+
+# Verify the BPM binary was installed correctly
+RUN /var/vcap/packages/bpm/bin/bpm version

apple-silicon/director/build.sh

@@ -0,0 +1,76 @@
+#!/bin/bash
+# Build a patched instant-bosh director image for Apple Silicon
+#
+# This script builds a Docker image that includes a patched BPM binary
+# compiled from a branch with Rosetta emulation detection. The patched BPM
+# automatically disables seccomp when running under architecture emulation.
+#
+# Usage:
+#   ./build.sh                                      # Build from latest
+#   ./build.sh ghcr.io/rkoster/instant-bosh:sha-xxx # Build from specific image
+#
+# Environment variables:
+#   OUTPUT_IMAGE - Override the output image name/tag
+#   BPM_BRANCH   - Override the BPM branch to build from (default: disable-seccomp-for-docker-cpi-on-apple-silicon)
+#   BPM_REPO     - Override the BPM repository URL (default: https://github.com/cloudfoundry/bpm-release.git)
+
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+# Default base image - use the argument or default to latest
+BASE_IMAGE="${1:-ghcr.io/rkoster/instant-bosh:latest}"
+
+# BPM build configuration
+BPM_BRANCH="${BPM_BRANCH:-disable-seccomp-for-docker-cpi-on-apple-silicon}"
+BPM_REPO="${BPM_REPO:-https://github.com/cloudfoundry/bpm-release.git}"
+
+# Output image - derive from base image or use override
+if [[ -n "${OUTPUT_IMAGE:-}" ]]; then
+    OUTPUT="${OUTPUT_IMAGE}"
+else
+    # Extract tag from base image and append -apple-silicon
+    if [[ "${BASE_IMAGE}" =~ :(.+)$ ]]; then
+        TAG="${BASH_REMATCH[1]}"
+        OUTPUT="${BASE_IMAGE/:${TAG}/:${TAG}-apple-silicon}"
+    else
+        OUTPUT="${BASE_IMAGE}:apple-silicon"
+    fi
+fi
+
+echo "=============================================="
+echo "Building Patched instant-bosh for Apple Silicon"
+echo "=============================================="
+echo ""
+echo "Base image:   ${BASE_IMAGE}"
+echo "Output image: ${OUTPUT}"
+echo "BPM branch:   ${BPM_BRANCH}"
+echo "BPM repo:     ${BPM_REPO}"
+echo ""
+
+# Build the image (multi-stage: compiles BPM from source)
+echo "Building image (this may take a few minutes to compile BPM)..."
+docker build \
+    --platform linux/amd64 \
+    --provenance=false \
+    --build-arg "BASE_IMAGE=${BASE_IMAGE}" \
+    --build-arg "BPM_BRANCH=${BPM_BRANCH}" \
+    --build-arg "BPM_REPO=${BPM_REPO}" \
+    -t "${OUTPUT}" \
+    -f "${SCRIPT_DIR}/Dockerfile" \
+    "${SCRIPT_DIR}"
+
+echo ""
+echo "=============================================="
+echo "Build Complete!"
+echo "=============================================="
+echo ""
+echo "Tagged as: ${OUTPUT}"
+echo ""
+echo "To use this image with instant-bosh, set IBOSH_IMAGE:"
+echo "  export IBOSH_IMAGE=${OUTPUT}"
+echo "  ibosh start"
+echo ""
+echo "Or push to a registry and use:"
+echo "  docker push ${OUTPUT}"
+echo ""

apple-silicon/director/wait-for-postgres-role.sh

@@ -0,0 +1,17 @@
+#!/bin/bash
+# Wait for postgres role to be created
+# The create-database script runs in the background and creates the postgres role.
+# We need to wait for it before running migrations.
+
+echo "Waiting for postgres role to be created..."
+for i in $(seq 1 60); do
+  if /var/vcap/packages/postgres-15/bin/psql -h 127.0.0.1 -U vcap -d postgres -tAc "SELECT 1 FROM pg_roles WHERE rolname='postgres'" 2>/dev/null | grep -q 1; then
+    echo "postgres role found"
+    exit 0
+  fi
+  echo "Waiting for postgres role... ($i/60)"
+  sleep 1
+done
+
+echo "ERROR: postgres role not found after 60 seconds"
+exit 1