diff --git a/apple-silicon/README.md b/apple-silicon/README.md new file mode 100644 index 0000000..1af45a2 --- /dev/null +++ b/apple-silicon/README.md @@ -0,0 +1,186 @@ +# Apple Silicon Support + +Run instant-bosh on Apple Silicon Macs using Colima with Rosetta x86_64 emulation. + +**Goal:** Get these fixes upstream and remove this directory. + +## Requirements + +**Colima with VZ + Rosetta is REQUIRED.** Podman Machine is NOT supported. + +### Why Podman Machine doesn't work + +The BPM fix detects Rosetta emulation by checking for `/proc/sys/fs/binfmt_misc/rosetta`. Podman Machine uses **QEMU user-mode emulation** (`qemu-x86_64-static`) instead of Rosetta, even when `"Rosetta": true` is set in the machine config. + +The `Rosetta: true` setting in Podman only affects the VM hypervisor layer, not container emulation inside the VM. This means: +- No `/proc/sys/fs/binfmt_misc/rosetta` exists (only `qemu-x86_64`) +- BPM cannot detect emulation and fails with seccomp errors + +See [containers/podman#28181](https://github.com/containers/podman/issues/28181) for details. + +### Supported Configuration + +```bash +# Start Colima with Rosetta (REQUIRED) +colima start --arch aarch64 --vm-type vz --vz-rosetta --cpu 4 --memory 8 +``` + +## Overview + +Two types of containers need patching: + +1. **Director container** - BPM seccomp filters fail on arm64 kernel +2. **Stemcell VMs** - systemd services crash due to `MemoryDenyWriteExecute=yes` conflicting with Rosetta JIT + +## Upstream Issues + +| Issue | Workaround | Upstream PR | +|-------|------------|-------------| +| Seccomp filters fail (x86_64 on arm64) | `director/Dockerfile` builds BPM with Rosetta detection | [cloudfoundry/bpm-release#201](https://github.com/cloudfoundry/bpm-release/pull/201) | +| systemd services crash in stemcell VMs | `stemcell/Dockerfile` adds drop-in overrides | Add `ConditionVirtualization=!container` or detect Rosetta | + +## Directory Structure + +``` +apple-silicon/ +├── director/ +│ ├── Dockerfile # Multi-stage build: compiles BPM from PR branch +│ ├── build.sh # Build script +│ └── wait-for-postgres-role.sh # Helper script for postgres startup +├── stemcell/ +│ ├── Dockerfile # Patches CloudFoundry stemcell images +│ └── build.sh # Build script +├── start-colima-rosetta.sh # Start Colima with Rosetta emulation +└── README.md +``` + +## Quick Start + +### 1. Start Colima with Rosetta + +```bash +./apple-silicon/start-colima-rosetta.sh +``` + +Or manually: +```bash +colima start --arch aarch64 --vm-type vz --vz-rosetta --cpu 4 --memory 8 +``` + +### 2. Build Patched Images + +Build the patched director image (compiles BPM from source): +```bash +./apple-silicon/director/build.sh ghcr.io/rkoster/instant-bosh:sha-6de5b3c +``` + +Build the patched stemcell image: +```bash +./apple-silicon/stemcell/build.sh ubuntu-noble latest +``` + +### 3. Start instant-bosh with Patched Director + +```bash +export IBOSH_IMAGE=ghcr.io/rkoster/instant-bosh:sha-6de5b3c-apple-silicon +ibosh start +``` + +### 4. Upload Patched Stemcell and Deploy + +```bash +# Upload the patched stemcell +ibosh upload-stemcell ghcr.io/rkoster/ubuntu-noble-stemcell:latest-apple-silicon + +# Deploy (example with zookeeper) +eval "$(ibosh print-env)" +bosh -d zookeeper deploy test/manifest/zookeeper.yml +``` + +## How the Patches Work + +### Director Patches (`director/Dockerfile`) + +The Dockerfile uses a **multi-stage build** to compile BPM from a branch that includes Rosetta emulation detection: + +1. **Stage 1 (bpm-builder)**: Clones [bpm-release PR #201](https://github.com/cloudfoundry/bpm-release/pull/201) and compiles the `bpm` binary +2. **Stage 2 (final)**: Copies the patched BPM binary into the instant-bosh image + +Note: The existing `runc` binary from the base image is kept as-is because the Rosetta detection logic is in the `bpm` binary's `sysfeat` package, not in runc. + +**How the BPM fix works:** +- BPM now includes a `sysfeat` package that detects Rosetta emulation by checking: + - `/proc/sys/fs/binfmt_misc/rosetta` (Rosetta binfmt registration) + - `/proc/cpuinfo` for `VirtualApple` vendor string +- When emulation is detected, BPM sets `SeccompSupported = false` +- This causes BPM to skip loading seccomp filters (which would fail with "invalid argument") +- Unlike the previous privileged mode workaround, this **only** disables seccomp without granting additional privileges + +**Environment variables for build.sh:** +- `BPM_BRANCH` - Override the BPM branch (default: `disable-seccomp-for-docker-cpi-on-apple-silicon`) +- `BPM_REPO` - Override the BPM repository URL +- `OUTPUT_IMAGE` - Override the output image name/tag + +### Stemcell Patches (`stemcell/Dockerfile`) + +The Dockerfile creates systemd drop-in overrides that disable security features incompatible with Rosetta JIT: + +```ini +[Service] +MemoryDenyWriteExecute=no +SystemCallArchitectures= +SystemCallFilter= +LockPersonality=no +NoNewPrivileges=no +``` + +Services patched: +- `systemd-journald` +- `systemd-resolved` +- `systemd-networkd` +- `systemd-logind` +- `systemd-timesyncd` +- `auditd` + +Also masks `systemd-binfmt.service` to prevent clearing host binfmt_misc registrations. + +## Pre-built Images + +If available, you can use pre-built images instead of building locally: + +```bash +# Director +export IBOSH_IMAGE=ghcr.io/rkoster/instant-bosh:sha-6de5b3c-apple-silicon + +# Stemcell +ibosh upload-stemcell ghcr.io/rkoster/ubuntu-noble-stemcell:latest-apple-silicon +``` + +## Troubleshooting + +### Check if director processes are running +```bash +docker exec -it $(docker ps -qf name=instant-bosh) bash +monit summary +``` + +### Verify BPM version (should show `-apple-silicon` suffix) +```bash +docker exec -it $(docker ps -qf name=instant-bosh) /var/vcap/packages/bpm/bin/bpm version +``` + +### Check stemcell VM systemd status +```bash +docker exec -it systemctl status +``` + +### View BPM logs +```bash +docker exec -it $(docker ps -qf name=instant-bosh) bash +tail -f /var/vcap/sys/log/*/bpm.log +``` + +### Build with a different BPM branch +```bash +BPM_BRANCH=my-feature-branch ./apple-silicon/director/build.sh +``` diff --git a/apple-silicon/director/Dockerfile b/apple-silicon/director/Dockerfile new file mode 100644 index 0000000..4e7b692 --- /dev/null +++ b/apple-silicon/director/Dockerfile @@ -0,0 +1,76 @@ +# Dockerfile for patching instant-bosh director for Apple Silicon +# +# This uses a multi-stage build to compile BPM from a branch that includes +# Rosetta emulation detection. When BPM detects it's running under Rosetta +# (x86_64 on ARM64 kernel), it automatically disables seccomp filters which +# would otherwise fail with "error loading seccomp filter: invalid argument". +# +# This approach is cleaner than the previous privileged mode workaround because +# it only disables seccomp (the actual problem) without granting additional +# privileges or requiring privilege dropping scripts. +# +# Usage: +# docker build --platform linux/amd64 \ +# --build-arg BASE_IMAGE=ghcr.io/rkoster/instant-bosh:sha-xxx \ +# -t my-patched-instant-bosh . +# +# Or use the build.sh script: +# ./build.sh ghcr.io/rkoster/instant-bosh:sha-xxx + +# Global ARGs - must be declared before first FROM to be used in FROM lines +ARG BASE_IMAGE=ghcr.io/rkoster/instant-bosh:latest +ARG BPM_BRANCH=disable-seccomp-for-docker-cpi-on-apple-silicon +ARG BPM_REPO=https://github.com/cloudfoundry/bpm-release.git + +# ============================================================================ +# Stage 1: Build BPM binary from PR branch with Rosetta detection +# ============================================================================ +FROM golang:1.25-bookworm AS bpm-builder + +# Re-declare ARGs needed in this stage (they don't persist across FROM) +ARG BPM_BRANCH +ARG BPM_REPO + +# Install git and clone the bpm-release repo +RUN apt-get update && apt-get install -y --no-install-recommends git ca-certificates \ + && rm -rf /var/lib/apt/lists/* + +WORKDIR /build +RUN git clone --depth 1 --branch ${BPM_BRANCH} ${BPM_REPO} bpm-release + +# Build the bpm binary +# Note: The Rosetta detection logic is in src/bpm/sysfeat/sysfeat.go +WORKDIR /build/bpm-release/src/bpm +RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \ + go build -mod=vendor \ + -ldflags "-X bpm/commands.Version=$(cat /build/bpm-release/src/version)-apple-silicon" \ + -o /build/bpm \ + ./cmd/bpm + +# ============================================================================ +# Stage 2: Final image with patched BPM binary +# ============================================================================ +# Note: We only replace the bpm binary. The existing runc binary from the base +# image is kept as-is because the Rosetta detection is in BPM, not runc. +# BPM decides whether to apply seccomp filters before invoking runc. +FROM ${BASE_IMAGE} + +LABEL maintainer="instant-bosh" +LABEL description="instant-bosh patched for Apple Silicon (Rosetta x86_64 emulation)" + +# Copy the patched BPM binary that includes Rosetta emulation detection +# This binary automatically disables seccomp when it detects architecture mismatch +COPY --from=bpm-builder /build/bpm /var/vcap/packages/bpm/bin/bpm + +# ============================================================================ +# Postgres startup fix: wait for postgres role before migrations +# ============================================================================ +# The create-database script runs in the background and creates the postgres role. +# We need to wait for it before running migrations. +COPY wait-for-postgres-role.sh /usr/local/bin/wait-for-postgres-role.sh +RUN chmod +x /usr/local/bin/wait-for-postgres-role.sh && \ + sed -i 's|/var/vcap/packages/director/bin/bosh-director-migrate|/usr/local/bin/wait-for-postgres-role.sh\n\n/var/vcap/packages/director/bin/bosh-director-migrate|' \ + /var/vcap/jobs/director/templates/director + +# Verify the BPM binary was installed correctly +RUN /var/vcap/packages/bpm/bin/bpm version diff --git a/apple-silicon/director/build.sh b/apple-silicon/director/build.sh new file mode 100755 index 0000000..53fe48f --- /dev/null +++ b/apple-silicon/director/build.sh @@ -0,0 +1,76 @@ +#!/bin/bash +# Build a patched instant-bosh director image for Apple Silicon +# +# This script builds a Docker image that includes a patched BPM binary +# compiled from a branch with Rosetta emulation detection. The patched BPM +# automatically disables seccomp when running under architecture emulation. +# +# Usage: +# ./build.sh # Build from latest +# ./build.sh ghcr.io/rkoster/instant-bosh:sha-xxx # Build from specific image +# +# Environment variables: +# OUTPUT_IMAGE - Override the output image name/tag +# BPM_BRANCH - Override the BPM branch to build from (default: disable-seccomp-for-docker-cpi-on-apple-silicon) +# BPM_REPO - Override the BPM repository URL (default: https://github.com/cloudfoundry/bpm-release.git) + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +# Default base image - use the argument or default to latest +BASE_IMAGE="${1:-ghcr.io/rkoster/instant-bosh:latest}" + +# BPM build configuration +BPM_BRANCH="${BPM_BRANCH:-disable-seccomp-for-docker-cpi-on-apple-silicon}" +BPM_REPO="${BPM_REPO:-https://github.com/cloudfoundry/bpm-release.git}" + +# Output image - derive from base image or use override +if [[ -n "${OUTPUT_IMAGE:-}" ]]; then + OUTPUT="${OUTPUT_IMAGE}" +else + # Extract tag from base image and append -apple-silicon + if [[ "${BASE_IMAGE}" =~ :(.+)$ ]]; then + TAG="${BASH_REMATCH[1]}" + OUTPUT="${BASE_IMAGE/:${TAG}/:${TAG}-apple-silicon}" + else + OUTPUT="${BASE_IMAGE}:apple-silicon" + fi +fi + +echo "==============================================" +echo "Building Patched instant-bosh for Apple Silicon" +echo "==============================================" +echo "" +echo "Base image: ${BASE_IMAGE}" +echo "Output image: ${OUTPUT}" +echo "BPM branch: ${BPM_BRANCH}" +echo "BPM repo: ${BPM_REPO}" +echo "" + +# Build the image (multi-stage: compiles BPM from source) +echo "Building image (this may take a few minutes to compile BPM)..." +docker build \ + --platform linux/amd64 \ + --provenance=false \ + --build-arg "BASE_IMAGE=${BASE_IMAGE}" \ + --build-arg "BPM_BRANCH=${BPM_BRANCH}" \ + --build-arg "BPM_REPO=${BPM_REPO}" \ + -t "${OUTPUT}" \ + -f "${SCRIPT_DIR}/Dockerfile" \ + "${SCRIPT_DIR}" + +echo "" +echo "==============================================" +echo "Build Complete!" +echo "==============================================" +echo "" +echo "Tagged as: ${OUTPUT}" +echo "" +echo "To use this image with instant-bosh, set IBOSH_IMAGE:" +echo " export IBOSH_IMAGE=${OUTPUT}" +echo " ibosh start" +echo "" +echo "Or push to a registry and use:" +echo " docker push ${OUTPUT}" +echo "" diff --git a/apple-silicon/director/wait-for-postgres-role.sh b/apple-silicon/director/wait-for-postgres-role.sh new file mode 100644 index 0000000..182aa02 --- /dev/null +++ b/apple-silicon/director/wait-for-postgres-role.sh @@ -0,0 +1,17 @@ +#!/bin/bash +# Wait for postgres role to be created +# The create-database script runs in the background and creates the postgres role. +# We need to wait for it before running migrations. + +echo "Waiting for postgres role to be created..." +for i in $(seq 1 60); do + if /var/vcap/packages/postgres-15/bin/psql -h 127.0.0.1 -U vcap -d postgres -tAc "SELECT 1 FROM pg_roles WHERE rolname='postgres'" 2>/dev/null | grep -q 1; then + echo "postgres role found" + exit 0 + fi + echo "Waiting for postgres role... ($i/60)" + sleep 1 +done + +echo "ERROR: postgres role not found after 60 seconds" +exit 1 diff --git a/apple-silicon/start-colima-rosetta.sh b/apple-silicon/start-colima-rosetta.sh new file mode 100755 index 0000000..3b900ab --- /dev/null +++ b/apple-silicon/start-colima-rosetta.sh @@ -0,0 +1,194 @@ +#!/bin/bash +# Script to start Colima with Rosetta emulation for Apple Silicon Macs +# This enables running x86_64 containers with better compatibility than QEMU +# +# Rosetta is required for instant-bosh because: +# - The instant-bosh Docker image is built for amd64 (x86_64) +# - QEMU emulation has issues with runc's /proc/self/exe cloning mechanism +# - Rosetta provides faster and more compatible x86_64 emulation +# +# Prerequisites: +# - macOS Ventura (13.0) or later +# - Apple Silicon Mac (M1/M2/M3) +# - Rosetta 2 installed (softwareupdate --install-rosetta) +# +# Usage: ./scripts/start-colima-rosetta.sh [--force] +# +# Options: +# --force Stop and restart Colima even if already running + +set -euo pipefail + +# Configuration - adjust these as needed +CPU="${COLIMA_CPU:-8}" +MEMORY="${COLIMA_MEMORY:-16}" +DISK="${COLIMA_DISK:-200}" + +# Parse arguments +FORCE_RESTART=false +while [[ $# -gt 0 ]]; do + case $1 in + --force) + FORCE_RESTART=true + shift + ;; + *) + echo "Unknown option: $1" + echo "Usage: $0 [--force]" + exit 1 + ;; + esac +done + +# Check if running on Apple Silicon +check_apple_silicon() { + if [[ "$(uname -m)" != "arm64" ]]; then + echo "Error: This script is intended for Apple Silicon Macs only." + echo "Current architecture: $(uname -m)" + exit 1 + fi +} + +# Check macOS version (need Ventura or later for vz + rosetta) +check_macos_version() { + local version + version=$(sw_vers -productVersion) + local major_version + major_version=$(echo "$version" | cut -d. -f1) + + if [[ "$major_version" -lt 13 ]]; then + echo "Error: macOS Ventura (13.0) or later is required for Rosetta + Virtualization.framework" + echo "Current version: $version" + exit 1 + fi +} + +# Check if Rosetta is installed +check_rosetta() { + if ! /usr/bin/pgrep -q oahd 2>/dev/null; then + echo "Rosetta 2 does not appear to be installed or running." + echo "Installing Rosetta 2..." + softwareupdate --install-rosetta --agree-to-license + fi +} + +# Check if Colima is installed +check_colima() { + if ! command -v colima &>/dev/null; then + echo "Error: Colima is not installed." + echo "Install it with: brew install colima" + exit 1 + fi +} + +# Stop Colima if running and force flag is set +stop_colima_if_needed() { + if colima status &>/dev/null; then + if [[ "$FORCE_RESTART" == "true" ]]; then + echo "==> Stopping existing Colima instance..." + colima stop + else + echo "Colima is already running." + echo "Use --force to stop and restart with Rosetta configuration." + + # Check if current config already uses Rosetta + if colima ssh -- cat /proc/sys/fs/binfmt_misc/rosetta &>/dev/null; then + echo "Rosetta emulation is already enabled." + exit 0 + else + echo "Warning: Current Colima instance is NOT using Rosetta." + echo "Run with --force to restart with Rosetta support." + exit 1 + fi + fi + fi +} + +# Start Colima with Rosetta +start_colima() { + echo "==> Starting Colima with Rosetta emulation..." + echo " CPU: ${CPU}" + echo " Memory: ${MEMORY}GiB" + echo " Disk: ${DISK}GiB" + echo "" + + # Key options: + # --arch aarch64 : Run VM natively on ARM (required for vz-rosetta) + # --vm-type vz : Use Apple's Virtualization.framework + # --vz-rosetta : Enable Rosetta for x86_64 binary translation + # --network-address : Assign a routable IP to the VM + # --mount-type virtiofs: Fast file sharing + # --binfmt : Register binfmt handlers (needed for multi-arch) + + colima start \ + --cpu "${CPU}" \ + --memory "${MEMORY}" \ + --disk "${DISK}" \ + --arch aarch64 \ + --vm-type vz \ + --vz-rosetta \ + --port-forwarder=ssh \ + --network-address \ + --mount-type=virtiofs \ + --mount "${TMPDIR}:w" \ + --mount "${HOME}:w" + + # Fix Docker socket permissions + echo "==> Fixing Docker socket permissions..." + colima ssh -- sudo chmod 666 /var/run/docker.sock +} + +# Verify Rosetta is working +verify_rosetta() { + echo "==> Verifying Rosetta emulation..." + + # Check if Rosetta binfmt is registered + if colima ssh -- cat /proc/sys/fs/binfmt_misc/rosetta &>/dev/null; then + echo " Rosetta binfmt handler: REGISTERED" + else + echo " Warning: Rosetta binfmt handler not found" + echo " Available handlers:" + colima ssh -- ls /proc/sys/fs/binfmt_misc/ + fi + + # Test running an x86_64 binary + echo "==> Testing x86_64 emulation..." + if docker run --rm --platform linux/amd64 alpine:latest uname -m 2>/dev/null | grep -q x86_64; then + echo " x86_64 container test: PASSED" + else + echo " Warning: x86_64 container test failed" + fi +} + +# Print summary +print_summary() { + echo "" + echo "==============================================" + echo "Colima started with Rosetta emulation" + echo "==============================================" + echo "" + colima list + echo "" + echo "Docker context is ready. You can now run:" + echo " go run ./cmd/ibosh/main.go docker start" + echo "" +} + +# Main +main() { + echo "==============================================" + echo "Starting Colima with Rosetta for Apple Silicon" + echo "==============================================" + echo "" + + check_apple_silicon + check_macos_version + check_rosetta + check_colima + stop_colima_if_needed + start_colima + verify_rosetta + print_summary +} + +main "$@" diff --git a/apple-silicon/start-podman-rosetta.sh b/apple-silicon/start-podman-rosetta.sh new file mode 100755 index 0000000..456586b --- /dev/null +++ b/apple-silicon/start-podman-rosetta.sh @@ -0,0 +1,236 @@ +#!/bin/bash +# Script to start Podman with Rosetta emulation for Apple Silicon Macs +# This enables running x86_64 containers with better compatibility than QEMU +# +# Rosetta is required for instant-bosh because: +# - The instant-bosh Docker image is built for amd64 (x86_64) +# - QEMU emulation has issues with runc's /proc/self/exe cloning mechanism +# - Rosetta provides faster and more compatible x86_64 emulation +# +# Prerequisites: +# - macOS Tahoe (26.0) or later (Rosetta fix requires Tahoe beta or newer) +# - Apple Silicon Mac (M1/M2/M3/M4) +# - Rosetta 2 installed (softwareupdate --install-rosetta) +# - Podman 5.6 or later +# +# Based on: https://blog.podman.io/2025/08/podman-5-6-released-rosetta-status-update/ +# +# Usage: ./apple-silicon/start-podman-rosetta.sh [--force] +# +# Options: +# --force Stop and restart Podman machine even if already running + +set -euo pipefail + +# Configuration - adjust these as needed +CPU="${PODMAN_CPU:-8}" +MEMORY="${PODMAN_MEMORY:-16384}" # In MB +DISK="${PODMAN_DISK:-200}" # In GB + +# Parse arguments +FORCE_RESTART=false +while [[ $# -gt 0 ]]; do + case $1 in + --force) + FORCE_RESTART=true + shift + ;; + *) + echo "Unknown option: $1" + echo "Usage: $0 [--force]" + exit 1 + ;; + esac +done + +# Check if running on Apple Silicon +check_apple_silicon() { + if [[ "$(uname -m)" != "arm64" ]]; then + echo "Error: This script is intended for Apple Silicon Macs only." + echo "Current architecture: $(uname -m)" + exit 1 + fi +} + +# Check macOS version (need Tahoe or later for Rosetta fix) +check_macos_version() { + local version + version=$(sw_vers -productVersion) + local major_version + major_version=$(echo "$version" | cut -d. -f1) + + if [[ "$major_version" -lt 26 ]]; then + echo "Warning: macOS Tahoe (26.0) or later is recommended for Rosetta compatibility." + echo "Current version: $version" + echo "Rosetta may not work correctly with Linux kernel 6.13+ on older macOS versions." + echo "" + read -p "Continue anyway? (y/N) " -n 1 -r + echo + if [[ ! $REPLY =~ ^[Yy]$ ]]; then + exit 1 + fi + fi +} + +# Check if Rosetta is installed +check_rosetta() { + if ! /usr/bin/pgrep -q oahd 2>/dev/null; then + echo "Rosetta 2 does not appear to be installed or running." + echo "Installing Rosetta 2..." + softwareupdate --install-rosetta --agree-to-license + fi +} + +# Check if Podman is installed +check_podman() { + if ! command -v podman &>/dev/null; then + echo "Error: Podman is not installed." + echo "Install it with: brew install podman" + exit 1 + fi + + # Check Podman version (need 5.6+) + local version + version=$(podman --version | grep -oE '[0-9]+\.[0-9]+' | head -1) + local major minor + major=$(echo "$version" | cut -d. -f1) + minor=$(echo "$version" | cut -d. -f2) + + if [[ "$major" -lt 5 ]] || { [[ "$major" -eq 5 ]] && [[ "$minor" -lt 6 ]]; }; then + echo "Error: Podman 5.6 or later is required for Rosetta support." + echo "Current version: $(podman --version)" + echo "Update with: brew upgrade podman" + exit 1 + fi + + echo "Podman version: $(podman --version)" +} + +# Check if Podman machine exists +machine_exists() { + podman machine list --format "{{.Name}}" 2>/dev/null | grep -q "^podman-machine-default$" +} + +# Check if Podman machine is running +machine_running() { + podman machine list --format "{{.Name}} {{.Running}}" 2>/dev/null | grep -q "^podman-machine-default true$" +} + +# Stop Podman machine if running and force flag is set +stop_machine_if_needed() { + if machine_running; then + if [[ "$FORCE_RESTART" == "true" ]]; then + echo "==> Stopping existing Podman machine..." + podman machine stop + else + echo "Podman machine is already running." + echo "Use --force to stop and restart with Rosetta configuration." + + # Check if Rosetta is already enabled + if podman machine ssh "cat /proc/sys/fs/binfmt_misc/rosetta" &>/dev/null; then + echo "Rosetta emulation is already enabled." + exit 0 + else + echo "Warning: Current Podman machine is NOT using Rosetta." + echo "Run with --force to restart with Rosetta support." + exit 1 + fi + fi + fi +} + +# Initialize Podman machine if it doesn't exist +init_machine() { + if ! machine_exists; then + echo "==> Initializing Podman machine..." + echo " CPU: ${CPU}" + echo " Memory: ${MEMORY}MB" + echo " Disk: ${DISK}GB" + echo "" + + podman machine init \ + --cpus "${CPU}" \ + --memory "${MEMORY}" \ + --disk-size "${DISK}" + fi +} + +# Start Podman machine +start_machine() { + echo "==> Starting Podman machine..." + podman machine start +} + +# Enable Rosetta in the Podman machine +# Based on: https://blog.podman.io/2025/08/podman-5-6-released-rosetta-status-update/ +enable_rosetta() { + echo "==> Enabling Rosetta emulation..." + + # Create the Rosetta enablement file + podman machine ssh "sudo touch /etc/containers/enable-rosetta" + + echo "==> Restarting Podman machine to apply Rosetta configuration..." + podman machine stop + podman machine start +} + +# Verify Rosetta is working +verify_rosetta() { + echo "==> Verifying Rosetta emulation..." + + # Check if Rosetta binfmt is registered + if podman machine ssh "cat /proc/sys/fs/binfmt_misc/rosetta" &>/dev/null; then + echo " Rosetta binfmt handler: REGISTERED" + else + echo " Warning: Rosetta binfmt handler not found" + echo " Available handlers:" + podman machine ssh "ls /proc/sys/fs/binfmt_misc/" + return 1 + fi + + # Test running an x86_64 binary + echo "==> Testing x86_64 emulation..." + if podman run --rm --platform linux/amd64 alpine:latest uname -m 2>/dev/null | grep -q x86_64; then + echo " x86_64 container test: PASSED" + else + echo " Warning: x86_64 container test failed" + fi +} + +# Print summary +print_summary() { + echo "" + echo "==============================================" + echo "Podman started with Rosetta emulation" + echo "==============================================" + echo "" + podman machine list + echo "" + echo "Podman is ready. You can now run:" + echo " go run ./cmd/ibosh/main.go docker start" + echo "" + echo "Or use podman directly:" + echo " podman run --rm --platform linux/amd64 ghcr.io/rkoster/instant-bosh" + echo "" +} + +# Main +main() { + echo "==============================================" + echo "Starting Podman with Rosetta for Apple Silicon" + echo "==============================================" + echo "" + + check_apple_silicon + check_macos_version + check_rosetta + check_podman + stop_machine_if_needed + init_machine + start_machine + enable_rosetta + verify_rosetta + print_summary +} + +main "$@" diff --git a/apple-silicon/stemcell/Dockerfile b/apple-silicon/stemcell/Dockerfile new file mode 100644 index 0000000..9c53080 --- /dev/null +++ b/apple-silicon/stemcell/Dockerfile @@ -0,0 +1,66 @@ +# Dockerfile for patching upstream BOSH stemcell images for Apple Silicon +# +# This patches the CloudFoundry stemcell Docker images to work under Rosetta +# x86_64 emulation on Apple Silicon Macs. +# +# The main issue is that systemd services with MemoryDenyWriteExecute=yes +# fail under Rosetta because Rosetta uses JIT compilation which requires +# writable+executable memory. +# +# Usage: +# docker build --build-arg STEMCELL_IMAGE=ghcr.io/cloudfoundry/ubuntu-jammy-stemcell:1.586 \ +# -t my-patched-stemcell:1.586 . +# +# Or use the build.sh script: +# ./build.sh ubuntu-jammy 1.586 + +ARG STEMCELL_IMAGE=ghcr.io/cloudfoundry/ubuntu-noble-stemcell:latest +FROM ${STEMCELL_IMAGE} + +LABEL maintainer="instant-bosh" +LABEL description="BOSH stemcell patched for Apple Silicon (Rosetta x86_64 emulation)" + +# Create systemd drop-in overrides to disable security features that conflict +# with Rosetta's JIT compilation. Rosetta needs writable+executable memory +# for its binary translation, which conflicts with MemoryDenyWriteExecute=yes. +# +# Services that need patching: +# - systemd-journald: Journal/logging service +# - systemd-resolved: DNS resolver +# - systemd-networkd: Network configuration (may be used) +# - systemd-logind: Login manager +# - systemd-timesyncd: Time synchronization +# - auditd: Audit daemon (required by bosh-agent bootstrap) + +# Create a common drop-in that can be applied to multiple services +RUN mkdir -p /etc/systemd/system/systemd-journald.service.d \ + /etc/systemd/system/systemd-resolved.service.d \ + /etc/systemd/system/systemd-networkd.service.d \ + /etc/systemd/system/systemd-logind.service.d \ + /etc/systemd/system/systemd-timesyncd.service.d \ + /etc/systemd/system/auditd.service.d + +# Rosetta compatibility: disable security features that conflict with JIT +# Write the config content to a temp file, then copy to each service directory +RUN printf '[Service]\n\ +# Disable security features that conflict with Rosetta x86_64 emulation\n\ +# Rosetta uses JIT compilation which requires writable+executable memory\n\ +MemoryDenyWriteExecute=no\n\ +SystemCallArchitectures=\n\ +SystemCallFilter=\n\ +LockPersonality=no\n\ +NoNewPrivileges=no\n' > /tmp/rosetta.conf && \ + cp /tmp/rosetta.conf /etc/systemd/system/systemd-journald.service.d/rosetta.conf && \ + cp /tmp/rosetta.conf /etc/systemd/system/systemd-resolved.service.d/rosetta.conf && \ + cp /tmp/rosetta.conf /etc/systemd/system/systemd-networkd.service.d/rosetta.conf && \ + cp /tmp/rosetta.conf /etc/systemd/system/systemd-logind.service.d/rosetta.conf && \ + cp /tmp/rosetta.conf /etc/systemd/system/systemd-timesyncd.service.d/rosetta.conf && \ + cp /tmp/rosetta.conf /etc/systemd/system/auditd.service.d/rosetta.conf && \ + rm /tmp/rosetta.conf + +# Also mask systemd-binfmt which fails under Rosetta (fixed upstream in PR #473) +RUN systemctl mask systemd-binfmt.service || true + +# Verify the drop-ins were created +RUN cat /etc/systemd/system/systemd-journald.service.d/rosetta.conf +RUN cat /etc/systemd/system/systemd-resolved.service.d/rosetta.conf diff --git a/apple-silicon/stemcell/build.sh b/apple-silicon/stemcell/build.sh new file mode 100755 index 0000000..d4f75b6 --- /dev/null +++ b/apple-silicon/stemcell/build.sh @@ -0,0 +1,75 @@ +#!/bin/bash +# Build a patched stemcell image for Apple Silicon (Rosetta x86_64 emulation) +# +# This script builds a Docker image from the upstream CloudFoundry stemcell +# with patches applied to work around Rosetta emulation issues. +# +# Usage: +# ./build.sh # Build ubuntu-noble:latest +# ./build.sh ubuntu-noble # Build ubuntu-noble:latest +# ./build.sh ubuntu-noble 1.586 # Build ubuntu-noble:1.586 +# ./build.sh ubuntu-jammy 1.586 # Build ubuntu-jammy:1.586 +# +# Environment variables: +# REGISTRY - Registry to push to (default: ghcr.io/rkoster) +# Set to your own registry namespace +# +# The BOSH Docker CPI needs to pull the image, so it must be pushed to a +# registry accessible from inside the instant-bosh container. + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +# Default values +OS="${1:-ubuntu-noble}" +VERSION="${2:-latest}" + +# Registry - default to rkoster's namespace, override with REGISTRY env var +REGISTRY="${REGISTRY:-ghcr.io/rkoster}" + +# Upstream image reference +UPSTREAM_IMAGE="ghcr.io/cloudfoundry/${OS}-stemcell:${VERSION}" + +# Output image - use custom registry with apple-silicon suffix +OUTPUT_TAG="${VERSION}-apple-silicon" +OUTPUT_IMAGE="${REGISTRY}/${OS}-stemcell:${OUTPUT_TAG}" + +echo "==============================================" +echo "Building Patched Stemcell for Apple Silicon" +echo "==============================================" +echo "" +echo "Upstream image: ${UPSTREAM_IMAGE}" +echo "Output image: ${OUTPUT_IMAGE}" +echo "" + +# Build the image +# Use --provenance=false to avoid creating a multi-arch manifest with attestations +# This ensures the image can be pulled without specifying --platform +echo "Building image..." +docker build \ + --platform linux/amd64 \ + --provenance=false \ + --build-arg "STEMCELL_IMAGE=${UPSTREAM_IMAGE}" \ + -t "${OUTPUT_IMAGE}" \ + -f "${SCRIPT_DIR}/Dockerfile" \ + "${SCRIPT_DIR}" + +echo "" +echo "==============================================" +echo "Build Complete!" +echo "==============================================" +echo "" +echo "Tagged as: ${OUTPUT_IMAGE}" +echo "" +echo "Next steps:" +echo "" +echo " 1. Push to registry (required for BOSH CPI to access):" +echo " docker push ${OUTPUT_IMAGE}" +echo "" +echo " 2. Upload to BOSH:" +echo " ibosh upload-stemcell ${OUTPUT_IMAGE}" +echo "" +echo "Note: The BOSH Docker CPI pulls images from inside the container," +echo "so the image must be in a registry, not just local." +echo ""