diff --git a/.gitattributes b/.gitattributes index e0d56685a954..47d6ced91289 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1 +1,2 @@ test-requirements.txt merge=union +docsource/modules180-190.rst merge=union diff --git a/.github/workflows/aggregate.yml b/.github/workflows/aggregate.yml new file mode 100644 index 000000000000..0916814e90c1 --- /dev/null +++ b/.github/workflows/aggregate.yml @@ -0,0 +1,165 @@ +name: Aggregate fork branches + +# Builds the throwaway `aggregated` branch = OCA/19.0 + every branch in the +# merge manifest, then force-pushes it. The build image and the lab's +# `make migrate` consume `aggregated`. +# +# SINGLE SOURCE OF TRUTH for the merge list: ledoent/openupgrade-lab `repos.yaml` +# (the `./openupgrade` block). It is NOT duplicated here — this workflow checks +# out the lab repo and reads it. The OpenUpgrade fork's working/migration +# branches stay clean of aggregation config (they are based on pristine +# OCA/19.0 and are upstream-ready). To change which branches aggregate, edit +# `repos.yaml` in the lab repo (and run `make check-fork-model`). See +# `docs/fork-aggregation-model.md` in the lab repo. + +on: + push: + branches: + - ledoent + - "19.0-mig-*" + - "19.0-fix-*" + schedule: + # Weekly, Sunday 06:00 UTC — one hour after mirror-upstream, deliberately. + # + # The ordering is the point, not the hour: pushes to clean migration + # branches don't carry this workflow file so they never fire the push + # trigger, and an upstream OCA merge can silently start conflicting with a + # repos.yaml branch (seen with l10n_es / OCA#5646). A scheduled rebuild + # after the mirror surfaces that instead of leaving it for the next manual + # push. Keep this cron strictly later than mirror-upstream's. + # + # Was daily; moved to weekly with the mirror when the fork stopped being an + # active contribution target. The detection window widens from a day to a + # week — acceptable now, and the reason is org-wide CI concurrency rather + # than anything about this workflow. See mirror-upstream.yml. + - cron: "0 6 * * 0" + workflow_dispatch: + +permissions: + contents: write + +jobs: + aggregate: + runs-on: ubuntu-latest + steps: + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: "3.10" + + - name: Install git-aggregator + run: pip install git-aggregator==4.1 + + - name: Configure git identity (gitaggregate needs this before merging) + run: | + git config --global user.name "github-actions[bot]" + git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Checkout lab repo (single source of the merge manifest) + # The aggregate merge list lives ONLY in the lab repo's repos.yaml so the + # OpenUpgrade fork's working/migration branches stay clean of any + # aggregation config (they are based on pristine OCA/19.0). This workflow + # reads that committed repos.yaml — it does not carry its own copy. + uses: actions/checkout@v4 + with: + repository: ledoent/openupgrade-lab + ref: main + path: lab + fetch-depth: 1 + # openupgrade-lab is private; the default GITHUB_TOKEN is scoped to THIS + # repo only and gets "Repository not found". LAB_READ_TOKEN is a + # fine-grained PAT with READ access to ledoent/openupgrade-lab. + # Create it once: + # gh secret set LAB_READ_TOKEN --repo ledoent/OpenUpgrade + token: ${{ secrets.LAB_READ_TOKEN }} + # persist-credentials defaults to true, which makes checkout configure + # the token for PUSH and fail a read-only PAT with 403 "Write access + # not granted". We only read repos.yaml, so turn it off. + persist-credentials: false + + - name: Extract the ./openupgrade block into the aggregate config + # gitaggregate wants a single-repo config; pull just the ./openupgrade + # top-level key out of the lab's multi-repo repos.yaml. + run: | + python3 - <<'PY' + import yaml + full = yaml.safe_load(open("lab/repos.yaml")) + block = full["./openupgrade"] + yaml.safe_dump({"./openupgrade": block}, open("aggregate.yml", "w"), + default_flow_style=False, sort_keys=False) + print(open("aggregate.yml").read()) + PY + + - name: Run gitaggregate + run: gitaggregate -c aggregate.yml + + - name: Dedup docsource module rows + # Each merged migration branch edits docsource/modules180-190.rst to mark + # its modules, so the same module's row recurs once per branch that touches + # it. gitaggregate replays those edits verbatim, leaving duplicate rows that + # make the coverage table unreadable and let a blank row mask a marked one. + # Collapse to first occurrence per module, preferring a marked row over a + # blank, and commit so the force-pushed aggregated tree carries the clean + # table. (Per-branch diffs stay untouched — this lives only on aggregated.) + working-directory: openupgrade + run: | + python3 - <<'PY' + import re + path = "docsource/modules180-190.rst" + row_re = re.compile(r"^\|\s+([a-z][\w.]+)\s+\|([^|]*)\|") + lines = open(path, encoding="utf-8").read().splitlines(keepends=True) + pos, out, dropped = {}, [], 0 + for ln in lines: + m = row_re.match(ln) + if not m: + out.append(ln); continue + mod, col2 = m.group(1), m.group(2).strip() + if mod not in pos: + pos[mod] = len(out); out.append(ln) + else: + dropped += 1 + if not row_re.match(out[pos[mod]]).group(2).strip() and col2: + out[pos[mod]] = ln + open(path, "w", encoding="utf-8").writelines(out) + print(f"dedup: dropped {dropped} duplicate module row(s)") + PY + if ! git diff --quiet -- docsource/modules180-190.rst; then + git commit -am "[CI] aggregate: dedup docsource module rows" + else + echo "no duplicate rows to collapse" + fi + + - name: Force-push aggregated + id: push + working-directory: openupgrade + env: + PUSH_TOKEN: ${{ secrets.AGGREGATE_PUSH_TOKEN || secrets.GITHUB_TOKEN }} + run: | + git remote set-url ledoent "https://x-access-token:${PUSH_TOKEN}@github.com/ledoent/OpenUpgrade.git" + git push --force ledoent HEAD:refs/heads/aggregated + sha=$(git rev-parse HEAD) + echo "Aggregated head: $sha" + echo "sha=$sha" >> "$GITHUB_OUTPUT" + + - name: Trigger build-image workflow + # GITHUB_TOKEN-driven branch pushes don't fire downstream workflows + # (loop protection). Use repository_dispatch with a PAT so build-image + # can react. Falls back to no-op if AGGREGATE_PUSH_TOKEN is unset. + # Uses curl, not the gh CLI: the self-hosted runner pool doesn't ship + # gh (the gh api call failed with "command not found"); curl is portable. + env: + DISPATCH_TOKEN: ${{ secrets.AGGREGATE_PUSH_TOKEN }} + SHA: ${{ steps.push.outputs.sha }} + run: | + if [ -z "$DISPATCH_TOKEN" ]; then + echo "AGGREGATE_PUSH_TOKEN not set; skipping repository_dispatch." + echo "Run the build-image workflow manually against the aggregated branch." + exit 0 + fi + curl -fsS -X POST \ + -H "Authorization: Bearer $DISPATCH_TOKEN" \ + -H "Accept: application/vnd.github+json" \ + -H "X-GitHub-Api-Version: 2022-11-28" \ + "https://api.github.com/repos/${{ github.repository }}/dispatches" \ + -d "{\"event_type\":\"aggregated-updated\",\"client_payload\":{\"sha\":\"$SHA\"}}" + echo "Fired repository_dispatch event_type=aggregated-updated" diff --git a/.github/workflows/build-image.yml b/.github/workflows/build-image.yml new file mode 100644 index 000000000000..3b684ed68606 --- /dev/null +++ b/.github/workflows/build-image.yml @@ -0,0 +1,58 @@ +name: Build openupgrade image + +# Builds an Odoo 19 image with the aggregated OpenUpgrade tree baked in +# and pushes to the ledoent Zot registry at registry.hz.ledoweb.com. +# +# Auth: basic-auth via the `robot-ci` Zot user. +# Secrets: ZOT_USERNAME (= robot-ci), ZOT_PASSWORD (set on this repo). +# +# Image: registry.hz.ledoweb.com/openupgrade/openupgrade +# Consumed by: openupgrade-lab/docker-compose.yml (odoo-19 service) + +on: + push: + branches: + - aggregated + repository_dispatch: + types: [aggregated-updated] + workflow_dispatch: + +concurrency: + group: build-image-${{ github.ref }} + cancel-in-progress: true + +env: + ZOT_HOST: registry.hz.ledoweb.com + ZOT_IMAGE: registry.hz.ledoweb.com/openupgrade/openupgrade + +jobs: + build: + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - uses: actions/checkout@v4 + with: + ref: aggregated + fetch-depth: 1 + + - uses: docker/login-action@v3 + with: + registry: ${{ env.ZOT_HOST }} + username: ${{ secrets.ZOT_USERNAME }} + password: ${{ secrets.ZOT_PASSWORD }} + + - uses: docker/setup-buildx-action@v3 + + - name: Build and push (amd64 only — cluster is amd64; M-series uses Rosetta) + uses: docker/build-push-action@v6 + with: + context: . + file: ./Dockerfile.openupgrade + push: true + platforms: linux/amd64 + tags: | + ${{ env.ZOT_IMAGE }}:latest + ${{ env.ZOT_IMAGE }}:${{ github.sha }} + cache-from: type=registry,ref=${{ env.ZOT_IMAGE }}:buildcache + cache-to: type=registry,ref=${{ env.ZOT_IMAGE }}:buildcache,mode=max diff --git a/.github/workflows/documentation-commit.yml b/.github/workflows/documentation-commit.yml index c7d445ee6521..84842b18ecf6 100644 --- a/.github/workflows/documentation-commit.yml +++ b/.github/workflows/documentation-commit.yml @@ -8,10 +8,15 @@ name: Build and commit documentation on: push: + branches: ["19.0"] paths: ["docsource/modules180-190.rst"] jobs: documentation-commit: + # Docs publishing only makes sense on OCA upstream (default branch + # publishes to https://oca.github.io/OpenUpgrade/). Forks don't have + # a `documentation` branch — checkout would fail. Restrict to OCA. + if: ${{ github.repository_owner == 'OCA' }} runs-on: ubuntu-latest steps: - name: Check out OpenUpgrade Documentation diff --git a/.github/workflows/mirror-upstream.yml b/.github/workflows/mirror-upstream.yml new file mode 100644 index 000000000000..cb532dd6e2ee --- /dev/null +++ b/.github/workflows/mirror-upstream.yml @@ -0,0 +1,80 @@ +name: Mirror upstream OCA/OpenUpgrade + +on: + schedule: + # Weekly, Sunday 05:00 UTC (01:00 US Eastern). + # + # This ran every 6h from 2026-07-13, because Tecnativa-pace upstream + # outran a daily tick and open PRs drifted from their base. That tempo + # was worth it while this fork was an active contribution target. It no + # longer is, and the cost is paid by the whole organisation: mirroring + # force-pushes 19.0, which fires aggregate.yml's push trigger, which + # rebuilds and force-pushes `aggregated`, which runs tests, pre-commit + # and the migration suite. Four ticks a day of that saturates the + # org-wide 20-concurrent-job cap on the Free plan and leaves unrelated + # repositories queueing for tens of minutes. + # + # If contribution restarts, raise this again — but prefer moving the + # heavy jobs onto the self-hosted runners first. + # + # workflow_dispatch below is the escape hatch for a manual catch-up. + - cron: "0 5 * * 0" + workflow_dispatch: + +permissions: + contents: write + issues: write + +jobs: + mirror: + runs-on: ubuntu-latest + steps: + - name: Checkout fork + uses: actions/checkout@v4 + with: + ref: ledoent + fetch-depth: 0 + token: ${{ secrets.GIT_PUSH_TOKEN || secrets.GITHUB_TOKEN }} + + - name: Fetch upstream + run: | + git remote add upstream https://github.com/OCA/OpenUpgrade.git + git fetch upstream 19.0 + + - name: Force-push upstream/19.0 to fork's 19.0 + run: | + git push origin "upstream/19.0:refs/heads/19.0" --force-with-lease || \ + git push origin "upstream/19.0:refs/heads/19.0" --force + + - name: Check whether ledoent has drifted from upstream/19.0 + id: drift + run: | + # ledoent should be upstream/19.0 + custom-CI commits. If a merge-base + # comparison shows ledoent missing upstream commits, we need a rebase. + if git merge-base --is-ancestor upstream/19.0 ledoent; then + echo "drift=no" >> "$GITHUB_OUTPUT" + echo "ledoent is up to date with upstream/19.0" + else + echo "drift=yes" >> "$GITHUB_OUTPUT" + behind=$(git rev-list --count ledoent..upstream/19.0) + echo "ledoent is behind upstream/19.0 by $behind commits — rebase needed" + fi + + - name: Open issue if drift detected + if: steps.drift.outputs.drift == 'yes' + continue-on-error: true + uses: actions/github-script@v7 + with: + script: | + const { owner, repo } = context.repo; + const title = 'ledoent branch needs rebase onto upstream/19.0'; + const existing = await github.rest.issues.listForRepo({ + owner, repo, labels: 'mirror-drift', state: 'open' + }); + if (existing.data.length === 0) { + await github.rest.issues.create({ + owner, repo, title, + labels: ['mirror-drift'], + body: 'upstream OCA/OpenUpgrade `19.0` has advanced. Rebase `ledoent` onto it and force-push.\n\n```bash\ngit fetch origin\ngit checkout ledoent\ngit rebase origin/19.0\ngit push -f origin ledoent\n```' + }); + } diff --git a/.github/workflows/test-migration-enriched.yml b/.github/workflows/test-migration-enriched.yml new file mode 100644 index 000000000000..df5f9af483cd --- /dev/null +++ b/.github/workflows/test-migration-enriched.yml @@ -0,0 +1,218 @@ +# Fork-only variant of test-migration.yml that exercises the migration +# against the enriched 18.0 seed (OCA's 18.0.psql + populate-factory +# volume + curated edge cases from scripts/seed-edge-cases.py in the +# lab repo). Built and uploaded by scripts/build-fork-seed.sh. +# +# Catches migration regressions on edge-case data that OCA's vanilla +# demo seed doesn't contain — VATEX_ legacy codes, crm.stage with +# team_id set, archived parents, etc. +# +# Runs in parallel with `test-migration.yml` so a green build means +# both the upstream baseline AND our enriched seed migrate cleanly. + +name: Test OpenUpgrade migration (enriched seed) + +on: + push: + branches: + - "19.0" + - "19.0-ocabot-*" + - "19.0-mig-*" + - "19.0-fix-*" + - "aggregated" + - "ledoent" + +jobs: + test: + # Fork-only: OCA upstream doesn't have our enriched seed. + if: ${{ github.repository_owner == 'ledoent' }} + runs-on: ubuntu-22.04 + env: + DB: "openupgrade" + DB_HOST: "localhost" + DB_PASSWORD: "odoo" + DB_PORT: 5432 + DB_USERNAME: "odoo" + # Enriched seed lives on ledoent/OpenUpgrade `databases` release. + DOWNLOADS: https://github.com/ledoent/OpenUpgrade/releases/download/databases + SEED_NAME: 18.0-ledoent.psql + ODOO: "./odoo/odoo-bin" + PGHOST: "localhost" + PGPASSWORD: "odoo" + PGUSER: "odoo" + OPENUPGRADE_USE_DEMO: "yes" + services: + postgres: + # PG16 to match the lab's local postgres used by build-fork-seed.sh. + # PG14 in OCA's test-migration.yml can't restore custom-format dumps + # written by PG16 (file header version 1.15 vs max-supported 1.14). + image: postgres:16 + env: + POSTGRES_USER: odoo + POSTGRES_PASSWORD: odoo + POSTGRES_DB: odoo + ports: + - 5432:5432 + steps: + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.10' + - name: Sleep for 10 seconds + run: sleep 10s + - name: Provision and restore the enriched migration DB + # Install client, raise the lock limit, create + restore in ONE step: + # some "oca forks" self-hosted runners use a fresh container per `run:`, + # so a postgresql-client installed in an earlier step is gone by + # `createdb` (exit 127, "command not found"). Keeping install → createdb + # → restore together guarantees the client is present where it's used. + # NB: PG16 bin must go on PATH in-step (export, not $GITHUB_PATH, which + # only affects *later* steps) since pg_restore must read PG16 dumps. + run: | + # Ubuntu 22.04 ships pg_restore v14 — can't read PG16 custom-format + # dumps from the lab's local postgres:16. Install matching client. + sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list' + wget -q -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add - + sudo apt-get update -qq + sudo apt-get install -y postgresql-client-16 + export PATH="/usr/lib/postgresql/16/bin:$PATH" + # the 226-module _process_end unlink sweep exhausts the default 64 + # max_locks_per_transaction ("out of shared memory" at the end of an + # otherwise complete migration) + psql -d postgres -c "ALTER SYSTEM SET max_locks_per_transaction = 1024" + docker restart ${{ job.services.postgres.id }} + until pg_isready -h localhost -U odoo; do sleep 1; done + createdb $DB + wget -q -O- $DOWNLOADS/$SEED_NAME | pg_restore -d $DB --no-owner + psql $DB -c "UPDATE ir_module_module SET demo=False" + # cloud_storage_google (installed in this seed) gains a hard google-auth + # external dependency in 19.0, and google-auth's cryptography>=38 is + # incompatible with Odoo 18's pinned pyopenssl/urllib3 stack (it breaks + # `import OpenSSL`, so base won't load). Nothing depends on it, so skip it: + # mark uninstalled pre-migration so OpenUpgrade doesn't load/upgrade it. + # Its 18->19 change needs live Google config to exercise anyway. + psql $DB -c "UPDATE ir_module_module SET state='uninstalled' WHERE name = 'cloud_storage_google'" + - name: Check out Odoo + uses: actions/checkout@v4 + with: + repository: odoo/odoo + ref: "19.0" + fetch-depth: 1 + path: odoo + - name: Check out previous Odoo + uses: actions/checkout@v4 + with: + repository: odoo/odoo + ref: "18.0" + fetch-depth: 1 + path: odoo-old + - name: Check out OpenUpgrade + uses: actions/checkout@v4 + with: + path: openupgrade + - name: Configuration + run: | + sudo apt update + sudo apt install \ + expect \ + expect-dev \ + libevent-dev \ + libldap2-dev \ + libsasl2-dev \ + libxml2-dev \ + libxslt1-dev \ + nodejs \ + python3-lxml \ + python3-passlib \ + python3-psycopg2 \ + python3-serial \ + python3-simplejson \ + python3-werkzeug \ + python3-yaml \ + unixodbc-dev + - name: Requirements Installation + run: | + sed -i -E "s/(gevent==)21\.8\.0( ; sys_platform != 'win32' and python_version == '3.10')/\122.10.2\2/;s/(greenlet==)1.1.2( ; sys_platform != 'win32' and python_version == '3.10')/\12.0.2\2/" odoo/requirements.txt + pip install -q -r odoo/requirements.txt + # Workaround for an upstream core bug (mirrors test-migration.yml): + # l10n_es_edi_verifactu's models/certificate.py does `_inherit = + # 'certificate.certificate'` but its manifest only declares + # `depends: ['l10n_es']`. The enriched seed force-updates verifactu + # (it joins MODULES_NEW), so the registry build dies with + # `Model 'certificate.certificate' does not exist in registry`. Add + # the missing manifest dependency on the fresh odoo checkout. + # Submitted upstream as odoo/odoo#271120; no-op once 19.0 has it. + vfm=$(find odoo -path '*/l10n_es_edi_verifactu/__manifest__.py' | head -1) + [ -n "$vfm" ] && sed -i "s/'depends': \['l10n_es'\],/'depends': ['l10n_es', 'certificate'],/" "$vfm" + pip install -r ./openupgrade/requirements.txt + pip install -U git+https://github.com/oca/openupgradelib + # this is for v18 l10n_eg_edi_eta which crashes without it + pip install asn1crypto + # required by v18 + pip install decorator + pip install coverage + # this is for account_peppol + pip install phonenumbers + # NB: google-auth is intentionally NOT installed here. It needs + # cryptography>=38, which evicts Odoo's pinned cryptography 3.4.8 and + # breaks the matched pyopenssl 21 / urllib3 1.26 stack (base won't + # import). cloud_storage_google/google_gmail (enriched seed only) need + # it — handle their dep without disturbing the base crypto stack. + pip install geoip2 + - name: Test data + run: | + if test -n "$(ls openupgrade/openupgrade_scripts/scripts/*/tests/data*.py 2> /dev/null)"; then + for snippet in openupgrade/openupgrade_scripts/scripts/*/tests/data*.py; do + odoo-old/odoo-bin shell -d $DB < $snippet + done + fi + - name: OpenUpgrade test (enriched) + id: run_migration + run: | + # select modules and perform the upgrade + MODULES_OLD=$(\ + sed -n '/^+========/,$p' \ + openupgrade/docsource/modules180-190.rst \ + | grep "Done\|Partial\|Nothing" \ + | grep -v "theme_" \ + | sed -rn 's/((^\| *\|del\| *)|^\| *)([0-9a-z_]*)[ \|].*/\3/g p' \ + | sed '/^\s*$/d' \ + | paste -d, -s) + MODULES_NEW=$(\ + sed -n '/^+========/,$p' \ + openupgrade/docsource/modules180-190.rst \ + | grep "Done\|Partial\|Nothing" \ + | grep -v "theme_" \ + | sed -rn 's/((^\| *\|new\| *)|^\| *)([0-9a-z_]*)[ \|].*/\3/g p' \ + | sed '/^\s*$/d' \ + | paste -d, -s) + echo "modules_old=$MODULES_OLD" >> $GITHUB_OUTPUT + echo "modules_new=$MODULES_NEW" >> $GITHUB_OUTPUT + if [ -z "$MODULES_NEW" ]; then + echo "No modules to test yet" + exit + fi + REQUEST="update ir_module_module set state='uninstalled' \ + where name not in ('$(echo $MODULES_OLD | sed -e "s/,/','/g")')" + echo Set the modules as not installable if they are not in the following list : $MODULES_OLD + echo Running $REQUEST + psql $DB -c "$REQUEST" + ADDONS_PATHS="\ + $GITHUB_WORKSPACE/odoo/addons \ + $GITHUB_WORKSPACE/odoo/odoo/addons \ + $GITHUB_WORKSPACE/openupgrade" + echo Execution of Openupgrade with the update of the following modules : $MODULES_NEW + $ODOO \ + --addons-path=`echo $ADDONS_PATHS | awk -v OFS="," '$1=$1'` \ + --database=$DB \ + --db_host=$DB_HOST \ + --db_password=$DB_PASSWORD \ + --db_port=$DB_PORT \ + --db_user=$DB_USERNAME \ + --load=base,web,openupgrade_framework \ + --test-enable \ + --test-tags openupgrade \ + --log-handler odoo.models.unlink:WARNING \ + --stop-after-init \ + --without-demo=$MODULES_NEW \ + --update=$MODULES_NEW diff --git a/.github/workflows/test-migration-real-oca.yml b/.github/workflows/test-migration-real-oca.yml new file mode 100644 index 000000000000..d6a60d59abc6 --- /dev/null +++ b/.github/workflows/test-migration-real-oca.yml @@ -0,0 +1,241 @@ +# MANUAL-ONLY variant of test-migration.yml. Runs the migration against +# the realistic SMB + OCA-stacked seed produced by: +# 1. scripts/install-targeted-modules.sh seed_18_woodbimble_real +# (25 curated CE modules — sale/purchase/stock/mrp/account/POS/HR + l10n_us) +# 2. scripts/seed-multicompany-orm.py (Wood + Showroom branch + Bimble) +# 3. scripts/seed-stock-topology.py (3-step receipt + 2-step ship on Wood, +# 1-step on Showroom, transit locations) +# 4. scripts/seed-operations-volume.py (96 SOs + 72 POs + 38 MOs + chatter) +# 5. scripts/clone-oca-18.sh + odoo -i for OCA modules +# 6. scripts/seed-oca-assets-rma.py (25 fixed assets + 150 depreciation +# lines + RMA cases on Wood; mis_builder + account_reconcile_oca +# + account_lock_date_update installed) +# +# Result: 18.0-ledoent-real-oca.psql (7.3 MB dump, ~50 MB restored). +# 137 modules — what real Ledo SMB customers run, not the 631-module +# CE-all/115-locale-demo bloat the previous mc workflow used. +# +# Workflow_dispatch only — no push trigger. Use on-demand: +# gh workflow run test-migration-real-oca.yml \ +# --repo ledoent/OpenUpgrade --ref +# +# Auto-CI still runs test-migration.yml (OCA baseline) + +# test-migration-enriched.yml (single-company edge cases). This one +# is the prod-confidence signal — run before any Ledo prod migration +# commitment or any OCA PR review where multi-company / asset +# depreciation / RMA / multi-step warehouse depth matters. + +name: Test OpenUpgrade migration (real SMB + OCA — manual) + +on: + workflow_dispatch: + inputs: + ref: + description: 'Branch to test (default: ledoent)' + required: false + default: 'ledoent' + +jobs: + test: + # Fork-only: OCA upstream doesn't have our enriched seed. + if: ${{ github.repository_owner == 'ledoent' }} + runs-on: ubuntu-22.04 + env: + DB: "openupgrade" + DB_HOST: "localhost" + DB_PASSWORD: "odoo" + DB_PORT: 5432 + DB_USERNAME: "odoo" + # Enriched seed lives on ledoent/OpenUpgrade `databases` release. + DOWNLOADS: https://github.com/ledoent/OpenUpgrade/releases/download/databases + SEED_NAME: 18.0-ledoent-real-oca.psql + ODOO: "./odoo/odoo-bin" + PGHOST: "localhost" + PGPASSWORD: "odoo" + PGUSER: "odoo" + OPENUPGRADE_USE_DEMO: "yes" + services: + postgres: + # PG16 to match the lab's local postgres used by build-fork-seed.sh. + # PG14 in OCA's test-migration.yml can't restore custom-format dumps + # written by PG16 (file header version 1.15 vs max-supported 1.14). + image: postgres:16 + env: + POSTGRES_USER: odoo + POSTGRES_PASSWORD: odoo + POSTGRES_DB: odoo + ports: + - 5432:5432 + steps: + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.10' + - name: Sleep for 10 seconds + run: sleep 10s + - name: Install postgresql-client-16 + run: | + # Ubuntu 22.04 ships pg_restore v14 — can't read PG16 custom-format + # dumps from the lab's local postgres:16. Install matching client. + sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list' + wget -q -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add - + sudo apt-get update -qq + sudo apt-get install -y postgresql-client-16 + echo "/usr/lib/postgresql/16/bin" >> $GITHUB_PATH + - name: DB Creation + run: createdb $DB + - name: DB Restore (real SMB + OCA seed) + run: | + wget -q -O- $DOWNLOADS/$SEED_NAME | pg_restore -d $DB --no-owner + psql $DB -c "UPDATE ir_module_module SET demo=False" + - name: Check out Odoo + uses: actions/checkout@v4 + with: + repository: odoo/odoo + ref: "19.0" + fetch-depth: 1 + path: odoo + - name: Check out previous Odoo + uses: actions/checkout@v4 + with: + repository: odoo/odoo + ref: "18.0" + fetch-depth: 1 + path: odoo-old + - name: Check out OpenUpgrade + uses: actions/checkout@v4 + with: + path: openupgrade + - name: Clone OCA modules at 18.0 (source) + 19.0 (target) + run: | + # The seed DB has OCA modules (account_asset_management, rma, + # mis_builder, etc.) marked installed. Odoo's registry load + # needs their manifests in addons-path or it errors with + # "Some modules are not loaded". Clone at BOTH 18.0 (so the + # source DB module records match a real install) AND 19.0 + # (so the target migration finds the upgraded modules). + REPOS_OCA="account-financial-reporting account-financial-tools \ + account-reconcile bank-statement-import mis-builder \ + reporting-engine rma server-tools server-ux" + mkdir -p oca-18 oca-19 + for repo in $REPOS_OCA; do + GIT_TERMINAL_PROMPT=0 git clone --depth=1 --branch 18.0 \ + "https://github.com/OCA/$repo.git" "oca-18/$repo" 2>&1 | tail -1 || \ + echo " WARN $repo @18.0 unavailable" + GIT_TERMINAL_PROMPT=0 git clone --depth=1 --branch 19.0 \ + "https://github.com/OCA/$repo.git" "oca-19/$repo" 2>&1 | tail -1 || \ + echo " WARN $repo @19.0 unavailable" + done + - name: Configuration + run: | + sudo apt update + sudo apt install \ + expect \ + expect-dev \ + libevent-dev \ + libldap2-dev \ + libsasl2-dev \ + libxml2-dev \ + libxslt1-dev \ + nodejs \ + python3-lxml \ + python3-passlib \ + python3-psycopg2 \ + python3-serial \ + python3-simplejson \ + python3-werkzeug \ + python3-yaml \ + unixodbc-dev + - name: Requirements Installation + run: | + sed -i -E "s/(gevent==)21\.8\.0( ; sys_platform != 'win32' and python_version == '3.10')/\122.10.2\2/;s/(greenlet==)1.1.2( ; sys_platform != 'win32' and python_version == '3.10')/\12.0.2\2/" odoo/requirements.txt + pip install -q -r odoo/requirements.txt + pip install -r ./openupgrade/requirements.txt + pip install -U git+https://github.com/oca/openupgradelib + # this is for v18 l10n_eg_edi_eta which crashes without it + pip install asn1crypto + # required by v18 + pip install decorator + pip install coverage + # this is for account_peppol + pip install phonenumbers + - name: Test data + run: | + # Pass OCA-18 addons-path so the 18.0 odoo-bin can load the + # registry with all installed modules (incl OCA ones present + # in the seed DB). Without this odoo-bin shell fails at + # registry load before any data snippet runs. + OCA_18_PATHS=$(ls -d $GITHUB_WORKSPACE/oca-18/* 2>/dev/null | tr '\n' ',') + OLD_ADDONS="odoo-old/addons,odoo-old/odoo/addons,${OCA_18_PATHS%,}" + # Filter data snippets to modules actually installed in the seed. + # The realistic SMB seed has 137 modules, not the full 500+ — + # data fixtures that env.ref records from uninstalled modules + # (like hr_expense) raise ValueError before any useful work. + INSTALLED=$(psql -At -d $DB -c \ + "SELECT name FROM ir_module_module WHERE state='installed'") + if test -n "$(ls openupgrade/openupgrade_scripts/scripts/*/tests/data*.py 2> /dev/null)"; then + for snippet in openupgrade/openupgrade_scripts/scripts/*/tests/data*.py; do + module=$(echo "$snippet" | sed -E 's|.*/scripts/([^/]+)/tests/.*|\1|') + if echo "$INSTALLED" | grep -qx "$module"; then + echo "==> $module: running $(basename $snippet)" + odoo-old/odoo-bin shell --addons-path="$OLD_ADDONS" -d $DB < $snippet + else + echo "==> $module: SKIP (not installed in this seed)" + fi + done + fi + - name: OpenUpgrade test (real SMB + OCA) + id: run_migration + run: | + # select modules and perform the upgrade + MODULES_OLD=$(\ + sed -n '/^+========/,$p' \ + openupgrade/docsource/modules180-190.rst \ + | grep "Done\|Partial\|Nothing" \ + | grep -v "theme_" \ + | sed -rn 's/((^\| *\|del\| *)|^\| *)([0-9a-z_]*)[ \|].*/\3/g p' \ + | sed '/^\s*$/d' \ + | paste -d, -s) + MODULES_NEW=$(\ + sed -n '/^+========/,$p' \ + openupgrade/docsource/modules180-190.rst \ + | grep "Done\|Partial\|Nothing" \ + | grep -v "theme_" \ + | sed -rn 's/((^\| *\|new\| *)|^\| *)([0-9a-z_]*)[ \|].*/\3/g p' \ + | sed '/^\s*$/d' \ + | paste -d, -s) + echo "modules_old=$MODULES_OLD" >> $GITHUB_OUTPUT + echo "modules_new=$MODULES_NEW" >> $GITHUB_OUTPUT + if [ -z "$MODULES_NEW" ]; then + echo "No modules to test yet" + exit + fi + REQUEST="update ir_module_module set state='uninstalled' \ + where name not in ('$(echo $MODULES_OLD | sed -e "s/,/','/g")')" + echo Set the modules as not installable if they are not in the following list : $MODULES_OLD + echo Running $REQUEST + psql $DB -c "$REQUEST" + # OCA modules at 19.0 take precedence (target version); + # 18.0 clones are referenced only for the Test data step + # via odoo-old in the next section. Glob each OCA repo as + # its own addons-path entry. + ADDONS_PATHS="\ + $GITHUB_WORKSPACE/odoo/addons \ + $GITHUB_WORKSPACE/odoo/odoo/addons \ + $GITHUB_WORKSPACE/openupgrade \ + $(ls -d $GITHUB_WORKSPACE/oca-19/* 2>/dev/null | tr '\n' ' ')" + echo Execution of Openupgrade with the update of the following modules : $MODULES_NEW + $ODOO \ + --addons-path=`echo $ADDONS_PATHS | awk -v OFS="," '$1=$1'` \ + --database=$DB \ + --db_host=$DB_HOST \ + --db_password=$DB_PASSWORD \ + --db_port=$DB_PORT \ + --db_user=$DB_USERNAME \ + --load=base,web,openupgrade_framework \ + --test-enable \ + --test-tags openupgrade \ + --log-handler odoo.models.unlink:WARNING \ + --stop-after-init \ + --without-demo=$MODULES_NEW \ + --update=$MODULES_NEW diff --git a/.github/workflows/test-migration.yml b/.github/workflows/test-migration.yml index bc425add5de4..99179412a217 100644 --- a/.github/workflows/test-migration.yml +++ b/.github/workflows/test-migration.yml @@ -13,6 +13,10 @@ on: branches: - "19.0" - "19.0-ocabot-*" + - "19.0-mig-*" + - "19.0-fix-*" + - "aggregated" + - "ledoent" jobs: test: @@ -23,7 +27,9 @@ jobs: DB_PASSWORD: "odoo" DB_PORT: 5432 DB_USERNAME: "odoo" - DOWNLOADS: https://github.com/${{github.repository}}/releases/download/databases + # Test databases live only on OCA upstream; forks reuse them so we + # don't have to mirror multi-GB psql dumps. + DOWNLOADS: https://github.com/OCA/OpenUpgrade/releases/download/databases ODOO: "./odoo/odoo-bin" PGHOST: "localhost" PGPASSWORD: "odoo" @@ -45,12 +51,27 @@ jobs: python-version: '3.10' - name: Sleep for 10 seconds run: sleep 10s - - name: DB Creation - run: createdb $DB - - name: DB Restore + - name: Provision and restore the migration DB + # Install client, raise the lock limit, create + restore in ONE step: + # some "oca forks" self-hosted runners use a fresh container per `run:`, + # so a postgresql-client installed in an earlier step is gone by + # `createdb` (exit 127, "command not found"). Keeping install → createdb + # → restore together guarantees the client is present where it's used. run: | + sudo apt-get update && sudo apt-get install -y postgresql-client wget + # the _process_end unlink sweep exhausts the default 64 + # max_locks_per_transaction (same as the enriched gate) + psql -d postgres -c "ALTER SYSTEM SET max_locks_per_transaction = 1024" + docker restart ${{ job.services.postgres.id }} + until pg_isready -h localhost -U odoo; do sleep 1; done + createdb $DB wget -q -O- $DOWNLOADS/18.0.psql | pg_restore -d $DB --no-owner psql $DB -c "UPDATE ir_module_module SET demo=False" + # cloud_storage_google gains a hard google-auth dependency in 19.0, + # and google-auth's cryptography>=38 breaks Odoo 18's pinned + # pyopenssl/urllib3 stack (same neutralization as the enriched + # gate); nothing depends on it, so skip it. + psql $DB -c "UPDATE ir_module_module SET state='uninstalled' WHERE name = 'cloud_storage_google'" - name: Check out Odoo uses: actions/checkout@v4 with: @@ -72,14 +93,29 @@ jobs: - name: Configuration run: | sudo apt update - sudo apt install \ + # The 'oca forks' self-hosted runner pool is leaner than GitHub's + # hosted ubuntu-22.04 image (which OCA upstream uses and which ships + # the build toolchain preinstalled). Install the full toolchain so + # every sdist in odoo/requirements.txt compiles: build-essential + + # python3-dev cover gcc/g++/make/Python.h; the lib*-dev headers cover + # psycopg2 (libpq), python-ldap (libldap2/libsasl2), lxml (libxml2/ + # libxslt1), cryptography (libffi/libssl), Pillow (libjpeg/zlib), + # PyYAML (libyaml). + sudo apt install -y \ + build-essential \ + python3-dev \ expect \ expect-dev \ libevent-dev \ + libffi-dev \ + libjpeg-dev \ libldap2-dev \ + libpq-dev \ libsasl2-dev \ + libssl-dev \ libxml2-dev \ libxslt1-dev \ + libyaml-dev \ nodejs \ python3-lxml \ python3-passlib \ @@ -88,7 +124,8 @@ jobs: python3-simplejson \ python3-werkzeug \ python3-yaml \ - unixodbc-dev + unixodbc-dev \ + zlib1g-dev - name: Requirements Installation run: | sed -i -E "s/(gevent==)21\.8\.0( ; sys_platform != 'win32' and python_version == '3.10')/\122.10.2\2/;s/(greenlet==)1.1.2( ; sys_platform != 'win32' and python_version == '3.10')/\12.0.2\2/" odoo/requirements.txt @@ -102,6 +139,12 @@ jobs: pip install coverage # this is for account_peppol pip install phonenumbers + # NB: google-auth is intentionally NOT installed here. It needs + # cryptography>=38, which evicts Odoo's pinned cryptography 3.4.8 and + # breaks the matched pyopenssl 21 / urllib3 1.26 stack (base won't + # import). cloud_storage_google/google_gmail (enriched seed only) need + # it — handle their dep without disturbing the base crypto stack. + pip install geoip2 - name: Test data run: | if test -n "$(ls openupgrade/openupgrade_scripts/scripts/*/tests/data*.py 2> /dev/null)"; then diff --git a/Dockerfile.openupgrade b/Dockerfile.openupgrade new file mode 100644 index 000000000000..5d946f732c28 --- /dev/null +++ b/Dockerfile.openupgrade @@ -0,0 +1,34 @@ +## Image: registry.ledoweb.com/openupgrade +## +## Built by .github/workflows/build-image.yml on every push to `aggregated`. +## Contains: +## - Odoo 19.0 base +## - Python deps the openupgrade-lab uses (s3fs, redis, sentry-sdk, etc.) +## - The full `aggregated` checkout of this repo as /opt/openupgrade +## The lab's docker-compose mounts the openupgradelib source over the +## pip-installed lib so it can hot-swap during development. + +FROM odoo:19.0 + +USER root + +# git needed for the openupgradelib install (no pinned PyPI release yet for 19.0). +RUN apt-get update && \ + apt-get install -y --no-install-recommends git && \ + rm -rf /var/lib/apt/lists/* + +RUN pip install --no-cache-dir --break-system-packages \ + fsspec>=2025.3.0 \ + s3fs \ + packaging \ + python-json-logger \ + redis \ + sentry-sdk \ + python-slugify \ + plaid-python \ + cssselect \ + "git+https://github.com/OCA/openupgradelib@master" + +COPY --chown=odoo:odoo . /opt/openupgrade + +USER odoo diff --git a/docs/ledoent-roadmap.md b/docs/ledoent-roadmap.md new file mode 100644 index 000000000000..6f599eb96b45 --- /dev/null +++ b/docs/ledoent-roadmap.md @@ -0,0 +1,159 @@ +# ledoent fork — internal roadmap + +**Scope**: this lives on the `ledoent` branch only. Not for upstream. +Captures fork-only validation work, OCA review backlog, and the +multi-tier seed plan. See `docsource/contributing.rst` for the upstream- +facing contribution guide. + +**Last update**: 2026-05-18 — scope trimmed. Seed Tier plans and +prod-readiness calibration migrated out of this doc into the lab +repo (see cross-links below). What remains here is what this fork +*alone* is the source of truth for: CI status, coverage map, +fork-test branch hygiene, unclaimed-module triage, and OCA PR +review backlog. + +Refresh after each major batch or weekly, whichever comes first. + +## Where related tracking lives (NOT in this doc) + +The OpenUpgrade fork should diverge from upstream for **code +reasons** only. Tracking docs that don't justify a code commit +inflate rebase pain, couple Ledo's cadence to OpenUpgrade-the- +project, and noise up every fork-vs-upstream diff. Migrated to: + +* **Seed composition + Tier 0/1/2/3** — + `openupgrade-lab/docs/seed-plan.md` +* **Prod-migration readiness (Kencove-scale calibration)** — + `openupgrade-lab/docs/prod-readiness-plan.md` +* **Agent-clerks regression harness roadmap** — + `odoo-agent-clerks/ROADMAP.md` +* **Migration gotchas catalog** — + `openupgrade-lab/docs/migration-gotchas.md` +* **Why this trim happened + remaining cleanup phases** — + `openupgrade-lab/docs/fork-roadmap-cleanup-plan.md` + +## Status snapshot + +| Concern | State | +|---|---| +| `ledoent` branch | rebased onto `origin/19.0@8b85f42` + 18 custom CI commits | +| `mirror-upstream` cron | **fixed** (was 403 on issues; added `issues: write` permission) | +| `test-migration` workflow | green (baseline OCA seed) | +| `test-migration-enriched` workflow | green (our `18.0-ledoent.psql` seed) | +| Fork-test branches green | 25/25 last push | +| Open OCA PRs (Ledo authored) | 5 drafts: #5633, #5634, #5635, #5636, #5637 — left as drafts per fork-only rule | + +## Coverage map (post-rebase, on `ledoent`) + +``` +309 modules with upgrade_analysis.txt (the upstream catalog) + 29 [MIG] PRs merged into origin/19.0 (~9%) + 48 with upgrade_analysis_work.txt on us + 36 with pre/post/end-migration scripts + 187 modules touched by our 25 fork branches (= what we've validated on CI) + 72 truly unclaimed — see "Unclaimed list" below +``` + +72 = 309 (catalog) − 187 (our fork) − 50 (overlap from upstream/our drafts). +71 of those 72 are non-US `l10n_*` (deliberately skipped — see +`CLAUDE.local.md` "skip non-US l10n_*" rule). **Functional unclaimed +count = 1**. + +## Fork hygiene TODO + +### Stale fork-test branches — rebase candidates (low priority) + +8 fork-test branches are `behind=5–6` vs `ledoent/aggregated` because they +predate the rollup merges. Code under test unaffected. Rebase for fresh +CI confidence only: +- `19.0-mig-test-account-trivial` +- `19.0-mig-test-auth-deps-trivial` +- `19.0-mig-test-comms-trivial` +- `19.0-mig-test-crm-delivery-sale-trivial` +- `19.0-mig-test-event-family-trivial` +- `19.0-mig-test-hr-trivial` +- `19.0-mig-test-payment-gateways` +- `19.0-mig-test-project-suite` + +## Unclaimed upstream — review list + +These have `upgrade_analysis.txt` in `origin/19.0` but **no work doc**, +**no pre/post script**, and **not touched by any of our fork branches**. +Use this list to plan the next pickup (or defer per skip-rule). + +### Functional priority (non-l10n) — 1 module +| Module | Status | Action | +|---|---|---| +| `partner_autocomplete` | analysis present, no claimant | scout — likely a Tier C annotation-only batch; check upstream PR before claiming | + +### Localizations skipped per fork rule — 71 modules +Non-US l10n_* are deliberately deferred. Pickup criterion: only when +Ledo onboards a customer in that locale. List preserved for completeness: + +`l10n_ae`, `l10n_ar`, `l10n_ar_stock`, `l10n_ar_website_sale`, `l10n_at`, +`l10n_br`, `l10n_br_website_sale`, `l10n_cd`, `l10n_ci`, `l10n_cl`, +`l10n_cr`, `l10n_cz`, `l10n_din5008_expense`, `l10n_dk_oioubl`, +`l10n_ec`, `l10n_ec_sale`, `l10n_ee`, `l10n_eg_edi_eta`, +`l10n_es_edi_facturae`, `l10n_fr`, `l10n_fr_account`, `l10n_gcc_invoice`, +`l10n_gcc_invoice_stock_account`, `l10n_gcc_pos`, `l10n_gr`, `l10n_hu`, +`l10n_id_efaktur_coretax`, `l10n_il`, `l10n_in`, `l10n_in_edi`, +`l10n_in_ewaybill`, `l10n_in_ewaybill_irn`, `l10n_in_ewaybill_stock`, +`l10n_in_hr_holidays`, `l10n_in_pos`, `l10n_in_sale`, `l10n_iq`, +`l10n_it`, `l10n_it_edi`, `l10n_jo`, `l10n_jo_edi`, `l10n_latam_base`, +`l10n_latam_check`, `l10n_latam_invoice_document`, `l10n_lk`, `l10n_ma`, +`l10n_ml`, `l10n_mx`, `l10n_my`, `l10n_my_edi`, `l10n_my_edi_pos`, +`l10n_nl`, `l10n_pe`, `l10n_ph`, `l10n_pk`, `l10n_ro_cpv_code`, +`l10n_ro_edi`, `l10n_sa`, `l10n_sa_edi`, `l10n_sg`, `l10n_si`, +`l10n_sk`, `l10n_th`, `l10n_tr_nilvera`, `l10n_tr_nilvera_edispatch`, +`l10n_tr_nilvera_einvoice`, `l10n_tr_nilvera_einvoice_extended`, +`l10n_tw`, `l10n_ug`, `l10n_uz`. + +### Likely-overlapping with `hr_recruitment` upstream PR +Already opened by hbrunn upstream as **#5612** (`19.0-hr_recruitment`). +Don't duplicate — review and watch that PR instead. + +## Open OCA PRs to review + +### Authored by Ledo (held as drafts per fork-only rule) +| # | Title | State | Note | +|---|---|---|---| +| #5633 | `[19.0][MIG] website_*`: 31 uncharted modules | draft | Validated on fork CI; held | +| #5634 | `[19.0][MIG] hr_*`: 13 simple submodules | draft | Validated on fork CI; held | +| #5635 | `[19.0][MIG] hr_*`: overtime + skills refactor | draft | Validated on fork CI; held | +| #5636 | `[19.0][MIG] event_*`: 5 simple submodules | draft | Validated on fork CI; held | +| #5637 | `[19.0][MIG] event`: slots + question m2m promotion | draft | Validated on fork CI; held | + +### Third-party open 19.0 PRs (as of 2026-05-16) + +The OCA upstream queue is quiet. **One** third-party 19.0 PR open: + +| # | Author | Title | Updated | Status | +|---|---|---|---|---| +| #5612 | hbrunn | `[19.0][MIG] hr_recruitment` | 2026-05-08 | Reviewed by us 2026-05-16; OCA CI red on stale `lift_constraints(cascade)` API; fork CI **green** on current openupgradelib master (hbrunn's own openupgradelib PR #446 added cascade support post-PR). PR needs only a rebase to re-trigger OCA CI. Multi-company branch traversal in `candidate_properties_definition` merge is a real flag worth raising. | + +Refresh by running: +```bash +gh pr list --repo OCA/OpenUpgrade --state open --search "19.0 in:title" \ + --json number,title,author,updatedAt +``` + +## CI infra reference + +| Workflow | Trigger | Purpose | +|---|---|---| +| `mirror-upstream.yml` | daily 06:00 UTC | Push `origin/19.0` → `ledoent/19.0`. Open issue if `ledoent` branch drifts. **Now has `issues: write`**. | +| `aggregate.yml` | push to `ledoent` / `19.0-fix-*` | Run gitaggregator → push `aggregated` branch. | +| `test-migration.yml` | push to `19.0-mig-*` / `19.0-fix-*` / `aggregated` / `ledoent` | Baseline OCA 18.0.psql migration test. | +| `test-migration-enriched.yml` | same | Same migration but on our enriched `18.0-ledoent.psql`. | +| `build-image.yml` | repository_dispatch from aggregate | Build & push `registry.hz.ledoweb.com/openupgrade/openupgrade:latest`. | +| `generate-analysis-cron.yml` | weekly | Refresh `upgrade_analysis.txt` files. | + +## Drift / hygiene rules + +- Re-check `ledoent` vs `origin/19.0` drift weekly. The mirror cron now + opens an issue automatically when behind. +- Don't open new OCA PRs (CLAUDE.local.md rule, set 2026-05-15). + Existing drafts stay drafts. +- Don't merge orphan-cleanup migrations (pedrobaeza policy — rejected + #5630–5632; database_cleanup handles residuals). +- Skip non-US `l10n_*` until business case appears. diff --git a/docs/proof-18-to-19-migration.md b/docs/proof-18-to-19-migration.md new file mode 100644 index 000000000000..6b5473917c57 --- /dev/null +++ b/docs/proof-18-to-19-migration.md @@ -0,0 +1,206 @@ +# Proof of work — OpenUpgrade 18.0 → 19.0 on demo + Ledo realistic SMB seed + +**Status**: 2026-05-16. This document captures the verifiable evidence +that the 18.0 → 19.0 OpenUpgrade migration works correctly for Ledoweb's +target customer profile. + +## TL;DR + +| Layer | Evidence | Result | +|---|---|---| +| Schema-level (OCA upstream demo seed) | `Test OpenUpgrade migration` fork workflow | ✅ green, 11m28s | +| Schema-level (Ledo edge-case seed) | `Test OpenUpgrade migration (enriched seed)` workflow | ✅ green, 9m03s | +| Schema-level (Ledo realistic SMB + OCA stack) | `Test OpenUpgrade migration (real SMB + OCA)` workflow | ✅ green, 5m50s | +| Behavioral AP workflow (XML-RPC) | AP-clerk migration loop, pre/post diff | ✅ NO DELTA | +| Behavioral AR workflow (XML-RPC) | AR-clerk migration loop, pre/post diff | ✅ NO DELTA | + +All five gates run against the **actual** migration image +(`registry.hz.ledoweb.com/openupgrade/openupgrade:latest`) — the same +image Ledo prod migration day will use. No mocking. + +## Schema-level proof — fork CI workflows + +Integrated test branch `ledoent/19.0-mig-test-allopen` includes: + +- `ledoent/aggregated` (origin/19.0 + our `ledoent` CI commits + all + `19.0-fix-*` rollups) — carries 6 of our 7 open OCA PRs via cherry-picks +- hbrunn's PR #5612 `hr_recruitment` cherry-picked on top — the only + third-party 19.0 PR currently open + +All three workflows fired automatically on push to that branch; all +green. See: +[run 25966950427](https://github.com/ledoent/OpenUpgrade/actions/runs/25966950427) · +[run 25966950429](https://github.com/ledoent/OpenUpgrade/actions/runs/25966950429) · +[run 25966954561 (dispatch)](https://github.com/ledoent/OpenUpgrade/actions/runs/25966954561). + +### Seeds used + +| Asset | Size | Modules | Purpose | +|---|---|---|---| +| `18.0.psql` | 88 MB | OCA-curated | Upstream baseline | +| `18.0-ledoent.psql` | 88 MB | OCA-curated + edge cases | Catches data-preservation regressions OCA's vanilla seed misses | +| `18.0-ledoent-real.psql` | 6.5 MB | 124 CE modules | Realistic Ledo SMB CE-only shape | +| `18.0-ledoent-real-oca.psql` | 7.3 MB | 137 (CE + OCA) | What Ledo prod actually runs (full OCA stack) | + +The `_real` seeds use targeted module installation (`scripts/install-targeted-modules.sh`) +instead of `--init=all` — avoids the 115-locale demo bloat that produced +1,965 picking types and 35k chart accounts on prior iterations. Resulting +DB matches realistic SMB shape: 2-3 companies, ~30 picking types, ~50 +accounts, ~150 stock moves. + +### Multi-company topology + +`scripts/seed-multicompany-orm.py` establishes: + +- **Wood Manufacturing Co.** (id=1, parent, US generic_coa) + - **Wood Co. – Showroom** (branch, `parent_id=1`, shares parent COA) +- **Bimble Design Services Co.** (separate entity, own generic_coa install) + +171 partners shared via `company_id=NULL`; admin user has +`allowed_company_ids=[1, 119, 120]`. + +## Behavioral proof — `odoo-agent-clerks` + +Private repo at `~/projects/ledoent/odoo-agent-clerks/`. Runs +deterministic per-role recipes against pre-migration + post-migration +DBs, diffs XML-RPC report snapshots, surfaces behavioral regressions +schema-level CI can't catch. + +### AP clerk loop (2026-05-16T17:29Z) + +Recipe: 3 vendor bills totaling $2,000. + +``` +## ap-clerk/ap-aging.csv ✓ 4 rows match +## ap-clerk/ap-control-balance.csv ✓ 2 rows match +## ap-clerk/bills.csv ✓ 5 rows match +## ap-clerk/payment-runs.csv ✓ 0 rows match + +Overall: ✅ NO DELTA +``` + +- Pre-migration: Wood AP balance = $2,000 credit on account `211000` +- Post-migration: same $2,000 credit, same aging bucket distribution +- AP control account, vendor names, payment_state, residual amounts: + all preserved exactly + +### AR clerk loop (2026-05-16T17:37Z) + +Recipe: 3 customer invoices totaling $2,300. + +``` +## ar-clerk/invoices.csv ✓ 4 rows match +## ar-clerk/ar-aging.csv ✓ 3 rows match +## ar-clerk/ar-control-balance.csv ✓ 3 rows match +## ar-clerk/customer-payments.csv ✓ 6 rows match + +Overall: ✅ NO DELTA +``` + +### Wall-clock budget per role + +| Step | Time | +|---|---| +| Reset 18.0 DB from canonical seed | ~5s | +| Provision demo users | ~3s | +| Recipe execution + snapshot on 18.0 | ~10s | +| Dump + restore as `agent_test_19_target` | ~30s | +| **Run OpenUpgrade migration image** | **~3.5 min** | +| Start odoo-19 service | ~10s | +| Recipe + snapshot on 19.0 | ~10s | +| Diff | <2s | +| **Total per role** | **~5–6 min** | + +## Coverage scope (honest) + +### Module coverage on the fork + +| Bucket | Count | +|---|---| +| `[19.0][MIG]` PRs merged into `origin/19.0` (all contributors) | 29 | +| Covered by our fork branches (not merged upstream) | ~158 | +| **Total with our coverage** | **~187 / 309** (~60%) | +| Deliberately skipped — non-US `l10n_*` (Ledo customers are US) | 71 | +| In flight upstream by others (hbrunn #5612 hr_recruitment) | 1 | +| Functional unclaimed (no fork branch, no upstream PR) | **1** — `partner_autocomplete` | + +### Open OCA PRs (as of 2026-05-16) + +Authored by Ledo (held as drafts per fork-only rule): + +| # | Title | Status | +|---|---|---| +| #5633 | `website_*`: 31 uncharted modules | draft, CI green on fork | +| #5634 | `hr_*`: 13 simple submodules | draft, CI green on fork | +| #5635 | `hr_*`: overtime + skills refactor (4 submodules) | draft, CI green on fork | +| #5636 | `event_*`: 5 simple submodules | draft, CI green on fork | +| #5637 | `event`: slots + question m2m promotion | draft, CI green on fork | +| #5628 | `[IMP] hr`: backfill NULL create_date/write_date | draft | + +Authored by others: + +| # | Author | Title | State | +|---|---|---|---| +| #5612 | hbrunn | `hr_recruitment` | CI red upstream on stale `lift_constraints(cascade)` API; CI green on our fork after hbrunn's openupgradelib PR #446 was merged. Needs a rebase to re-trigger OCA CI. | + +## What is NOT yet tested + +Honest gap list — these would back stronger claims if we did them, but +they're separate work: + +1. **UI navigation regression** — "does every menu still open and let + the user progress a record post-migration?" We exercised XML-RPC + recipes only. The chrome-devtools MCP UI path is Phase 5 of + `odoo-agent-clerks` (deferred). +2. **7 of 9 behavioral roles** — only AP + AR currently have recipes + and migration loops. Phase 3 of `odoo-agent-clerks` adds purchasing, + production planner, inventory planner, sales, shipping, receiving, + accounting. +3. **Parallel multi-role run** — single-role serial only. Lock + contention and race conditions during concurrent role activity + are surfaced by Phase 2. +4. **Real Kencove sanitized prod migration** — never executed. The + 30 GB Kencove DB is the actual prod-confidence test; this + document covers demo + seeded SMB only. +5. **Non-US localizations** — 71 modules deferred. Any future + non-US Ledo customer would need their locale's coverage filled in. + +## Reproducing this proof + +### Schema-level + +```bash +# Watch the most recent run on the integrated branch +gh run list --repo ledoent/OpenUpgrade --branch 19.0-mig-test-allopen --limit 3 + +# Re-trigger by pushing a no-op to ledoent or re-dispatch the manual workflow: +gh workflow run test-migration-real-oca.yml \ + --repo ledoent/OpenUpgrade --ref 19.0-mig-test-allopen +``` + +### Behavioral + +```bash +cd ~/projects/ledoent/odoo-agent-clerks +bash scripts/run-migration-loop.sh ap-clerk +bash scripts/run-migration-loop.sh ar-clerk +# Reports land in reports/migration-loop-.md +``` + +Each loop produces a `reports/diff-loop--postwork-18-vs-postwork-19.md` +file. Green means migration preserved the role's workflow output. + +## Provenance / pinning + +For exact-reproducibility across future runs: + +- Migration image: `registry.hz.ledoweb.com/openupgrade/openupgrade:latest` + (built from `ledoent/aggregated` via fork's `build-image.yml`) +- Source seed: built from `odoo:18.0` Docker image + (currently `18.0-20260513`) via `scripts/install-targeted-modules.sh` +- OCA modules at 18.0 pinned via `scripts/clone-oca-18.sh` — + commit SHAs captured at clone time (see `/tmp/erp-src/*/` on runner) + +Pinning at `:latest` is acceptable for now since we're not yet +running Ledo prod migration; pin to specific SHAs before any real +prod migration commitment. diff --git a/docs/upstream-pr-feedback.md b/docs/upstream-pr-feedback.md new file mode 100644 index 000000000000..7a9748b9d7e0 --- /dev/null +++ b/docs/upstream-pr-feedback.md @@ -0,0 +1,78 @@ +# Posting fork-CI feedback on OCA PRs + +When you've run an OCA PR through `scripts/test-upstream-pr.sh `, +paste one of these templates into the OCA PR thread. Adjust the bullets +to whatever's actually relevant for that PR's surface. + +The point of these comments is **real-data evidence**, not style review: +"your migration ran against cancelled-state moves, branch trees, and +cross-company partners — here's what happened." OCA reviewers see plenty +of `lgtm`; almost none of them see "tested against multi-company prod +shape." + +## Both fork CI jobs green + +> Ran this PR through our fork's enriched migration CI on top of the standard 18.0 seed: +> +> - **Baseline migration** ([run](LINK)) — green. +> - **Enriched migration** ([run](LINK)) — green, against `18.0-ledoent.psql` which adds: +> - 10 `account.tax` rows with legacy `VATEX_*` selection values +> - `crm.stage.team_id` populated on demo stages +> - cancelled `account.move` rows (the demo has zero) +> - archived `res.partner` + `product.template` (active=FALSE edge case) +> - `res.partner.bank.aba_routing` populated for preservation testing +> - `im_livechat.channel.rule.chatbot_only_if_no_operator = TRUE` +> - `pos.payment.method` rows with `viva_wallet_*` credentials +> - *(multi-company tier coming once `18.0-ledoent-mc.psql` lands)* +> +> Code looks good to me. LGTM for the data-preservation surface. + +## Fork CI red — actionable + +> Pulled this PR into our fork and ran it through our enriched seed CI: +> +> - **Baseline migration** ([run](LINK)) — `STATUS_HERE` +> - **Enriched migration** ([run](LINK)) — `STATUS_HERE` +> +> The enriched run hit ``. Reproducer in the log around line ``. Looks like the migration assumes `` but our seed has ``. Repro locally with our seed dump: +> +> ``` +> wget https://github.com/ledoent/OpenUpgrade/releases/download/databases/18.0-ledoent.psql +> # restore + re-run migration +> ``` +> +> Happy to test any follow-up commits. + +## Multi-company / branch surface (once `seed_18_woodbimble` lands) + +> Tested this PR against our multi-company fork seed (`18.0-ledoent-mc.psql`) on top of the standard runs: +> +> - **Wood Manufacturing Co.** (US, generic_coa) parent +> - **Wood Co. — Showroom** (branch, `parent_id` set, shares parent COA) +> - **Bimble Design Services Co.** (separate entity, own COA install) +> - 20 partners with cross-company reach via `res_partner_res_company_rel` +> - Admin user with `allowed_company_ids` spanning all three +> +> Run: [LINK] — `STATUS`. +> +> Notable: `` / ``. Catches the surface that OCA's vanilla demo doesn't have (single flat company, zero branches). + +## After OCA-stacked seed lands + +> Also tested against our OCA-stacked seed (`18.0-ledoent-mc-oca.psql`) which adds the typical Ledo prod OCA stack: `account-financial-reporting`, `account-financial-tools`, `account-reconcile`, `bank-statement-import`, `mis-builder`, `reporting-engine`, `server-tools`, `server-ux`, `web`, `social`, etc. on top of multi-company. +> +> Run: [LINK] — `STATUS`. +> +> This is the closest signal we have to a real Ledo prod migration. ``. + +## Tone notes + +- Lead with the result (green/red), not the methodology. +- Link runs by full URL so reviewers can click without `gh` access. +- If red, **never** speculate on the fix in the comment — say what + broke and where, let the PR author decide. Speculation reads as + "do my thinking for me." +- Don't comment on style or conventions in these reports — that's + OCA reviewers' job. We're providing data, not opinion. +- If their PR is already approved upstream, our comment is "extra + signal, not a blocker." Phrase accordingly.