Skip to content

Stabilization of cardano db v2 (aggregator) #2610

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
+196 −111
Open

Stabilization of cardano db v2 (aggregator) #2610

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3bd5ace
feature(aggregator, signer, common): keep only MithrilStakeDistributi…
turmelclem Jun 25, 2025
54f3a59
refactor(client-cli, e2e): rename cardano-db download directory
turmelclem Jun 26, 2025
216f579
feat(docs): add cardano-db download example with a range of immutables
turmelclem Jun 27, 2025
a037108
fix(docs): default backend for cardano-db verify is v2
turmelclem Jun 27, 2025
a31cdad
feat(client-cli): remove unstable in CardanoDbCommandsBackend V2 clap…
turmelclem Jun 27, 2025
df7ee5b
WIP: switch immutableFilesFull to cardanoDatabase and update mock
turmelclem Jul 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/backward-compatibility.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ on:
description: "Signed entity types parameters (discriminants names in an ordered comma separated list)"
required: true
type: string
default: "CardanoTransactions,CardanoStakeDistribution,CardanoDatabase"
default: "CardanoTransactions,CardanoStakeDistribution,CardanoDatabase,CardanoImmutableFilesFull"
workflow_call:
inputs:
total-releases:
Expand All @@ -28,7 +28,7 @@ on:
default: "10.4.1"
signed-entity-types:
type: string
default: "CardanoTransactions,CardanoStakeDistribution,CardanoDatabase"
default: "CardanoTransactions,CardanoStakeDistribution,CardanoDatabase,CardanoImmutableFilesFull"

jobs:
prepare-env-variables:
Expand Down
22 changes: 11 additions & 11 deletions .github/workflows/test-client.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ jobs:
- name: Cardano Database Snapshot / download & restore latest
shell: bash
working-directory: ./bin
run: ./mithril-client ${{ steps.prepare.outputs.debug_level }} --origin-tag CI cardano-db download $CDB_SNAPSHOT_DIGEST --backend v1 ${{ matrix.extra_args }} 2>&1 | tee cdb-download-output.txt
run: ./mithril-client ${{ steps.prepare.outputs.debug_level }} --origin-tag CI cardano-db download $CDB_SNAPSHOT_DIGEST --backend v1 --download-dir v1 ${{ matrix.extra_args }} 2>&1 | tee cdb-download-output.txt

- name: Cardano Database Snapshot / verify Cardano node starts successfully
if: runner.os == 'Linux'
Expand All @@ -173,19 +173,19 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
shell: bash
working-directory: ./bin
run: ./mithril-client ${{ steps.prepare.outputs.debug_level }} --unstable tools utxo-hd snapshot-converter --db-directory db --cardano-node-version latest --utxo-hd-flavor LMDB --commit
run: ./mithril-client ${{ steps.prepare.outputs.debug_level }} --unstable tools utxo-hd snapshot-converter --db-directory v1/db --cardano-node-version latest --utxo-hd-flavor LMDB --commit

- name: Remove downloaded artifacts to free up disk space (Linux, Windows)
if: runner.os != 'macOS'
shell: bash
working-directory: ./bin
run: rm --force db/immutable/*.{chunk,primary,secondary}
run: rm --force v1/db/immutable/*.{chunk,primary,secondary}

- name: Remove downloaded artifacts to free up disk space (macOs)
if: runner.os == 'macOS'
shell: bash
working-directory: ./bin
run: sudo rm -rf db/
run: sudo rm -rf v1/db/

- name: Mithril Stake Distribution / list and get last hash
shell: bash
Expand Down Expand Up @@ -259,13 +259,13 @@ jobs:
if: steps.aggregator_capability_unix.outputs.cardano_database_v2_enabled == 'true' || steps.aggregator_capability_windows.outputs.cardano_database_v2_enabled == 'true'
shell: bash
working-directory: ./bin
run: ./mithril-client ${{ steps.prepare.outputs.debug_level }} --origin-tag CI cardano-db download --backend v2 $CARDANO_DATABASE_V2_SNAPSHOT_HASH ${{ matrix.extra_args }} 2>&1 | tee cdb-v2-download-output.txt
run: ./mithril-client ${{ steps.prepare.outputs.debug_level }} --origin-tag CI cardano-db download --backend v2 --download-dir v2 $CARDANO_DATABASE_V2_SNAPSHOT_HASH ${{ matrix.extra_args }} 2>&1 | tee cdb-v2-download-output.txt

- name: Cardano Database V2 Snapshot / verify immutables
if: steps.aggregator_capability_unix.outputs.cardano_database_v2_enabled == 'true' || steps.aggregator_capability_windows.outputs.cardano_database_v2_enabled == 'true'
shell: bash
working-directory: ./bin
run: ./mithril-client ${{ steps.prepare.outputs.debug_level }} --origin-tag CI cardano-db verify --backend v2 --db-dir db_v2 $CARDANO_DATABASE_V2_SNAPSHOT_HASH | tee cdb-v2-verify-output.txt
run: ./mithril-client ${{ steps.prepare.outputs.debug_level }} --origin-tag CI cardano-db verify --backend v2 --db-dir v2/db $CARDANO_DATABASE_V2_SNAPSHOT_HASH | tee cdb-v2-verify-output.txt

- name: Cardano Database V2 Snapshot / verify Cardano node starts successfully
if: runner.os == 'Linux' && steps.aggregator_capability_unix.outputs.cardano_database_v2_enabled == 'true'
Expand Down Expand Up @@ -329,16 +329,16 @@ jobs:
- name: Cardano Database Snapshot / download & restore latest
shell: bash
run: ${{ steps.command.outputs.mithril_client }} ${{ steps.prepare.outputs.debug_level }} --origin-tag CI cardano-db download $CDB_SNAPSHOT_DIGEST --backend v1 --download-dir /app/data ${{ matrix.extra_args }}
run: ${{ steps.command.outputs.mithril_client }} ${{ steps.prepare.outputs.debug_level }} --origin-tag CI cardano-db download $CDB_SNAPSHOT_DIGEST --backend v1 --download-dir /app/data/v1 ${{ matrix.extra_args }}

- name: Ledger state snapshot conversion from InMemory to LMDB
if: matrix.extra_args == '--include-ancillary'
shell: bash
run: ${{ steps.command.outputs.mithril_client }} ${{ steps.prepare.outputs.debug_level }} --unstable tools utxo-hd snapshot-converter --db-directory /app/data/db --cardano-node-version latest --utxo-hd-flavor LMDB --commit
run: ${{ steps.command.outputs.mithril_client }} ${{ steps.prepare.outputs.debug_level }} --unstable tools utxo-hd snapshot-converter --db-directory /app/data/v1/db --cardano-node-version latest --utxo-hd-flavor LMDB --commit

- name: Remove downloaded artifacts to free up disk space
shell: bash
run: sudo rm -rf $PWD/data/db/
run: sudo rm -rf $PWD/data/v1/db/

- name: Mithril Stake Distribution / list and get last hash
shell: bash
Expand Down Expand Up @@ -401,12 +401,12 @@ jobs:
- name: Cardano Database V2 Snapshot / download & restore latest (Full restoration)
if: steps.aggregator_capability.outputs.cardano_database_v2_enabled == 'true'
shell: bash
run: ${{ steps.command.outputs.mithril_client }} ${{ steps.prepare.outputs.debug_level }} --origin-tag CI cardano-db download --backend v2 $CARDANO_DATABASE_V2_SNAPSHOT_HASH --download-dir /app/data ${{ matrix.extra_args }}
run: ${{ steps.command.outputs.mithril_client }} ${{ steps.prepare.outputs.debug_level }} --origin-tag CI cardano-db download --backend v2 $CARDANO_DATABASE_V2_SNAPSHOT_HASH --download-dir /app/data/v2 ${{ matrix.extra_args }}

- name: Cardano Database V2 Snapshot / verify immutables
if: steps.aggregator_capability.outputs.cardano_database_v2_enabled == 'true'
shell: bash
run: ${{ steps.command.outputs.mithril_client }} ${{ steps.prepare.outputs.debug_level }} --origin-tag CI cardano-db verify --backend v2 $CARDANO_DATABASE_V2_SNAPSHOT_HASH --db-dir /app/data/db_v2
run: ${{ steps.command.outputs.mithril_client }} ${{ steps.prepare.outputs.debug_level }} --origin-tag CI cardano-db verify --backend v2 $CARDANO_DATABASE_V2_SNAPSHOT_HASH --db-dir /app/data/v2/db

test-mithril-client-wasm:
strategy:
Expand Down
Loading
Loading