DV Setup: Charon Integration with Sedge

[sugh01]

Changes:

Charon is used by stakers to distribute the responsibility of running Ethereum Validators across a number of different instances and client implementations. Setting up and running a full ethereum node with charon, needs some learning curve and compatibility knowledge, in order for the setup to be fully compliant with the charon configuration requirements for different BN-VC combinations. We want to provide a better and guided user experience for setting up a DV with Charon.

Sedge, is a one-click ethereum node setup tool which focuses on ease of use and, as such, lowers the entry barrier for non-technical users. With Sedge, one can run a node or a validator on Ethereum with little to no technical background.
Integrating Charon with Sedge would make it easy for stakers to setup and run a DV with Charon without having to go through each individual client setup docs and their compatibility with DVT.

Please refer Design Doc here - https://docs.google.com/document/d/1nQsOmkd-od3-lej3OcdUgVZ0GYXAHK9GZqlwEoyIzSY/edit

Types of changes

What types of changes does your code introduce?
Put an x in the boxes that apply

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation Update
• Code style update (formatting, renaming)
• Refactoring (no functional changes, no api changes)
• Build related changes
• Other (please describe):

Testing

Requires testing

• Yes
• No

In case you checked yes, did you write tests?

• Yes
• No

Comments about testing , should you have some (optional)

Further comments (optional)

How to test

Checkout branch
Perform Charon DKG process, a .charon folder is created

make compile
sudo cp build/sedge /usr/local/bin/
sedge generate full-node --validator=lodestar --network=holesky --distributed
sedge import-key --from keystore --distributed -n holesky lodestar
sedge run

[sugh01]

I was having a problem importing keys.
Charon DKG process outputs a folder which has a subfolder validator_keys inside it.
I saved the folder in the root.
And then ran the commands -
sedge generate full-node --validator=lodestar --network=holesky --distributed
sudo sedge import-key --from node0/validator_keys -n holesky lodestar
But i got the below error -
2024-04-05 13:57:51 -- [INFO] [Logger Init] Log level: info 2024-04-05 13:57:51 -- [WARN] A new Version of sedge is available. Please update to the latest Version. See https://github.com/NethermindEth/sedge/releases for more information. Latest detected tag: v0.0.0 2024-04-05 13:57:52 -- [INFO] Setting up containers 2024-04-05 13:57:52 -- [INFO] Running command: docker compose -f /Users/sg/obol/nethermind/sedge/sedge-data/docker-compose.yml build validator 2024-04-05 13:57:52 -- [INFO] Running command: docker compose -f /Users/sg/obol/nethermind/sedge/sedge-data/docker-compose.yml pull validator [+] Pulling 0/0 ⠋ validator Pulling 0.1s [+] Pulling 0/17:52 must use ASL logging (which requires CGO) if running ⠙ validator Pulling 0.2s error getting credentials - err: exit status 1, out: `` 2024-04-05 13:57:52 -- [FATA] exit status 1

node0.zip

[adriantpaez]

Hi @sugh01. The issue is related to the use of sudo with the sedge import-key command. I recreated the scenario locally (without sudo) and attempted to import the keys from the provided node0 directory, which resulted in the following error:

$ sedge import-key --from node0/validator_keys -n holesky lodestar
2024-05-16 12:40:34 -- [INFO] [Logger Init] Log level: info
2024-05-16 12:40:34 -- [INFO] You are running the latest version of sedge. Version:  v1.3.2
2024-05-16 12:40:35 -- [INFO] Setting up containers
2024-05-16 12:40:35 -- [INFO] Running command: docker compose -f /abs/path/to/sedge-data/docker-compose.yml build validator
2024-05-16 12:40:35 -- [INFO] Running command: docker compose -f /abs/path/to/sedge-data/docker-compose.yml pull validator
[+] Pulling 1/1
 ✔ validator Pulled                                                                                           1.3s 
2024-05-16 12:40:36 -- [INFO] Running command: docker compose -f /abs/path/to/sedge-data/docker-compose.yml create validator
[+] Creating 2/0
 ✔ Container sedge-validator-blocker  Created                                                                 0.0s 
 ✔ Container sedge-validator-client   Created                                                                 0.0s 
2024-05-16 12:40:36 -- [WARN] The keys path is not the default one, copying the keys to the default path /abs/path/to/sedge-data/keystore
2024-05-16 12:40:36 -- [INFO] Importing validator keys
2024-05-16 12:40:36 -- [INFO] The keys import container is starting
2024-05-16 12:40:38 -- [FATA] validator import container exited with non-zero exit code: validator-import service container with id 85b3e4a24962a0923f93754a8855c0575d768f7a75bb4191718ba926b8cbafc8 ends with status code 1. Here are the logs for more details:
 ✖ Error: ENOENT: no such file or directory, open '/keystore/keystore_password.txt'
    at Object.readFileSync (node:fs:453:20)
    at readPassphraseFile (file:///usr/app/packages/cli/src/util/passphrase.ts:9:19)
    at readPassphraseOrPrompt (file:///usr/app/packages/cli/src/cmds/validator/signers/importExternalKeystores.ts:30:12)
    at Object.handler (file:///usr/app/packages/cli/src/cmds/validator/import.ts:61:23)
    at file:///usr/app/node_modules/yargs/build/lib/command.js:206:54
    at maybeAsyncResult (file:///usr/app/node_modules/yargs/build/lib/utils/maybe-async-result.js:9:15)
    at CommandInstance.handleValidationAndGetResult (file:///usr/app/node_modules/yargs/build/lib/command.js:205:25)
    at CommandInstance.applyMiddlewareAndGetResult (file:///usr/app/node_modules/yargs/build/lib/command.js:245:20)
    at CommandInstance.runCommand (file:///usr/app/node_modules/yargs/build/lib/command.js:128:20)
    at YargsInstance.[runYargsParserAndExecuteCommands] (file:///usr/app/node_modules/yargs/build/lib/yargs-factory.js:1386:105)

To import the keys correctly, provide the keystore password in the keystore_password.txt file.

Example of successfully import

This is a keystore directory keystore.zip, generated with sedge keys command, which will be used for the imprt-key in this example. This is the structure of the keystore directory:

$ tree keystore 
keystore
├── deposit_data.json
├── keystore_password.txt
└── validator_keys
    └── keystore-m_12381_3600_0_0_0.json

2 directories, 3 files

1. sedge generate full-node --validator=lodestar --network=holesky --distributed. This command generates the following files in the sedge-data directory:

$ tree sedge-data
sedge-data
├── docker-compose.yml
└── jwtsecret

1 directory, 2 files

2. sedge import-key --from keystore -n holesky lodestar This command produces these logs:

sedge import-key --from keystore -n holesky lodestar
2024-05-16 12:59:10 -- [INFO] [Logger Init] Log level: info
2024-05-16 12:59:10 -- [INFO] You are running the latest version of sedge. Version:  v1.3.2
2024-05-16 12:59:10 -- [INFO] Setting up containers
2024-05-16 12:59:10 -- [INFO] Running command: docker compose -f /abs/path/to/sedge-data/docker-compose.yml build validator
2024-05-16 12:59:10 -- [INFO] Running command: docker compose -f /abs/path/to/sedge-data/docker-compose.yml pull validator
[+] Pulling 1/1
 ✔ validator Pulled                                                                                           1.4s 
2024-05-16 12:59:12 -- [INFO] Running command: docker compose -f /abs/path/to/sedge-data/docker-compose.yml create validator
[+] Creating 2/0
 ✔ Container sedge-validator-blocker  Created                                                                 0.1s 
 ✔ Container sedge-validator-client   Created                                                                 0.0s 
2024-05-16 12:59:12 -- [WARN] The keys path is not the default one, copying the keys to the default path /abs/path/to/sedge-data/keystore
2024-05-16 12:59:12 -- [INFO] Importing validator keys
2024-05-16 12:59:12 -- [INFO] The keys import container is starting
2024-05-16 12:59:15 -- [INFO] Validator keys imported successfully

And the sedge-data directory is updated with the imported keystore.

$ tree sedge-data 
sedge-data
├── docker-compose.yml
├── jwtsecret
├── keystore
│   ├── deposit_data.json
│   ├── keystore_password.txt
│   └── validator_keys
│       └── keystore-m_12381_3600_0_0_0.json
└── validator-data
    ├── keystores
    │   └── 0xb53a59a1a5f59adc0d33686164bb90dcadfabbe3000bc95d3ac738b8269d80acb1bdb081a8f23592b6cd21e00bb86f94
    │       └── voting-keystore.json
    └── secrets
        └── 0xb53a59a1a5f59adc0d33686164bb90dcadfabbe3000bc95d3ac738b8269d80acb1bdb081a8f23592b6cd21e00bb86f94

7 directories, 7 files

[adriantpaez]

It appears that the import process is working fine, but the provided keystore directory structure is not correct

[sugh01]

Hi @adriantpaez
While testing the key import process, i ran into an error when trying with Lighthouse.

Steps:

1. sedge generate full-node --validator=lighthouse --network=holesky
2. sedge keys

sedge-data
├── docker-compose.yml
├── jwtsecret
└── keystore
    ├── deposit_data.json
    ├── keystore_password.txt
    └── validator_keys
        └── keystore-m_12381_3600_0_0_0.json

3. sedge import-key -n holesky lighthouse

This step does not complete.

2024-06-19 13:38:45 -- [INFO] [Logger Init] Log level: info
2024-06-19 13:38:45 -- [INFO] You are running the latest version of sedge. Version:  v1.3.2
2024-06-19 13:38:45 -- [INFO] Setting up containers
2024-06-19 13:38:45 -- [INFO] Running command: docker compose -f /Users/sg/sedge/sedge/sedge-data/docker-compose.yml build validator
2024-06-19 13:38:45 -- [INFO] Running command: docker compose -f /Users/sg/sedge/sedge/sedge-data/docker-compose.yml pull validator
[+] Pulling 1/1
 ✔ validator Pulled                                                                                                                                    1.3s 
2024-06-19 13:38:55 -- [INFO] Running command: docker compose -f /Users/sg/sedge/sedge/sedge-data/docker-compose.yml create validator
[+] Creating 3/0
 ✔ Network sedge-network              Created                                                                                                          0.0s 
 ✔ Container sedge-validator-blocker  Created                                                                                                          0.0s 
 ✔ Container sedge-validator-client   Created                                                                                                          0.0s 
2024-06-19 13:38:55 -- [INFO] Running command: docker build -t sedge/validator-import-lighthouse --build-arg LH_VERSION=sigp/lighthouse:v5.0.0 --build-arg NETWORK=holesky /var/folders/ng/dv8zg6ld6nvdrbqmysj1klxh0000gn/T/sedge-validator-import-context-3819293176
[+] Building 1.0s (11/11) FINISHED                                                                                                     docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                   0.0s
 => => transferring dockerfile: 529B                                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/debian:stable-20221004-slim                                                                         0.9s
 => [internal] load .dockerignore                                                                                                                      0.0s
 => => transferring context: 2B                                                                                                                        0.0s
 => [1/7] FROM docker.io/library/debian:stable-20221004-slim@sha256:92ed72016796475ea1f18f84cd8e2f8519ece3a9ea27fdde1157464078ea5371                   0.0s
 => [internal] load build context                                                                                                                      0.0s
 => => transferring context: 677B                                                                                                                      0.0s
 => CACHED [2/7] RUN apt-get update && apt-get install -y curl                                                                                         0.0s
 => CACHED [3/7] RUN version=$(echo sigp/lighthouse:v5.0.0 | cut -d':' -f 2) && curl -L https://github.com/sigp/lighthouse/releases/download/$version  0.0s
 => CACHED [4/7] RUN tar -xvf lh.gz                                                                                                                    0.0s
 => CACHED [5/7] COPY validator-init.sh .                                                                                                              0.0s
 => CACHED [6/7] RUN chmod +x validator-init.sh                                                                                                        0.0s
 => exporting to image                                                                                                                                 0.0s
 => => exporting layers                                                                                                                                0.0s
 => => writing image sha256:68726c147b82af43d6a88a143b1d5560d3f67fee8e14f14950824c8a965c22c8                                                           0.0s
 => => naming to docker.io/sedge/validator-import-lighthouse                                                                                           0.0s

View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/yknf6rtsfipgqfwzfzjngzr8d

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview 
2024-06-19 13:38:56 -- [INFO] Importing validator keys
2024-06-19 13:38:56 -- [INFO] The keys import container is starting
2024-06-19 13:38:57 -- [FATA] validator import container exited with non-zero exit code: validator-import service container with id 67c460dfabc79a3402901c19e6dbfb08572829c58bf3156ad84e20744699df8a ends with status code 255. Here are the logs for more details:
Found validator key in /keystore/validator_keys/keystore-m_12381_3600_0_0_0.json
Importing validator...
qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory

The same steps run successful for Teku, Lodestar and Prysm.
Do i need to preform some additional steps for Lighthouse?

[AntiD2ta]

Thank you so much for this feature, @sugh01, and for your patience throughout the entire process. We are very close to merging this PR. Please address the few open comments, and we will be good to go.

[khalifaa55]

Created Issue #475 to add E2E tests for this integration later on.

[stdevMac]

We found one error while running:

make compile
sudo cp build/sedge /usr/local/bin/
sedge generate full-node --validator=lodestar --network=holesky --distributed
sedge import-key --from keystore --distributed -n holesky lodestar

and this is the error message
on Windows:

2024-11-07 00:27:19 -- [INFO] Copying the keys from D:\Desktop\nethermind\projects\sedge\sedge\keystore
2024-11-07 00:27:19 -- [INFO] Copying Charon contents to the default path D:\Desktop\nethermind\projects\sedge\sedge\sedge-data\.charon charon                                                                                                                             fied.
2024-11-07 00:27:19 -- [FATA] CreateFile D:\Desktop\nethermind\projects\sedge\sedge\keystore: The system cannot find the file specicified.

on WSL:

2024-11-07 00:30:58 -- [INFO] Copying the keys from /mnt/d/Desktop/nethermind/projects/sedge/sedge/keystore
2024-11-07 00:30:58 -- [INFO] Copying Charon contents to the default path /mnt/d/Desktop/nethermind/projects/sedge/sedge/sedge-data/.charon
2024-11-07 00:30:58 -- [FATA] lstat /mnt/d/Desktop/nethermind/projects/sedge/sedge/keystore: no such file or directory