Adding Secret support to Lepton Executor by zoeyz101 · Pull Request #382 · NVIDIA-NeMo/Run

zoeyz101 · 2025-11-12T19:30:18Z

No description provided.

Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

roclark

LGTM!

* Add logs dir to container mount for ray slurm Signed-off-by: Hemil Desai <hemild@nvidia.com> * fix tests Signed-off-by: Hemil Desai <hemild@nvidia.com> --------- Signed-off-by: Hemil Desai <hemild@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

…DIA-NeMo#286) * finetune on dgxcloud with nemo-run and deploy on bedrock example Signed-off-by: Zoey Zhang <zozhang@nvidia.com> * removing trailing slash Signed-off-by: Zoey Zhang <zozhang@nvidia.com> * reformatting notebook Signed-off-by: Zoey Zhang <zozhang@nvidia.com> * adding EOF Signed-off-by: Zoey Zhang <zozhang@nvidia.com> --------- Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Signed-off-by: Roee Landesman <roeeland@cisco.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

…#285) * fix docs tutorial links and add intro to guides/index.md Signed-off-by: Hemil Desai <hemild@nvidia.com> * Adding project.json/versions1.json, and update conf.py Signed-off-by: Andrew Schilling <aschilling@nvidia.com> * fixes Signed-off-by: Hemil Desai <hemild@nvidia.com> --------- Signed-off-by: Hemil Desai <hemild@nvidia.com> Signed-off-by: Andrew Schilling <aschilling@nvidia.com> Co-authored-by: Andrew Schilling <aschilling@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Signed-off-by: Andrew Schilling <aschilling@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Signed-off-by: Hemil Desai <hemild@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

…es (NVIDIA-NeMo#295) * Add thread pool to get status of jobs inside experiment Signed-off-by: Hemil Desai <hemild@nvidia.com> * Add thread pools to experiment run Signed-off-by: Hemil Desai <hemild@nvidia.com> * fixes Signed-off-by: Hemil Desai <hemild@nvidia.com> * fix Signed-off-by: Hemil Desai <hemild@nvidia.com> * fix Signed-off-by: Hemil Desai <hemild@nvidia.com> * fix Signed-off-by: Hemil Desai <hemild@nvidia.com> * fix Signed-off-by: Hemil Desai <hemild@nvidia.com> --------- Signed-off-by: Hemil Desai <hemild@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

…NeMo#251) - do prepare stage only from single process or rank - for --node-rank, also look for SLURM_NODEID Signed-off-by: Pramod Kumbhar <prkumbhar@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

* Upgrade skypilot to v0.10.0, introduce network_tier Signed-off-by: Roee Landesman <roeeland@cisco.com> * add unit tests Signed-off-by: Roee Landesman <roeeland@cisco.com> --------- Signed-off-by: Roee Landesman <roeeland@cisco.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

* ci(fix): Use GITHUB_TOKEN for community bot Signed-off-by: oliver könig <okoenig@nvidia.com> * f Signed-off-by: oliver könig <okoenig@nvidia.com> --------- Signed-off-by: oliver könig <okoenig@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Signed-off-by: Pablo Garay <palenq@gmail.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

* remove breaking torchrun config for single-node runs Signed-off-by: Roee Landesman <roeeland@cisco.com> * fix lint Signed-off-by: Roee Landesman <roeeland@cisco.com> --------- Signed-off-by: Roee Landesman <roeeland@cisco.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

* update lepton executor to include custom prelaunch commands section Signed-off-by: ansjindal <ansjindal@nvidia.com> * add test for prelaunch section Signed-off-by: ansjindal <ansjindal@nvidia.com> * add more tests for checking the pre-launch-commands section Signed-off-by: ansjindal <ansjindal@nvidia.com> * update lepton executor tests Signed-off-by: ansjindal <ansjindal@nvidia.com> --------- Signed-off-by: ansjindal <ansjindal@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

* Create CHANGELOG.md Signed-off-by: Pablo Garay <palenq@gmail.com> * Add entries to CHANGELOG.md Signed-off-by: Pablo Garay <palenq@gmail.com> * Update CHANGELOG.md Signed-off-by: Pablo Garay <palenq@gmail.com> * Update CHANGELOG.md Co-authored-by: oliver könig <okoenig@nvidia.com> Signed-off-by: Pablo Garay <palenq@gmail.com> * add links --------- Signed-off-by: Pablo Garay <palenq@gmail.com> Co-authored-by: oliver könig <okoenig@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

* Correctly append tar files for packaging Signed-off-by: Sahil Modi <samodi@nvidia.com> * tests Signed-off-by: Hemil Desai <hemild@nvidia.com> --------- Signed-off-by: Sahil Modi <samodi@nvidia.com> Signed-off-by: Hemil Desai <hemild@nvidia.com> Co-authored-by: Hemil Desai <hemild@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

…IA-NeMo#319) Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

…VIDIA-NeMo#320) * Specify nodes for gpu metrics collection and split data to each rank Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> * Fix unit test Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Signed-off-by: Pablo Garay <palenq@gmail.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Signed-off-by: Hemil Desai <hemild@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

* Fixing documentation layout Signed-off-by: Andrew Schilling <aschilling@nvidia.com> * documentation.md Signed-off-by: Andrew Schilling <aschilling@nvidia.com> * Removing live-server Signed-off-by: Andrew Schilling <aschilling@nvidia.com> * Correctin .vscode Signed-off-by: Andrew Schilling <aschilling@nvidia.com> --------- Signed-off-by: Andrew Schilling <aschilling@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Signed-off-by: Andrew Schilling <aschilling@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Signed-off-by: Pablo Garay <pagaray@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

* fix: Emit exit-code of docker runs Signed-off-by: oliver könig <okoenig@nvidia.com> * fix test Signed-off-by: oliver könig <okoenig@nvidia.com> * fixes Signed-off-by: oliver könig <okoenig@nvidia.com> * refactor Signed-off-by: oliver könig <okoenig@nvidia.com> * cleanup Signed-off-by: oliver könig <okoenig@nvidia.com> * add scheduler test Signed-off-by: oliver könig <okoenig@nvidia.com> * more scheduler tests Signed-off-by: oliver könig <okoenig@nvidia.com> * test executor Signed-off-by: oliver könig <okoenig@nvidia.com> * formatting Signed-off-by: oliver könig <okoenig@nvidia.com> --------- Signed-off-by: oliver könig <okoenig@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Signed-off-by: Pablo Garay <pagaray@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

* [🤖]: Howdy folks, let's bump NeMo Run to `0.8.0rc0.dev0` ! Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * lintfix double quotes Signed-off-by: Pablo Garay <pagaray@nvidia.com> --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Pablo Garay <pagaray@nvidia.com> Co-authored-by: pablo-garay <7166088+pablo-garay@users.noreply.github.com> Co-authored-by: Pablo Garay <pagaray@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

* feat: add copyright check * feat: add copyright check * feat: add copyright check * feat: add copyright check * feat: add copyright check Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

* Add port parameter to SSHTunnel Signed-off-by: Igor Gitman <igitman@nvidia.com> * Fix ci Signed-off-by: Igor Gitman <igitman@nvidia.com> * Fix copyright Signed-off-by: Igor Gitman <igitman@nvidia.com> * Fix copyright Signed-off-by: Igor Gitman <igitman@nvidia.com> * Fix copyright Signed-off-by: Igor Gitman <igitman@nvidia.com> --------- Signed-off-by: Igor Gitman <igitman@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Signed-off-by: Wei Du <wedu@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Signed-off-by: Pablo Garay <pagaray@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

* Update ray template Signed-off-by: Hemil Desai <hemild@nvidia.com> * add ray enroot exec template Signed-off-by: Hemil Desai <hemild@nvidia.com> * fix Signed-off-by: Hemil Desai <hemild@nvidia.com> * fix Signed-off-by: Hemil Desai <hemild@nvidia.com> * fix Signed-off-by: Hemil Desai <hemild@nvidia.com> * fix Signed-off-by: Hemil Desai <hemild@nvidia.com> --------- Signed-off-by: Hemil Desai <hemild@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

…Mo#380) Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

adding secret variable in Lepton

a908d0b

Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

zoeyz101 requested a review from roclark November 12, 2025 19:30

zoeyz101 had a problem deploying to public November 12, 2025 19:31 — with GitHub Actions Failure

zoeyz101 had a problem deploying to public November 12, 2025 19:32 — with GitHub Actions Failure

roclark previously approved these changes Nov 12, 2025

View reviewed changes

zoeyz101 dismissed roclark’s stale review via f335356 November 12, 2025 20:08

zoeyz101 had a problem deploying to public November 12, 2025 20:10 — with GitHub Actions Failure

hemildesai and others added 22 commits November 12, 2025 12:12

Fix skypilot archive mount bug (NVIDIA-NeMo#288)

8fc9c42

Signed-off-by: Roee Landesman <roeeland@cisco.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

docs: Fixing doc build issue (NVIDIA-NeMo#290)

7561f5d

Signed-off-by: Andrew Schilling <aschilling@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Add option to specify --container-env for srun (NVIDIA-NeMo#293)

e4f1fb0

Signed-off-by: Hemil Desai <hemild@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Fixes for multi-node execution with torchrun + LocalExecutor (NVIDIA-…

d895c1d

…NeMo#251) - do prepare stage only from single process or rank - for --node-rank, also look for SLURM_NODEID Signed-off-by: Pramod Kumbhar <prkumbhar@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

ci: Add community-bot (NVIDIA-NeMo#300)

84e6fbf

Signed-off-by: oliver könig <okoenig@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Update release.yml (NVIDIA-NeMo#306)

224f43e

Signed-off-by: Pablo Garay <palenq@gmail.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

changelog workflow (NVIDIA-NeMo#315)

f4b7872

Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Add nsys patch in ray sub template (NVIDIA-NeMo#318)

ae34ded

Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Apply '_enable_goodbye_message' check to both goodbye messages. (NVID…

f26da41

…IA-NeMo#319) Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Update package_info.py (NVIDIA-NeMo#322)

18f0439

Signed-off-by: Pablo Garay <palenq@gmail.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Add ray head start timeout (NVIDIA-NeMo#324)

7e514e8

Signed-off-by: Hemil Desai <hemild@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

aschilling-nv and others added 27 commits November 12, 2025 12:14

remove custom dir (NVIDIA-NeMo#351)

748d7a7

Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Bumping to 0.5.0 (NVIDIA-NeMo#352)

7453de0

Signed-off-by: Andrew Schilling <aschilling@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Update release notes header in changelog build (NVIDIA-NeMo#355)

52d3b10

Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

add changelog-config (NVIDIA-NeMo#356)

cf26040

Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

Changelog 0.6.0 (NVIDIA-NeMo#357)

874eec9

Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

spelling (NVIDIA-NeMo#359)

f692e2a

Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

spelling (NVIDIA-NeMo#359)

51f1173

Signed-off-by: Pablo Garay <pagaray@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

new changelog-build (NVIDIA-NeMo#367)

400e1d5

Signed-off-by: Pablo Garay <pagaray@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

feat: add copyright check (NVIDIA-NeMo#369)

d39b377

Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

feat: copyright check (NVIDIA-NeMo#370)

de66b3a

* feat: add copyright check * feat: add copyright check * feat: add copyright check * feat: add copyright check * feat: add copyright check Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

fix host (NVIDIA-NeMo#373)

65fff82

Signed-off-by: Wei Du <wedu@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

update copyright check version

179d931

Signed-off-by: Pablo Garay <pagaray@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

fix

69811a7

Signed-off-by: Pablo Garay <pagaray@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

fix2

2bab921

Signed-off-by: Pablo Garay <pagaray@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

add copyright notice

f20ff39

Signed-off-by: Pablo Garay <pagaray@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

lintfix

42010c5

Signed-off-by: Pablo Garay <pagaray@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

undo

66b8757

Signed-off-by: Pablo Garay <pagaray@nvidia.com> Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

fix ray templates by using --exclusive to launch ray nodes (NVIDIA-Ne…

d7289b3

…Mo#380) Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

fix(typo): exit_code prints empty (NVIDIA-NeMo#379)

1538fc6

Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

fix: limit docker hostname to 32 characters (NVIDIA-NeMo#378)

9a294bf

Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

added tests for env var

6a13a75

Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

linting

9cf1d77

Signed-off-by: Zoey Zhang <zozhang@nvidia.com>

zoeyz101 force-pushed the zozhang/lepton-secrets branch from 90478e0 to 9cf1d77 Compare November 12, 2025 20:16

zoeyz101 closed this Nov 12, 2025

zoeyz101 deleted the zozhang/lepton-secrets branch November 12, 2025 20:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding Secret support to Lepton Executor#382

Adding Secret support to Lepton Executor#382
zoeyz101 wants to merge 64 commits intoNVIDIA-NeMo:mainfrom
zoeyz101:zozhang/lepton-secrets

zoeyz101 commented Nov 12, 2025

Uh oh!

roclark left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

zoeyz101 commented Nov 12, 2025

Uh oh!

roclark left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants