Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NMT 'test' script would benefit from an option to run the test on ClearML #627

Open
mmartin9684-sil opened this issue Jan 14, 2025 · 3 comments
Assignees
Labels
enhancement New feature or request pipeline 5: test Issue relating to testing a model quality with Bleu or other metrics.

Comments

@mmartin9684-sil
Copy link
Collaborator

Most of the NMT scripts that require a GPU have an option (--clearml-queue xxx) to send the job to a ClearML queue so that it can be executed on a shared GPU. The 'test' script does not support this command line option. This would be helpful to have since most models can't be run locally.

@mmartin9684-sil mmartin9684-sil added enhancement New feature or request pipeline 5: test Issue relating to testing a model quality with Bleu or other metrics. labels Jan 14, 2025
@ddaspit ddaspit moved this from 🆕 New to 🔖 Ready in SIL-NLP Research Jan 15, 2025
@ddaspit ddaspit assigned mshannon-sil and unassigned ddaspit Jan 15, 2025
@mshannon-sil
Copy link
Collaborator

I see that the train.py script also does not include a --clearml-queue flag. The only part of the experiment pipeline that includes a --clearml-queue flag is translate.py, which was only added 7 months ago. @ddaspit would you like me to add --clearml-queue flags to both train.py and test.py, or should we direct users to specify --train and/or --test in experiment.py if they'd like to run on ClearML?

@ddaspit
Copy link
Collaborator

ddaspit commented Jan 22, 2025

Does it work if you run the experiment script with just the --test flag?

@mmartin9684-sil
Copy link
Collaborator Author

Yes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request pipeline 5: test Issue relating to testing a model quality with Bleu or other metrics.
Projects
Status: 🔖 Ready
Development

No branches or pull requests

3 participants