Skip to content

Prevent CI from running OOM when oversubscribing GPUs #629

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
abouteiller merged 2 commits into from
Feb 3, 2024
Merged

Prevent CI from running OOM when oversubscribing GPUs #629

abouteiller merged 2 commits into from
Feb 3, 2024

Conversation

@abouteiller
Copy link
Contributor

@abouteiller abouteiller commented Jan 31, 2024

No description provided.

@abouteiller abouteiller mentioned this pull request Jan 31, 2024
Open
@abouteiller
Copy link
Contributor Author

abouteiller commented Feb 1, 2024

Allocating 79855878144 bytes of memory on the GPU device failed
that's 79GB, wat?

@abouteiller abouteiller requested a review from therault February 1, 2024 21:12
@abouteiller
Copy link
Contributor Author

abouteiller commented Feb 1, 2024 •
edited
Loading

Even if we want to pursue #615, there is no drawbacks to merging this.

I created #630 to track the more serious defect identified in the CI runtest.

@abouteiller abouteiller marked this pull request as ready for review February 1, 2024 21:14
@bosilca
Copy link
Contributor

bosilca commented Feb 1, 2024

not not of the not not. What was that supposed to mean ?

@abouteiller abouteiller merged commit b304378 into ICLDisco:master Feb 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bosilca bosilca bosilca approved these changes

@therault therault Awaiting requested review from therault

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@abouteiller @bosilca