Skip to content

New executor for Fujitsu Technical Computing Suite (TCS) #5928

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 26 commits into from
Sep 2, 2025
Merged

New executor for Fujitsu Technical Computing Suite (TCS) #5928

merged 26 commits into from
Sep 2, 2025
+442 −1

Conversation

exthnet
Copy link
Contributor

@exthnet exthnet commented Mar 27, 2025

This is a new Executor for Fujitsu Technical Computing Suite (TCS).

Some supercomputers build by Fujitsu are operated (mainly in Japan).
Most of they use Technical Computing Suite (TCS) job management system, they are not supported by current Nextflow.

Therefore, I wrote new Executor citing existing executors.
We tested simple jobs and confirmed the correct behaviors on on Fugaku (Riken), Flow (Naogya Univ.), and Genkai (Kyushu Univ.) supercomputers.

Note that I am not familiar with all the features of Nextflow, so there may be some lack of implementation.

@netlify Netlify
Copy link

netlify bot commented Mar 27, 2025 •
edited
Loading

✅ Deploy Preview for nextflow-docs-staging ready!

Name Link
🔨 Latest commit 519d02b
🔍 Latest deploy log https://app.netlify.com/projects/nextflow-docs-staging/deploys/68b6f9b7e800340008877ac5
😎 Deploy Preview https://deploy-preview-5928--nextflow-docs-staging.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

pditommaso
pditommaso requested changes Mar 27, 2025
View reviewed changes
Copy link
Member

@pditommaso pditommaso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for this contribution. Please consider adding some unit tests. You can reference to other executors for some examples. Also please sign your commits as example in the contributing docs

@bentsherman bentsherman linked an issue Mar 28, 2025 that may be closed by this pull request
New feature: TCS Executor (for Supercomputers of Fujitsu) #5916
Closed
@exthnet exthnet requested a review from a team as a code owner April 3, 2025 04:49
@exthnet
Copy link
Contributor Author

exthnet commented Apr 3, 2025 •
edited
Loading

Thanks a lot for this contribution. Please consider adding some unit tests. You can reference to other executors for some examples. Also please sign your commits as example in the contributing docs

Thanks for the advice. I'll try to add unit tests and add sign.

@pditommaso
Copy link
Member

This PR looks stalled. Feel free to reopen if you are keen to contribute to this feature.

@pditommaso pditommaso closed this May 24, 2025
@exthnet
Copy link
Contributor Author

exthnet commented Jun 10, 2025

I have been too busy to take the time.

I am going to resume my work. I have already successfully run a simple workflow job, but I am not sure what additional tests I should run.
I know that there are many test sets in the test directories (tests, tests-v1, test-e2e), but I can't find any explanations.
Could you please advise?

@pditommaso pditommaso reopened this Jun 10, 2025
@pditommaso
Copy link
Member

Ideally it should be added a TcsExecutorTest class validating the executor logic. You can following the the SlurmExecutorTest as as template

@exthnet
Copy link
Contributor Author

exthnet commented Jun 11, 2025

Ideally it should be added a TcsExecutorTest class validating the executor logic. You can following the the SlurmExecutorTest as as template

Thanks, I'll try it.

pditommaso and others added 13 commits July 3, 2025 10:47
@pditommaso @exthnet
Revert "Fix JWT refresh and use TowerClient to handle Fusion token re…
470a055 
…quests nextflow-io#6207" [ci fast]

This reverts commit 9883b4d.

Signed-off-by: Paolo Di Tommaso <[email protected]>
Signed-off-by: Satoshi OHSHIMA <[email protected]>
@adamrtalbot @exthnet
Minor Azure Batch disk slot calculation demoted to debug (nextflow-io…
445813b 
…#6234) [ci skip]

Signed-off-by: adamrtalbot <[email protected]>
Signed-off-by: Satoshi OHSHIMA <[email protected]>
@pditommaso @exthnet
Fix JWT token refresh to Fusion validation request (nextflow-io#6231)…
4fd5fcf 
… [ci fast]

Signed-off-by: Paolo Di Tommaso <[email protected]>
Signed-off-by: Satoshi OHSHIMA <[email protected]>
@ewels @exthnet
Fix docs rendering on config page [ci fast] (nextflow-io#6236)
5e42bbb 
Minor formatting bug when rendering the AWS config reference.

Signed-off-by: Phil Ewels <[email protected]>
Signed-off-by: Satoshi OHSHIMA <[email protected]>
@bentsherman @exthnet
Document secrets config variable (nextflow-io#6230) [ci fast]
c5384b9 
Signed-off-by: Ben Sherman <[email protected]>
Signed-off-by: Paolo Di Tommaso <[email protected]>
Signed-off-by: Satoshi OHSHIMA <[email protected]>
@pditommaso @exthnet
Remove deprecated nexus-staging plugin [ci fast]
1e89771 
Signed-off-by: Paolo Di Tommaso <[email protected]>
Signed-off-by: Satoshi OHSHIMA <[email protected]>
@bentsherman @exthnet
Fix false warning in config schema validator (nextflow-io#6240)
175a617 
Signed-off-by: Ben Sherman <[email protected]>
Signed-off-by: Satoshi OHSHIMA <[email protected]>
@pditommaso @exthnet
Remove deprecated GsonHelper class
586c8d0 
Signed-off-by: Paolo Di Tommaso <[email protected]>
Signed-off-by: Satoshi OHSHIMA <[email protected]>
@exthnet
add new executor for Fujitsu Technical Computing Suite (TCS)
b9c514e 
Signed-off-by: Satoshi OHSHIMA <[email protected]>
@exthnet
update comment for TCS Executor
7d8e394 
Signed-off-by: Satoshi Ohshima <[email protected]>>
Signed-off-by: Satoshi OHSHIMA <[email protected]>
@exthnet
Update TcsExecutor
d1b3e34 
Add TcsExecutorTest

Signed-off-by: Satoshi OHSHIMA <[email protected]>
@exthnet
Update TcsExecutor
a4edb55 
Add TcsExecutorTest

Signed-off-by: Satoshi OHSHIMA <[email protected]>
@exthnet
Update test cases for TcsExecutor.
cd5935c 
Signed-off-by: Satoshi OHSHIMA <[email protected]>
@exthnet exthnet requested a review from a team as a code owner July 3, 2025 02:24
christopher-hakkaart
christopher-hakkaart requested changes Jul 8, 2025
View reviewed changes
Copy link
Collaborator

@christopher-hakkaart christopher-hakkaart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've requested some changes in the documentation. It's mostly formatting and changing the voice and making the example more generic.

@bentsherman bentsherman mentioned this pull request Jul 10, 2025
Closed
pditommaso and others added 4 commits July 15, 2025 15:38
@pditommaso @christopher-hakkaart
Update docs/executor.md [ci skip]
1ace909 
Co-authored-by: Chris Hakkaart <[email protected]>
Signed-off-by: Paolo Di Tommaso <[email protected]>
@pditommaso @christopher-hakkaart
Update docs/executor.md [ci skip]
e20a4a5 
Co-authored-by: Chris Hakkaart <[email protected]>
Signed-off-by: Paolo Di Tommaso <[email protected]>
@pditommaso @christopher-hakkaart
Update docs/executor.md [ci skip]
e011736 
Co-authored-by: Chris Hakkaart <[email protected]>
Signed-off-by: Paolo Di Tommaso <[email protected]>
@bentsherman
cleanup
eb6a226 
Signed-off-by: Ben Sherman <[email protected]>
pditommaso
pditommaso requested changes Jul 15, 2025
View reviewed changes
@manuel-g-castro manuel-g-castro mentioned this pull request Sep 1, 2025
Closed
12 tasks
@pditommaso pditommaso merged commit dd87b2f into nextflow-io:master Sep 2, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pditommaso pditommaso pditommaso approved these changes

@bentsherman bentsherman bentsherman approved these changes

@christopher-hakkaart christopher-hakkaart Awaiting requested review from christopher-hakkaart

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

New feature: TCS Executor (for Supercomputers of Fujitsu)
6 participants
@exthnet @pditommaso @bentsherman @christopher-hakkaart @adamrtalbot @ewels