Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify when --tmpdir-prefix is necessary #65

Open
dleehr opened this issue Mar 20, 2019 · 1 comment
Open

Clarify when --tmpdir-prefix is necessary #65

dleehr opened this issue Mar 20, 2019 · 1 comment

Comments

@dleehr
Copy link
Member

dleehr commented Mar 20, 2019

More context in #30

In #62, I changed the behavior for each container to get a local emptyDir volume mounted as /tmp. This means that the calrissian/cwltool generated tmpdir provided to the CalrissianCommandLineJob object usually goes unused and does not need to be located on a persistent volume. Prior to this change, a job's tmpdir would be a subdirectory inside of --tmpdir-prefix, so we required that that be on a persistent volume that could be mounted by the container.

After the change, that behavior no longer applies. The /tmp directory inside a container is local to that container, and should improve performance (since /tmp is no longer a network share)

However, there are still cases that would require a --tmpdir-prefix to be provided to calrissian. Specifically, tools that use the InitialWorkDirRequirement with writable files or directories. This feature is implemented in calrissian by making a copy of the original required file into the job's tmpdir (from the calrissian python process). So, that location must be a place that calrissian can write it and the container can modify it. I assume tmpdir was chosen since this this writable file is temporary and isolated:

If true, the file or directory must be writable by the tool. Changes to the file or directory must be isolated and not visible by any other CommandLineTool process. This may be implemented by making a copy of the original file or directory.

@dleehr
Copy link
Member Author

dleehr commented Mar 20, 2019

At this point I'm not sure what the action item is - raise an exception when this case is encountered? informing the user they must provide --tmpdir-prefix with a PVC mount?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant