docs: [WIP] Add a README with tutorial on how to use the job attachment CLIs #502
Conversation
examples/job_attachments/README.md
Outdated
|
|
||
| Deadline Cloud offers the [Job Attachments](https://docs.aws.amazon.com/deadline-cloud/latest/userguide/storage-job-attachments.html) feature to transfer files back and forth between your workstation and AWS Deadline Cloud. For an indepth look into the Job Attachments feature, please refer to the Job Attachments [developer guide](https://docs.aws.amazon.com/deadline-cloud/latest/developerguide/submitting-files-with-a-job.html). | ||
|
|
||
| Where does Job Attachments come into use in the Deadline Cloud job life cycle? 1) Job Submission, 2) Moving data to workers for job execution, 3) Copying job execution output back to the Job Attachments bucket and 4) Downloading Job Outputs. |
There was a problem hiding this comment.
nit - inconsistent cap
Submission->submissionDownloading Job Outputs->Downloading job outputs
Would it be helpful to specify the persona for the action?
There was a problem hiding this comment.
Thanks, changed!
In terms of persona, I think only technical members will read this doc. It is unlikely an artist will run the CLIs unless we have a nice button click in the DCM. We can add more details if there's interest.
examples/job_attachments/README.md
Outdated
| --exclude "*.tmp" (Optional) | ||
| ``` | ||
|
|
||
| ### Uploading a manifest file to S3, useful for subsequent job submissions. |
examples/job_attachments/README.md
Outdated
| --s3-manifest-prefix where/to/store/manifests \ (Optional) | ||
| --json \ (Optional) | ||
| --profile \ (Optional) | ||
| /path/for/snapshot/manifest/data-snapshot-2024-11-15T15-46-40.manifest |
There was a problem hiding this comment.
Is this supposed to be output from the command, maybe we can make that explicit?
There was a problem hiding this comment.
Its an (Argument), so I'll add it (Argument)
examples/job_attachments/README.md
Outdated
| ``` | ||
| deadline attachment upload \ | ||
| --manifests "/path/to/job/attachment.manifest" \ | ||
| --s3-root-uri s3://my-queue/DeadlineCloud \ |
There was a problem hiding this comment.
This is optional too, when not provided, the command would look for the default Queue and use GetQueue to retrieve JA bucket configurations.
There was a problem hiding this comment.
Right, I'll mark it as (Optional, farm-id and queue-id) alternative.
I'll keep the TLDR simple, the examples section can have both.
|
|
||
| ### Uploading files captured by Job Attachment "snapshot" to S3: | ||
| ``` | ||
| deadline attachment upload \ |
There was a problem hiding this comment.
One of --root-dirs or --path-mapping-rules is required, but not both. They're used for determining where the attachments are for upload.
There was a problem hiding this comment.
Right! I need to add that.
| --farm-id farm-abcdabcdabcdabcdabcdabcdabcdabcd \ | ||
| --queue-id queue-11111111111111111111111111111111 \ | ||
| --job-id job-22222222222222222222222222222222 \ | ||
| --step-id step-33333333333333333333333333333333 \ (Optional) |
There was a problem hiding this comment.
This command should support a list of step ids for Step-Step dependencies?
There was a problem hiding this comment.
No, this is the step to download for, the command line internally takes care of finding the step-step dependency of the argument step.
| --farm-id farm-abcdabcdabcdabcdabcdabcdabcdabcd \ | ||
| --queue-id queue-11111111111111111111111111111111 \ | ||
| --job-id job-22222222222222222222222222222222 \ |
There was a problem hiding this comment.
farm, queue and job should not be optional? Manifest download keys on the farm, queue and job.
| --job-id job-22222222222222222222222222222222 \ | ||
| --step-id step-33333333333333333333333333333333 \ (Optional) | ||
| --json \ (Optional) | ||
| --profile \ (Optional) |
There was a problem hiding this comment.
When present, the credentials associated with this profile should grant access to the s3 bucket.
There was a problem hiding this comment.
Oh great comment - super important info.
There was a problem hiding this comment.
Added as note for all CLIs.
examples/job_attachments/README.md
Outdated
| - `deadline manifest diff` | ||
| Render a file folder structure tree showing new, modified and deleted files compared to a manifest. | ||
| - `deadline manifest download` | ||
| Download a manifest file from S3. Assumes Deadline Queue Crentials to access S3 bucket. |
There was a problem hiding this comment.
Crentials -> Credentials
suggestion - Assumes Deadline Queue Credentials to access S3 bucket, or use profile specified via input.
There was a problem hiding this comment.
Great suggestion on --profile! Added and fixed typo.,
examples/job_attachments/README.md
Outdated
| - `deadline manifest download` | ||
| Download a manifest file from S3. Assumes Deadline Queue Crentials to access S3 bucket. | ||
| - `deadline manifest upload` | ||
| Upload a manifest file to S3. Uploaded manifests can be used for subsequent programatic job sumission. Assumes Deadline Queue Crentials to access S3 bucket. |
There was a problem hiding this comment.
| Upload a manifest file to S3. Uploaded manifests can be used for subsequent programatic job sumission. Assumes Deadline Queue Crentials to access S3 bucket. | |
| Upload a manifest file to S3. Uploaded manifests can be used for subsequent programmatic job submission. Assumes Deadline Queue Credentials to access S3 bucket, or use profile specified via input. |
leongdl
force-pushed
the
ja-readme
branch
2 times, most recently
from
06a50e8 to
7af0c9d
Compare
Signed-off-by: David Leong <[email protected]>
|
epmog
added
job attachments
4 participants
Fixes:
What was the problem/requirement? (What/Why)
What was the solution? (How)
What is the impact of this change?
How was this change tested?
See DEVELOPMENT.md for information on running tests.
downloadorasset_syncmodules? If so, then it is highly recommendedthat you ensure that the docker-based unit tests pass.
Was this change documented?
Does this PR introduce new dependencies?
This library is designed to be integrated into third-party applications that have bespoke and customized deployment environments. Adding dependencies will increase the chance of library version conflicts and incompatabilities. Please evaluate the addition of new dependencies. See the Dependencies section of DEVELOPMENT.md for more details.
Is this a breaking change?
A breaking change is one that modifies a public contract in a way that is not backwards compatible. See the
Public Contracts section
of the DEVELOPMENT.md for more information on the public contracts.
If so, then please describe the changes that users of this package must make to update their scripts, or Python applications.
Does this change impact security?
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.