Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add,copy: if multiple sources are used in ADD or COPY then dest must end with a slash #4183

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

flouthoc
Copy link
Collaborator

When using ADD or COPY enforce the condition that destination must
look like a directory by making sure that destination ends with a slash
/.

This ensures that we don't hit undefined behaviour for example with
the use case COPY some/* /oncontainer it is not clear that
oncontainer will be a directory or a file, users expect it to be a
directory if * wildcard resolves to multiple files and logically if
wildcard resolves to single file then it should be a directory, since
this condition is undefined and not in parity with docker so lets drop
support for it.

Docker's defined rule:

  • If multiple <src> resources are specified, either directly or due to the
    use of a wildcard, then <dest> must be a directory, and it must end with a slash /.

Reference: https://docs.docker.com/engine/reference/builder/#add

Closes: #4167

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 11, 2022

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: flouthoc

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

add.go Outdated
// Check length before accessing `0`
// to avoid `index out of range`.
if len(sources) == 1 {
if strings.HasSuffix(sources[0], "*") {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should you check for len(sources)==0 and return an error?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is a suffix of "*" the special case here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Special case is here to match with the current behavior of docker #4167 (comment)

}
}
if multipleSources {
if !strings.HasSuffix(destination, "/") {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does docker enforce this? Or does it just check if destination is not a directory then throw an error?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just chiming in that if the directory already exists, podman/buildah does the expected (?) thing and copies all the matches of the wildcard into the existing directory

Re-using the reproducer from #4167, if you modify the Dockerfile to this

FROM alpine:latest as builder

RUN mkdir /spam
COPY spam/*.txt /spam

Then it copies all the files

Copy link
Collaborator Author

@flouthoc flouthoc Aug 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rhatdan I think docker checks this cause rule is

* If multiple <src> resources are specified, either directly or due to the use of a wildcard, then <dest> must be a directory, and it must end with a slash /.

https://docs.docker.com/engine/reference/builder/#add

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still believe this should be done on in the copier.

Basically if the SRC contains more then one file and the DEST is not a directory, then it should return an error.

The directory should not be checked for ending with /

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rhatdan Yes the function is validateSourceAndDest to validates this using filepath.Glob( there is also a test which verifies the same, i can add few more test cases if this is a concern and move it to copier.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're already globbing inside a chroot further down, when we call copier.Stat(), and we're probably better off doing this sort of check just after that, when we have all of the facts. Doing this before that means that the current iteration of this patch passes URLs for remote locations to filepath.Glob(), which is a bug.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess they do document the following:

If <dest> does not end with a trailing slash, it will be considered a regular file and the contents of <src> will be written at <dest>.

But @nalind question above has not been addressed

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically if the SRC contains more then one file and the DEST is not a directory, then it should return an error.

The directory should not be checked for ending with /

+1 to this, it sounds reasonable to me that if the destination already exists then the trailing slash doesn't need to be enforced. It wouldn't be 100% consistent with docker, but isn't it likely that some users are already doing this? E.g. copying things to /etc or other standard dirs

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chmeliik Yes that is already addressed I just need to move code around as per @nalind 's suggestion here #4183 (comment) , I'll amend this.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done moved to just after we split local and remote sources and only validating local sources now.

tests/add.bats Outdated Show resolved Hide resolved
tests/add.bats Outdated Show resolved Hide resolved
@flouthoc flouthoc force-pushed the add-disallow-dest-dir-without-slash branch 3 times, most recently from e60a9ef to f1d47cc Compare August 23, 2022 17:31
@TomSweeneyRedHat
Copy link
Member

I know I lost my mind a long time ago, but didn't we enforce an ending slash at one point and then removed the enforcement as it was counter to Docker?

@github-actions
Copy link

A friendly reminder that this PR had no activity for 30 days.

@rhatdan
Copy link
Member

rhatdan commented Sep 26, 2022

@flouthoc any chance you get get back to this one?

@flouthoc
Copy link
Collaborator Author

@flouthoc any chance you get get back to this one?

Sure yes, I'll amend this again in this week.

@github-actions github-actions bot removed the stale-pr label Oct 15, 2022
@github-actions
Copy link

A friendly reminder that this PR had no activity for 30 days.

@rhatdan
Copy link
Member

rhatdan commented Nov 15, 2022

@flouthoc Any movement on this?

@rhatdan rhatdan removed the stale-pr label Nov 15, 2022
@github-actions
Copy link

A friendly reminder that this PR had no activity for 30 days.

@rhatdan
Copy link
Member

rhatdan commented Dec 17, 2022

@flouthoc Reminder

@github-actions github-actions bot removed the stale-pr label Dec 18, 2022
@github-actions
Copy link

A friendly reminder that this PR had no activity for 30 days.

@rhatdan
Copy link
Member

rhatdan commented Jan 19, 2023

@flouthoc after the release lets get back to this one.

@flouthoc
Copy link
Collaborator Author

@rhatdan Yes getting back on this and closing this one.

@flouthoc flouthoc force-pushed the add-disallow-dest-dir-without-slash branch from f1d47cc to 33bcb72 Compare January 26, 2023 11:12
When using `ADD` or `COPY` enforce the condition that destination must
look like a directory by making sure that destination ends with a slash
`/`.

This ensures that we don't hit undefined behaviour for example with
the use case `COPY some/* /oncontainer` it is not clear that
`oncontainer` will be a directory or a file, users expect it to be a
directory if `*` wildcard resolves to multiple files and logically if
wildcard resolves to single file then it should be a directory, since
this condition is undefined and not in parity with docker so lets drop
support for it.

Docker's defined rule:
* If multiple <src> resources are specified, either directly or due to the use of a wildcard, then <dest> must be a directory, and it must end with a slash /.

Reference: https://docs.docker.com/engine/reference/builder/#add
Closes: containers#4167

Signed-off-by: Aditya R <[email protected]>
Accoring to new validation in ADD and COPY if there are multiple sources
then destination must specify that it is a directory

Signed-off-by: Aditya R <[email protected]>
@flouthoc flouthoc force-pushed the add-disallow-dest-dir-without-slash branch from 33bcb72 to 86a9ac4 Compare January 26, 2023 11:14
@flouthoc
Copy link
Collaborator Author

@nalind @rhatdan PTAL

@github-actions
Copy link

A friendly reminder that this PR had no activity for 30 days.

@rhatdan
Copy link
Member

rhatdan commented Mar 1, 2023

LGTM
@nalind @vrothberg @giuseppe @umohnani8 PTAL

@TomSweeneyRedHat
Copy link
Member

LGTM, but would like a head nod from @vrothberg or @nalind to make sure comments were addressed.

@github-actions github-actions bot removed the stale-pr label Mar 2, 2023
@github-actions
Copy link

github-actions bot commented Apr 2, 2023

A friendly reminder that this PR had no activity for 30 days.

@rhatdan
Copy link
Member

rhatdan commented Apr 3, 2023

This still needs to be reworked.

@github-actions github-actions bot removed the stale-pr label Apr 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Inconsistent behavior when COPYing multiple files to /dest without trailing slash
5 participants