Security regarding tasks/migrations/extensions

[sisp]

I'd like to follow up on prior discussions (#376 (comment) & #1130 (reply in thread)) related to Copier's security when tasks/migrations and custom extensions are used by a template, and open a dedicated discussion on this topic.

To summarize previous discussions, the concern has been the execution of untrusted code from template tasks/migrations and custom Jinja extensions which is run in the user's context and, thus, has the same permissions as the user generating/updating a project. Some initial ideas to reduce this risk have been:

1. Request for user's confirmation that tasks/migrations shall be executed (e.g. for each task/migration independently), or passing a flag like --UNSAFE for auto-confirmation.
2. Mark a template as trusted or warn about possible risks (tasks, migrations, extensions).

I think both options are fine and should be doable. In addition, I think we could recommend running Copier only from inside an ephemeral container into which the destination folder is mounted. This way, malicious code could only cause harm inside the container which would get removed immediately after Copier has completed its work. Assuming the container runtime and the host kernel has no exploitable security vulnerability, the havoc should be minimal. In addition, gVisor might be used to strengthen security further.

That said, I'd like to challenge the existence of the problem itself.

1. Searching the internet for "executing untrusted code" (or similar), I find discussions related to, e.g., REPL services or other kinds of server-side code execution of user-provided code. Those scenarios are obviously very dangerous when not handled well because a compromised service/system would affect all users of the service/system. But with Copier, the damage would typically not affect other users.
2. I don't quite see the difference between running tasks/migrations/extensions and installing and using other 3rd party dependencies in any software project. With Copier, a template is like a dependency, somebody chooses to use this dependency and generate/update a project. With other 3rd party dependencies, somebody chooses to use this dependency and use it in a project – which means executing foreign code that I bet nobody reads first.
3. pre-commit – when used in the officially suggested (but IMHO wrong) way – installs hooks/packages into independent virtual envs, and then some executables are run. I bet nobody reads the code (and that of the entire dependency tree) first. I'm not saying Copier shouldn't be better than pre-commit, but I notice that other projects have comparable situations and don't seem to consider them problematic.

So with these arguments, I'm not sure Copier has a security problem at all, or else we'd also need to change the way we integrate 3rd party dependencies in general.

I realize this point of view might be controversial, so I'm really curious about others' opinions.

/cc @yajo @pawamoy

[yajo]

I've been thinking more about this... 🤔 El lun., 8 may. 2023 12:39, Sigurd Spieckermann ***@***.***> escribió:

I'd like to follow up on prior discussions (#376 (comment) <#376 (comment)> & #1130 (reply in thread) <#1130 (reply in thread)>) related to Copier's security when tasks/migrations and custom extensions are used by a template, and open a dedicated discussion on this topic. To summarize previous discussions, the concern has been the execution of untrusted code from template tasks/migrations and custom Jinja extensions which is run in the user's context and, thus, has the same permissions as the user generating/updating a project. Some initial ideas to reduce this risk have been: 1. Request for user's confirmation that tasks/migrations shall be executed (e.g. for each task/migration independently), or passing a flag like --UNSAFE for auto-confirmation. 2. Mark a template as trusted or warn about possible risks (tasks, migrations, extensions). I think both options are fine and should be doable.

The problem is, templates just need these features. Writing good migrations takes time, but they provide smooth updates; but what if users are able to turn them off? Then you can't predict the output. Tasks also are needed to replay last render when updating. And jinja would die if you use it without the required extensions. So, although it's OK to warn the user, the features are needed for copier to do its work. Basically, running Copier without --UNSAFE would only work with templates without these features. At least, we shouldn't require that flag when the operation won't involve any of those. Also we should provide a different exit code for this exit path. I think it'd be OK to present a question before starting, which tells all dangers at once, so user allows them all or exits. In the case of updates, we need to include replay dangers and migrations if there are any between the two versions involved.

In addition, I think we could recommend running Copier only from inside an ephemeral container into which the destination folder is mounted. This way, malicious code could only cause harm inside the container which would get removed immediately after Copier has completed its work. Assuming the container runtime and the host kernel has no exploitable security vulnerability, the havoc should be minimal. In addition, gVisor <https://github.com/google/gvisor> might be used to strengthen security further.

Copier is a development tool, so it will usually need access to other development tools expected in that environment. I don't recommend running malicious templates in a container. I just recommend not running them 😆. However, it would be good to have a page in the docs dedicated to our threat model and possible workarounds. And that could happen even without implementing any other fix. It could even be "the fix". 🤔

*That said, I'd like to challenge the existence of the problem itself.* 1. Searching the internet for "executing untrusted code" (or similar), I find discussions related to, e.g., REPL services or other kinds of server-side code execution of user-provided code. Those scenarios are obviously very dangerous when not handled well because a compromised service/system would affect *all* users of the service/system. But with Copier, the damage would typically not affect other users. 2. I don't quite see the difference between running tasks/migrations/extensions and installing and using other 3rd party dependencies in any software project. With Copier, a template is like a dependency, somebody chooses to use this dependency and generate/update a project. With other 3rd party dependencies, somebody chooses to use this dependency and use it in a project – which means executing foreign code that I bet nobody reads first. 3. pre-commit – when used in the officially suggested (but IMHO wrong) way – installs hooks/packages into independent virtual envs, and then some executables are run. I bet nobody reads the code (and that of the entire dependency tree) first. I'm not saying Copier shouldn't be better than pre-commit, but I notice that other projects have comparable situations and don't seem to consider them problematic. So with these arguments, I'm not sure Copier has a security problem at all, or else we'd also need to change the way we integrate 3rd party dependencies in general.

Yes... in reality, I guess most users just will say always yes. Let's go for the docs route then. It seems the easiest MVP for increasing security. Also let's link to those docs from the warning we already print. Seems good?

…

[sisp]

The problem is, templates just need these features. Writing good migrations
takes time, but they provide smooth updates; but what if users are able to
turn them off? Then you can't predict the output. Tasks also are needed to
replay last render when updating. And jinja would die if you use it without
the required extensions.

Good point!

So, although it's OK to warn the user, the features are needed for copier
to do its work. Basically, running Copier without --UNSAFE would only work
with templates without these features. At least, we shouldn't require that
flag when the operation won't involve any of those. Also we should provide
a different exit code for this exit path.

👌

think it'd be OK to present a question before starting, which tells all
dangers at once, so user allows them all or exits. In the case of updates,
we need to include replay dangers and migrations if there are any between
the two versions involved.

👍

Copier is a development tool, so it will usually need access to other
development tools expected in that environment.

Also good point! I hadn't considered that.

I don't recommend running malicious templates in a container. I just
recommend not running them 😆.

However, it would be good to have a page in the docs dedicated to our
threat model and possible workarounds. And that could happen even without
implementing any other fix. It could even be "the fix". 🤔

Yes, I think docs plus the additional question when tasks/migrations/extensions are present or the --UNSAFE flag in case of non-interactive usage, as you described above.

I think this sounds like a good actions plan to raise awareness of the dangers and let users make conscious decisions. 👍

[yajo]

I created #1137