Cross-platform POSIX shell for tasks/migrations

[sisp]

Creating a Copier template with tasks/migrations that shall be compatible with Linux, macOS, and Windows is a bit cumbersome because there are numerous shells (especially Windows' CMD and PowerShell with completely different syntaxes and tools than Unix shells). Currently, there are two ways to handle this problem:

1. Write cross-platform Python scripts instead of using native shell commands and syntax.
2. Use Jinja conditionals to render different task commands for each operating system. Currently, a Jinja extension is necessary to provide OS information, but I've created feat: add OS identifier to render context #1188 to make it available through Copier directly.

Unless there are semantic differences in the task commands between the operating systems, I think those two approaches introduce unnecessary complexity because a template creator needs to deal explicitly with the different shells.

So, how about using a cross-platform shell instead? Then, template creators could write tasks independent of the template consumer's operating system and shell. I've drawn some inspiration from Task which uses mvdan.cc/sh as a shell interpreter:

Task uses mvdan.cc/sh, a native Go sh interpreter. So you can write sh/bash commands, and it will work even on Windows, where sh or bash are usually not available. Just remember any executable called must be available by the OS or in PATH.

— https://taskfile.dev/usage/#getting-started

mvdan.cc/sh is written in Go and I haven't found an equivalent library for Python, but it might be possible to create a Python binding using gopy.

WDYT?

[pawamoy]

Very personal opinion based on past experiences: I'm sick of having to go to such lengths to support Windows 😂

More seriously now. We should keep in mind that Windows users can install Git-for-Windows without admin rights which comes bundled with a Bash port for Windows. There's also WSL2, and other virtualization solutions. Template writers who want to support Windows can ask their users to use one of these solutions, or they can write their tasks in Python (or Ruby, JS, or any other language). Bundling a Python binding of a Go binary inside the Copier package seems a bit far fetched when these solutions are already available to template writers and users IMO.

But don't let me deter you from working towards a bundled cross-platform sh interpreter, that would still be a very cool feature.

[sisp]

Thanks for sharing your thoughts, @pawamoy!

I know what you mean. 😆 But unfortunately Windows is very dominant in some corporate environments.

I don't envision bundling the Go binding within Copier but rather creating a new (small) Python package on which Copier (and any other project that has a similar problem) would/could depend.

[pawamoy]

Oh OK, that would be better yes.

[sisp]

Perhaps even the sh project could provide this Python package: mvdan/sh#1011

[pawamoy]

Wouldn't providing such an interpreter, with shell constructs, but without usual executables (ls, grep, etc.) be confusing to users? They'd expect to be able to use the full potential of the shell, but they couldn't as executable presence would again depend on the OS?

[sisp]

Hm, maybe. 🤔 I'm not sure yet.

Getting wild now: https://github.com/uutils/coreutils/ 😆

[yajo]

There exists invoke. It is a task manager that is written in Python and lets you supply tasks written in that same language. In Doodba, I solved the problem long ago by just asking users to install it before using the template. You can do the same with Task if that's what you prefer.

There is plumbum too. With it, you write pure python, but can use handy shell-like syntax if needed. Plumbum is one of Copier dependencies, so it's always available for your tasks.

Finally, you can just use commands instead of tasks. Commands don't use a shell and thus work on any os.

So it seems to me like this problem has a lot of possible solutions without much work on your side!