Support non-entry point declared executables in wheels

[rickeylev]

While helping a user, I was pointed to an older dtschema wheel: https://pypi-browser.org/package/dtschema/dtschema-2021.10-py3-none-any.whl

Instead of having formally declared entry_points, it has e.g. {name}.data/scripts/{script-name} entries e.g. dtschema-2021.10.data/scripts/dt-doc-validate. I'm guessing its some older way of packaging entry points?

These are #!python shebang scripts, so obviously they're expecting an installer to rewrite them. They have custom startup logic.

Normally we point users to py_console_script_binary, however, using it in this case is tought because of the custom startup logic. Similarly, re-exporting the file directory doesn't help because of the shebang.

I think, ideally, we could detect these during the repo phase and have a rule that wraps or transforms them. All it really has to do is copy the content into a .py file, then stick that in a py_binary.

More short term, some docs or helper rule for how to get these to be executable and runnable.

[rickeylev]

A user on slack found the spec: https://packaging.python.org/en/latest/specifications/binary-distribution-format/#recommended-installer-features

In wheel, scripts are packaged in {distribution}-{version}.data/scripts/. If the first line of a file in scripts/ starts with exactly b'#!python', rewrite to point to the correct interpreter.

Rewriting it to the interpreter isn't really possible/useful, but detecting these and doing something automatically sounds very feasible.

[aignas]

#1000 is related.

[rickeylev]

A variation of OP: ruff. It has ruff.dist-info/scripts/ruff, but its a native executable, not a shell wrapper for a python bootstrap.

For this, the best thing to do is probably to just re-export the file directly.

Proposal:

Have the repo phase inspect files in scripts/. If it's a #!python script, then wrap it in something py_binary-like. If it's a native executable, create a target for it. The define_targets() function in the pypi code then looks something like:

def define_targets(py_shebangs):
  # glob returns list from loading phase.
  for f in glob("scripts/*"):
    if f in py_shebang:
      py_whl_script_binary(f)
    else:
      native.filegroup(f)

The nice thing is this detection should be easy in the repo phase code. It just has to check for #!python in the file, otherwise treat it as a native executable.