Skip to content

robotpy-build meson rewrite #252

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: next
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
61 changes: 29 additions & 32 deletions .github/workflows/dist.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ jobs:
with:
submodules: true

- uses: actions/setup-python@v4
- uses: actions/setup-python@v5
with:
python-version: 3.8
- name: Sphinx
Expand All @@ -67,7 +67,7 @@ jobs:
submodules: recursive
fetch-depth: 0

- uses: actions/setup-python@v4
- uses: actions/setup-python@v5
with:
python-version: 3.8

Expand Down Expand Up @@ -97,12 +97,12 @@ jobs:
- '3.11'
- '3.12'
- '3.13'
architecture: [x86, x64]
exclude:
- os: macos-13
architecture: x86
- os: ubuntu-22.04
architecture: x86
architecture: [x64]
# exclude:
# - os: macos-13
# architecture: x86
# - os: ubuntu-22.04
# architecture: x86
include:
- os: macos-14
python_version: 3.9
Expand All @@ -116,7 +116,7 @@ jobs:
with:
submodules: true

- uses: actions/setup-python@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python_version }}
architecture: ${{ matrix.architecture }}
Expand All @@ -133,25 +133,32 @@ jobs:
key: ${{ matrix.os }}-${{ matrix.architecture }}-${{ matrix.python_version }}
variant: ccache

- name: Setup MSVC
uses: bus1/cabuild/action/msdevshell@e22aba57d6e74891d059d66501b6b5aed8123c4d # v1
with:
architecture: x64
if: runner.os == 'Windows'

- name: Configure ccache for MSVC
shell: bash
# https://github.com/mesonbuild/meson/issues/10423
run: |
echo "CC=ccache.exe cl" >> $GITHUB_ENV
echo "CXX=ccache.exe cl" >> $GITHUB_ENV
if: runner.os == 'Windows'

- name: Install
shell: bash
working-directory: dist
run: python -m pip --disable-pip-version-check install *.whl

- name: Install test dependencies
shell: bash
working-directory: tests
run: python -m pip --disable-pip-version-check install -r requirements.txt

- name: Test wheel
shell: bash
env:
RPYBUILD_PARALLEL: 1
RPYBUILD_STRIP_LIBPYTHON: 1
RPYBUILD_CC_LAUNCHER: ccache
working-directory: tests
run: |
python run_tests.py
run: python run_tests.py

cross-build:
runs-on: ubuntu-latest
Expand All @@ -160,8 +167,8 @@ jobs:
max-parallel: ${{ fromJSON(needs.setup_concurrency.outputs.max-parallel).v }}
matrix:
container:
- wpilib/roborio-cross-ubuntu:2024-22.04-py312
- wpilib/raspbian-cross-ubuntu:bullseye-22.04-py312
- wpilib/roborio-cross-ubuntu:2025-22.04-py313
- wpilib/raspbian-cross-ubuntu:2025-bookworm-22.04-py313

container:
image: "${{ matrix.container }}"
Expand All @@ -183,33 +190,23 @@ jobs:
name: dist
path: dist

- name: Install setuptools
run: |
/build/venv/bin/build-pip --disable-pip-version-check install setuptools
/build/venv/bin/cross-pip --disable-pip-version-check install setuptools
- name: Setup cross environment
uses: robotpy/build-actions/setup-cross-meson@semiwrap

- name: Install
working-directory: dist
run: |
# install to both cross and build in case things get mixed up
/build/venv/bin/build-pip --disable-pip-version-check install *.whl
/build/venv/bin/cross-pip --disable-pip-version-check install *.whl

- name: Install test dependencies
shell: bash
working-directory: tests
run: |
# install to both cross and build in case things get mixed up
/build/venv/bin/build-pip --disable-pip-version-check install -r requirements.txt
/build/venv/bin/cross-pip --disable-pip-version-check install -r requirements.txt

- name: Build cross wheel
env:
RPYBUILD_PARALLEL: 1
RPYBUILD_STRIP_LIBPYTHON: 1
RPYBUILD_CC_LAUNCHER: ccache
working-directory: tests/cpp
run: /build/venv/bin/cross-python run_install.py wheel
run: /build/venv/bin/cross-python run_install.py --config-settings=setup-args=--cross-file=cross.txt

publish:
runs-on: ubuntu-latest
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ __pycache__
/dist
/build

/robotpy_build/version.py
/src/semiwrap/version.py
4 changes: 2 additions & 2 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[submodule "robotpy_build/pybind11"]
path = robotpy_build/pybind11
[submodule "src/semiwrap/pybind11"]
path = src/semiwrap/pybind11
url = https://github.com/pybind/pybind11.git
2 changes: 1 addition & 1 deletion LICENSE.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
Copyright (c) 2020 Dustin Spicuzza <[email protected]>, All rights reserved.
Copyright (c) 2020-2025 Dustin Spicuzza <[email protected]>, All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
Expand Down
2 changes: 0 additions & 2 deletions MANIFEST.in

This file was deleted.

29 changes: 16 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,29 @@
robotpy-build
=============
semiwrap
========

This is a build tool intended to be generally useful for any python project
that has binary dependencies. It is especially designed to meet the needs
of RobotPy's various wrapper libraries, chiefly around:
semiwrap is a build tool that makes it simpler to wrap C/C++ libraries with
pybind11 by automating large portions of the wrapping process and handling some
of the more complex aspects of creating pybind11 based wrappers (especially with
trampolines to allow inheriting from C++ classes from Python).

* Managing upstream binary dependencies
* Autogenerating pybind11 wrappers around those dependencies
* Building wheels from those generated wrappers
semiwrap includes a hatchling plugin that autogenerates `meson.build` files that
can be built using meson, and those build files parse your wrapped headers and
generate/compile pybind11 based wrappers into python extension modules.

Requires Python 3.8+

Documentation
-------------

Documentation can be found at https://robotpy-build.readthedocs.io/
Documentation can be found at https://semiwrap.readthedocs.io/

Author
------

Dustin Spicuzza is the primary author of robotpy-build, but it is the
culmination of many years of experimentation with automated wrapper
generation by members of the RobotPy community.
Dustin Spicuzza is the primary author of semiwrap.

robotpy-build is available under the BSD 3-clause license.
Semiwrap is a direct decendant of the robotpy-build project, and is culmination
of many years of experimentation with automated wrapper generation by members of
the RobotPy community.

semiwrap is available under the BSD 3-clause license.
19 changes: 10 additions & 9 deletions docs/autowrap.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
Automated C++ header wrapping
=============================

robotpy-build can be told to parse C/C++ headers and automatically generate
semiwrap can be told to parse C/C++ headers and automatically generate
:std:doc:`pybind11 <pybind11:basics>` wrappers around the functions
and objects found in that header.

Expand All @@ -15,9 +15,9 @@ and objects found in that header.
C++ Features
------------

robotpy-build uses a pure python C++ parser and macro processor to attempt to
semiwrap uses a pure python C++ parser and macro processor to attempt to
parse header files. As a result, a full AST of the header files is not created.
This means particularly opaque code might confuse the parser, as robotpy-build
This means particularly opaque code might confuse the parser, as semiwrap
only receives the names, not the actual type information.

However, most basic features typically work without needing to coerce the
Expand All @@ -34,6 +34,7 @@ generator into working correctly, including:
* final classes/methods - cannot be overridden from Python code
* Enumerations
* Global variables
* Many many more weird edge cases too

Additionally, the following features are supported, but require some manual
intervention:
Expand All @@ -46,22 +47,22 @@ to your package in ``pyproject.toml``:

.. code-block:: toml

[tool.robotpy-build.wrappers."MYPACKAGE".autogen_headers]
[tool.semiwrap.extension_modules."PACKAGE.NAME".headers]
demo = "demo.h"

That causes ``demo.h`` to be parsed and wrapped.

.. note:: If you're importing a large number of headers, the
``robotpy-build scan-headers`` tool can generate this for you
``semiwrap scan-headers`` tool can generate this list for you
automatically.

Documentation
-------------

robotpy-build will find doxygen documentation comments on many types of elements
and use sphinxify to translate them into python docstrings. All elements that
support documentation strings can have their docstrings set explicitly using
a ``doc`` value in the YAML file.
semiwrap will find doxygen documentation comments on many types of elements
and use sphinxify to translate them into python docstrings. If this is not
sufficient, all elements that support documentation strings can have their
docstrings set explicitly using a ``doc`` value in the YAML file.

.. code-block:: yaml

Expand Down
6 changes: 3 additions & 3 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

os.environ["GENERATING_DOCUMENTATION"] = "True"

__version__ = pkg_resources.get_distribution("robotpy-build").version
__version__ = pkg_resources.get_distribution("semiwrap").version


# -- RTD configuration ------------------------------------------------
Expand All @@ -25,8 +25,8 @@

# -- Project information -----------------------------------------------------

project = "robotpy-build"
copyright = "2020, RobotPy Development Team"
project = "semiwrap"
copyright = "2025, RobotPy Development Team"
author = "RobotPy Development Team"

# The full version, including alpha/beta/rc tags
Expand Down
56 changes: 29 additions & 27 deletions docs/config/autowrap.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,66 +4,68 @@
Generator Customization
=======================

Because robotpy-build's code generation is intended to be a semi-automated
Because semiwrap's code generation is intended to be a semi-automated
process (except for simple C++ code), a rich set of per-{class/function/parameter}
configuration options can be specified in per-file YAML configurations.

Additionally, some headers are too complex for the autogenerator to
completely process, so when this occurs you must manually specify required
information in the YAML file.

Most files generated by robotpy-build are customizable.
Most files generated by semiwrap are customizable.

.. note:: robotpy-build is designed for the RobotPy project and may contain
.. note:: semiwrap is designed for the RobotPy project and may contain
defaults that aren't appropriate for all projects. If you find that
you need more customization, file an issue on github and let's talk
about it!

Location of customization file
------------------------------

In your ``pyproject.toml``, you can specify either a single YAML file with
customizations, or you can specify a directory that robotpy-build will search
for YAML files.

Single file:

.. code-block:: toml

[tool.robotpy-build.wrappers."PACKAGENAME"]
generation_data = "gen/data.yml"

Multiple files:
By default customization files are located in the ``semiwrap`` directory. The name
of the YAML file is the ``key`` in the extension module headers table:

.. code-block:: toml

[tool.robotpy-build.wrappers."PACKAGENAME"]
generation_data = "gen"

When a directory is specified, pybind11 will search for YAML files in the
directory based on the header filename. In the above example, customization
data for ``header.h`` could be specified in ``gen/header.yml``.
[tool.semiwrap.extension_modules."PACKAGE.NAME".headers]
# yaml file is `semiwrap/demo.yml`
demo = "include/demo.h"


Autogeneration
--------------

The default values for these YAML files can be generated via the robotpy-build
command line tool:
The default values for these YAML files should be generated via the semiwrap command
line tool:

.. code-block:: sh

robotpy-build create-gen --write
semiwrap create-yaml --write

This can be a good way to get the boilerplate out of the way when you need to
provide customizations.

Manual pybind11 APIs
--------------------

You can write your own custom .cpp files and add them to the python extensions
generated by semiwrap. In your ``meson.build`` after you include the generated
``semiwrap`` subdir, add files to ``NAME_sources``.

.. code-block:: meson

subdir('semiwrap')

my_module_sources += files(
'src/my_module/main.cpp',
)


Reference
---------

The following strctures describe the dictionaries that are read from the YAML
file. The toplevel structure is :class:`.AutowrapConfigYaml`.
files. The toplevel structure is :class:`.AutowrapConfigYaml`.

.. automodule:: robotpy_build.config.autowrap_yml
.. automodule:: semiwrap.config.autowrap_yml
:members:
:undoc-members:
Loading