Skip to content

setup.md

Latest commit

 

History

History
155 lines (99 loc) · 8.88 KB

setup.md

File metadata and controls

155 lines (99 loc) · 8.88 KB
Raw

Rez Setup

System Requirements

We expect to have the following packages, our reference OS is Fedora:

sudo dnf -y install @development-tools
sudo dnf -y install gcc-c++ nasm patchelf perl strace tcsh
sudo dnf -y install libffi-devel

sudo dnf -y install wayland-devel
sudo dnf -y install mesa-libGL-devel mesa-libGLU-devel mesa-libEGL-devel
sudo dnf -y install libXxf86vm-devel libXext-devel libXi-devel libXrandr-devel libXinerama-devel libXcursor-devel

Python

For rez itself, we have to install python on the OS.

The latest compatible version can be found in the installation documentation.

As we manage python installations itself via REZ packages, this 'base' python version is only used to drive REZ.

sudo python <rez>/install.py

As rez is based on python, we have to make sure that it doesn't start conflicting itself when we use different python versions and container runtimes.

On Linux this means we have to self-compile and rpath link the resulting libs as otherwise we get conflicts from conflicting libraries.

On Windows we are able to use the pre-compiled standalone releases.

Linking

We currently don't follow the best practice of leveraging CMake config/find functionality. Instead we favor explicit linking of lib/include dir paths.

Package Setup

We run our package build commands through python due to its easy-to-use nature.

The python that executes the command should be either the rez-python or the python variant of the resolved build context. To achieve this we must dynamically lookup the python version. This is achieved via the setup_package_utils.get_build_command function. This extracts the detected python version and then runs the actual build command with it.

To streamline the package definition setup, we follow a convention over configuration approach:

  • We automatically derive the package name and version from the file structure.
  • All package versions run through package-shared build.py.
  • This results in packages only defining the most important fields: description, private_build_requires, requires, variants and commands.

This reduces the surface area rez tries to cover and makes files more readable.

Tips & Tricks

Keep temporary files: export REZ_KEEP_TMPDIRS=1

OS & Containerization

Preamble

As our recipes support containerized environments, we need to specify what OS to use.

Before we breakdown our implementation, here is an summary of what we are achieving.

Terminology:

  • os: An operating system (e.g. fedora, rocky, almalinux).
  • flavor: A hint which the os container can decide to consume to determine the image name. This allows os to specify the OS, but switch between different variations of it.

Restrictions:

  • Ephemeral (packages starting with .) can't be used in variants.
  • We can't specify a build_requires and a variant of the same package type at the same time.

Requirements:

  • We need to be able to specify separate build and runtime container environments.
  • We need to bootstrap rez within the container, regardless of the 'flavor' used. This means there needs to be a rez-compatible python and a rez installation for that OS. This needs to be created as if installing rez on a bare-metal install of the OS. The location is defined via the container_bash shell plugin config.

Setup

We have the following packages to supply the relevant information to the resolve:

  • os: Define the OS to run in. The package must follow the os-<distro-name>-<version> syntax. If a package supports only a specific OS, it should require or variant ["os-<distro-name>-<version>"]. We also use the os package to correctly bootstrap rez within the containerized environment via the rez_container_bootstrap package.
  • .os_flavor_build: Override the build-time flavor. This can only be supplied in a private_build_requires. The package must follow the .os_flavor_build-(<flavor>) syntax. This allows us to customize the build environment on a per package level.
  • .os_flavor_run: Override the run-time flavor. This can only be supplied in a requires. The package must follow the .os_flavor_run-(<flavor>) syntax. This allows us to customize the runtime environment on a per package level.

We use ephemeral packages to avoid building dedicated packages for every flavor as well as automatically blocking us from creating flavor package variants. Using packages to drive the flavor also ensures that only a signle flavor is "active", as package resolution otherwise fails.

We use a separate build vs runtime override ephemeral packages, so that one can specify both simultaneously.

A default (fallback) build/run flavor can be specified through the global rezconfig.py shell plugin configuration.

Currently the config holds templates for the containerization command that then get populated with the above OS name/version/flavor values. This is static by design to avoid complexity, but can easily be extended to have dynamic processing.

To then achieve 'clean' environment construction, we have a dedicated rez shell plugin (subclassed from the bash shell), that manages building the command and injecting it in the right way.

Build workflows

As stated above we support:

  • requires, private_build_requires, variants: ["os-<distro-name>-<version>"]
  • private_build_requires: [.os_flavor_build-<flavor name>]
  • requires: [.os_flavor_run-<flavor name>]

Below are examples of supported build workflows.

Workflow A: Build on one OS with a devel environment, distribute to an OS-less runtime.

private_build_requires = ["os-fedora-43", ".os_flavor_build-devel"]
variants = []

Workflow B: Build on multiple OS' with a devel environment, distribute to a OS runtime.

private_build_requires = [".os_flavor_build-devel"]
variants = [["os-fedora-43"], ["os-rocky-9.6"]]

Workflow C: Build and run on a specific OS distribution:

requires = ["platform-linux", "arch-x86_64", "os-<distro-name>-<version>"]

We recommend adding the OS as a preventive measure for most packages to enable future multi-os package distribution (if we intend on not versioning up if a new OS is added).

Our rez_container_bootstrap package needs to be built in order for the container_shell to have access to rez within the shell.

⚠️ In order to build the rez_container_bootstrap package, we must temporarily disable it as an implicit package in the rezconfig.py, as otherwise there is a (recursive) conflict.

Rez, the PATH environment variable and containers

Rez inherits the environment via the usual shell inheritance rules, except for the PATH environment variable (and variables that are explicitly overridden).

The default behaviour is:

  • Start with an empty PATH list and apply the list pre/append operations as defined in the package command sections.
  • Append the system PATH directories (e.g. /usr/bin:/bin)
  • Prepend/Append the rez installation bin directory if rez_tools_visibility is configured.
  • In the case of interactive shell, source the shell standard profiles (e.g. .bashrc).

We disable rez adding itself to the path via the rez_tools_visibility='prepend' mode, as this injects the wrong rez path within the container. This happens because the rez path gets extracted from the current environment prior to entering the context.

With rez we have a chicken-and-egg situation of how to:

In order to have access to rez, we usually add rez to the PATH env var in our shell configuration files dynamically (e.g. .bash_profile/.bash_rc) or via system level environment sourcing files. If statically defined, the rez_tools_visibility=never setting, erases rez from the path (since it overrides the PATH value completely).

When running rez-env <packages ...>, our current shell environment profile sourcing mechanism gets executed when entering the shell, as it spawns a new interactive shell. This means in the case of dynamic allocation, our mechanism here needs to either a) not source rez, if it already finds its executables or b) append to it, so that the rez_container_bootstrap PATH edit has a higher priority. Note that Rez does some level of environment isolation in the shell (e.g. rez-shell.sh/context.sh) scripts to circumvent the system standard way of sourcing the shell environment, but it still more or less follows the 'standard' execution model of spawning a interactive shell.

When running rez-env <packages ...> -- <some command> (or rez-env <packages ...> -c <some command>), we don't enter an interactive shell. This means that our shell specific sourcing mechanism is not run. Therefore (if we use that mechanism to put rez on the PATH env var), we don't get rez in non-interactive shells. The rez_tools_visibility='prepend' setting by default would prevent this problem, but since that is sourced from the active environment, we would be getting the wrong rez installation. Therefore we bootstrap it ourselves via the rez_container_bootstrap package.