Automate building packages with different feature sets

[Vrixyz]

Current status

This pr adds a rust project to generate the current rapier2d and rapier3d folders, which will be removed once this PR is stable.

no-compat builds

When running ./prepare_builds/prepare_all_projects.sh, you should see multiple folders created under ./builds:

Run ./prepare_builds/build_all_projects.sh to build them. Each package will be in their respective pkg folder.

CI has been tested on a different branch on my repository .

Differences with master

It's interesting to see a few typos which have gone through manual copying, this PR helps with detecting and fixing those:

Differences for generated rapier2d with current master version:

diff rapier2d/build_rust.sh builds/rapier2d/build_rust.sh
1c1
< #! /bin/sh
---
> #!/bin/sh
3c3
< npx wasm-pack build
---
>  npx wasm-pack build
diff rapier2d/build_typescript.sh builds/rapier2d/build_typescript.sh
4c4
< cp -r ../src.ts/* pkg/src/.
---
> cp -r ../../src.ts/* pkg/src/.
diff rapier2d/Cargo.toml builds/rapier2d/Cargo.toml
2c2
< name = "dimforge_rapier2d_old" # Can't be named rapier2d which conflicts with the dependency.
---
> name = "dimforge_rapier2d" # Can't be named rapier2d which conflicts with the dependency.
20c20
< path = "../src/lib.rs"
---
> path = "../../src/lib.rs"
30c30
< rapier2d = { version = "0.22", features = [
---
> rapier2d = { version = "0.22.0", features = [
33d32
<     "enhanced-determinism",
37c36
< wasm-bindgen = "0.2.90"
---
> wasm-bindgen = "^0.2.90"
44,45d42
< libm = "0.2"
< parry2d = "0.17" # Use at least this version to fix a regression in 0.15.0
Only in rapier2d/: docs
diff rapier2d/package.json builds/rapier2d/package.json
13a14
>     "//": "Better keep rimraf version in sync with wasm-pack, see https://github.com/rustwasm/wasm-pack/issues/1444",
24c25
< }
---
> }
\ No newline at end of file
Only in rapier2d/: pkg
diff rapier2d/typedoc.json builds/rapier2d/typedoc.json
3c3,5
<     "entryPoints": ["./pkg/rapier.d.ts"],
---
>     "entryPoints": [
>         "./pkg/rapier.d.ts"
>     ],
40c42
< }
---
> }
\ No newline at end of file

Differences for generated rapier3d with current master version:

diff rapier3d/build_rust.sh builds/rapier3d/build_rust.sh
1c1
< #! /bin/sh
---
> #!/bin/sh
3c3
< npx wasm-pack build
---
>  npx wasm-pack build
12c12
< rm pkg/.gitignore
\ No newline at end of file
---
> rm pkg/.gitignore
diff rapier3d/build_typescript.sh builds/rapier3d/build_typescript.sh
4c4
< cp -r ../src.ts/* pkg/src/.
---
> cp -r ../../src.ts/* pkg/src/.
diff rapier3d/Cargo.toml builds/rapier3d/Cargo.toml
2c2
< name = "dimforge_rapier3d_old" # Can't be named rapier3d which conflicts with the dependency.
---
> name = "dimforge_rapier3d" # Can't be named rapier3d which conflicts with the dependency.
6c6
< documentation = "https://rapier.rs/rustdoc/rapier2d/index.html"
---
> documentation = "https://rapier.rs/rustdoc/rapier3d/index.html"
20c20
< path = "../src/lib.rs"
---
> path = "../../src/lib.rs"
33d32
<     "enhanced-determinism",
44d42
< parry2d = "0.17" # Use at least this version to fix a regression in 0.15.0
Only in rapier3d/: docs
diff rapier3d/package.json builds/rapier3d/package.json
13a14
>     "//": "Better keep rimraf version in sync with wasm-pack, see https://github.com/rustwasm/wasm-pack/issues/1444",
20,21c21,25
<     }
< }
---
>     },
>     "sideEffects": [
>         "./*.js"
>     ]
> }
\ No newline at end of file
Only in rapier3d/: pkg
diff rapier3d/typedoc.json builds/rapier3d/typedoc.json
3c3,5
<     "entryPoints": ["./pkg/rapier.d.ts"],
---
>     "entryPoints": [
>         "./pkg/rapier.d.ts"
>     ],
40c42
< }
---
> }
\ No newline at end of file

compat builds

When running npm run build in rapier-compat directory, a subdirectory builds for each feature set will be created to fit all built files into each. A final pkg folder is created in each of those directory.

gen2d and gen2d is similar for each folders, so it is generated once, then copied into each folders. the original is not removed to ease debugging and understanding each steps.

CI has been tested on a different branch on my repository

Adoption strategy

We want to move away from enhanced-determinism (cross-platform) to be the default behaviour, as it's a more advanced feature, and it would mean "negative features" packages being created.

This PR changes rapier2d and rapier3d packages behaviour to be now deterministic locally (opposed to cross-platform), this PR should be blocked until we do at least a release of current versions, warning users about this upcoming change, so they can prepare to migrate. Changelog should be clear about that.

Current limitations:

npm ci

• npm ci is difficult to use because package-lock.json should be existing before calling it.
  • workaround: use npm i

workspace

It's practical to use workspace to share options in the different rust projects (we've been using codegen unit).

• A rust workspace cannot have multiple projects with a similar name. previously, we've been using "dimforge_rapier2d, ...". Now with multiple feature flags, we have a few options:
  • we use more project names
  • we exclude generated projects, but workspace exclude does not support globs (search/create upstream cargo issue?)
  • we change hierarchy to have builds outside of workspace tree

I went with using more project names, but changing hierarchy is also a solid option: I concatenated additional features to the name, but we may want to customize it further through an explicit name or a hash when generating projects, let's see.

rapier-compat hardcoded projects list

Current rollup hardcodes all features, it would be cleaner to list all packages in runtime, from previous build step or with a config file shared with rust wasm builds.

Error handling

Scripts should stop on an error, or report all errors at the end. Currently some error may occur and be lost in the noise.

Other notes

• wasm-pack is bringing rimraf 3 and inflight 1.0.6 through their binary-install, see Deprecated packages issuing from wasm-pack rustwasm/wasm-pack#1444
• CI is a bit long (~15 minutes ; I think making it parallel would yield great results but logs would be difficult to follow as-is, let's discuss that as an improvement)

[Vrixyz]

It was brought up that a reader may infer that deterministic feature means the default is locally non-deterministic.

That's wrong: the additional feature deterministic refers here to cross-platform determinism, so the naming and changelog wording should be updated.

[Vrixyz]

todo : re integrate check on mac