Skip to content

Commit ca372c9

Browse files
complexspacescomplexspaces
committed
Distribute Android component through pre-built Maven local repository
1 parent 5ea4c11 commit ca372c9

File tree

12 files changed

+229
-62
lines changed

12 files changed

+229
-62
lines changed

‎.gitignore

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,8 @@
33
/.idea
44

55
/android/verification/
6+
7+
# Ignore all generated Maven local repository files and folders
8+
/android-release-support/maven/pom.xml
9+
/android-release-support/maven/rustls/rustls-platform-verifier/**/
10+
/android-release-support/maven/rustls/rustls-platform-verifier/maven-metadata-local.xml

‎Cargo.lock

Lines changed: 5 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,7 @@
11
[workspace]
2-
members = ["rustls-platform-verifier"]
2+
members = [
3+
"android-release-support",
4+
"rustls-platform-verifier",
5+
]
36

47
resolver = "2"

‎README.md

Lines changed: 32 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -47,64 +47,55 @@ component must be included in your app's build to support `rustls-platform-verif
4747
#### Gradle Setup
4848

4949
`rustls-platform-verifier` bundles the required native components in the crate, but the project must be setup to locate them
50-
automatically and correctly.
50+
automatically and correctly. These steps assume you are using `.gradle` Groovy files because they're the most common, but everything
51+
is 100% applicable to Kotlin script (`.gradle.kts`) configurations too with a few replacements.
5152

52-
Firstly, create an [init script](https://docs.gradle.org/current/userguide/init_scripts.html) in your Android
53-
Gradle project, with a filename of `init.gradle`. This is generally placed in your project's root. In your project's `settings.gradle`, add these lines:
53+
Inside of your project's `build.gradle` file, add the following code and Maven repository definition. If applicable, this should only be the one "app" sub-project that
54+
will actually be using this crate at runtime. With multiple projects running this, your Gradle configuration performance may degrade.
5455

55-
```groovy
56-
apply from: file("./init.gradle");
57-
// Cargo automatically handles finding the downloaded crate in the correct location
58-
// for your project.
59-
def veifierProjectPath = findRustlsPlatformVerifierProject()
60-
includeBuild("${verifierProjectPath}/android/")
61-
```
62-
63-
Next, the `rustls-platform-verifier` external dependency needs to be setup. Open the `init.gradle` file and add the following:
64-
`$PATH_TO_DEPENDENT_CRATE` is the relative path to the Cargo manifest (`Cargo.toml`) of any crate in your workspace that depends on `rustls-platform-verifier`
65-
from the location of your `init.gradle` file.
66-
67-
Alternatively, you can use `cmdProcessBuilder.directory(File("PATH_TO_ROOT"))` to change the working directory instead.
56+
`$PATH_TO_DEPENDENT_CRATE` is the relative path to the Cargo manifest (`Cargo.toml`) of any crate in your workspace that depends on `rustls-platform-verifier` from
57+
the location of your `build.gradle` file:
6858

6959
```groovy
70-
ext.findRustlsPlatformVerifierProject = {
71-
def cmdProcessBuilder = new ProcessBuilder(new String[] { "cargo", "metadata", "--format-version", "1", "--manifest-path", "$PATH_TO_DEPENDENT_CRATE" })
72-
def dependencyInfoText = new StringBuffer()
60+
import groovy.json.JsonSlurper
7361
74-
def cmdProcess = cmdProcessBuilder.start()
75-
cmdProcess.consumeProcessOutput(dependencyInfoText, null)
76-
cmdProcess.waitFor()
62+
// ...Your own script code could be here...
7763
78-
def dependencyJson = new groovy.json.JsonSlurper().parseText(dependencyInfoText.toString())
79-
def manifestPath = file(dependencyJson.packages.find { it.name == "rustls-platform-verifier" }.manifest_path)
80-
return manifestPath.parent
64+
repositories {
65+
// ... Your other repositories could be here...
66+
maven {
67+
url = findRustlsPlatformVerifierProject()
68+
metadataSources.artifact()
69+
}
8170
}
82-
```
8371
84-
This script can be tweaked as best suits your project, but the `cargo metadata` invocation must be included so that the Android
85-
implementation source can be located on disk.
72+
String findRustlsPlatformVerifierProject() {
73+
def dependencyText = providers.exec {
74+
it.workingDir = new File("../")
75+
commandLine("cargo", "metadata", "--format-version", "1", "--manifest-path", "$PATH_TO_DEPENDENT_CRATE/Cargo.toml")
76+
}.standardOutput.asText.get()
8677
87-
If your project often updates its Android Gradle Plugin versions, you should additionally consider setting your app's project
88-
up to override `rustls-platform-verifier`'s dependency versions. This allows your app to control what versions are used and avoid
89-
conflicts. To do so, advertise a `versions.path` system property from your `settings.gradle`:
90-
91-
```groovy
92-
ext.setVersionsPath = {
93-
System.setProperty("versions.path", file("your/versions/path.toml").absolutePath)
78+
def dependencyJson = new JsonSlurper().parseText(dependencyText)
79+
def manifestPath = file(dependencyJson.packages.find { it.name == "rustls-platform-verifier-android" }.manifest_path)
80+
return new File(manifestPath.parentFile, "maven").path
9481
}
82+
```
9583

96-
setVersionsPath()
84+
Then, wherever you declare your dependencies, add the following:
85+
```groovy
86+
implementation "rustls:rustls-platform-verifier:latest.release"
9787
```
9888

99-
Finally, sync your gradle project changes. It should pick up on the `rustls-platform-verifier` Gradle project. It should finish
100-
successfully, resulting in a `rustls` group appearing in Android Studio's project view.
101-
After this, everything should be ready to use. Future updates of `rustls-platform-verifier` won't need any maintenance beyond the
102-
expected `cargo update`.
89+
Cargo automatically handles finding the downloaded crate in the correct location for your project. It also handles updating the version when
90+
new releases of `rustls-platform-verifier` are published. If you only use published releases, no extra maintenance should be required.
91+
92+
These script snippets can be tweaked as best suits your project, but the `cargo metadata` invocation must be included so that the Android
93+
implementation part can be located on-disk.
10394

10495
#### Proguard
10596

10697
If your Android application makes use of Proguard for optimizations, its important to make sure that the Android verifier component isn't optimized
107-
out because it looks like dead code. Proguard is unable to see any JNI usage, so your rules must manually opt into keeping it. THe following rule
98+
out because it looks like dead code. Proguard is unable to see any JNI usage, so your rules must manually opt into keeping it. The following rule
10899
can do this for you:
109100
```text
110101
-keep, includedescriptorclasses class org.rustls.platformverifier.** { *; }

‎admin/RELEASING.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# How-to release `rustls-platform-verifier`
2+
3+
This document records the steps to publish new versions of the crate since it requires non-trivial preparation and ordering
4+
that needs to be remembered due to the Android component's distribution.
5+
6+
## Steps
7+
8+
1. Update main crate's version in `rustls-platform-verifier/Cargo.toml`.
9+
2. If any non-test changes have been made to the `android` directory since the last release:
10+
1. Update Android artifact version in `android-release-support/Cargo.toml`
11+
2. Bump dependency version of the Android support crate in `rustls-platform-verifier/Cargo.toml` to match the new one
12+
3. Commit version increase changes on the release branch
13+
* We typically name these branches `rel-xxx` where `xxx` is the major version.
14+
* We typically leave these branches around for future maintenance releases.
15+
4. Run `ci/package_android_release.sh` in a UNIX compatible shell
16+
5. (Optional) `cargo publish -p rustls-platform-verifier-android --dry-run --alow-dirty`
17+
<!---
18+
TODO: Consider instead making tag-specific commits that check-in the artifacts. For now, the
19+
seamless AAR reproducibility makes this a non-issue.
20+
-->
21+
* `--allow-dirty` is required because we don't check-in the generated Maven local repository at this time.
22+
6. (Optional) Inspect extracted archive to ensure the local Maven repository artifacts are present
23+
1. Un-tar the `rustls-platform-verifier-android-*.crate` file inside of `target/package`.
24+
2. Verify `maven/rustls/rustls-platform-verifier` contains a single `*.RELEASE` directory and that contains a `.aar` file.
25+
3. (Optional) If the releaser has an external Gradle project that uses the configuration from the README, paste the path to the
26+
unzipped package's `Cargo.toml` as a replacement for the `manifestPath` variable. Run a Gradle Sync and observe everything works.
27+
7. Publish the Android artifacts' new version: `cargo publish -p rustls-platform-verifier-android --alow-dirty`
28+
3. Commit main crate's version increase on the release branch
29+
4. Publish the main crate's new version: `cargo publish -p rustls-platform-verifier`
30+
* Do **not** use `--allow-dirty` for the main crate. Only the Android component requires it and a dirty workspace elsewhere is an error.
31+
32+
See the Rustls repo [RELEASING] guidance for more information (e.g. on best practices for creating a GitHub release with a changelog).
33+
34+
[RELEASING]: https://github.com/rustls/rustls/blob/main/RELEASING.md

‎android-release-support/Cargo.toml

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
[package]
2+
name = "rustls-platform-verifier-android"
3+
version = "0.1.0"
4+
description = "The internal JVM support component of the rustls-platform-verifier crate. You shouldn't depend on this directly."
5+
repository = "https://github.com/rustls/rustls-platform-verifier"
6+
license = "MIT OR Apache-2.0"
7+
edition = "2021"
8+
9+
# Explicitly include the Maven local repository for the Android component.
10+
# While not checked into the repository, it is generated for releases and other contexts.
11+
include = [
12+
"src/*",
13+
"maven/pom.xml",
14+
"maven/rustls/rustls-platform-verifier/**/",
15+
"maven/rustls/rustls-platform-verifier/maven-metadata-local.xml",
16+
]
17+
18+
[dependencies]

‎android-release-support/maven/rustls/rustls-platform-verifier/.gitkeep

Whitespace-only changes.

‎android-release-support/pom-template.xml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
3+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
4+
<modelVersion>4.0.0</modelVersion>
5+
<groupId>rustls</groupId>
6+
<artifactId>rustls-platform-verifier</artifactId>
7+
<version>$VERSION</version>
8+
<packaging>aar</packaging>
9+
<description>The internal JVM support component of the rustls-platform-verifier Rust crate</description>
10+
</project>

‎android-release-support/src/lib.rs

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
//! # rustls-platform-verifier-android
2+
//!
3+
//! This crate is an implementation detail of the actual [rustls-platform-verifier](https://github.com/rustls/rustls-platform-verifier) crate.
4+
//!
5+
//! It contains no Rust code and is solely intended as a convenient delivery mechanism for the supporting Kotlin code that the main crate
6+
//! requires to perform TLS certificate validation using Android's APIs.
7+
//!
8+
//! Other crates should not directly depend on this crate in any way, as nothing about it is considered stable and it is probably useless elsewhere.
9+
//!
10+
//! ## Details
11+
//!
12+
//! Note: Everything in this section is subject to change at any time. Semver may not be followed.
13+
//!
14+
//! ### Why?
15+
//!
16+
//! It was the best middle ground between several tradeoffs. The important ones, in priority order, are:
17+
//! - Automatically keeping component versions in sync
18+
//! - Allowing well-tested and well-known `cargo` dependency management patterns to apply everywhere
19+
//! - Providing a smooth developer experience as an Android consumer of `rustls-platform-verifier`
20+
//!
21+
//! Firstly, what alternatives are available for distributing the component? The other two known are source distribution in some form (here, it will be through crates.io)
22+
//! and Maven Central. Starting with the first, its become infeasible due to toolchain syncing requirements. If the Android component is
23+
//! built as part of the host app's Gradle build, then it becomes subject to any Gradle or Android Gradle Plugin incompatibilities/requirements. In practice this means
24+
//! the AGP version between this project and the main application have to match all the time. Sometimes this works, but it becomes challenging/unfeasible
25+
//! during yearly toolchain/SDK upgrades and is not maintainable long term. Note that this is the _only_ option in this section which retains compatibility
26+
//! with Cargo's Git dependency patching.
27+
//!
28+
//! Next, Maven Central. This is considered the standard way of distributing public Android dependencies. There are two downsides to this
29+
//! approach: version synchronization and publishing overhead. Version syncing is the hardest part: There's not a good way to know what version
30+
//! a crate is that doesn't hurt the Cargo part of the build or damage functionality. So instead of making assumptions at runtime, we would need to do
31+
//! clunky and manual version counting with an extra error case. Less importantly, the admin overhead of Maven Central is non-zero so its good to avoid
32+
//! if possible for such a small need.
33+
//!
34+
//! It is also worth calling out a third set of much worse options: requiring users to manually download and install the Android component
35+
//! on each update, which magnifies the version syncing problem with lots of user overhead and then deleting the component outright. A rewrite
36+
//! could be done with raw JNI calls, but this would easily be 3x the size of the existing implementation and require huge amounts of `unsafe`
37+
//! to review then audit.
38+
//!
39+
//! ### The solution
40+
//!
41+
//! The final design was built to avoid the pitfalls the previous two options mentioned. To build it, we rely on CI and packaging scripts to build
42+
//! the Android component into a prebuilt AAR file before creating a release. Next, a [on-disk Maven repository](https://maven.apache.org/repositories/local.html)
43+
//! is hosted inside of this repository. Only the unchanging file structure of it is kept checked-in, to avoid churn. The remaining parts are filled in
44+
//! during the packaging/release process, before being included in `cargo package` via an `include` Cargo.toml directive. Finally, once the repository has had
45+
//! its artifacts added the crate containing the Maven repository is published to crates.io. Then, the main crate ensures it's downloaded when an Android target
46+
//! is compiled via a platform-specific dependency.
47+
//!
48+
//! On [the Gradle side](https://github.com/rustls/rustls-platform-verifier/tree/main#gradle-setup), we include a very small snippet of code for users to include in their `settings.gradle` file
49+
//! to dynamically locate the local maven repository on disk automatically based off Cargo's current version of it. The script is configuration cache friendly and
50+
//! doesn't impact performance either. When the script is run, it finds the cargo-cached download of the crate and tells Gradle it can find the Android component there
51+
//! when it gets sourced into the hosting application's build tree.
52+
//!
53+
//! Assuming a properly configured Gradle project, the slow (~500ms) script should only run once per Gradle sync while the `android-release-support` crate
54+
//! remains untouched. This is due to the configuration cache previously mentioned and is what ensures performance on-par with a "normal" Maven repository.
55+
//! Upon any version updates (semver, Git refs, etc), the change will be detected as-intended by Gradle, break the cache, and the project will update the dependency reference to the new AAR file.
56+
//!
57+
//! ### Precompiled artifacts?
58+
//!
59+
//! For some, the notion of shipping something pre-compiled with an existing source distribution might seem incorrect, or insecure. However in this specific case,
60+
//! putting aside the fact shipping Kotlin code doesn't work (see above), there are many reasons this isn't the case:
61+
//! - Shipping pre-compiled artifacts is normal in the Java ecosystem. Maven Central and other package repositories do the same thing and serve `.jar` downloads.
62+
//! - Those not using Android will never download the pre-compiled AAR file.
63+
//! - The artifacts are incredibly easy to reproduce given an identical compilation toolchain.
64+
//! - The artifacts are not native executables, or raw `.jar` files, so they can't be accidentally executed on a host system.
65+
//!
66+
//! ## Summary
67+
//!
68+
//! In summary, the selected distribution method avoids most of the previous pitfalls while still balancing a good experience for `cargo` and Gradle users. Some of its
69+
//! positive properties include:
70+
//! - Full compatibility with Cargo's dependency management, including Git patching[^1]
71+
//! - No version checking or synchronization required
72+
//! - Painless and harmless to integrate into an Android app's build system
73+
//! - Low maintenance for the main crate maintainers'
74+
//!
75+
//! [^1]: The Git reference being used must have the local maven repository built and checked-in first.

‎android/settings.gradle

Lines changed: 1 addition & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -12,19 +12,9 @@ dependencyResolutionManagement {
1212
mavenCentral()
1313
}
1414

15-
// We use a version catalog for two reasons:
16-
// 1. Ease of dependency management
17-
// 2. Supporting having our versions overridden by a larger project including us
18-
// as a composite build. This lets versions stay in sync when they would otherwise become
19-
// incompatible. Examples of this include AGP, where an actual app might have a newer one
20-
// then this library (which doesn't need to).
21-
//
22-
// This works by first trying to read global property that a parent module could advertise
23-
// and then falling back to our local definitions. In combination, both project-local tasks
24-
// still work as intended and use as a library in a full application.
2515
versionCatalogs {
2616
libs {
27-
from(files(System.getProperty("versions.path", "gradle/libraries.versions.toml")))
17+
from(files("gradle/libraries.versions.toml"))
2818
}
2919
}
3020
}

‎ci/package_android_release.sh

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
#!/usr/bin/env bash
2+
3+
# This script's purpose is to automate the build + packaging steps for the pre-compiled Android verifier component.
4+
# It works with template files and directories inside the `android-release-support/` part of the repository to setup
5+
# a Maven local repository and then add the pre-compiled AAR file into it for distribution. The results of this packaging
6+
# are then included by `cargo` when publishing `rustls-platform-verifier-android`.
7+
8+
set -euo pipefail
9+
10+
if ! type mvn > /dev/null; then
11+
echo "The maven CLI, mvn, is required to run this script."
12+
echo "Download it from: https://maven.apache.org/download.cgi"
13+
exit 1
14+
fi
15+
16+
version=$(grep -m 1 "version = " android-release-support/Cargo.toml | tr -d "version= " | tr -d '"')
17+
18+
echo "Packaging v$version of the Android support component"
19+
20+
pushd ./android
21+
22+
./gradlew assembleRelease
23+
24+
popd
25+
26+
artifact_name="rustls-platform-verifier-release.aar"
27+
28+
pushd ./android-release-support
29+
30+
artifact_path="../android/rustls-platform-verifier/build/outputs/aar/$artifact_name"
31+
32+
# Ensure no prior artifacts are present
33+
git clean -dfX "./maven/"
34+
35+
cp ./pom-template.xml ./maven/pom.xml
36+
37+
# This sequence is meant to workaround the incompatibilites between macOS's sed
38+
# command and the GNU command. Referenced from the following:
39+
# https://stackoverflow.com/questions/5694228/sed-in-place-flag-that-works-both-on-mac-bsd-and-linux
40+
sed -i.bak "s/\$VERSION/$version/" ./maven/pom.xml
41+
rm ./maven/pom.xml.bak
42+
43+
mvn install:install-file -Dfile="$artifact_path" -Dpackaging="aar" -DpomFile="./maven/pom.xml" -DlocalRepositoryPath="./maven/"

0 commit comments

Comments
 (0)