[AINFRA-1533] Adopt git-conceal in this repo
#14979
There are no files selected for viewing
|
Contributor
Author
There was a problem hiding this comment. 🎗️ TODO: Point back to stable version once Automattic/a8c-ci-toolkit-buildkite-plugin#195 has been merged and released |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,13 @@ | ||
| RELEASE-NOTES.txt merge=union | ||
|
|
||
| ######################################### | ||
| # Secrets files encrypted with git-conceal | ||
| ######################################### | ||
|
|
||
| secrets.properties filter=git-conceal diff=git-conceal | ||
| sentry.properties filter=git-conceal diff=git-conceal | ||
| google-services.json filter=git-conceal diff=git-conceal | ||
| firebase.secrets.json filter=git-conceal diff=git-conceal | ||
| google-upload-credentials.json filter=git-conceal diff=git-conceal | ||
| *.keystore filter=git-conceal diff=git-conceal | ||
| *.jks filter=git-conceal diff=git-conceal |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -17,7 +17,7 @@ plugins { | |
|
|
||
| fladle { | ||
| variant = "vanillaDebug" | ||
| serviceAccountCredentials = rootProject.file("firebase.secrets.json") | ||
| testTargets = [ | ||
| "notPackage com.woocommerce.android.e2e.tests.screenshot", | ||
| "notClass com.woocommerce.android.e2e.tests.ui.OrdersRealAPI", | ||
|
|
@@ -498,12 +498,13 @@ android.buildTypes.all { buildType -> | |
| } | ||
|
|
||
| // If Google services file doesn't exist, copy example file. | ||
| def googleServicesFile = file("google-services.json") | ||
| if (!googleServicesFile.exists() || isFileEncrypted(googleServicesFile)) { | ||
| tasks.copyGoogleServicesExampleFile.copy() | ||
| } | ||
AliSoftware marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| // Print warning message if example Google services file is used. | ||
| if ((googleServicesFile.text) == (file("google-services.json-example").text)) { | ||
| println("WARNING: You're using the example google-services.json file. Google login will fail.") | ||
| } | ||
| } | ||
|
|
@@ -516,6 +517,13 @@ static def loadPropertiesFromFile(inputFile) { | |
| return properties | ||
| } | ||
|
|
||
| static def isFileEncrypted(File file) { | ||
| def gitconcealHeader = [0x00, 0x61, 0x38, 0x63, 0x63, 0x72, 0x79, 0x70, 0x74] as byte[] // git-conceal header | ||
| def header = new byte[gitconcealHeader.length] | ||
| file.withInputStream { stream -> stream.read(header) } | ||
| return Arrays.equals(header, gitconcealHeader) | ||
| } | ||
|
|
||
| def isLeakCanaryEnabled() { | ||
| return developerProperties.get("enable_leak_canary") ?: true | ||
| } | ||
|
|
||
|
Contributor
There was a problem hiding this comment. 💭 Again thinking out loud, not sure if it's worth the trouble: what do you think of grouping encrypted / secret files in the same folder and making a convention out of this for all projects (well, kinda similar to what we had before but making it more obvious)?
Contributor
Author
There was a problem hiding this comment. Yeah, I thought about that. I think that for some particular secret files, they do need to be at specific locations in the project structure for the compilation to work (e.g. For others ( The only trick is that if they were already existing in the repo a particular location on disk before that PR, and we move them elsewhere as part of this PR, we'd then need to add their old location back to I still see the appeal of having secret files grouped nicely in a dedicated folder rather than keeping them at the root of the repo still. But I'm not sure moving them is worth the potential risk (or having to keep legacy entries in
Contributor
There was a problem hiding this comment.
👍 I'm also leaning towards that. It's also annoying that they can't be in the same place, which would be the ideal scenario and perhaps would make the change worth it. |
|
Contributor
There was a problem hiding this comment. 💭 I was wondering if there could be a way to be sure, in reviews, that this has in fact been encrypted or not specially given files like this are binary files. Then I've noticed all
Contributor
Author
There was a problem hiding this comment. You mean like I do here? 😛 Indeed maybe we can write a Dangermattic plugin to detect which file in the PR are encrypted and add an inline comment on the file if so as an extra information? Is that what you meant? As for manually testing locally if a file is properly encrypted before pushing a commit to the remote, one can use
Contributor
There was a problem hiding this comment.
Ha, yes, I realized that function was doing that after I posted the comment 😂
Yeah, though at the same time I find it a bit difficult to do that in a systemic way that will be useful (as we don't add secrets that often)...
👍 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -13,11 +13,8 @@ When creating your application, you should select "**Native client**" for the ap | |
| The "**Website URL**", "**Redirect URLs**", and "**Javascript Origins**" fields are required but not used for | ||
| the mobile apps. Just use "**[https://localhost](https://localhost)**". | ||
|
|
||
| Once you've created your application in the [applications manager][wp-com-apps], you'll need to update the `wc.oauth.app_id` and `wc.oauth.app_secret` fields in `defaults.properties` (copied from `defaults-example.properties`). See [setup instructions][setup] for more details. | ||
| Then you can compile and run the app on a device or an emulator and try to login with a WordPress.com account. Note that authenticating to WordPress.com via Google is not supported in development builds of the app, only in the official release. | ||
|
|
||
| Note that credentials created with our [WordPress.com applications manager][wp-com-apps] | ||
| allow login only and not signup. New accounts must be created using the [official app][wp-app] | ||
|
|
@@ -35,7 +32,12 @@ Read more about [OAuth2][oauth] and the [WordPress.com REST endpoint][wp-api]. | |
|
|
||
| #### `secrets.properties` | ||
|
|
||
| The `secrets.properties` file is used to store sensitive information that should not be checked into version control in clear text. | ||
| This file is encrypted (using [`git-conceal`](https://github.com/Automattic/git-conceal)), and only developers working at Automattic have the decryption key. | ||
|
|
||
| If you are a developer working at Automattic, follow the steps [in the README.md](../README.md#-setup-instructions) to decrypt the encrypted files (including `secrets.properties`). | ||
|
|
||
| If you are an external contributor, provide those variables in your `defaults.properties` instead: | ||
|
|
||
| | Property | Description | | ||
| |:---------------------------|:------------| | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -62,7 +62,7 @@ GLOTPRESS_APP_STRINGS_PROJECT_URL = 'https://translate.wordpress.com/projects/wo | |
| GLOTPRESS_PLAYSTORE_METADATA_PROJECT_URL = "#{GLOTPRESS_APP_STRINGS_PROJECT_URL}/release-notes/".freeze | ||
|
|
||
| APP_PACKAGE_NAME = 'com.woocommerce.android' | ||
| GOOGLE_FIREBASE_SECRETS_PATH = File.join(PROJECT_ROOT_FOLDER, 'firebase.secrets.json') | ||
|
|
||
| # Instantiate versioning classes | ||
| VERSION_CALCULATOR = Fastlane::Wpmreleasetoolkit::Versioning::MarketingVersionCalculator.new | ||
|
|
@@ -80,7 +80,7 @@ DEFAULT_BRANCH = 'trunk' | |
| REPOSITORY_NAME = 'woocommerce-android' | ||
| GH_ORG_NAME = 'woocommerce' | ||
|
|
||
| UPLOAD_TO_PLAY_STORE_JSON_KEY = File.join(PROJECT_ROOT_FOLDER, 'google-upload-credentials.json') | ||
|
|
||
| SUPPORTED_LOCALES = [ | ||
| { glotpress: 'ar', android: 'ar', google_play: 'ar', promo_config: {} }, | ||
|
|
@@ -476,8 +476,6 @@ platform :android do | |
| UI.important("Downloading latest translations for release: #{release_version_current}") | ||
| UI.user_error!("Terminating as requested. Don't forget to run the remainder of this automation manually.") unless skip_confirm || UI.confirm('Do you want to continue?') | ||
|
|
||
| # Don't check translation coverage in CI | ||
| check_translation_progress_all unless is_ci | ||
| download_translations | ||
|
|
@@ -511,8 +509,6 @@ platform :android do | |
| UI.important("Finalizing release: #{release_version_current}") | ||
| UI.user_error!("Terminating as requested. Don't forget to run the remainder of this automation manually.") unless skip_confirm || UI.confirm('Do you want to continue?') | ||
|
|
||
| # Bump the release version and build code | ||
| UI.message 'Bumping final release version and build code...' | ||
| VERSION_FILE.write_version( | ||
|
|
@@ -1485,8 +1481,11 @@ platform :android do | |
| end | ||
|
|
||
| def firebase_secret(name:) | ||
| begin | ||
| key_file_secrets = JSON.parse(File.read(GOOGLE_FIREBASE_SECRETS_PATH)) | ||
| rescue StandardError | ||
| UI.user_error!('Unable to read Firebase Secrets File – did you run `echo "…encryption-key…" | base64 -d | git-conceal unlock -` on the repo?') | ||
| end | ||
| UI.user_error!("Unable to find key `#{name}` in #{GOOGLE_FIREBASE_SECRETS_PATH}") if key_file_secrets[name].nil? | ||
| key_file_secrets[name] | ||
| end | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Having the repo
git-crypt-unlocked during the branch dance that is done internally bycomment_with_manifest_diffcausedYour local changes to the following files would be overwritten by checkoutissues, especially if thegit-crypt'd files listed in.gitattributeson the HEAD branch are not the same as the ones in the BASE branch1Since we don't need any secret in practice to generate the manifest and call
process{variant}Manifest, the solution is simple: just don't bother unlocking the repo's secrets for that task.A better long-term solution to make
comment_with_manifest_diffmore resilient to situations like this would be to make it usegit worktreeinstead of switching branches in-place:git worktree add $TMP_DIR_FOR_BASE $BASE_BRANCH && cd $TMP_DIR_FOR_BASEthen run./gradlew process{variant}Manifesttherecd $CHECKOUT_DIR && rm $TMP_DIR_FOR_BASE && git worktree prunethen run./gradlew process{variant}ManifestthereThat way each checkout is done in independent folders, eliminating the risk of conflicts during the branch dance.
Footnotes
like will be the case during that transition to
git-crypt, or when we'll add a new secret file, especially if that secret file previously existed unencrypted in the BASE branch as an example file for external contributors I think? ↩