Repository navigation
[skills] Make the Firebase AI Logic skill platform-independent. - #225
Merged
Merged
Conversation
`SKILL.md` mixed Web SDK specifics (the `npm install` step, Chrome hybrid inference, reCAPTCHA Enterprise) into the shared setup flow, and its description mentioned only web apps even though the skill also covers Android, iOS, and Flutter.
`SKILL.md` now keeps only cross-platform guidance: provisioning the backend with the Firebase CLI via `npx`, which every platform needs, followed by a pointer to the platform reference for adding the SDK. The Web specifics move to the web reference, and the iOS reference now routes SDK installation to the `xcode-project-setup` skill. App Check debug-provider instructions, previously inline for Web, Android, and iOS only, now live in each platform reference; the iOS and new Flutter sections cover the required dependency, call order, debug-only gating, and CI token setup.
The same split applies to content from the latest GitHub import, cl/996819095. Its three per-platform debug token churn tips in `SKILL.md` become one platform-neutral tip, and their details merge into the debug token persistence steps of the Android, iOS, and Web references, which now sit in each reference's App Check section. Its new Text-to-Speech section in `SKILL.md` drops the Swift and JavaScript syntax and the platform audio APIs, which `references/tts_generation.md` already covers.
The frontmatter description now lists all supported platforms and says when to use the skill. The Node.js minimum moves from 16 to 20 to match `firebase-basics`, and the hybrid and App Check links point to the platform-neutral AI Logic docs.
Agents no longer install Node.js themselves; they ask the user to install it as `firebase-basics` describes. In an earlier eval run, an agent following the old "Install them if they aren't already available" line ran `sudo apt-get install nodejs npm` on the machine running the eval. The provisioning warning now also covers apps that already use Firebase, and agents that can't run `init ailogic` tell the user to run it, since leaving it out was the most common eval failure.
Evals after syncing past the import, baseline (`local_base`) vs. this CL, 3 runs per case with all 12 Firebase skills loaded:
- `EVAL.txtpb`: 5/18 → 4/18 passed,
https://evalin.corp.google.com/compare/5d831db67f0142cb/ff8e0caaf5754ce6
- `activationEval.txtpb`: 63/63 → 63/63 passed,
https://evalin.corp.google.com/compare/9c60d402ec5e4e9a/292460151a314091
Most of the `EVAL.txtpb` failures come from the eval setup, not the skill. In both arms, all nine runs of the three Flutter cases failed before the agent started, because `eval_fixtures/setup_flutter.sh` from cl/996169567 has a Python syntax error; pending cl/996505101 fixes it. All `integrate_ai_logic_complex` runs in both arms hit the case's 300-second limit, one CL run after meeting every expectation. The only difference is `basic_text_generation`, 2/3 → 1/3, and every failure of that case in both arms is the agent choosing `gemini-2.5-flash`.
Before the import, when the Flutter cases still ran, `EVAL.txtpb` results varied a lot between runs: across the four base-vs-CL runs on snapshots 3 to 6, the baseline passed 9 to 13 of 18 cases and this CL 12 to 15. The snapshot 6 run was 13/18 → 13/18: https://evalin.corp.google.com/compare/d7c79edc6c054110/6be69aba9a9341a2. The reviewer edits in snapshot 5 (numbered provisioning steps, section rules, and heading changes) scored the same as snapshot 4 in a head-to-head run, 13/18 → 13/18 with identical per-case results: https://evalin.corp.google.com/compare/299f1ff5d5ba4ac6/03d9c6874b3a406e.
PiperOrigin-RevId: 996079357
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
paulb777
approved these changes
Oct 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[skills] Make the Firebase AI Logic skill platform-independent.
SKILL.mdmixed Web SDK specifics (thenpm installstep, Chrome hybrid inference, reCAPTCHA Enterprise) into the shared setup flow, and its description mentioned only web apps even though the skill also covers Android, iOS, and Flutter.SKILL.mdnow keeps only cross-platform guidance: provisioning the backend with the Firebase CLI vianpx, which every platform needs, followed by a pointer to the platform reference for adding the SDK. The Web specifics move to the web reference, and the iOS reference now routes SDK installation to thexcode-project-setupskill. App Check debug-provider instructions, previously inline for Web, Android, and iOS only, now live in each platform reference; the iOS and new Flutter sections cover the required dependency, call order, debug-only gating, and CI token setup.The same split applies to content from the latest GitHub import, cl/996819095. Its three per-platform debug token churn tips in
SKILL.mdbecome one platform-neutral tip, and their details merge into the debug token persistence steps of the Android, iOS, and Web references, which now sit in each reference's App Check section. Its new Text-to-Speech section inSKILL.mddrops the Swift and JavaScript syntax and the platform audio APIs, whichreferences/tts_generation.mdalready covers.The frontmatter description now lists all supported platforms and says when to use the skill. The Node.js minimum moves from 16 to 20 to match
firebase-basics, and the hybrid and App Check links point to the platform-neutral AI Logic docs.Agents no longer install Node.js themselves; they ask the user to install it as
firebase-basicsdescribes. In an earlier eval run, an agent following the old "Install them if they aren't already available" line ransudo apt-get install nodejs npmon the machine running the eval. The provisioning warning now also covers apps that already use Firebase, and agents that can't runinit ailogictell the user to run it, since leaving it out was the most common eval failure.Evals after syncing past the import, baseline (
local_base) vs. this CL, 3 runs per case with all 12 Firebase skills loaded:EVAL.txtpb: 5/18 → 4/18 passed,https://evalin.corp.google.com/compare/5d831db67f0142cb/ff8e0caaf5754ce6
activationEval.txtpb: 63/63 → 63/63 passed,https://evalin.corp.google.com/compare/9c60d402ec5e4e9a/292460151a314091
Most of the
EVAL.txtpbfailures come from the eval setup, not the skill. In both arms, all nine runs of the three Flutter cases failed before the agent started, becauseeval_fixtures/setup_flutter.shfrom cl/996169567 has a Python syntax error; pending cl/996505101 fixes it. Allintegrate_ai_logic_complexruns in both arms hit the case's 300-second limit, one CL run after meeting every expectation. The only difference isbasic_text_generation, 2/3 → 1/3, and every failure of that case in both arms is the agent choosinggemini-2.5-flash.Before the import, when the Flutter cases still ran,
EVAL.txtpbresults varied a lot between runs: across the four base-vs-CL runs on snapshots 3 to 6, the baseline passed 9 to 13 of 18 cases and this CL 12 to 15. The snapshot 6 run was 13/18 → 13/18: https://evalin.corp.google.com/compare/d7c79edc6c054110/6be69aba9a9341a2. The reviewer edits in snapshot 5 (numbered provisioning steps, section rules, and heading changes) scored the same as snapshot 4 in a head-to-head run, 13/18 → 13/18 with identical per-case results: https://evalin.corp.google.com/compare/299f1ff5d5ba4ac6/03d9c6874b3a406e.