Skip to content

[skills] Make the Firebase AI Logic skill platform-independent. - #225

Merged
paulb777 merged 1 commit into
mainfrom
99989F3414F5C213CB70A29B36593F38
Oct 10, 2026
Merged

paulb777 merged 1 commit into
mainfrom
99989F3414F5C213CB70A29B36593F38

Conversation

@copybara-service

Copy link
Copy Markdown
Contributor

[skills] Make the Firebase AI Logic skill platform-independent.

SKILL.md mixed Web SDK specifics (the npm install step, Chrome hybrid inference, reCAPTCHA Enterprise) into the shared setup flow, and its description mentioned only web apps even though the skill also covers Android, iOS, and Flutter.

SKILL.md now keeps only cross-platform guidance: provisioning the backend with the Firebase CLI via npx, which every platform needs, followed by a pointer to the platform reference for adding the SDK. The Web specifics move to the web reference, and the iOS reference now routes SDK installation to the xcode-project-setup skill. App Check debug-provider instructions, previously inline for Web, Android, and iOS only, now live in each platform reference; the iOS and new Flutter sections cover the required dependency, call order, debug-only gating, and CI token setup.

The same split applies to content from the latest GitHub import, cl/996819095. Its three per-platform debug token churn tips in SKILL.md become one platform-neutral tip, and their details merge into the debug token persistence steps of the Android, iOS, and Web references, which now sit in each reference's App Check section. Its new Text-to-Speech section in SKILL.md drops the Swift and JavaScript syntax and the platform audio APIs, which references/tts_generation.md already covers.

The frontmatter description now lists all supported platforms and says when to use the skill. The Node.js minimum moves from 16 to 20 to match firebase-basics, and the hybrid and App Check links point to the platform-neutral AI Logic docs.

Agents no longer install Node.js themselves; they ask the user to install it as firebase-basics describes. In an earlier eval run, an agent following the old "Install them if they aren't already available" line ran sudo apt-get install nodejs npm on the machine running the eval. The provisioning warning now also covers apps that already use Firebase, and agents that can't run init ailogic tell the user to run it, since leaving it out was the most common eval failure.

Evals after syncing past the import, baseline (local_base) vs. this CL, 3 runs per case with all 12 Firebase skills loaded:

Most of the EVAL.txtpb failures come from the eval setup, not the skill. In both arms, all nine runs of the three Flutter cases failed before the agent started, because eval_fixtures/setup_flutter.sh from cl/996169567 has a Python syntax error; pending cl/996505101 fixes it. All integrate_ai_logic_complex runs in both arms hit the case's 300-second limit, one CL run after meeting every expectation. The only difference is basic_text_generation, 2/3 → 1/3, and every failure of that case in both arms is the agent choosing gemini-2.5-flash.

Before the import, when the Flutter cases still ran, EVAL.txtpb results varied a lot between runs: across the four base-vs-CL runs on snapshots 3 to 6, the baseline passed 9 to 13 of 18 cases and this CL 12 to 15. The snapshot 6 run was 13/18 → 13/18: https://evalin.corp.google.com/compare/d7c79edc6c054110/6be69aba9a9341a2. The reviewer edits in snapshot 5 (numbered provisioning steps, section rules, and heading changes) scored the same as snapshot 4 in a head-to-head run, 13/18 → 13/18 with identical per-case results: https://evalin.corp.google.com/compare/299f1ff5d5ba4ac6/03d9c6874b3a406e.

`SKILL.md` mixed Web SDK specifics (the `npm install` step, Chrome hybrid inference, reCAPTCHA Enterprise) into the shared setup flow, and its description mentioned only web apps even though the skill also covers Android, iOS, and Flutter.

`SKILL.md` now keeps only cross-platform guidance: provisioning the backend with the Firebase CLI via `npx`, which every platform needs, followed by a pointer to the platform reference for adding the SDK. The Web specifics move to the web reference, and the iOS reference now routes SDK installation to the `xcode-project-setup` skill. App Check debug-provider instructions, previously inline for Web, Android, and iOS only, now live in each platform reference; the iOS and new Flutter sections cover the required dependency, call order, debug-only gating, and CI token setup.

The same split applies to content from the latest GitHub import, cl/996819095. Its three per-platform debug token churn tips in `SKILL.md` become one platform-neutral tip, and their details merge into the debug token persistence steps of the Android, iOS, and Web references, which now sit in each reference's App Check section. Its new Text-to-Speech section in `SKILL.md` drops the Swift and JavaScript syntax and the platform audio APIs, which `references/tts_generation.md` already covers.

The frontmatter description now lists all supported platforms and says when to use the skill. The Node.js minimum moves from 16 to 20 to match `firebase-basics`, and the hybrid and App Check links point to the platform-neutral AI Logic docs.

Agents no longer install Node.js themselves; they ask the user to install it as `firebase-basics` describes. In an earlier eval run, an agent following the old "Install them if they aren't already available" line ran `sudo apt-get install nodejs npm` on the machine running the eval. The provisioning warning now also covers apps that already use Firebase, and agents that can't run `init ailogic` tell the user to run it, since leaving it out was the most common eval failure.

Evals after syncing past the import, baseline (`local_base`) vs. this CL, 3 runs per case with all 12 Firebase skills loaded:

-   `EVAL.txtpb`: 5/18 → 4/18 passed,
    https://evalin.corp.google.com/compare/5d831db67f0142cb/ff8e0caaf5754ce6
-   `activationEval.txtpb`: 63/63 → 63/63 passed,
    https://evalin.corp.google.com/compare/9c60d402ec5e4e9a/292460151a314091

Most of the `EVAL.txtpb` failures come from the eval setup, not the skill. In both arms, all nine runs of the three Flutter cases failed before the agent started, because `eval_fixtures/setup_flutter.sh` from cl/996169567 has a Python syntax error; pending cl/996505101 fixes it. All `integrate_ai_logic_complex` runs in both arms hit the case's 300-second limit, one CL run after meeting every expectation. The only difference is `basic_text_generation`, 2/3 → 1/3, and every failure of that case in both arms is the agent choosing `gemini-2.5-flash`.

Before the import, when the Flutter cases still ran, `EVAL.txtpb` results varied a lot between runs: across the four base-vs-CL runs on snapshots 3 to 6, the baseline passed 9 to 13 of 18 cases and this CL 12 to 15. The snapshot 6 run was 13/18 → 13/18: https://evalin.corp.google.com/compare/d7c79edc6c054110/6be69aba9a9341a2. The reviewer edits in snapshot 5 (numbered provisioning steps, section rules, and heading changes) scored the same as snapshot 4 in a head-to-head run, 13/18 → 13/18 with identical per-case results: https://evalin.corp.google.com/compare/299f1ff5d5ba4ac6/03d9c6874b3a406e.

PiperOrigin-RevId: 996079357
@google-cla

google-cla Bot commented Oct 10, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@paulb777
paulb777 merged commit b5735e5 into main Oct 10, 2026
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant