Add AI integration tests #7038

emilypgoogle · 2025-06-17T15:37:13Z

Currently lacking mechanism to get API key from GitHub workflow and the workflow to regularly run the tests.
This functionality will be added in a follow up PR.

Includes a modification to an internal serializable type to better interact with failure cases and prevent crashes.

github-actions · 2025-06-17T15:37:29Z

Firebase AI Mock Responses Check ⚠️

A newer major version of the mock responses for Firebase AI unit tests is available. update_responses.sh should be updated to clone the latest version of the responses: v14.1

github-actions · 2025-06-17T15:37:35Z

📝 PRs merging into main branch

Our main branch should always be in a releasable state. If you are working on a larger change, or if you don't want this change to see the light of the day just yet, consider using a feature branch first, and only merge into the main branch when the code complete and ready to be released.

gemini-code-assist

Summary of Changes

Hello @emilypgoogle, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a comprehensive suite of integration tests for the AI module, covering core functionalities like content generation, token counting, and tool usage. It also includes a minor but important fix to improve the robustness of the CountTokensResponse type by making a field nullable to prevent crashes.

Highlights

New Integration Test Suite: I've added a comprehensive suite of integration tests for the AI module, including AiIntegrationTests, CountTokensTests, GenerateContentTests, and ToolTests, to validate core functionalities.
Core AI Functionality Testing: The new tests cover basic content generation, multi-modal content handling (text and images), and various countTokens scenarios, ensuring the AI model behaves as expected.
Advanced Tooling/Function Calling Tests: I've included robust tests for the AI model's function calling capabilities, covering interpretation of prompts, correct argument passing to declared functions, and chained tool calls.
Test Utility Setup: I introduced AIModels.kt to centralize the initialization and access of FirebaseApp and GenerativeModel instances, streamlining test setup.
Build Configuration Update: I modified firebase-ai.gradle.kts to include src/testUtil in the androidTest source set, which enables shared test utilities to be used within instrumented tests.
Robustness Improvement: I updated CountTokensResponse to make the totalTokens field nullable in its internal representation, allowing it to gracefully handle API responses where this field might be absent and preventing potential crashes.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

google-oss-bot · 2025-06-17T15:37:52Z

	1 Warning
⚠️	Did you forget to add a changelog entry? (Add the 'no-changelog' label to the PR to silence this warning.)

Generated by 🚫 Danger

gemini-code-assist

Code Review

The pull request introduces integration tests for the Firebase AI SDK, covering content generation, token counting, and tool usage. It also modifies CountTokensResponse.Internal to enhance robustness. The review focuses on the correctness and reliability of the new tests and the safety of the type modification. Identified issues include a potential NullPointerException, IndexOutOfBoundsException, and incomplete test.

firebase-ai/src/androidTest/kotlin/com/google/firebase/ai/AIModels.kt

firebase-ai/src/androidTest/kotlin/com/google/firebase/ai/TypesValidator.kt

firebase-ai/src/androidTest/kotlin/com/google/firebase/ai/AIModels.kt

firebase-ai/src/androidTest/kotlin/com/google/firebase/ai/GenerateContentTests.kt

firebase-ai/src/androidTest/kotlin/com/google/firebase/ai/AIModels.kt

github-actions · 2025-06-17T15:46:08Z

Test Results

24 files - 1 034 24 suites - 1 034 37s ⏱️ - 35m 15s
102 tests - 5 883 102 ✅ - 5 861 0 💤 - 22 0 ❌ ±0
204 runs - 11 829 204 ✅ - 11 785 0 💤 - 44 0 ❌ ±0

Results for commit 21ff798. ± Comparison against base commit f08e0c7.

This pull request removes 5883 tests.

com.google.android.datatransport.cct.CctBackendFactoryTest ‑ create_returnCCTBackend_WhenBackendNameIsCCT
com.google.android.datatransport.cct.CctDestinationTest ‑ cctDestination_shouldOnlySupportProtoAndJson
com.google.android.datatransport.cct.CctDestinationTest ‑ cctDestination_shouldSupportProtoAndJson
com.google.android.datatransport.cct.CctTransportBackendTest ‑ decorate_whenOffline_shouldProperlyPopulateNetworkInfo
com.google.android.datatransport.cct.CctTransportBackendTest ‑ decorate_whenOnline_shouldProperlyPopulateNetworkInfo
com.google.android.datatransport.cct.CctTransportBackendTest ‑ schedule_shouldAddCookieOnPseudonymousIds
com.google.android.datatransport.cct.CctTransportBackendTest ‑ schedule_shouldDropCookieOnMixedPseudonymousIds
com.google.android.datatransport.cct.CctTransportBackendTest ‑ send_CompressedResponseIsUncompressed
com.google.android.datatransport.cct.CctTransportBackendTest ‑ send_whenBackendRedirectsMoreThan5Times_shouldOnlyRedirect4Times
com.google.android.datatransport.cct.CctTransportBackendTest ‑ send_whenBackendRedirects_shouldCorrectlyFollowTheRedirectViaPost
…

♻️ This comment has been updated with latest results.

google-oss-bot · 2025-06-17T15:46:09Z

Coverage Report ¹

Affected Products

No changes between base commit (3b7bfef) and merge commit (74ef93b).

Test Logs

https://storage.googleapis.com/firebase-sdk-metric-reports/CeN3feC9W8.html

google-oss-bot · 2025-06-17T15:56:41Z

Size Report ¹

Affected Products

firebase-ai
Type Base (3b7bfef) Merge (74ef93b) Diff
aar 829 kB 829 kB +15 B (+0.0%)
apk (aggressive) 1.56 MB 1.56 MB +164 B (+0.0%)
apk (release) 9.53 MB 9.53 MB +300 B (+0.0%)

Test Logs

https://storage.googleapis.com/firebase-sdk-metric-reports/Gxc4eTnpx0.html

Add AI integration tests

ccf0d50

emilypgoogle requested review from rlazo and davidmotson June 17, 2025 15:37

gemini-code-assist bot reviewed Jun 17, 2025

View reviewed changes

davidmotson approved these changes Jun 17, 2025

View reviewed changes

emilypgoogle added 2 commits June 18, 2025 15:58

Adjustments and format

b3320e6

Headers

21ff798

emilypgoogle merged commit ece0efd into main Jun 18, 2025
32 checks passed

emilypgoogle deleted the ep/ai-integ-tests branch June 18, 2025 21:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add AI integration tests #7038

Add AI integration tests #7038

Uh oh!

emilypgoogle commented Jun 17, 2025

Uh oh!

github-actions bot commented Jun 17, 2025

Uh oh!

github-actions bot commented Jun 17, 2025 •

edited

Loading

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

google-oss-bot commented Jun 17, 2025

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Jun 17, 2025 •

edited

Loading

Uh oh!

google-oss-bot commented Jun 17, 2025 •

edited

Loading

Uh oh!

google-oss-bot commented Jun 17, 2025 •

edited

Loading

`firebase-ai`

Uh oh!

Uh oh!

Uh oh!

Add AI integration tests #7038

Add AI integration tests #7038

Uh oh!

Conversation

emilypgoogle commented Jun 17, 2025

Uh oh!

github-actions bot commented Jun 17, 2025

Firebase AI Mock Responses Check ⚠️

Uh oh!

github-actions bot commented Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📝 PRs merging into main branch

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Summary of Changes

Highlights

Footnotes

Uh oh!

google-oss-bot commented Jun 17, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

google-oss-bot commented Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage Report 1

Affected Products

Test Logs

Uh oh!

google-oss-bot commented Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Size Report 1

Affected Products

firebase-ai

Test Logs

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Jun 17, 2025 •

edited

Loading

github-actions bot commented Jun 17, 2025 •

edited

Loading

google-oss-bot commented Jun 17, 2025 •

edited

Loading

Coverage Report ¹

google-oss-bot commented Jun 17, 2025 •

edited

Loading

Size Report ¹

`firebase-ai`