This repo is maintained, but not warrantied by Google. Issues and feature requests can be reported via https://github.com/looker-open-source/ca-api-quickstarts/issues.
The Conversational Analytics API provides a natural language interface to query BigQuery and Looker data programmatically. The API enables data access through multiple integration points including embedded Looker dashboards, chat applications, and custom web applications. The API helps organizations streamline ad-hoc data requests and provides self-service analytics capabilities.
This repository serves as a quick start app to integrate with the Conversational Analytics API. You can either deploy this example implementation as-is for testing purposes or adapt it to your specific production requirements. The application currently can only be deployed locally for development and testing. Check out the API documentation for additional enablement, setup, and use.
Try out this app to:
- Create, update, list, and delete data agents managed/used by the Conversational Analytics API
- Configure BQ as a data source for the data agents
- Configure Looker as a data sourc for the data agents to benefit from Looker's semantic modeling improved context and accuracy in conversations
- Hold multi-turn conversations with the data agents
- View past conversations with the data agents
- Learn more about the extensible API architecture for your own custom application development
Note: This is a pre-GA product intended for test environments only. It should not be used in production environments or to process personal data or other data subject to legal or regulatory compliance requirements. This repository is subject to the Pre-GA Offerings Terms of the Service Specific Terms and the Consent Addendum for Gemini for Google Cloud Trusted Tester Program
The local deployment option is ideal for:
- Development and testing
- Customizing the application
- Individual developer use
- Demonstrating capabilities in a controlled environment
You must have the following prerequisites installed in your local environment:
- Python 3.11 or higher
- Git
- Google Cloud SDK (gcloud CLI)
- Determine your Google Cloud billing project you will use for the quickstart app. Enable the required APIs on the Google Cloud billing project (charges may apply):
gcloud services enable geminidataanalytics.googleapis.com bigquery.googleapis.com cloudaicompanion.googleapis.com --project=YOUR_PROJECT_ID
- Determine the user or service account the quickstart app will use to access cloud resources. Setup gcloud CLI application default credentials for the account. Check out these steps for more context.
gcloud auth application-default login
gcloud auth application-default set-quota-project YOUR_PROJECT_ID
-
Set the correct IAM roles on the user or service account from step 2 depending on the type of the data source the app will query:
Data source Roles BigQuery roles/bigquery.dataViewer
BigQuery Data Viewer
roles/bigquery.user
BigQuery UserLooker roles/looker.instanceUser
Looker Instance User
Clone the repository and navigate to the project directory:
git clone https://github.com/looker-open-source/ca-api-quickstarts.git
cd ca-api-quickstarts
Create a secrets.toml
file in the .streamlit
directory:
[cloud]
project_id = "YOUR_PROJECT_ID"
# Uncomment next 3 lines if using Looker as data source
#[looker]
#client_id = "YOUR_LOOKER_CLIENT_ID"
#client_secret = "YOUR_LOOKER_CLIENT_SECRET"
If you will use Looker as a data source:
- Determine the Looker account that will access Looker.
- Ensure the Looker account has the access_data and gemini_in_looker permissions.
- Retrieve the Looker account's client id and client secret and set it in the secrets.toml file.
The quickstart app auths with a Looker API key. The app DOES NOT use a Looker access token.
Install the app's dependencies:
pip install -r requirements.txt
Start the app locally:
streamlit run app.py
Access the app at http://localhost:8501 in your web browser.
- Navigate to the "Agents" page
- Scroll down to "Create Agent" form.
- Enter the "display name", "description", and "system instructions". Tips for writing system instructions
- If you want the agent to query Looker as a data source:
- Select "Looker" as the data source
- Enter the Looker instance url. e.g. "myinstance.looker.com"
- Enter the Looker model name
- Enter the Looker explore name
- Or, if you want the agent to query BigQuery as a data source:
- Select "BigQuery" as the DataSource
- Enter the id of the project containing the BigQuery dataset. e.g. "bigquery-public-data"
- Enter the name of the dataset. e.g. "san_francisco_trees"
- Enter the name of the table. e.g. "street_trees"
- Select "Create"
- View the data agents you've created in the agents page.
- Select a data agent to expand it.
- You can change all fields except "Data Source". Select "Update agent" after you've made your changes to save your changes to the agent.
- You can select "Delete agent" to delete the agent.
Once your agent is configured:
- Navigate to the "Chat" page
- The last created agent is automatically selected.
- Ask a question in the chat prompt field. A conversation will automatically be started
- View responses in text, table, and chart formats.
- Ask follow-up questions to hold a multi-turn conversation that builds on previous context.
Example queries:
- "How many products are in each category?"
- "What were our top 5 customers by revenue last quarter?"
- "Show me a bar chart of monthly sales trends"
- "Compare performance across regions in a table"
- Navigate to the "Chat" page.
- Select the agent you'd like to see past conversations with in the dropdown in the top bar.
- Select a past conversation from the dropdown.
- Check out the past messages from the selected conversation.
- You can continue the past conversation by asking another question in the chat prompt field.
A critical component of the Conversational Analytics API is its semantic layer implementation, which dramatically improves query accuracy and contextual understanding.
Semantic layers act as a translation layer between raw data and business users, providing:
- Business Context: Mapping technical fields to business terminology
- Relationship Modeling: Pre-defining table joins and relationships
- Metric Definitions: Establishing consistent calculations and aggregations
- Enhanced Accuracy: Providing guardrails for query generation
Internal testing shows that Looker's semantic layer reduces data errors in GenAI natural language queries by two-thirds compared to ungoverned data warehouse tables. As use cases become more complex—involving multiple tables, joins, and complex calculations—semantic layers like Looker's excel by offloading reasoning complexity from the AI model.
The application supports two types of semantic layers:
-
YAML-Based Metadata Layer (BigQuery)
- Automatically generated with table/field descriptions, synonyms, and relationships
- Can be manually edited to improve performance for specific use cases
- Gives some benefits of a formal semantic model
-
LookML Semantic Layer (Looker)
- Leverages existing LookML models with rich metadata
- Includes pre-defined joins and consistent calculation definitions
- Provides field-level permissions and governance controls
- Offers superior accuracy for complex, multi-table queries
- For Existing Looker Users: Leverage your existing LookML investment for highest accuracy
- For BigQuery-Only Users: Use the YAML generation capabilities with manual refinement
- For Complex Use Cases: Consider developing LookML models for critical data domains