Skip to content

jsparhamii/databricks_bakehouse_graphql

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 

Repository files navigation

GraphQL GSA - Bakehouse Analytics

A full-stack GraphQL application for exploring Databricks datasets, featuring a React frontend and Python backend with support for multiple database connectors.

Architecture

┌─────────────────────────────────────────────────────────────┐
│                    React Frontend                            │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐          │
│  │  Dashboard  │  │ Sales Table │  │   Schema    │          │
│  │   Charts    │  │   Filters   │  │  Explorer   │          │
│  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘          │
│         └────────────────┼────────────────┘                  │
│                          ▼                                   │
│              ┌───────────────────────┐                       │
│              │    Apollo Client      │                       │
│              └───────────┬───────────┘                       │
└──────────────────────────┼──────────────────────────────────┘
                           │ GraphQL
┌──────────────────────────┼──────────────────────────────────┐
│                          ▼                                   │
│              ┌───────────────────────┐                       │
│              │ Strawberry GraphQL    │                       │
│              │      + FastAPI        │                       │
│              └───────────┬───────────┘                       │
│                          │                                   │
│              ┌───────────▼───────────┐                       │
│              │  Database Abstraction │                       │
│              │       Layer           │                       │
│              └───────────┬───────────┘                       │
│                          │                                   │
│    ┌─────────────────────┼─────────────────────┐            │
│    ▼                     ▼                     ▼            │
│ ┌──────────┐      ┌──────────┐          ┌──────────┐        │
│ │Databricks│      │PostgreSQL│          │  MySQL   │        │
│ │Connector │      │(planned) │          │(planned) │        │
│ └──────────┘      └──────────┘          └──────────┘        │
│                    Python Backend                            │
└─────────────────────────────────────────────────────────────┘

Quick Start

Prerequisites

  • Python 3.11+
  • Node.js 18+
  • Access to a Databricks workspace with SQL warehouse

Backend Setup

  1. Navigate to backend directory:

    cd backend
  2. Create virtual environment:

    python -m venv venv
    source venv/bin/activate  # On Windows: venv\Scripts\activate
  3. Install dependencies:

    pip install -r requirements.txt
  4. Configure environment:

    cp env.example .env

    Edit .env with your Databricks credentials:

    DATABRICKS_SERVER_HOSTNAME=your-workspace.cloud.databricks.com
    DATABRICKS_HTTP_PATH=/sql/1.0/warehouses/your-warehouse-id
    DATABRICKS_ACCESS_TOKEN=your-access-token
    
  5. Start the server:

    uvicorn app.main:app --reload

    The GraphQL playground will be available at: http://localhost:8000/graphql

Frontend Setup

  1. Navigate to frontend directory:

    cd frontend
  2. Install dependencies:

    npm install
  3. Start development server:

    npm run dev

    The app will be available at: http://localhost:5173

Features

Dashboard

  • Real-time statistics (total orders, revenue, customers, products)
  • Revenue by category bar chart
  • Order distribution pie chart
  • Top performing category highlight

Sales Orders

  • Paginated data table
  • Filter by customer name
  • Filter by product category
  • Sortable columns

Franchise Stores

  • Card grid view of all locations
  • Store details including address and phone
  • Geographic statistics

Schema Explorer

  • Browse available tables
  • View column definitions and types
  • Run custom SQL queries

GraphQL API

Example Queries

Get Dashboard Stats:

query {
  dashboardStats {
    totalOrders
    totalRevenue
    uniqueCustomers
    uniqueProducts
    topCategory
  }
}

Get Sales Orders with Filters:

query {
  salesOrders(limit: 25, productCategory: "Breads") {
    orderNumber
    customerName
    productName
    orderAmount
  }
}

Explore Table Schema:

query {
  tableInfo(tableName: "sales_orders") {
    name
    columns {
      name
      type
      nullable
    }
  }
}

Adding New Database Connectors

The application is designed to support multiple databases. To add a new connector:

  1. Create a new file in backend/app/connectors/ (e.g., postgresql.py)

  2. Implement the BaseConnector interface:

    from .base import BaseConnector
    
    class PostgreSQLConnector(BaseConnector):
        def connect(self) -> None:
            # Implementation
            pass
        
        def disconnect(self) -> None:
            # Implementation
            pass
        
        def execute_query(self, query: str, params=None) -> list[dict]:
            # Implementation
            pass
        
        def get_tables(self, catalog=None, schema=None) -> list[str]:
            # Implementation
            pass
        
        def get_table_schema(self, table_name: str, catalog=None, schema=None) -> list[dict]:
            # Implementation
            pass
  3. Register the connector in backend/app/connectors/__init__.py

  4. Update resolvers to use the appropriate connector based on configuration

Tech Stack

Backend

  • FastAPI - Modern Python web framework
  • Strawberry GraphQL - Python GraphQL library with type safety
  • databricks-sql-connector - Official Databricks SQL connector
  • Pydantic - Data validation and settings management

Frontend

  • React 18 - UI library
  • Apollo Client - GraphQL client
  • Tailwind CSS - Utility-first CSS framework
  • Recharts - Charting library
  • Vite - Build tool and dev server
  • TypeScript - Type-safe JavaScript

Project Structure

graphql_gsa/
├── backend/
│   ├── app/
│   │   ├── connectors/       # Database connectors
│   │   │   ├── base.py       # Abstract base class
│   │   │   └── databricks.py # Databricks implementation
│   │   ├── resolvers/        # GraphQL resolvers
│   │   │   ├── queries.py    # Query resolvers
│   │   │   └── mutations.py  # Mutation resolvers
│   │   ├── schemas/          # GraphQL types
│   │   │   └── types.py      # Type definitions
│   │   ├── config.py         # Settings management
│   │   └── main.py           # FastAPI application
│   ├── requirements.txt
│   └── env.example
├── frontend/
│   ├── src/
│   │   ├── components/       # React components
│   │   ├── graphql/          # Apollo client & queries
│   │   ├── App.tsx
│   │   └── main.tsx
│   ├── package.json
│   └── tailwind.config.js
└── README.md

License

MIT

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published