Skip to content

feat: Add Market Explorer - voluntary carbon market data across 5 registries#5889

Open
gautamp8 wants to merge 16 commits intohashgraph:developfrom
gautamp8:market-explorer
Open

feat: Add Market Explorer - voluntary carbon market data across 5 registries#5889
gautamp8 wants to merge 16 commits intohashgraph:developfrom
gautamp8:market-explorer

Conversation

@gautamp8
Copy link
Copy Markdown
Contributor

@gautamp8 gautamp8 commented Mar 26, 2026

Summary

Adds Market Explorer to Carbon Atlas — a full-stack feature for exploring voluntary carbon market data across all major registries. Built on a FastAPI + PostgreSQL backend with an ETL pipeline that harmonizes data from 5 registries into a unified schema.

Live demo: https://atlas.carbonmarketshq.com/market

Depends on #5834 (Carbon Atlas base). This PR includes those commits since #5834 is still under review — once merged, this diff will reduce to the Market Explorer additions only.

Issues

Closes #5875 - Core feature integrating all major registries, harmonizing data and API on top
Closes #5876 - Frontend for all harmonized data with inline search
Closes #5877 - Regularly updated analytics dashboard with key KPIs and world map of projects
Closes #5878 - Project Proponent data refinement, listing apis and connection to projects

What's included

Backend (FastAPI + PostgreSQL)

  • RESTful API with 14 endpoints — projects, credits, developers, events, and 8 analytics charts
  • ETL pipeline harmonizing data from Verra, Gold Standard, ACR, CAR, and ART TREES
  • 53 tests (pytest + pytest-asyncio)
  • Docker Compose for self-contained deployment
  • Alembic migrations for schema management

Frontend (Next.js)

  • Market Overview dashboard with stat cards, interactive charts, and world map
  • All Projects page with search, filtering by registry/status/category/country, and sorting
  • Project detail pages with credit history, SDG goals, and developer info
  • Project Developer List
  • Responsive design with dark/light theme support

Analytics & Charts

  • Issuances by vintage year
  • Credits over time (monthly time series)
  • Projects by country (bar chart + world map with ISO3 geocoding)
  • Projects by category (donut chart)
  • Registry × status breakdown
  • Credits remaining by vintage
  • Reduction vs removal classification

Data Coverage

  • ~10,570 projects across 5 registries
  • ~2.46B credits issued, ~1.25B retired
  • 147 countries
  • 10 project categories
  • 8+ project statuses

Test plan

  • npm run build passes (Next.js production build)
  • npm test passes (Vitest — frontend logic tests)
  • pytest passes (53 backend tests)
  • Market Overview loads at /market with stats and charts
  • All Projects page at /market/projects supports search, filtering, pagination
  • Project detail pages render credit tables, SDG goals, developer info
  • World map renders with correct country geocoding
  • Dark/light theme works across all market pages
  • Existing Carbon Atlas pages (dashboard, issuances, projects, verify) are unaffected

gautamp8 and others added 13 commits March 24, 2026 21:34
Carbon Atlas is a Next.js dashboard that translates Guardian's on-chain
Verifiable Credentials into business-friendly views for carbon market
stakeholders.

Currently supports the Gold Standard MECD 431 policy (testnet) with:
- Per-policy dashboards with emission reduction stats, charts, and tables
- Trust chain explorer tracing issuances through the full VC relationship graph
- Dedicated VC-type renderers for monitoring reports, verification reports,
  projects, device MRV data, and VVB registrations
- Searchable device data table (3,254 metered cooking devices)
- Server-side auth proxy (API tokens never exposed to client)
- Hedera proof links to HashScan for every document
- Document verification by consensus timestamp

Tech stack: Next.js 16 (App Router), React 19, TanStack Query, shadcn/ui,
Tailwind CSS 4, Vitest

Multi-policy support (VM0033/Verra mainnet) planned as next phase.

Signed-off-by: Gautam Prajapati <gautamprajapati06@gmail.com>
…opy UX, auth restructure

- Trust chain follows standard carbon credit lifecycle (PDD → Calculated Project → Validation → Validated Project → dMRV → MR → Verification → MR Approved)
- VVB registration/approval excluded from chain; shown as inline assignment chips
- Ghost "Token Minting" pending node; removed broken minted token link
- Projects page deduplicates PDD + Validated Project into one row per project
- ATEC badge removed from projects list (policy-level page, multiple developers possible)
- ATEC logo increased to h-20 for visibility (source images are 2000x2000 squares)
- CopyableId component with visible copy icon replaces hover-to-copy
- Auth restructured: 3-step MGS SSO chain with auto-refresh, static token fallback
- All 28 review checklist items addressed; 38 tests passing

Signed-off-by: Gautam Prajapati <gautamprajapati06@gmail.com>
… emission reductions

- Sidebar title: MECD Indexer → Carbon Atlas
- Page title: MECD Indexer → Carbon Atlas
- Hedera Policy URL updated to guardian.hedera.com
- Emission reductions card/chart renamed to "Projected Emission Reductions"
- Footer text clarifies partial issuances to date

Signed-off-by: Gautam Prajapati <gautamprajapati06@gmail.com>
…hain fixes

- Multi-network switching: NetworkProvider context with mainnet/testnet live toggle,
  per-network config (policy IDs, token IDs), proxy routes by _network param
- Mainnet entity type normalization: multi-pass algorithm derives entityType from
  schemaName + documentStatus + relationship graph when indexer doesn't populate it
- Trust chain chip placement: MR Approved and Project Validated chips now appear
  after VVB report submission (above the report step in newest-first display)
- VCRenderer entityType override: expanded trust chain steps on mainnet now render
  with correct specialized views (device table, monitoring report, etc.)
- Dashboard performance: disable TanStack Query retries on 500s, non-blocking MRV
  fallback for device count
- Null-safe detail pages: prevent crash when switching networks on detail pages

Signed-off-by: Gautam Prajapati <gautamprajapati06@gmail.com>
…che errors

- useVcDocument: add retry: false to prevent repeated calls on upstream 500s
- Proxy: only cache successful responses (no-store on errors) so Vercel
  doesn't serve stale 500s for 10 minutes
- ValidationReportView: dynamically render all credential subject fields
  instead of hardcoded 3 fields (mainnet has different schema)
- ProjectView: handle non-string values in project_details fields
  (e.g. field0 may be object on some indexer versions)

Signed-off-by: Gautam Prajapati <gautamprajapati06@gmail.com>
…tract fNRB from mainnet

- ChainStep: pre-fetch verification_report and validation_report to show
  document dates in collapsed view (e.g. "v2.0 · Completed 26/02/2026")
- Project detail page: look up entityType from allPolicyVcs so VCRenderer
  gets correct type on mainnet (fixes raw JSON display)
- ProjectView + MonitoringReportView: extract fNRB, NCVb, EF_CO2 from
  per-fuel baseline_emission_case_common_values array (mainnet structure)

Signed-off-by: Gautam Prajapati <gautamprajapati06@gmail.com>
…ion and more

Signed-off-by: Gautam Prajapati <gautamprajapati06@gmail.com>
…duction/removal classification

Full-stack carbon market explorer integrated into Carbon Atlas sidebar:

Backend (FastAPI + PostgreSQL):
- 14 API endpoints: projects, credits, events, developers, stats, 8 chart endpoints
- CarbonPlan offsets-db harmonization pipeline with extended schema (SDGs, crediting periods, certifications)
- Reduction/removal classification via static project_type mapping (domain knowledge)
- Change detection, developer aggregation, Alembic migrations
- 105 tests (pytest-asyncio + httpx ASGI transport)

Frontend (Next.js + React + shadcn/ui + Recharts):
- Dashboard with stat cards, world map choropleth (react-simple-maps), 6 chart types
- Credits remaining by vintage, reduction vs removal breakdown
- Filterable/searchable/sortable projects table with pagination
- Project detail page with credit transactions, SDGs, developers
- Responsive layout with truncated project names and visible Issued/Retired columns

Data: 8,941 projects (Verra + Gold Standard), ~482K credit transactions, 146 countries
Signed-off-by: Gautam Prajapati <gautamprajapati06@gmail.com>
- Header shows Market Explorer title on /market/* routes, hides Hedera network selector
- Add data disclaimer footer on all market pages
- Replace Methodology sidebar link with Digitize Methodologies handbook
- Remove Hedera Policy sidebar link
- Compact registry names in stat cards (GS, ART)
- Make docker-compose ports configurable via env vars for shared hosting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
API serves public market data with no authentication. Allow all
origins with GET/OPTIONS only.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
react-simple-maps doesn't support React 19 peer dep yet.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Prevents Vercel from detecting pyproject.toml and installing Python
deps — the API runs on a separate VM, not on Vercel.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…nd app/api

Only exclude Python-specific files (pipeline, alembic, pyproject.toml,
uv.lock) to prevent Vercel from detecting a Python project.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@gautamp8 gautamp8 requested review from a team as code owners March 26, 2026 11:08
@gautamp8 gautamp8 requested review from mgarbs and rbarker-dev March 26, 2026 11:08
Rewrite README to cover both Market Explorer and MECD Indexer as
sections of the broader Carbon Atlas project. Add acknowledgements
for CarbonPlan and Berkeley. Consolidate all screenshots to
docs/screenshots/ with fresh captures from the live site.
@gautamp8 gautamp8 changed the title feat: Add Market Explorer — voluntary carbon market data across 5 registries feat: Add Market Explorer - voluntary carbon market data across 5 registries Mar 26, 2026
- Developer list page with search, registry/country/category filters
- Developer detail page with project listing
- Developer name normalization pipeline (merge variant spellings)
- Fix JSONB filter bugs (unique param names, CAST syntax)
- Add CORSIA eligible boolean to projects (derived from raw credit CSVs
  for ACR/CAR/ART TREES and additional_certifications for Verra)
- Add country filter combobox on developer list
- Add special developer annotations with tooltips
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant