Skip to content

UI-Research/affh-tool-data-exploration

Repository files navigation

Fair Housing Data Mapping Tool

An R Shiny application for exploring Affirmatively Furthering Fair Housing (AFFH) data with interactive maps and statistics.

For a walkthrough of the dashboard's features and controls, see the User Guide.

Feature Overview

The dashboard provides interactive exploration of AFFH demographic indicators and opportunity metrics at State, County, and Census Tract levels.

Maps

  • Multi-geography choropleth maps with automatic zoom and basemap label overlay (city/road names rendered above polygons)
  • R/ECAP tract overlay highlighting with red dashed borders (Tract level only)
  • Address search via US Census Bureau geocoder (no API key required); red circle marker persists after search bar collapses
  • PNG map export via html2canvas with non-blocking toast notification on capture failure
  • Ghosted previous-state overlay during geography transitions (30% opacity, still pan/zoomable)
  • Fullscreen mode with invalidateSize on exit to recalculate Leaflet dimensions

Selection and Navigation

  • Cascading dropdown selectors (Geography → Focus Area → Category → Variable) with step progress indicator
  • Interactive tooltips with context-aware formatting (area name, variable label, value)
  • Map click populates all downstream outputs (value boxes, charts, tables)

Charts

  • Time-series bar charts with parent-geography comparison overlays (up to 3 levels at Tract: tract, county, state)
  • Category comparison horizontal bar charts for race/ethnicity-disaggregated variables

Tables

  • Accordion-style indicator tables organized by category, with year-range column headers (e.g., "2009-2012")
  • Table filter toggle to isolate the selected variable's category and row
  • Automatic dropping of all-NA year columns
  • CSV download combining all categories for the clicked area

Data Notes

  • Sources and methodology notes accordion with pulse animation on variable change
  • Value boxes with parent-geography benchmark comparisons

Getting Started

# Load package code and data
devtools::load_all()

# Run the application
shiny::runApp()

# Run tests
devtools::test()

# Regenerate internal data objects (if needed)
source("data-raw/data_prep.R")

Project Structure

R/
├── affh_shinyApp.R      # Main app entry point (Data Dashboard + User Guide)
├── server.R             # Server logic (reactives, observers, outputs)
├── config.R             # Data loading and configuration
├── sidebar.R            # Geography/variable selection UI
├── summary_stats_panel.R # Indicator tables UI
├── map_helpers.R        # Leaflet proxy update functions
├── map_output.R         # Initial leaflet render
├── data_helpers.R       # Pure data transformation functions
├── comparison_data.R    # Comparison benchmark data
├── geography_names.R    # FIPS-to-name conversion
├── affh_transform.R     # AFFH data pivoting/splitting
├── load_affh_data.R     # CSV/Parquet data loading with error handling
├── get_fips_geometry.R  # Geometry extraction by FIPS
├── table.R              # Reactable rendering (year-range labels, all-NA column dropping)
├── step_progress.R      # Step progress indicator builder
├── barplot_output.R     # Plotly bar charts (time series + category comparison)
├── observe_selectize.R  # Cascading dropdown helper
├── text.R               # UI text constants
├── user_guide.R         # User Guide page content builder
├── notes_accordion.R    # Sources accordion UI
├── style_constants.R    # Color and styling constants
└── constants.R          # App constants (TABLE_CONFIG, geographies)

data/
├── final/               # AFFH metrics (CSV for State/County, Parquet for Tract)
├── lookup/              # Variable and metric lookup tables
│   ├── input_geo_var_lookup.csv        # Maps variables → geographies and categories
│   ├── metric_table_lookup.csv         # Metric display configuration
│   ├── tooltip_var_lookup.csv          # Tooltip variable definitions
│   ├── race_ethnicity_vars_lookup.csv  # Race/ethnicity comparison groups
│   └── data_years_lookup.csv           # Year → display label mapping for table headers
├── counties_sf.parquet  # County geometries
├── tracts_sf.parquet    # Tract geometries
├── states_sf.rda        # State geometries (shifted for national view)
└── state_county_xwalk.rda  # FIPS crosswalk table

tests/testthat/          # Unit tests (42 tests across 8 files)

www/
├── map-loading.js       # Client-side JS handlers (overlays, map capture, selectize)
└── styles.css           # Custom CSS (step progress, animations, map overlays)

docs/
├── user-guide/          # End-user dashboard walkthrough
├── testing/             # QA checklist and technical debt tracker
├── audit/               # Codebase architecture audit (historical)
├── prompts/             # Claude Code development prompts
├── presentation/        # Project presentation (Quarto + reveal.js)
├── blog/                # Blog post draft
└── exploratory/         # Early exploratory notebooks

deploy.R                 # Deployment script (manifest generation + rsconnect deploy)

Architecture

Control Flow Diagram

flowchart TB
    subgraph UI["UI Layer"]
        sidebar["sidebar.R<br/>Geography & Variable Selection"]
        map_panel["affh_shinyApp.R<br/>Map + Value Box"]
        stats_panel["summary_stats_panel.R<br/>Indicator Tables"]
    end

    subgraph Server["Server Layer (server.R)"]
        subgraph Reactives["Reactives"]
            geo_select["Geography Selection<br/>focus_geo_config, selected_poly"]
            var_select["Variable Selection<br/>variable_choices, selected_metric_data"]
            click_state["Click State<br/>click_id, clicked_leaflet_geo"]
            data_pipe["Data Pipelines<br/>map_data, barplot_data, clicked_geo_data"]
        end

        subgraph Observers["Observers"]
            obs["UI Updates<br/>observe_selectize, update_map"]
        end

        subgraph Outputs["Outputs"]
            out["Render Functions<br/>map, tables, barplot, text"]
        end
    end

    subgraph Helpers["Helper Functions"]
        data_helpers["data_helpers.R<br/>prepare_barplot_data<br/>prepare_title_text"]
        comparison["comparison_data.R<br/>prepare_comparison_data<br/>prepare_category_comparison_data"]
        geo_names["geography_names.R<br/>get_geography_display_name"]
        step_prog["step_progress.R<br/>build_step_progress_ui"]
        map_helpers["map_helpers.R<br/>update_map"]
        geo_helpers["get_fips_geometry.R<br/>get_fips_geometry"]
        transform["affh_transform.R<br/>affh_transform"]
    end

    subgraph Data["Data Layer"]
        config["config.R<br/>counties_sf, tracts_sf, states_sf"]
        load["load_affh_data.R<br/>AFFH CSVs + Parquet"]
        lookup["Lookup Tables<br/>input_geo_var_lookup<br/>metric_table_lookup<br/>race_ethnicity_vars_lookup<br/>data_years_lookup"]
    end

    sidebar --> geo_select
    sidebar --> var_select
    geo_select --> data_pipe
    var_select --> data_pipe
    click_state --> data_pipe

    data_pipe --> obs
    obs --> out

    data_pipe --> data_helpers
    data_pipe --> comparison
    data_pipe --> transform
    geo_select --> geo_helpers

    config --> geo_helpers
    load --> data_pipe
    lookup --> transform

    out --> map_panel
    out --> stats_panel
Loading

Data Flow

flowchart LR
    A[User selects geography] --> B[selected_poly]
    B --> C[get_fips_geometry]
    C --> D[counties_sf / tracts_sf]

    E[User selects variable] --> F[selected_metric_data]
    F --> G[load_affh_data]

    B --> H[map_data]
    F --> H
    H --> J[update_map]
    J --> K[Leaflet Proxy]

    L[User clicks map] --> M[click_id]
    M --> N[clicked_geo_data]
    N --> O[affh_transform]
    O --> P[Reactable Tables]

    M --> Q[barplot_data]
    Q --> R[prepare_barplot_data]
    R --> S[Time-Series Chart]

    M --> T[category_comparison_data]
    T --> U[prepare_category_comparison_data]
    U --> V[Category Comparison Chart]
Loading

Key Concepts

Reactive Flow

  1. Geography Selection: User picks State/County/Tract → triggers geometry load
  2. Variable Selection: Cascading dropdowns filter available variables
  3. Map Click: Clicking a polygon triggers data table and chart updates
  4. Proxy Updates: Map redraws via leafletProxy() without full re-render

FIPS Code Handling

All FIPS codes are stored as character strings with leading zeros: - State: 2 digits ("01" for Alabama) - County: 5 digits ("01001" for Autauga County) - Tract: 11 digits ("01001020100")

File Organization

  • UI builders: build_*() functions return bslib/htmltools objects
  • Pure helpers: prepare_*() functions in data_helpers.R are testable without Shiny
  • Internal helpers: .function_name() convention for non-exported functions

Key Dependencies

Package Purpose
bslib Bootstrap 5 UI components
leaflet Interactive maps
plotly Interactive charts
reactable Interactive tables
data.table Fast data filtering
sf Spatial data handling
arrow Parquet I/O
logger Console logging

External Services

Service Usage API Key Required
US Census Bureau Geocoder Address search via leaflet.extras No
CartoDB Positron Basemap tiles (labels + no-labels layers) No
html2canvas CDN PNG map export in browser No
Census TIGER/Line Geometry regeneration (data-raw/ scripts only) No

Data Sources

  • Geographic boundaries: Census TIGER/Line via tigris package
  • AFFH metrics: Pre-processed CSVs and Parquet files in data/final/

Configuration

All configuration is in R files (no environment variables):

File What it controls
R/config.R Data file paths, lookup table loading, lazy geometry caching
R/style_constants.R Brand color palette, map styling, chart colors, table styling
R/constants.R TABLE_CONFIG (table output IDs/titles), EXCLUDED_STATE_FIPS (territory filtering)
R/text.R UI labels and display text

Deployment

The app is deployed via Posit Connect or shinyapps.io using rsconnect. The deploy.R script is the single entry point for deployment — it defines the explicit list of app files, generates manifest.json, and calls rsconnect::deployApp().

# Deploy (generates manifest + pushes to Posit Connect)
source("deploy.R")

When R source files, data files, or web assets are added or removed, update the app_files vector in deploy.R to match.

Data Update Procedures

  1. Update CSV/Parquet files in data/final/
  2. If lookup tables change: source("data-raw/generate_lookup_tables.R")
  3. If Census boundaries change: source("data-raw/data_prep.R")
  4. Clear memoization caches: clear_affh_cache()
  5. Run tests: devtools::test()
  6. Deploy: source("deploy.R")

Documentation

Document Path Description
User Guide docs/user-guide/ Dashboard walkthrough for end users
User Testing Guide docs/testing/ 68-test manual QA checklist
Technical Debt docs/testing/ Flagged items for future improvement
Codebase Audit docs/audit/ Architecture audit (historical, pre-cleanup)
Claude Code Prompts docs/prompts/ Reusable prompts used during development

Known Issues and Limitations

  • Loading overlay timeout: The MutationObserver intended to detect Leaflet render completion doesn't fire; the loading overlay always falls back to a 3-second timeout. See www/map-loading.js.
  • Connecticut geometries: Uses 2021-vintage Census geometries because the 2022 vintage replaced CT's 8 historical counties with 9 planning regions that don't match the AFFH data's FIPS codes. See data-raw/data_prep.R.
  • Data load errors: load_affh_data() wraps file reads in tryCatch and shows a red toast notification on failure instead of surfacing raw R errors.

Troubleshooting

Symptom Cause Fix
White map after tab switch Stale Leaflet container dimensions Already fixed — invalidateSize() called before fitBounds() on tab return
Wrong county names for Connecticut 2022+ Census vintage returns planning regions data_prep.R fetches year = 2021 for CT; re-run if geometries were regenerated without this
Map zooms to entire US instead of selected state Stale bounding box after data.table subset rebuild_sf_bbox() reconstructs the sfc column; already applied in geometry pipeline
Variable not appearing in dropdown Missing from lookup table Add entry to data/lookup/input_geo_var_lookup.csv
Data not updating after file change Memoization cache serving stale results Run clear_affh_cache()

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages