Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file removed .DS_Store
Binary file not shown.
168 changes: 116 additions & 52 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,7 @@
<img style="width:100%" alt="knowledge_graph_banner" src="https://raw.githubusercontent.com/learning-commons-org/.github/refs/heads/main/assets/kg_hero.png" />

<p align="center">
<a href="https://docs.learningcommons.org/knowledge-graph/v1-1-0/getting-started/download-the-data/" target="_blank">Getting set up</a>
<a href="https://docs.learningcommons.org/knowledge-graph/v1-1-0/getting-started/tutorials/tutorial-overview" target="_blank">Tutorials</a>
<a href="https://docs.learningcommons.org/knowledge-graph/v1-3-0/getting-started/download-the-data/" target="_blank">Getting set up</a>
</p>


Expand All @@ -17,90 +15,156 @@ Key use cases include:
* **Compare state standards**: Adapt content aligned to one state standard to other states, initially in math across Common Core State Standards and 15+ additional states
* **Curriculum alignment:** Align your content or create additional materials aligned to curriculum (private-beta access only \- details below on how to join)

Knowledge Graph is distributed as CSV and JSON export files, making it accessible without specialized infrastructure. These files reflect a graph-based model, allowing developers to work with the data in relational databases or other environments. This structure enables rich querying and supports AI-enhanced educational applications.
Knowledge Graph is distributed as graph-native JSONL export files, making it accessible without specialized infrastructure. These files directly represent the underlying graph model, enabling developers to work with the data in graph databases, while remaining easy to ingest with standard data-processing tools. Developers can load the data into graph databases for relationship-centric querying or transform it for use in relational databases and data pipelines. This structure enables rich querying and supports AI-enhanced educational applications.

For complete setup instructions and usage examples, see the [full docs](https://docs.learningcommons.org/knowledge-graph/).

## **Repository contents**

| Path | Description |
| :---- | :---- |
| [import\_scripts/](./import_scripts/) | Helper scripts to import Knowledge Graph data into relational databases |
| [sample\_queries/](./sample_queries/) | Example SQL queries for exploring Knowledge Graph data |
| [tutorials/](./tutorials/) | Standalone example apps to demonstrate how Knowledge Graph data could be applied to solve different use cases |
| [LICENSE](./LICENSE.md) | Open source license details |

## **Quick Start**

**v1.2.0 will be the final version where Knowledge Graph can be downloaded as CSV flat files. Starting from v1.3.0 onwards, Knowledge Graph will be accessible as graph-native JSON flat files and we’ll be starting to grant access to our REST API in early 2026. Any CSV and JSON flat files that were previously downloaded will be unaffected.**
You can access the Knowledge Graph data using:

The knowledge graph data is available for download in both CSV and JSON formats. The graph data is exported with each file representing a specific entity type, and a relationships file capturing the connections between entities.
- **REST API**: Authenticate and make HTTP requests to retrieve academic standards directly. Best for applications that need real-time access. *(Currently available only to private beta users)*
- **MCP Server**: AI models can reliably work with academic standards, learning components, and learning progressions. They can resolve standards, decompose them into granular learning components, and trace progressions across standards. *(Currently available only to private beta users)*
- **Local JSONL**: Download local JSONL files and query them directly. Best for offline access, custom processing, or complex queries. *(Publicly available)*

**CSV files:** UTF-8 encoded with comma delimiters and quoted fields. All CSV files include header rows with column names.
### REST API

**JSON files:** Newline delimited JSON format with UTF-8 encoding.
> **Note:** The API is in limited early release and is only available to some private beta users. Because the API is an early release, current users should expect occasional breaking changes.

## Files
#### What you'll do

* `StandardsFramework`: Educational standards frameworks
* `StandardsFrameworkItem`: Individual standards and learning objectives within frameworks
* `LearningComponent`: Granular, precise representations of individual skills or concepts
* `Relationships`: Connections and associations between all entity types
- Authenticate using an API key
- Get a standards framework identifier (CASE UUID) for Multi-State Mathematics
- Retrieve a list of academic standards using that framework identifier

## Download options
#### What you'll need

There are two options to download the files: direct s3 links, or using curl commands.
- A Learning Commons Platform account
- An API key generated in the Learning Commons Platform

### Direct S3 links
#### Base URL

Click links to download files directly. Files will download to your browser's default location (typically `~/Downloads`).
All REST API requests should be sent to:

**CSV files:**
- [StandardsFramework.csv](https://aidt-knowledge-graph-datasets-public-prod.s3.us-west-2.amazonaws.com/knowledge-graph/v1.2.0/csv/StandardsFramework.csv?ref=github)
- [StandardsFrameworkItem.csv](https://aidt-knowledge-graph-datasets-public-prod.s3.us-west-2.amazonaws.com/knowledge-graph/v1.2.0/csv/StandardsFrameworkItem.csv?ref=github)
- [LearningComponent.csv](https://aidt-knowledge-graph-datasets-public-prod.s3.us-west-2.amazonaws.com/knowledge-graph/v1.2.0/csv/LearningComponent.csv?ref=github)
- [Relationships.csv](https://aidt-knowledge-graph-datasets-public-prod.s3.us-west-2.amazonaws.com/knowledge-graph/v1.2.0/csv/Relationships.csv?ref=github)
```
https://api.learningcommons.org/knowledge-graph/v0
```

**For SQL database imports:** Move the downloaded CSV files to `/tmp/kg-data/` to use the import scripts without modification:
#### Authentication

```bash
mkdir -p /tmp/kg-data
mv ~/Downloads/StandardsFramework.csv ~/Downloads/StandardsFrameworkItem.csv ~/Downloads/LearningComponent.csv ~/Downloads/Relationships.csv /tmp/kg-data/
```
Include your API key in the `x-api-key` header on every request:

```
x-api-key: YOUR_API_KEY
```

#### STEP 1: Get a standards framework identifier

**JSON files:**
- [StandardsFramework.json](https://aidt-knowledge-graph-datasets-public-prod.s3.us-west-2.amazonaws.com/knowledge-graph/v1.2.0/json/StandardsFramework.json?ref=github)
- [StandardsFrameworkItem.json](https://aidt-knowledge-graph-datasets-public-prod.s3.us-west-2.amazonaws.com/knowledge-graph/v1.2.0/json/StandardsFrameworkItem.json?ref=github)
- [LearningComponent.json](https://aidt-knowledge-graph-datasets-public-prod.s3.us-west-2.amazonaws.com/knowledge-graph/v1.2.0/json/LearningComponent.json?ref=github)
- [Relationships.json](https://aidt-knowledge-graph-datasets-public-prod.s3.us-west-2.amazonaws.com/knowledge-graph/v1.2.0/json/Relationships.json?ref=github)
Use your preferred HTTP client to send a GET request to the standards frameworks endpoint to get the CASE UUID for Multi-State Mathematics.

### Using curl commands
```bash
curl -X GET \
-H "x-api-key: YOUR_API_KEY" \
"https://api.learningcommons.org/knowledge-graph/v0/standards-frameworks?academicSubject=Mathematics&jurisdiction=Multi-State"
```

If you don't have `curl` installed, see [installation instructions](https://github.com/curl/curl).
You should receive a 200 response with the CCSS Math framework for Multi-State, including the framework name, jurisdiction, adoption status, and a `caseIdentifierUUID` (GUID). Copy the `caseIdentifierUUID` from the response for the next step.

**Recommended**: Download files to `/tmp/kg-data/` for compatibility with the SQL import scripts:
#### STEP 2: Retrieve academic standards

Use the `caseIdentifierUUID` you copied from Step 1's response with the academic standards endpoint to retrieve the individual standards for that framework.

```bash
# Create directory and download CSV files
mkdir -p /tmp/kg-data
cd /tmp/kg-data

curl -L "https://aidt-knowledge-graph-datasets-public-prod.s3.us-west-2.amazonaws.com/knowledge-graph/v1.2.0/csv/StandardsFramework.csv?ref=gh_curl" -o StandardsFramework.csv
curl -L "https://aidt-knowledge-graph-datasets-public-prod.s3.us-west-2.amazonaws.com/knowledge-graph/v1.2.0/csv/StandardsFrameworkItem.csv?ref=gh_curl" -o StandardsFrameworkItem.csv
curl -L "https://aidt-knowledge-graph-datasets-public-prod.s3.us-west-2.amazonaws.com/knowledge-graph/v1.2.0/csv/LearningComponent.csv?ref=gh_curl" -o LearningComponent.csv
curl -L "https://aidt-knowledge-graph-datasets-public-prod.s3.us-west-2.amazonaws.com/knowledge-graph/v1.2.0/csv/Relationships.csv?ref=gh_curl" -o Relationships.csv
curl -X GET \
-H "x-api-key: YOUR_API_KEY" \
"https://api.learningcommons.org/knowledge-graph/v0/academic-standards?standardsFrameworkCaseIdentifierUUID=YOUR_UUID_FROM_STEP_1"
```

> **Note:** If you skipped Step 1, you can use the CCSS Math framework UUID: `c6496676-d7cb-11e8-824f-0242ac160002` in place of `YOUR_UUID_FROM_STEP_1`.

You should see a paginated list of academic standards aligned to that framework, including statement codes, descriptions, grade levels, and subject information.

Example response:

```json
{
"data": [
{
"identifier": "e1755456-c533-5a84-891e-59725c0479e0",
"caseIdentifierURI": "https://satchelcommons.com/ims/case/v1p0/CFItems/6b9bf846-d7cc-11e8-824f-0242ac160002",
"caseIdentifierUUID": "6b9bf846-d7cc-11e8-824f-0242ac160002",
"name": null,
"statementCode": "3.NF.A.1",
"description": "Understand a fraction $\\frac{1}{b}$ as the quantity formed by 1 part when a whole is partitioned into b equal parts; understand a fraction $\\frac{a}{b}$ as the quantity formed by a parts of size $\\frac{1}{b}$.",
"statementType": "Standard",
"normalizedStatementType": "Standard",
"jurisdiction": "Multi-State",
"academicSubject": "Mathematics",
"gradeLevel": ["3"],
"inLanguage": "en-US",
"dateCreated": null,
"dateModified": "2025-02-05",
"notes": null,
"author": "1EdTech",
"provider": "Learning Commons",
"license": "https://creativecommons.org/licenses/by/4.0/",
"attributionStatement": "Knowledge Graph is provided by Learning Commons under the CC BY-4.0 license. Learning Commons received state standards and written permission under CC BY-4.0 from 1EdTech."
}
],
"pagination": {
"limit": 1,
"nextCursor": "eyJpZGVudGlmaWVyIjogImUxNzU1NDU2LWM1MzMtNWE4NC04OTFlLTU5NzI1YzA0NzllMCJ9",
"hasMore": true
}
}
```

### Local JSONL Files

The Knowledge Graph data is available for download in newline delimited JSONL format with UTF-8 encoding. The graph data is exported with `nodes.jsonl` representing the nodes of the knowledge graph and the `relationships.jsonl` file capturing the connections between nodes.

#### Files

- `nodes.jsonl`: Contains graph node records, defining each node by a unique identifier, labels, and a set of associated properties.
- `relationships.jsonl`: Contains graph relationship records, describing how nodes are connected, including the relationship type, properties, and the source and target nodes.

#### Download options

You can download the files through direct links or using curl commands.

**Direct links**

- [nodes.jsonl](https://cdn.learningcommons.org/knowledge-graph/v1.3.0/exports/nodes.jsonl?ref=github)
- [relationships.jsonl](https://cdn.learningcommons.org/knowledge-graph/v1.3.0/exports/relationships.jsonl?ref=github)

**Using curl commands**

If you don't have curl installed, visit https://github.com/curl/curl for installation instructions.

```bash
# Download JSON files
curl -L "https://aidt-knowledge-graph-datasets-public-prod.s3.us-west-2.amazonaws.com/knowledge-graph/v1.2.0/json/StandardsFramework.json?ref=gh_curl" -o StandardsFramework.json
curl -L "https://aidt-knowledge-graph-datasets-public-prod.s3.us-west-2.amazonaws.com/knowledge-graph/v1.2.0/json/StandardsFrameworkItem.json?ref=gh_curl" -o StandardsFrameworkItem.json
curl -L "https://aidt-knowledge-graph-datasets-public-prod.s3.us-west-2.amazonaws.com/knowledge-graph/v1.2.0/json/LearningComponent.json?ref=gh_curl" -o LearningComponent.json
curl -L "https://aidt-knowledge-graph-datasets-public-prod.s3.us-west-2.amazonaws.com/knowledge-graph/v1.2.0/json/Relationships.json?ref=gh_curl" -o Relationships.json
curl -L "https://cdn.learningcommons.org/knowledge-graph/v1.3.0/exports/nodes.jsonl?ref=gh_curl" -o nodes.jsonl
curl -L "https://cdn.learningcommons.org/knowledge-graph/v1.3.0/exports/relationships.jsonl?ref=gh_curl" -o relationships.jsonl
```

### **Next steps**
#### Querying with jq

One option for querying the JSONL files is to use [jq](https://jqlang.github.io/jq/). Example to extract Common Core math standards:

```bash
jq -c 'select((.labels | contains(["StandardsFrameworkItem"])) and .properties.jurisdiction == "Multi-State" and .properties.academicSubject == "Mathematics")' nodes.jsonl > common_core_math_standards.jsonl
```

This quick start walked you through how to download Knowledge Graph so you can start using it. You can explore next steps under [import\_scripts/](./import_scripts/), [sample\_queries/](./sample_queries/) and [tutorials/](./tutorials/).
This filters for nodes with:
- **Label:** `StandardsFrameworkItem`
- **Jurisdiction:** `Multi-State` (Common Core)
- **Academic Subject:** `Mathematics`

## **Support & Feedback**

Expand Down
Binary file removed import_scripts/.DS_Store
Binary file not shown.
71 changes: 0 additions & 71 deletions import_scripts/mysql/README.md

This file was deleted.

75 changes: 0 additions & 75 deletions import_scripts/mysql/create_tables.sql

This file was deleted.

32 changes: 0 additions & 32 deletions import_scripts/mysql/load_data.sql

This file was deleted.

Loading