Updated the docs for app-architecture-v3 with Java and Jena

Author: cjoakim

impl/docs/application_architecture.md

@@ -1,16 +1,16 @@
 # CosmosAIGraph : Application Architecture
 
 <p align="center">
-  <img src="img/app-architecture-v2.png" width="90%">
+  <img src="img/app-architecture-v3.png" width="90%">
 </p>
 
 ---
 
 ## Application Components
 
 - Microservices
-  - web microervice - UI front end
-  - graph microservice - Contains the in-memory rdflib graph and AI functionality
+  - web microervice - UI front end with AI functionality
+  - graph microservice - Contains the in-memory graph
 - Azure Container App - Runtime orchestrator for the above two microservices
 - Cosmos DB NoSQL or Mongo vCore API - Domain data and conversational AI documents, embeddings
 - Azure OpenAI - completions and embeddings service

impl/docs/cosmos_design_modeling.md

@@ -46,16 +46,17 @@ This domain of software libraries was chosen because it should be **relatable**
 to most customers, and it also suitable for **Bill-of-Materials** graphs.
 
 The PyPi JSON files were obtained with HTTP requests to public URLs such as 
-**https://pypi.org/pypi/{libname}/json**, and their HTML contents were tranformed into JSON.
+**https://pypi.org/pypi/{libname}/json**, and their HTML contents were transformed into JSON.
 
 Subsequent data wrangling fetched referenced HTML documentation, produced 
 **text summarization with Azure OpenAI and semantic-kernel** and produced
-a **vectorized embedding value** from several concatinated text attributes
+a **vectorized embedding value** from several concatenated text attributes
 within each library JSON document.  A full description of this data wrangling
 process is beyond the scope of this documentation, but the process itself
 is in file 'impl/app/wrangle.py' in the repo.
 
 ## Next Steps: Load Cosmos DB with Library Documents
 
+Depending on the Cosmos DB API you chose:
 - See [Load Azure Cosmos DB vCore](load_cosmos_vcore.md)
 - See [Load Azure Cosmos DB NoSQL](load_cosmos_nosql.md)

impl/docs/environment_variables.md

@@ -23,10 +23,16 @@ All of these begin with the prefix `CAIG_`.
 | CAIG_AZURE_REGION | The Azure region where the ACA app is deployed to |
 | CAIG_CONFIG_CONTAINER | The vCore container for configuration JSON values |
 | CAIG_CONVERSATIONS_CONTAINER | The vCore container where the chat conversations and history are persisted |
+| CAIG_COSMOSDB_NOSQL_ACCT | The Name of your Cosmos DB NoSQL account |
+| CAIG_COSMOSDB_NOSQL_AUTH_MECHANISM | The Cosmos DB NoSQL authentication mechanism; key or rbac |
 | CAIG_COSMOSDB_NOSQL_KEY1 | The key of your Cosmos DB NoSQL account |
+| CAIG_COSMOSDB_NOSQL_RG | The Resource Group of your Cosmos DB NoSQL account |
 | CAIG_COSMOSDB_NOSQL_URI | The URI of your Cosmos DB NoSQL account |
 | CAIG_DEFINED_AUTH_USERS |  |
+| CAIG_ENCRYPTION_SYMMETRIC_KEY | optional symmetric key for encryption/decryption |
 | CAIG_FEEDBACK_CONTAINER | The vCore container where user feedback is persisted |
+| CAIG_GRAPH_DUMP_OUTFILE | The file to write to if CAIG_GRAPH_DUMP_UPON_BUILD is true |
+| CAIG_GRAPH_DUMP_UPON_BUILD | Boolean true/false to dump the Java/Jena model to CAIG_GRAPH_DUMP_OUTFILE after GraphBuilder completes |
 | CAIG_GRAPH_NAMESPACE | The custom namespace for the RED graph |
 | CAIG_GRAPH_SERVICE_NAME |  |
 | CAIG_GRAPH_SERVICE_PORT |  |
@@ -39,9 +45,6 @@ All of these begin with the prefix `CAIG_`.
 | CAIG_HOME | Root directory of the CosmosAIGraph GitHub repository on your system |
 | CAIG_LA_WORKSPACE_NAME | The Log Analytics workspace name used by the Azure Container App (ACA) |
 | CAIG_LOG_LEVEL | a python logging standard-lib level name: notset, debug, info, warning, error, or critical |
-| CAIG_PG_FLEX_PASS | Azure PostgreSQL Flex Server user password |
-| CAIG_PG_FLEX_SERVER | Azure PostgreSQL Flex Server hostname |
-| CAIG_PG_FLEX_USER | Azure PostgreSQL Flex Server user |
 | CAIG_WEBSVC_AUTH_HEADER | x-caig-auth |
 | CAIG_WEBSVC_AUTH_VALUE | K6ZQw!81 |
 | CAIG_WEB_APP_NAME |  |

impl/docs/img/app-architecture-v3.png

@@ -1,4 +1,4 @@
-# CosmosAIGraph : Load Azure Cosmos DB NoSQL
+# CosmosAIGraph : Load Azure Cosmos DB for NoSQL
 
 ## Configuration
 
@@ -8,6 +8,7 @@ This page assumes that you have set the following environment variables:
 CAIG_GRAPH_SOURCE_TYPE              <-- must be set to 'cosmos_nosql'
 CAIG_COSMOSDB_NOSQL_URI             <-- this value is unique to your Azure deployment
 CAIG_COSMOSDB_NOSQL_KEY1            <-- Read/Write key value
+CAIG_COSMOSDB_NOSQL_AUTH_MECHANISM  <-- Authentication mechanism - key or RBAC (Entra ID)
 
 CAIG_GRAPH_SOURCE_DB                <-- defaults to 'caig'
 CAIG_GRAPH_SOURCE_CONTAINER         <-- defaults to 'libraries'
@@ -54,6 +55,18 @@ solution.
 
 ---
 
+Navigate to the **impl\app** directory of this repo and execute
+the following commands:
+
+```
+> .\venv.ps1                      <-- create the python virtual environment
+
+> .\venv\Scripts\Activate.ps1     <-- activate the python virtual environment
+```
+
+---
+
+
 ## Load the entities document into the config container
 
 This step will load one document into the **config** container.
@@ -90,13 +103,9 @@ For your CosmosAIGraph implementation, create and upload a similar file.
 
 ## Load the Library data into Cosmos DB NoSQL
 
-Navigate to the **impl\app** directory of this repo and execute
-the following commands:
+This step will load the main dataset into a libraries container :
 
 ```
-> .\venv.ps1                      <-- create the python virtual environment
-
-> .\venv\Scripts\Activate.ps1     <-- activate the python virtual environment
 
 > python main_nosql.py load_libraries caig libraries 999999
 
@@ -120,7 +129,7 @@ the following commands:
 
 ### Execute a Vector Search with the loaded data
 
-First generate an embedding value from the words: 
+First generate an embedding value from the words:
 "asynchronous web framework with pydantic".
 Then use that embedding in a vector search vs the Cosmos DB
 libraries container.
@@ -136,5 +145,4 @@ doc 3: {'pk': 'pypi', 'id': 'pypi_async_asgi_testclient', 'name': 'async-asgi-te
 ...
 ```
 
-Notice that the **FastAPI** library is correctly identified as the top semantic search result.
-
+Notice that the **FastAPI** library is correctly identified as the top semantic search result.

impl/docs/load_cosmos_nosql.md

@@ -39,19 +39,30 @@ This approach executes the application packaged as **Docker Containers** rather
 
 Start your **Docker Desktop** application if it's not already running.
 
-Be sure to modify your environment variables in the **docker-compose.yml** 
-before starting the microservices.
+Be sure to modify your environment variables in the appropriate
+**docker-compose-xxx.yml** ile before starting the microservices.
+
+Two docker-compose yml files are available:
+
+- docker/docker-compose-with-rdflib.yml
+  - This uses the Python-based web application
+  - This uses the Python-based graph microservice using rdflib
+
+- docker-compose-with-jena.yml 
+  - This also uses the same Python-based web application
+  - This uses the Java-based graph microservice using Apache Jena
 
 Create two PowerShell Terminal windows, and navigate to the **impl/app/** directory in each.
 
-In the first terminal window, execute the following command to start the application
-(both microservices).
+In the first terminal window, execute the following command to start the application (both microservices).
 
 ```
-docker compose -f docker/docker-compose.yml up
+docker compose -f docker/docker-compose-with-rdflib.yml up
+or
+docker compose -f docker/docker-compose-with-jena.yml up
 ```
 
-You should see verbose output that includes the following:
+You should see similar verbose output that includes the following:
 
 <p align="center">
   <img src="img/docker-compose-up.png" width="50%">
@@ -62,10 +73,12 @@ You should see verbose output that includes the following:
 In the second terminal window, execute the following command to terminate the application.
 
 ```
-docker compose -f docker/docker-compose.yml down
+docker compose -f docker/docker-compose-with-rdflib.yml down
+or
+docker compose -f docker/docker-compose-with-jena.yml down
 ```
 
-You should see verbose output that includes the following:
+You should see similar verbose output that includes the following:
 
 <p align="center">
   <img src="img/docker-compose-down.png" width="40%">
@@ -75,15 +88,20 @@ You should see verbose output that includes the following:
 
 ### The Docker Containers
 
-These two pre-built Docker containers exist on **DockerHub**:
+These three pre-built Docker containers exist on **DockerHub**:
 
 - cjoakim/caig_web_v2:latest
 - cjoakim/caig_graph_v2:latest
+- cjoakim/caig_graph_java_jena_v1:latest
 
-These are used by default by the above **docker-compose** script
+These are used by default by the above **docker-compose** scripts
 and also by the **Azure Container App** deployment process.
 
-If you wish to rebuild these containers and deploy them to your own Container Registry,
-please see the **impl/app/docker-builds.ps1** and **impl/app/docker-builds.sh**
-scripts in this repository.  You're free to modify these as necessary.
+If you wish to rebuild these containers and deploy them to your own
+Container Registry, please see the following Dockerfiles in this repo.
+You're free to modify these as necessary.
 Please change the **cjoakim** prefix to your own identifier.
+
+- impl/app/docker/Dockerfile_graph
+- impl/app/docker/Dockerfile_web
+- impl/java_jena_graph_websvc/Dockerfile

impl/docs/local_execution.md

@@ -1,6 +1,6 @@
 
 <p align="center">
-  <img src="img/app-architecture-v2.png" width="90%">
+  <img src="img/app-architecture-v3.png" width="90%">
 </p>
 
 

impl/docs/readme.md

@@ -11,7 +11,7 @@ or none.
 
 ## The impl\app directory
 
-**This directory contains the current codebase.**
+**This directory contains the Python codebase.**
 
 It contains these directories:
 
@@ -24,10 +24,11 @@ It contains these directories:
 
 ## The impl\app directory
 
-This is where the implementation code and scripts are.  It contains these directories.
+This is where the Python implementation code and scripts are.
+It contains these directories.
 
 ```
-├── docker              Dockerfiles and docker-compose.yml
+├── docker              Dockerfiles and docker-compose yml files
 ├── keys                Future use
 ├── ontologies          OWL schema files, with *.owl file suffix
 ├── rdf                 RDF graph data files in "triples" format, with *.nt file suffix
@@ -275,3 +276,127 @@ personal coding style preferences.  In addition to reformatting the source
 code, black can also identify some problems/errors in the code.
 
 See the **code-reformat.ps1** and **code-reformat.sh** scripts in the impl directory.
+
+--- 
+
+## The impl\java_jena_graph_websvc directory
+
+This directory contains the newer Graph Microservice implemented
+with Java, Spring Boot, and Apache Jena.  Gradle is used as the
+build tool.
+
+Please see the readme.md file in this directory regarding
+building and executing this version of the Graph Microservice.
+
+It contains these directories:
+
+```
+├── build                Output of the Gradle-based compilation and packaging process
+├── data
+├── data/cosmosdb_documents.json   
+├── ontologies
+├── rdf                  RDF files optionally used to load the graph
+├── src                  Standard Java source code directory structure
+│   ├── main
+│   └── test
+└── tmp                  Create this directory if it doesn't already exist
+```
+
+### Key Classes in the Java/Jena implementation
+
+This section describes the primary Java classes in the Java/Jena implementation.
+
+#### com.microsoft.cosmosdb.caig.WebApp
+
+This is the Spring Boot entry point for the application, per the 
+@SpringBootApplication annotation.
+
+#### com.microsoft.cosmosdb.caig.web.GraphRestController
+
+Spring @RestController that implements the **/sparql_query** HTTP endpoint.
+This endpoint is invoked by the Python-based Web UI when
+executing graph queries.
+
+#### com.microsoft.cosmosdb.caig.web.HealthRestController
+
+Spring @RestController that implements the **/health** HTTP endpoint.
+This endpoint that can optionally be invoked by your container
+orchestrator runtime environment - such as Azure Container Apps (ACA)
+or Azure Kubernetes Service (AKS).
+
+#### com.microsoft.cosmosdb.caig.web.PingRestController
+
+Spring @RestController that implements the **/** and **/ping** HTTP endpoints.
+The / endpoint simply returns the epoch time, while /ping returns
+uptime and JVM memory information.
+
+#### com.microsoft.cosmosdb.caig.web.AppStartup
+
+This contains the application startup logic per the Spring
+ApplicationListener interface.  It contains this logic which 
+loads the in-memory graph in class AppGraph.
+
+```
+  AppGraph g = AppGraphBuilder.build(null);
+  AppGraph.setSingleton(g);```
+```
+
+#### com.microsoft.cosmosdb.caig.util.AppConfig
+
+This static class returns almost all configuration values for the 
+application, such as from **environment variables**.
+
+The environment variables begin with the **CAIG_** prefix and are described
+in the [Environment Variables](environment_variables.md) page
+
+The Spring Boot framework also uses the src/main/resources/application.properties
+file for some configuration values.  But all **application coniguration**
+is done with environment variables and class AppConfig.  This approach 
+is typically used by Docker containerized applications.
+
+#### com.microsoft.cosmosdb.caig.graph.AppGraphBuilder
+
+This class creates and populates the in-memory graph object
+which is an instance of class **org.apache.jena.rdf.model.Model**.
+
+AppGraphBuilder can populate the graph from one of several 
+sources per the CAIG_GRAPH_SOURCE_TYPE environment variable.
+CAIG_GRAPH_SOURCE_TYPE may have one of the following values:
+
+- json_docs_file - the graph is sourced from ile cosmosdb_documents.json in the repo
+- rdf_file - the graph is sourced from the file specified in CAIG_GRAPH_SOURCE_RDF_FILENAME
+- cosmos_nosql - the graph is sourced from your Cosmos DB NoSQL account
+- cosmos_vcore - the graph is sourced from your Cosmos DB Mongo vCore account
+
+After the Jena graph is populated, you can optionally dump that
+graph to a file per the CAIG_GRAPH_DUMP_UPON_BUILD and CAIG_GRAPH_DUMP_OUTFILE
+environment variables.
+
+#### com.microsoft.cosmosdb.caig.graph.AppGraph
+
+This class contains the singleton instance of class
+org.apache.jena.rdf.model.Model in the Apache Jena SDK.
+
+It implements this primary message signature:
+
+```
+    public synchronized SparqlQueryResponse query(SparqlQueryRequest request) {
+    }
+```
+
+The method is **synchronized** so as to be thread safe.  Each HTTP request 
+to the Spring Boot application runs in its' own Thread.
+
+Classes SparqlQueryRequest and SparqlQueryResponse are simple
+JSON serializable classes used to receive the HTTP POSTed query
+and to return the JSON response to the query. 
+
+#### com.microsoft.cosmosdb.caig.graph.LibrariesGraphTriplesBuilder
+
+For the cases where AppGraphBuilder sources the graph from Cosmos DB,
+instances of LibrariesGraphTriplesBuilder are uses to populate the
+graph from each appropriate Cosmos DB document.
+
+Customers should implement their own GraphTriplesBuilder class for
+their needs and implement as necessary per the shape of your Cosmos DB
+documents and graph schema.