feat(endpoints): Add OpenAI Responses API endpoint with fixes and integration tests

.github/workflows/docs.yml

@@ -0,0 +1,51 @@
+name: Publish docs
+
+on:
+  release:
+    types: [published]
+  push:
+    branches: [main]
+    paths:
+      - 'docs/**'
+      - 'mkdocs.yml'
+      - '.github/workflows/docs.yml'
+
+permissions:
+  contents: read
+  id-token: write
+  pages: write
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    environment:
+      name: github-pages
+      url: ${{ steps.deploy.outputs.page_url }}
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v4
+
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version: "3.12"
+
+    - name: Install uv
+      uses: astral-sh/setup-uv@v7
+      with:
+        enable-cache: true
+
+    - name: Install dependencies
+      run: uv sync --only-group docs
+
+    - name: Build
+      run: uv run --no-sync zensical build --clean
+
+    - name: Upload
+      uses: actions/upload-pages-artifact@v4
+      with:
+        path: site
+
+    - name: Deploy
+      id: deploy
+      uses: actions/deploy-pages@v4

.github/workflows/pypi.yml

@@ -23,37 +23,29 @@ jobs:
       - name: Checkout code
         uses: actions/checkout@v4
 
-      - name: Set up Python 3.10
+      - name: Set up Python 3.12
         uses: actions/setup-python@v5
         with:
-          python-version: "3.10"
+          python-version: "3.12"
 
-      - name: Install Poetry
-        run: |
-          curl -sSL https://install.python-poetry.org | python - -y
-
-      - name: Update PATH
-        run: echo "$HOME/.local/bin" >> $GITHUB_PATH
-
-      - name: Update Poetry configuration
-        run: poetry config virtualenvs.create false
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+        with:
+          enable-cache: true
 
       - name: Configure pyproject version
         run: |
           if [[ $GITHUB_REF_TYPE == "tag" ]] && [[ $GITHUB_REF_NAME == v* ]]
           then
             # Drop leading 'v'
-            poetry version ${GITHUB_REF_NAME:1}
+            uv version --no-sync ${GITHUB_REF_NAME:1}
           else
             echo "Invalid tag trigger\nGITHUB_REF_TYPE=${GITHUB_REF_TYPE}\nGITHUB_REF_NAME=${GITHUB_REF_NAME}"
             exit 1
           fi
 
-      - name: Install dependencies
-        run: poetry install --sync --no-interaction
-
       - name: Package project
-        run: poetry build
+        run: uv build
 
       - name: Publish package distributions to PyPI
         uses: pypa/gh-action-pypi-publish@release/v1

.gitignore

@@ -99,7 +99,7 @@ ipython_config.py
 # currently check in any lockfiles:
 pdm.lock
 Pipfile.lock
-poetry.lock
+uv.lock
 
 # pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
 # in version control.
@@ -134,7 +134,7 @@ venv.bak/
 # Rope project settings
 .ropeproject
 
-# mkdocs documentation
+# mkdocs/zensical documentation
 /site
 
 # mypy

CONTRIBUTING.md

@@ -15,6 +15,7 @@ information to effectively respond to your bug report or contribution.
     - [Best practices](#best-practices)
     - [Getting Started](#getting-started)
       - [Linting/Formatting](#lintingformatting)
+      - [Documentation](#documentation)
       - [Testing](#testing)
   - [Finding Contributions to Work On](#finding-contributions-to-work-on)
   - [Code of Conduct](#code-of-conduct)
@@ -53,10 +54,26 @@ Contributions via pull requests are much appreciated. Before sending us a pull r
 
 ### Getting Started
 
-We recommend installing the package locally in editable mode for ease of development. To install the package in editable mode along with the development dependencies, run the following:
+We recommend installing the package locally in editable mode for ease of development.
+
+First, ensure you have [uv](https://docs.astral.sh/uv/) installed. Then, to install the package in editable mode along with the development dependencies and all optional dependencies, run:
+
+```bash
+uv sync --all-extras
+```
+
+This will install all dependencies (including optional ones like plotting, openai, litellm, and mlflow) and create a virtual environment in `.venv/`.
+
+For a minimal installation with just dev and test dependencies:
+
+```bash
+uv sync --all-groups
+```
+
+For just dev dependencies without test dependencies:
 
 ```bash
-pip install -e ".[dev]"
+uv sync --group dev
 ```
 
 #### Linting/Formatting
@@ -68,17 +85,60 @@ The tools below are used for linting and formatting the codebase.
 To check for linting and formatting issues, you can run the following:
 
 ```bash
-ruff check  llmeter/ &&  ruff format llmeter/ 
+uv run ruff check llmeter/ && uv run ruff format llmeter/
+```
+
+Or if not using uv:
+
+```bash
+ruff check llmeter/ && ruff format llmeter/
 ```
 
+#### Documentation
+
+This project uses [Zensical](https://zensical.org/) (A compatible [alernative to MkDocs](https://squidfunk.github.io/mkdocs-material/blog/2025/11/05/zensical/)) to power an interactive docs website, hosted on GitHub pages. This documentation should be updated as part of contributed changes, wherever appropriate.
+
+To work on docs locally, install only the docs dependencies (this avoids pulling in the full project dependency tree):
+
+```bash
+uv sync --only-group docs
+```
+
+Then preview with the dev server:
+
+```bash
+uv run --no-sync zensical serve
+```
+
+This starts a preview at http://localhost:8000/ that auto-refreshes as you edit `docs/`.
+
+To do a clean production build:
+
+```bash
+uv run --no-sync zensical build --clean
+```
+
+⚠️ Note - The following processes are stil **manual** for now (contributions to automate this work are much appreciated!):
+- When adding/renaming/deleting .py modules in LLMeter, you need to update the corresponding API reference folders and .md files under `docs/reference`
+  - `index.md` files in the API reference should include a top-level title equal to the module name, so they don't just show as "index" in the site. For example, `# callbacks` in [docs/reference/callbacks/index.md](docs/reference/callbacks/index.md).
+- The table of contents in [mkdocs.yaml](mkdocs.yaml) should be updated to include all the .md files under `docs/`.
+
 #### Testing
 
 This project uses [pytest](https://docs.pytest.org/en/8.2.x/) for unit testing, which you can invoke using the following:
 
 ```bash
-python -m pytest .
+uv run pytest
+```
+
+Or if not using uv:
+
+```bash
+python -m pytest
 ```
 
+⚠️ Note that integration tests are skipped by default. More details on testing in [tests/README.md](tests/README.md).
+
 ## Finding Contributions to Work On
 
 Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels, looking for any issues labeled `good first issue` or `help wanted` is a great place to start.

README.md

@@ -4,23 +4,32 @@
 **Measuring large language models latency and throughput**
 
 [![Latest Version](https://img.shields.io/pypi/v/llmeter.svg)](https://pypi.python.org/pypi/llmeter)
+[![Documentation: Online](https://img.shields.io/badge/Documentation-online-green)](https://awslabs.github.io/llmeter)
 [![Supported Python Versions](https://img.shields.io/pypi/pyversions/llmeter)](https://pypi.python.org/pypi/llmeter)
 [![Code Style: Ruff](https://img.shields.io/badge/code_style-ruff-000000.svg)](https://github.com/astral-sh/ruff)
 
 </div>
 
 LLMeter is a pure-python library for simple latency and throughput testing of large language models (LLMs). It's designed to be lightweight to install; straightforward to run standard tests; and versatile to integrate - whether in notebooks, CI/CD, or other workflows.
 
+📖 For full details, check out our documentation at: **https://awslabs.github.io/llmeter**
+
 ## 🛠️ Installation
 
 LLMeter requires `python>=3.10`, please make sure your current version of python is compatible.
 
-To install the basic metering functionalities, you can install the minimum package using pip install:
+To install the basic metering functionalities, you can install the minimum package using pip or uv:
 
 ```terminal
 pip install llmeter
 ```
 
+Or with uv (recommended for faster installation):
+
+```terminal
+uv pip install llmeter
+```
+
 LLMeter also offers extra features that require additional dependencies. Currently these extras include:
 
 - **plotting**: Add methods to generate charts to summarize the results
@@ -34,6 +43,12 @@ You can install one or more of these extra options using pip:
 pip install 'llmeter[plotting,openai,litellm,mlflow]'
 ```
 
+Or with uv:
+
+```terminal
+uv pip install 'llmeter[plotting,openai,litellm,mlflow]'
+```
+
 ## 🚀 Quick-start
 
 At a high level, you'll start by configuring an LLMeter "Endpoint" for whatever type of LLM you're connecting to: