The SDK can be installed with either npm, pnpm, bun or yarn package managers.
npm add honeyhivepnpm add honeyhivebun add honeyhiveyarn add honeyhive zod
# Note that Yarn does not install peer dependencies automatically. You will need
# to install zod as shown above.This SDK is also an installable MCP server where the various SDK methods are exposed as tools that can be invoked by AI applications.
Node.js v20 or greater is required to run the MCP server from npm.
Claude installation steps
Add the following server definition to your claude_desktop_config.json file:
{
"mcpServers": {
"HoneyHive": {
"command": "npx",
"args": [
"-y", "--package", "honeyhive",
"--",
"mcp", "start",
"--bearer-auth", "..."
]
}
}
}Cursor installation steps
Create a .cursor/mcp.json file in your project root with the following content:
{
"mcpServers": {
"HoneyHive": {
"command": "npx",
"args": [
"-y", "--package", "honeyhive",
"--",
"mcp", "start",
"--bearer-auth", "..."
]
}
}
}You can also run MCP servers as a standalone binary with no additional dependencies. You must pull these binaries from available Github releases:
curl -L -o mcp-server \
https://github.com/{org}/{repo}/releases/download/{tag}/mcp-server-bun-darwin-arm64 && \
chmod +x mcp-serverIf the repo is a private repo you must add your Github PAT to download a release -H "Authorization: Bearer {GITHUB_PAT}".
{
"mcpServers": {
"Todos": {
"command": "./DOWNLOAD/PATH/mcp-server",
"args": [
"start"
]
}
}
}For a full list of server arguments, run:
npx -y --package honeyhive -- mcp start --helpimport { HoneyHive } from "honeyhive";
const honeyHive = new HoneyHive({
bearerAuth: "<YOUR_BEARER_TOKEN_HERE>",
});
async function run() {
const result = await honeyHive.session.startSession({
session: {
project: "Simple RAG Project",
sessionName: "Playground Session",
source: "playground",
sessionId: "caf77ace-3417-4da4-944d-f4a0688f3c23",
childrenIds: [
"7f22137a-6911-4ed3-bc36-110f1dde6b66",
],
inputs: {
"context": "Hello world",
"question": "What is in the context?",
"chat_history": [
{
"role": "system",
"content":
"Answer the user's question only using provided context.\n"
+ "\n"
+ "Context: Hello world",
},
{
"role": "user",
"content": "What is in the context?",
},
],
},
outputs: {
"role": "assistant",
"content": "Hello world",
},
error: "<value>",
duration: 824.8056,
userProperties: {
"user": "google-oauth2|111840237613341303366",
},
startTime: 1712025501605,
endTime: 1712025499832,
},
});
// Handle the result
console.log(result);
}
run();Available methods
- getConfigurations - Retrieve a list of configurations
- createConfiguration - Create a new configuration
- updateConfiguration - Update an existing configuration
- deleteConfiguration - Delete a configuration
- getDatapoints - Retrieve a list of datapoints
- createDatapoint - Create a new datapoint
- getDatapoint - Retrieve a specific datapoint
- updateDatapoint - Update a specific datapoint
- deleteDatapoint - Delete a specific datapoint
- getDatasets - Get datasets
- createDataset - Create a dataset
- updateDataset - Update a dataset
- deleteDataset - Delete a dataset
- addDatapoints - Add datapoints to a dataset
- createEvent - Create a new event
- updateEvent - Update an event
- getEvents - Retrieve events based on filters
- createModelEvent - Create a new model event
- createEventBatch - Create a batch of events
- createModelEventBatch - Create a batch of model events
- createRun - Create a new evaluation run
- getRuns - Get a list of evaluation runs
- getRun - Get details of an evaluation run
- updateRun - Update an evaluation run
- deleteRun - Delete an evaluation run
- getExperimentResult - Retrieve experiment result
- getExperimentComparison - Retrieve experiment comparison
- getMetrics - Get all metrics
- createMetric - Create a new metric
- updateMetric - Update an existing metric
- deleteMetric - Delete a metric
- getProjects - Get a list of projects
- createProject - Create a new project
- updateProject - Update an existing project
- deleteProject - Delete a project
- startSession - Start a new session
- getSession - Retrieve a session
- getTools - Retrieve a list of tools
- createTool - Create a new tool
- updateTool - Update an existing tool
- deleteTool - Delete a tool
Some methods specify known errors which can be thrown. All the known errors are enumerated in the models/errors/errors.ts module. The known errors for a method are documented under the Errors tables in SDK docs. For example, the createEventBatch method may throw the following errors:
| Error Type | Status Code | Content Type |
|---|---|---|
| errors.CreateEventBatchResponseBody | 500 | application/json |
| errors.SDKError | 4XX, 5XX | */* |
If the method throws an error and it is not captured by the known errors, it will default to throwing a SDKError.
import { HoneyHive } from "honeyhive";
import {
CreateEventBatchResponseBody,
SDKValidationError,
} from "honeyhive/models/errors";
const honeyHive = new HoneyHive({
bearerAuth: "<YOUR_BEARER_TOKEN_HERE>",
});
async function run() {
let result;
try {
result = await honeyHive.events.createEventBatch({
events: [
{
project: "Simple RAG",
source: "playground",
eventName: "Model Completion",
eventType: "model",
eventId: "7f22137a-6911-4ed3-bc36-110f1dde6b66",
sessionId: "caf77ace-3417-4da4-944d-f4a0688f3c23",
parentId: "caf77ace-3417-4da4-944d-f4a0688f3c23",
childrenIds: [],
config: {
"model": "gpt-3.5-turbo",
"version": "v0.1",
"provider": "openai",
"hyperparameters": {
"temperature": 0,
"top_p": 1,
"max_tokens": 1000,
"presence_penalty": 0,
"frequency_penalty": 0,
"stop": [],
"n": 1,
},
"template": [
{
"role": "system",
"content":
"Answer the user's question only using provided context.\n"
+ "\n"
+ "Context: {{ context }}",
},
{
"role": "user",
"content": "{{question}}",
},
],
"type": "chat",
},
inputs: {
"context": "Hello world",
"question": "What is in the context?",
"chat_history": [
{
"role": "system",
"content":
"Answer the user's question only using provided context.\n"
+ "\n"
+ "Context: Hello world",
},
{
"role": "user",
"content": "What is in the context?",
},
],
},
outputs: {
"role": "assistant",
"content": "Hello world",
},
error: "<value>",
startTime: 1714978764301,
endTime: 1714978765301,
duration: 999.8056,
metadata: {
"cost": 0.00008,
"completion_tokens": 23,
"prompt_tokens": 35,
"total_tokens": 58,
},
metrics: {
"Answer Faithfulness": 5,
"Answer Faithfulness_explanation":
"The AI assistant's answer is a concise and accurate description of Ramp's API. It provides a clear explanation of what the API does and how developers can use it to integrate Ramp's financial services into their own applications. The answer is faithful to the provided context.",
"Number of words": 18,
},
userProperties: {
"user": "google-oauth2|111840237613341303366",
},
},
],
sessionProperties: {
sessionName: "Playground Session",
source: "playground",
sessionId: "caf77ace-3417-4da4-944d-f4a0688f3c23",
inputs: {
"context": "Hello world",
"question": "What is in the context?",
"chat_history": [
{
"role": "system",
"content":
"Answer the user's question only using provided context.\n"
+ "\n"
+ "Context: Hello world",
},
{
"role": "user",
"content": "What is in the context?",
},
],
},
outputs: {
"role": "assistant",
"content": "Hello world",
},
error: "<value>",
userProperties: {
"user": "google-oauth2|111840237613341303366",
},
},
});
// Handle the result
console.log(result);
} catch (err) {
switch (true) {
// The server response does not match the expected SDK schema
case (err instanceof SDKValidationError): {
// Pretty-print will provide a human-readable multi-line error message
console.error(err.pretty());
// Raw value may also be inspected
console.error(err.rawValue);
return;
}
case (err instanceof CreateEventBatchResponseBody): {
// Handle err.data$: CreateEventBatchResponseBodyData
console.error(err);
return;
}
default: {
// Other errors such as network errors, see HTTPClientErrors for more details
throw err;
}
}
}
}
run();Validation errors can also occur when either method arguments or data returned from the server do not match the expected format. The SDKValidationError that is thrown as a result will capture the raw value that failed validation in an attribute called rawValue. Additionally, a pretty() method is available on this error that can be used to log a nicely formatted multi-line string since validation errors can list many issues and the plain error string may be difficult read when debugging.
In some rare cases, the SDK can fail to get a response from the server or even make the request due to unexpected circumstances such as network conditions. These types of errors are captured in the models/errors/httpclienterrors.ts module:
| HTTP Client Error | Description |
|---|---|
| RequestAbortedError | HTTP request was aborted by the client |
| RequestTimeoutError | HTTP request timed out due to an AbortSignal signal |
| ConnectionError | HTTP client was unable to make a request to a server |
| InvalidRequestError | Any input used to create a request is invalid |
| UnexpectedClientError | Unrecognised or unexpected error |
The default server can be overridden globally by passing a URL to the serverURL: string optional parameter when initializing the SDK client instance. For example:
import { HoneyHive } from "honeyhive";
const honeyHive = new HoneyHive({
serverURL: "https://api.honeyhive.ai",
bearerAuth: "<YOUR_BEARER_TOKEN_HERE>",
});
async function run() {
const result = await honeyHive.session.startSession({
session: {
project: "Simple RAG Project",
sessionName: "Playground Session",
source: "playground",
sessionId: "caf77ace-3417-4da4-944d-f4a0688f3c23",
childrenIds: [
"7f22137a-6911-4ed3-bc36-110f1dde6b66",
],
inputs: {
"context": "Hello world",
"question": "What is in the context?",
"chat_history": [
{
"role": "system",
"content":
"Answer the user's question only using provided context.\n"
+ "\n"
+ "Context: Hello world",
},
{
"role": "user",
"content": "What is in the context?",
},
],
},
outputs: {
"role": "assistant",
"content": "Hello world",
},
error: "<value>",
duration: 824.8056,
userProperties: {
"user": "google-oauth2|111840237613341303366",
},
startTime: 1712025501605,
endTime: 1712025499832,
},
});
// Handle the result
console.log(result);
}
run();The TypeScript SDK makes API calls using an HTTPClient that wraps the native
Fetch API. This
client is a thin wrapper around fetch and provides the ability to attach hooks
around the request lifecycle that can be used to modify the request or handle
errors and response.
The HTTPClient constructor takes an optional fetcher argument that can be
used to integrate a third-party HTTP client or when writing tests to mock out
the HTTP client and feed in fixtures.
The following example shows how to use the "beforeRequest" hook to to add a
custom header and a timeout to requests and how to use the "requestError" hook
to log errors:
import { HoneyHive } from "honeyhive";
import { HTTPClient } from "honeyhive/lib/http";
const httpClient = new HTTPClient({
// fetcher takes a function that has the same signature as native `fetch`.
fetcher: (request) => {
return fetch(request);
}
});
httpClient.addHook("beforeRequest", (request) => {
const nextRequest = new Request(request, {
signal: request.signal || AbortSignal.timeout(5000)
});
nextRequest.headers.set("x-custom-header", "custom value");
return nextRequest;
});
httpClient.addHook("requestError", (error, request) => {
console.group("Request Error");
console.log("Reason:", `${error}`);
console.log("Endpoint:", `${request.method} ${request.url}`);
console.groupEnd();
});
const sdk = new HoneyHive({ httpClient });This SDK supports the following security scheme globally:
| Name | Type | Scheme |
|---|---|---|
bearerAuth |
http | HTTP Bearer |
To authenticate with the API the bearerAuth parameter must be set when initializing the SDK client instance. For example:
import { HoneyHive } from "honeyhive";
const honeyHive = new HoneyHive({
bearerAuth: "<YOUR_BEARER_TOKEN_HERE>",
});
async function run() {
const result = await honeyHive.session.startSession({
session: {
project: "Simple RAG Project",
sessionName: "Playground Session",
source: "playground",
sessionId: "caf77ace-3417-4da4-944d-f4a0688f3c23",
childrenIds: [
"7f22137a-6911-4ed3-bc36-110f1dde6b66",
],
inputs: {
"context": "Hello world",
"question": "What is in the context?",
"chat_history": [
{
"role": "system",
"content":
"Answer the user's question only using provided context.\n"
+ "\n"
+ "Context: Hello world",
},
{
"role": "user",
"content": "What is in the context?",
},
],
},
outputs: {
"role": "assistant",
"content": "Hello world",
},
error: "<value>",
duration: 824.8056,
userProperties: {
"user": "google-oauth2|111840237613341303366",
},
startTime: 1712025501605,
endTime: 1712025499832,
},
});
// Handle the result
console.log(result);
}
run();For supported JavaScript runtimes, please consult RUNTIMES.md.
All the methods listed above are available as standalone functions. These functions are ideal for use in applications running in the browser, serverless runtimes or other environments where application bundle size is a primary concern. When using a bundler to build your application, all unused functionality will be either excluded from the final bundle or tree-shaken away.
To read more about standalone functions, check FUNCTIONS.md.
Available standalone functions
configurationsCreateConfiguration- Create a new configurationconfigurationsDeleteConfiguration- Delete a configurationconfigurationsGetConfigurations- Retrieve a list of configurationsconfigurationsUpdateConfiguration- Update an existing configurationdatapointsCreateDatapoint- Create a new datapointdatapointsDeleteDatapoint- Delete a specific datapointdatapointsGetDatapoint- Retrieve a specific datapointdatapointsGetDatapoints- Retrieve a list of datapointsdatapointsUpdateDatapoint- Update a specific datapointdatasetsAddDatapoints- Add datapoints to a datasetdatasetsCreateDataset- Create a datasetdatasetsDeleteDataset- Delete a datasetdatasetsGetDatasets- Get datasetsdatasetsUpdateDataset- Update a dataseteventsCreateEvent- Create a new eventeventsCreateEventBatch- Create a batch of eventseventsCreateModelEvent- Create a new model eventeventsCreateModelEventBatch- Create a batch of model eventseventsGetEvents- Retrieve events based on filterseventsUpdateEvent- Update an eventexperimentsCreateRun- Create a new evaluation runexperimentsDeleteRun- Delete an evaluation runexperimentsGetExperimentComparison- Retrieve experiment comparisonexperimentsGetExperimentResult- Retrieve experiment resultexperimentsGetRun- Get details of an evaluation runexperimentsGetRuns- Get a list of evaluation runsexperimentsUpdateRun- Update an evaluation runmetricsCreateMetric- Create a new metricmetricsDeleteMetric- Delete a metricmetricsGetMetrics- Get all metricsmetricsUpdateMetric- Update an existing metricprojectsCreateProject- Create a new projectprojectsDeleteProject- Delete a projectprojectsGetProjects- Get a list of projectsprojectsUpdateProject- Update an existing projectsessionGetSession- Retrieve a sessionsessionStartSession- Start a new sessiontoolsCreateTool- Create a new tooltoolsDeleteTool- Delete a tooltoolsGetTools- Retrieve a list of toolstoolsUpdateTool- Update an existing tool
Some of the endpoints in this SDK support retries. If you use the SDK without any configuration, it will fall back to the default retry strategy provided by the API. However, the default retry strategy can be overridden on a per-operation basis, or across the entire SDK.
To change the default retry strategy for a single API call, simply provide a retryConfig object to the call:
import { HoneyHive } from "honeyhive";
const honeyHive = new HoneyHive({
bearerAuth: "<YOUR_BEARER_TOKEN_HERE>",
});
async function run() {
const result = await honeyHive.session.startSession({
session: {
project: "Simple RAG Project",
sessionName: "Playground Session",
source: "playground",
sessionId: "caf77ace-3417-4da4-944d-f4a0688f3c23",
childrenIds: [
"7f22137a-6911-4ed3-bc36-110f1dde6b66",
],
inputs: {
"context": "Hello world",
"question": "What is in the context?",
"chat_history": [
{
"role": "system",
"content":
"Answer the user's question only using provided context.\n"
+ "\n"
+ "Context: Hello world",
},
{
"role": "user",
"content": "What is in the context?",
},
],
},
outputs: {
"role": "assistant",
"content": "Hello world",
},
error: "<value>",
duration: 824.8056,
userProperties: {
"user": "google-oauth2|111840237613341303366",
},
startTime: 1712025501605,
endTime: 1712025499832,
},
}, {
retries: {
strategy: "backoff",
backoff: {
initialInterval: 1,
maxInterval: 50,
exponent: 1.1,
maxElapsedTime: 100,
},
retryConnectionErrors: false,
},
});
// Handle the result
console.log(result);
}
run();If you'd like to override the default retry strategy for all operations that support retries, you can provide a retryConfig at SDK initialization:
import { HoneyHive } from "honeyhive";
const honeyHive = new HoneyHive({
retryConfig: {
strategy: "backoff",
backoff: {
initialInterval: 1,
maxInterval: 50,
exponent: 1.1,
maxElapsedTime: 100,
},
retryConnectionErrors: false,
},
bearerAuth: "<YOUR_BEARER_TOKEN_HERE>",
});
async function run() {
const result = await honeyHive.session.startSession({
session: {
project: "Simple RAG Project",
sessionName: "Playground Session",
source: "playground",
sessionId: "caf77ace-3417-4da4-944d-f4a0688f3c23",
childrenIds: [
"7f22137a-6911-4ed3-bc36-110f1dde6b66",
],
inputs: {
"context": "Hello world",
"question": "What is in the context?",
"chat_history": [
{
"role": "system",
"content":
"Answer the user's question only using provided context.\n"
+ "\n"
+ "Context: Hello world",
},
{
"role": "user",
"content": "What is in the context?",
},
],
},
outputs: {
"role": "assistant",
"content": "Hello world",
},
error: "<value>",
duration: 824.8056,
userProperties: {
"user": "google-oauth2|111840237613341303366",
},
startTime: 1712025501605,
endTime: 1712025499832,
},
});
// Handle the result
console.log(result);
}
run();You can setup your SDK to emit debug logs for SDK requests and responses.
You can pass a logger that matches console's interface as an SDK option.
Warning
Beware that debug logging will reveal secrets, like API tokens in headers, in log messages printed to a console or files. It's recommended to use this feature only during local development and not in production.
import { HoneyHive } from "honeyhive";
const sdk = new HoneyHive({ debugLogger: console });This SDK is in beta, and there may be breaking changes between versions without a major version update. Therefore, we recommend pinning usage to a specific package version. This way, you can install the same version each time without breaking changes unless you are intentionally looking for the latest version.
While we value open-source contributions to this SDK, this library is generated programmatically. Feel free to open a PR or a Github issue as a proof of concept and we'll do our best to include it in a future release!