Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,6 @@
- `src/api.ts`: minimal fetch client for the TypeSafe System One API (`typesafe({ apiKey })` → `systemOne`, `models`), plus `noul`/`choice`/`score` builders and typed answers. Reads `TYPESAFE_API_KEY`, `TYPESAFE_BASE_URL`, `TYPESAFE_DEFAULT_MODEL` from `globalThis.process?.env` when options are omitted. With only `AI_GATEWAY_API_KEY` or `VERCEL_OIDC_TOKEN` set (or `provider: "vercel"`), it speaks the Vercel AI Gateway evaluation protocol instead and maps answers back to the System One shape (`confidence` computed from the top probability, `legend` built from criteria, `models()` throws). Validates API limits (choice 2–255 options, score 2–10 levels). No retries or logging.
- `src/ask.ts`: `ask(state, questions, options?)` from `IDEA.md`. Bare strings, `ask.choice`/`ask.score`/`ask.chance` tags (also callable as `ask.choice(instructions, criteria, options?)`, where instructions may be a JSON object or array), and plain client question objects go out as one request; answers come back under the same keys with `chance` (noul) and `ratio` (score scaled to 0–1) added. Every tag returns an `Askable`: the plain question object plus a hidden (non-enumerable) `then` that sends it alone under the key `input` and resolves to its answer, so anything that unwraps promises sends a request. Interpolated objects and arrays are kept in a `WeakMap` off the wire; when sent, the distinct objects of all tags in the request go into `input` (one directly, several as an array) and each slot is rendered as its path there. Inside `ask` they are added to an object state (which must not have `input`; empty or `null` becomes `{}`), and a text or array state goes first in that `input` array. The key stays `input` on purpose: on the real API, question-derived keys primed answers toward yes, and a named key for the text did worse than text-first. Options for a standalone send go in the criteria call.
- `askIf` (also `ask.if`) is a tag with no criteria call for one yes/no question resolved to a boolean (`askIf(options)` binds options; `threshold` defaults to 0.5, strict `>`); inside `ask` it goes out as a noul question and resolves to a boolean under its key.
- `src/index.ts` exports `ask`, `askIf`, and the tags; the client's `noul`/`choice`/`score` builders stay internal.
- `askSwitch` (also `ask.switch`) takes the same forms as `ask.choice`, goes out as a choice question, and resolves to the selected label alone. Named exports because `if` and `switch` are reserved words. Both tags accept a label array as criteria, normalized to `{ label: null }` before the wire so the answer type keeps the label union.
- `src/index.ts` exports `ask`, `askIf`, `askSwitch`, and the tags; the client's `noul`/`choice`/`score` builders stay internal.
- `playground/`: private workspace package with a GitHub issue triage CLI (`ask` over `octokit`). `pnpm-workspace.yaml` overrides `advocaat` to `link:.`, and `package.json` `exports` points at `src/` (swapped to `dist/` by `publishConfig` on publish). Node runs the source directly, so `src/` must stay erasable-syntax only (enforced by tsconfig).
37 changes: 26 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,12 +54,11 @@ console.log(severity.ratio); // 0...1

Answers use the same keys as your questions. Mix any of these forms in one `ask` call:

| Want | Use | Read |
| ----------- | ---------------------- | --------------- |
| Boolean | `ask.if` | Answer directly |
| Probability | String or `ask.chance` | `.chance` |
| Category | `ask.choice` | `.choice` |
| Rating | `ask.score` | `.ratio` |
| Want | Use | Read | Or, for the value alone |
| ------ | ---------------------- | --------- | ----------------------- |
| Yes/no | String or `ask.chance` | `.chance` | `ask.if` → `boolean` |
| Option | `ask.choice` | `.choice` | `ask.switch` → label |
| Rating | `ask.score` | `.ratio` | |

The examples below reuse `ask` and `issue` from the quick start.

Expand Down Expand Up @@ -103,6 +102,19 @@ console.log(kind.choice); // "bug" | "other"

Returns `{ type: "choice", choice, confidence, probabilities }`. `choice` is the selected label, typed as `"bug" | "other"` here. `probabilities` contains a probability for each label. `confidence` (0...1) is high when one label stands out and low when they are close.

When the labels speak for themselves, pass them as an array instead of an object with `null` descriptions: ``ask.choice`What kind of issue is this?`(["bug", "other"])``.

Use `ask.switch` with the same options when you only need the label:

```ts
switch (await ask.switch`What kind of issue is ${issue}?`(["bug", "other"])) {
case "bug":
return label(issue, "bug");
case "other":
return triage(issue);
}
```

### Scores

Use `ask.score` with 2–10 levels, ordered from lowest to highest:
Expand All @@ -126,7 +138,8 @@ Returns `{ type: "score", score, ratio, confidence, legend, probabilities }`. `s
For a single question, interpolate the data into a tag and await it. You get the same answer as you would under its key in `ask`:

```ts
const kind = await ask.choice`What kind of issue is ${issue}?`({ bug: null, other: null });
const kind = await ask.choice`What kind of issue is ${issue}?`(["bug", "other"]);
const label = await ask.switch`What kind of issue is ${issue}?`(["bug", "other"]);
const severity = await ask.score`How severe is ${issue}?`(["Cosmetic", "Blocks production"]);
const { chance } = await ask.chance`Does ${issue} need immediate attention?`();
const security = await ask.if`Does ${issue} describe a security vulnerability?`;
Expand Down Expand Up @@ -237,7 +250,8 @@ For standalone requests, pass client options after the criteria. `ask.chance` ac
```ts
const options = { model: "jev-latest" };

await ask.choice`What kind of issue is ${issue}?`({ bug: null, other: null }, options);
await ask.choice`What kind of issue is ${issue}?`(["bug", "other"], options);
await ask.switch`What kind of issue is ${issue}?`(["bug", "other"], options);
await ask.score`How severe is ${issue}?`(["Cosmetic", "Blocks production"], options);
await ask.chance`Does ${issue} need immediate attention?`(undefined, options);
```
Expand All @@ -249,7 +263,7 @@ const strictIf = ask.if({ threshold: 0.8 });
const security = await strictIf`Does ${issue} describe a security vulnerability?`;
```

Inside `ask`, set client options on `ask` itself; tags use the batch's settings. A threshold set with `ask.if({ threshold })` still applies to that question. `ask.if` is also exported as `askIf`.
Inside `ask`, set client options on `ask` itself; tags use the batch's settings. A threshold set with `ask.if({ threshold })` still applies to that question. `ask.if` and `ask.switch` are also exported as `askIf` and `askSwitch`.

Tags are promise-like: awaiting them, passing them to `Promise.all`, or returning them from an async function sends a request. Awaiting the same tag again sends another request; it does not reuse an earlier answer.

Expand All @@ -258,10 +272,11 @@ Tags are promise-like: awaiting them, passing them to `Promise.all`, or returnin
When you build questions without template strings, use:

- `ask.choice(instructions, criteria, options?)`
- `ask.switch(instructions, criteria, options?)`
- `ask.score(instructions, levels, options?)`
- `ask.chance(instructions, criteria?, options?)`

These return the same awaitable questions as the tags. Instructions and criteria descriptions accept text, JSON objects or arrays, or `null`:
These return the same awaitable questions as the tags. Instructions and criteria descriptions accept text, JSON objects or arrays, or `null`; choice and switch criteria may also be an array of labels:

```ts
const { kind } = await ask(issue, {
Expand Down Expand Up @@ -291,7 +306,7 @@ const { urgent } = await ask(issue, {
console.log(urgent.chance); // 0–1
```

Optional `instructions` accept text, a JSON object or array, or `null`. `criteria` follows the corresponding helper's shape and is required for choices and scores. Answers have the same shape as with the helpers, including `{ type: "chance", chance }` for `"noul"` questions.
Optional `instructions` accept text, a JSON object or array, or `null`. `criteria` follows the corresponding helper's shape and is required for choices and scores. Answers have the same shape as with the helpers, including `{ type: "chance", chance }` for `"noul"` questions. The tag shapes work too: `{ type: "if", instructions, threshold }` resolves to a boolean and `{ type: "switch", instructions, criteria }` to the selected label.

## Vercel AI Gateway

Expand Down
4 changes: 3 additions & 1 deletion skills/advocaat/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,12 +121,14 @@ Choose by what the answer means, then read the relevant primitive page:
| One of a defined set | ``ask.choice`…`({ a: "…", b: null })`` ([Choice](https://docs.typesafe.ai/primitives/choice.md)) | `{ choice, confidence, probabilities }` | Picks one option (2–255); `probabilities` compares competing options |
| Whether a condition holds | `"Is it …?"`, ``ask.chance`…`({ true: "…", false: "…" })`` ([Noul](https://docs.typesafe.ai/primitives/noul.md)) | `{ chance }` | Probability of yes; no separate confidence; use one per label when several may apply |
| A yes/no you act on directly | `` ask.if`…` `` | `boolean` | `chance` above `threshold` (default `0.5`); pass `ask.if({ threshold })` to bind another |
| A label you branch on directly | ``ask.switch`…`(["a", "b"])`` | `"a" \| "b"` | Same options as `ask.choice`; resolves to the selected label alone, without `confidence` or `probabilities` |
| Degree along a described dimension | ``ask.score`…`(["low …", "mid …", "high …"])`` ([Score](https://docs.typesafe.ai/primitives/score.md)) | `{ score, ratio, confidence, legend, probabilities }` | Probability-weighted position on 2–10 ordered levels; `ratio` scales it to 0–1; use comparable per-item Scores for graded ranking |

Every tag also takes plain arguments when the question is built elsewhere, for
example `ask.choice(instructions, criteria)`, and `ask` accepts plain question
objects (`type: "noul" | "choice" | "score"`). Instructions and criteria values
can be strings, JSON objects, or arrays.
can be strings, JSON objects, or arrays. Choice and switch criteria can also be
an array of labels when no descriptions are needed.

Give each question enough relevant **state** to answer: source text, identities,
relationships, policies, and current facts. Pass a named JSON object as the
Expand Down
90 changes: 72 additions & 18 deletions src/ask.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import {
} from "./api.ts";

/** A bare string is a yes/no question. */
export type AskQuestion = string | Question | IfQuestion;
export type AskQuestion = string | Question | IfQuestion | SwitchQuestion;

export type AskQuestions = { [name: string]: AskQuestion };

Expand All @@ -32,20 +32,24 @@ export interface ChanceAnswer {

export type Answer<Q extends AskQuestion> = Q extends IfQuestion
? boolean
: Q extends string | NoulQuestion
? ChanceAnswer
: Q extends ScoreQuestion<infer S>
? ScoreAnswer<S> & { /** Score scaled to 0–1. */ readonly ratio: number }
: Q extends ChoiceQuestion<infer C>
? ChoiceAnswer<C>
: never;
: Q extends SwitchQuestion<infer C>
? keyof C & string
: Q extends string | NoulQuestion
? ChanceAnswer
: Q extends ScoreQuestion<infer S>
? ScoreAnswer<S> & { /** Score scaled to 0–1. */ readonly ratio: number }
: Q extends ChoiceQuestion<infer C>
? ChoiceAnswer<C>
: never;

export type Answers<Q extends AskQuestions> = { readonly [K in keyof Q]: Answer<Q[K]> };

export type AskOptions = TypeSafeOptions & RequestOptions;

type Tagged = Question | IfQuestion | SwitchQuestion;

/** A question from a tag: a key in `ask`, or awaited on its own to send it with its interpolated state. */
export type Askable<Q extends Question | IfQuestion> = Q & PromiseLike<Answer<Q>>;
export type Askable<Q extends Tagged> = Q & PromiseLike<Answer<Q>>;

/** A yes/no question from `ask.if` that resolves to a boolean. */
export interface IfQuestion {
Expand All @@ -54,6 +58,24 @@ export interface IfQuestion {
readonly threshold: number;
}

/** A choice from `ask.switch` that resolves to the selected label. */
export interface SwitchQuestion<T extends ChoiceCriteria = ChoiceCriteria> {
readonly type: "switch";
readonly instructions: Entry;
readonly criteria: T;
}

/** Two or more option labels without descriptions. */
export type ChoiceLabels = readonly [string, string, ...string[]];

/** Labels as criteria with `null` descriptions; an object stays as it is. */
export type Named<T> = T extends ChoiceLabels ? { [K in T[number]]: null } : T;

const named = <T extends ChoiceCriteria | ChoiceLabels>(criteria: T) =>
(Array.isArray(criteria)
? Object.fromEntries(criteria.map((label) => [label, null]))
: criteria) as Named<T>;

// Interpolated objects of tagged questions, kept off the wire until sent.
const states = new WeakMap<object, Parsed>();

Expand All @@ -72,7 +94,12 @@ export async function ask<const Q extends AskQuestions>(
}
const own = states.get(q);
if (own !== undefined) tagged.push([name, own]);
wire[name] = q.type === "if" ? noul(q.instructions) : q;
wire[name] =
q.type === "if"
? noul(q.instructions)
: q.type === "switch"
? choiceQuestion(q.instructions, q.criteria)
: q;
}
// Interpolated objects of all tags go once each into `input`, and their slots become its paths.
// A text or array state goes first among them.
Expand Down Expand Up @@ -103,7 +130,9 @@ export async function ask<const Q extends AskQuestions>(
: { type: "chance", chance: a.noul }
: a.type === "score"
? { ...a, ratio: a.score / ((wire[name] as ScoreQuestion).criteria.length - 1) }
: a;
: typeof q === "object" && q.type === "switch"
? a.choice
: a;
}
return out as Answers<Q>;
}
Expand Down Expand Up @@ -142,7 +171,7 @@ function parse(strings: Strings, values: unknown[]): Parsed {
}

// Adds a hidden `then` that sends the question alone under `input`, so the object stays a plain question.
function askable<Q extends Question | IfQuestion>(q: Q, parsed?: Parsed, options?: AskOptions) {
function askable<Q extends Tagged>(q: Q, parsed?: Parsed, options?: AskOptions) {
if (parsed?.parts.length) states.set(q, parsed);
const then: PromiseLike<Answer<Q>>["then"] = (ok, fail) =>
ask("", { input: q }, options)
Expand All @@ -154,7 +183,7 @@ function askable<Q extends Question | IfQuestion>(q: Q, parsed?: Parsed, options

// Each tag works both as ask.choice`...`(criteria, options?) and ask.choice(instructions, criteria, options?),
// where plain-call instructions may be a JSON object or array (see .agents/typesafe.md).
function tag<C, Q extends Question>(build: (instructions: Entry, criteria: C) => Q) {
function tag<C, Q extends Tagged>(build: (instructions: Entry, criteria: C) => Q) {
return (first: Entry | Strings, ...rest: unknown[]) => {
if (!isTag(first)) return askable(build(first, rest[0] as C), undefined, rest[1] as AskOptions);
const parsed = parse(first, rest);
Expand All @@ -163,19 +192,21 @@ function tag<C, Q extends Question>(build: (instructions: Entry, criteria: C) =>
};
}

export const choice = tag(choiceQuestion) as {
<const T extends ChoiceCriteria>(
export const choice = tag((instructions: Entry, criteria: ChoiceCriteria | ChoiceLabels) =>
choiceQuestion(instructions, named(criteria)),
) as {
<const T extends ChoiceCriteria | ChoiceLabels>(
instructions: Entry,
criteria: T,
options?: AskOptions,
): Askable<ChoiceQuestion<T>>;
): Askable<ChoiceQuestion<Named<T>>>;
(
strings: Strings,
...values: unknown[]
): <const T extends ChoiceCriteria>(
): <const T extends ChoiceCriteria | ChoiceLabels>(
criteria: T,
options?: AskOptions,
) => Askable<ChoiceQuestion<T>>;
) => Askable<ChoiceQuestion<Named<T>>>;
};

export const score = tag(scoreQuestion) as {
Expand All @@ -193,6 +224,28 @@ export const score = tag(scoreQuestion) as {
) => Askable<ScoreQuestion<T>>;
};

/** Like `ask.choice`, but resolves to the selected label alone. */
export const askSwitch = tag(
(instructions: Entry, criteria: ChoiceCriteria | ChoiceLabels): SwitchQuestion => ({
type: "switch",
instructions,
criteria: named(criteria),
}),
) as {
<const T extends ChoiceCriteria | ChoiceLabels>(
instructions: Entry,
criteria: T,
options?: AskOptions,
): Askable<SwitchQuestion<Named<T>>>;
(
strings: Strings,
...values: unknown[]
): <const T extends ChoiceCriteria | ChoiceLabels>(
criteria: T,
options?: AskOptions,
) => Askable<SwitchQuestion<Named<T>>>;
};

export const chance = tag(noul) as {
(
instructions: Entry,
Expand Down Expand Up @@ -236,3 +289,4 @@ ask.choice = choice;
ask.score = score;
ask.chance = chance;
ask.if = askIf;
ask.switch = askSwitch;
5 changes: 4 additions & 1 deletion src/index.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
export { ask, askIf, chance, choice, score } from "./ask.ts";
export { ask, askIf, askSwitch, chance, choice, score } from "./ask.ts";
export type {
Answer,
Answers,
Expand All @@ -7,8 +7,11 @@ export type {
AskQuestion,
AskQuestions,
ChanceAnswer,
ChoiceLabels,
Askable,
IfQuestion,
Named,
SwitchQuestion,
} from "./ask.ts";
export { APIError, typesafe } from "./api.ts";
export type * from "./api.ts";
Loading
Loading