feat(runtime): add Grok subscriptions and preserve transcript timestamps

.pylon/features.yaml

@@ -496,3 +496,32 @@ decisions:
     upstream_support: Isolate normal Agent credentials, team selection and endpoints from CLI profile changes; import only validated production CLI credentials during explicit login.
     revisit_when:
       - Prime changes credential storage, endpoint overrides or explicit CLI import semantics.
+
+  grok-subscription-authentication:
+    area: provider-authentication
+    state: candidate
+    owner: shared
+    decision: hybridize
+    pylon_refs:
+      - https://github.com/pylon-code/prime-agent/issues/68
+    upstream_refs:
+      - https://github.com/PrimeIntellect-ai/prime-agent/pull/2252
+    fork_change: grok-subscription-with-owned-provider-routing
+    upstream_support: Adopt device-code authentication and auth-specific Responses routing while retaining Pylon scoped provider hooks, recovery cursors and the three-tab configuration menu.
+    revisit_when:
+      - The complete upstream integration includes equivalent subscription routing and preserves Pylon ownership contracts.
+
+  custom-message-persistence-identity:
+    area: runtime-reliability
+    state: candidate
+    owner: pylon-fork
+    decision: retain
+    pylon_refs:
+      - https://github.com/pylon-code/prime-agent/issues/76
+      - https://github.com/pylon-code/pylon/issues/552
+    upstream_refs:
+      - https://github.com/PrimeIntellect-ai/prime-agent/tree/f5859162c
+    fork_change: preserve-custom-message-timestamps
+    upstream_support: Current upstream allocates a second timestamp when persisting custom messages, so reloading or compacting changes live transcript identities. Preserve the original message timestamp at every AgentSession persistence boundary.
+    revisit_when:
+      - Upstream preserves exact custom-message identities across live emission, persistence and reconstruction.

.pylon/upstream-review.md

@@ -364,3 +364,17 @@ Follow-up: Task10 builds/packs the exact merged tree into a private prefix and r
 - Normal authentication, team selection and endpoints are Agent-owned. Explicit login may import a CLI key/team snapshot only with production endpoint checks and successful scope validation. Agent login/logout/team selection never writes CLI configuration. Injected custom auth storage/agent directories do not opt into ambient CLI import.
 - This changes intentional credential precedence rather than daemon protocol, capability or session ownership. Users who relied only on CLI credentials must explicitly log in once; documentation covers that migration, endpoint overrides, team billing and trace-specific credential precedence. Pylon's server has no dependency on the removed CLI config helpers; native auth/model discovery continues through the installed SDK/daemon boundary.
 - Validation: 249 focused auth storage, login, scope validation, SDK services, trace, model registry and session runtime tests pass using isolated files and mocked services. `npm run check` passes Biome, TypeScript, installer and browser-bundle checks. Preserve the existing MCP login coverage while leaving unrelated xAI subscription assertions to #68. Installed-runtime verification remains Pylon #557.
+
+## 2026-09-14 — Grok subscription authentication
+
+- Tracking #68. Hybridize upstream #2252 (`ca67580b524800a971f92420d4c665bd83df1dfa`), reviewed against current upstream `f5859162c`. Retain scoped provider hooks, owned-session recovery and Pylon's three-tab configuration menu. The full integration checkpoint remains `1eee2938b4eeb7a4d72e17035adda669a89b63de`.
+- Device-code OAuth and subscription refresh select xAI Responses request models; API keys retain their original models. Authentication changes refresh active, maintenance and daemon model views. Failed subscription refresh does not silently fall back to an API key. Shared response streaming handles interleaved tool arguments and encrypted reasoning while preserving Codex behavior.
+- No daemon commands, events, response shapes, schema revision or negotiated capabilities change. Existing model/catalog refresh commands carry the updated values. No credentials or installed runtime are modified by this adoption.
+- Validation: 580 focused tests pass across AI streaming/OAuth and coding-agent authentication, model selection, maintenance and daemon/in-process connections; four optional AI tests are skipped. `npm run check` passes formatting, types, installer and browser smoke. Live paid-provider access is not part of these faux-provider tests. Required hosted checks and final review accompany the PR.
+
+## 2026-09-14 — preserve custom message timestamps
+
+- Tracking #76 and Pylon #552. **Retain** a native persistence correction after a real isolated faux-provider compaction exposed different timestamps for the same live and persisted harness digest. Current upstream `f5859162c` still mints a new timestamp in `appendCustomMessageEntry`; its timestamp-related history has no equivalent correction. The complete integration checkpoint remains `1eee2938b4eeb7a4d72e17035adda669a89b63de`.
+- Add an optional original timestamp to both custom append methods and pass it from every AgentSession custom-message write: delivered/queued messages, direct messages, session commands, Python state, harness digests, refinement and compaction outcomes. Calls that originate durable markers without a live message retain their existing current-time default. Invalid or lossy timestamps fail before publication.
+- No daemon command, event, response shape, protocol/schema revision, artifact recipe or capability changes. Historical files are not rewritten. An older custom record whose live and persisted timestamps already disagree remains unproved; Pylon must not relax history matching to accept it. New managed bytes are required for the correction.
+- Validation: 165 focused tests pass across the timestamp regression, session persistence/rollback, compaction and input queue. The regression deliberately advances the clock at the persistence boundary and verifies the exact live, pre-compaction tree, current context and reopened session messages. `npm run check` and required hosted checks accompany the PR.

packages/ai/.changes/eng-6059-xai-subscription.md

@@ -0,0 +1 @@
+- Added xAI subscription device-code authentication and Responses support for all bundled Grok tool models while preserving API-key access.

packages/ai/README.md

@@ -76,7 +76,7 @@ Unified LLM API with automatic model discovery, provider configuration, token an
 - **Cerebras**
 - **Cloudflare AI Gateway**
 - **Cloudflare Workers AI**
-- **xAI**
+- **xAI** (API key or Grok/X subscription)
 - **OpenRouter**
 - **Vercel AI Gateway**
 - **MiniMax**
@@ -1126,14 +1126,23 @@ const key = getEnvApiKey('openai');  // checks OPENAI_API_KEY
 
 ## OAuth Providers
 
-Several providers require OAuth authentication instead of static API keys:
+OAuth authentication is available for these providers and subscriptions:
 
 - **Anthropic** (Claude Pro/Max subscription)
 - **OpenAI Codex** (ChatGPT Plus/Pro subscription, access to GPT-5.x Codex models)
 - **GitHub Copilot** (Copilot subscription)
+- **xAI** (Grok/X subscription, device-code login)
 
 For paid Cloud Code Assist subscriptions, set `GOOGLE_CLOUD_PROJECT` or `GOOGLE_CLOUD_PROJECT_ID` to your project ID.
 
+### xAI subscription
+
+In Prime Agent, run `/login` and select the **xAI subscription** entry. Open the displayed HTTPS URL and enter the device code. `XAI_API_KEY` and API-key login remain supported. Stored subscription credentials take priority over `XAI_API_KEY`; an explicit `--api-key` override uses API-key routing.
+
+All bundled xAI tool-capable language models can use subscription authentication through the Responses API. Each model keeps its own reasoning and input capabilities; only verified reasoning-effort controls are sent. Account eligibility, available models, and usage limits are controlled by xAI; signing in does not guarantee access or unlimited usage.
+
+SDK callers can use `loginXai(callbacks)` and `getOAuthApiKey("xai", credentials)` from `prime-agent-ai/oauth`. Persist refreshed credentials securely. When the effective credential is an xAI subscription, pass `getXaiSubscriptionModel(getModel("xai", "grok-4.5"))` to `stream` or `complete` with the resolved access token. The helper accepts configured xAI model descriptors and returns `undefined` for other providers. Do not apply this projection to API keys: the generated API-key models remain unchanged. Subscription requests use `https://api.x.ai/v1`.
+
 ### Vertex AI
 
 Vertex AI models support either a Google Cloud API key or Application Default Credentials (ADC):

packages/ai/src/providers/openai-responses-shared.ts

@@ -164,6 +164,7 @@ export function convertResponsesMessages<TApi extends Api>(
 				if (block.type === "thinking") {
 					if (block.thinkingSignature) {
 						const reasoningItem = JSON.parse(block.thinkingSignature) as ResponseReasoningItem;
+						if (model.provider === "xai") delete reasoningItem.status;
 						output.push(reasoningItem);
 					}
 				} else if (block.type === "text") {
@@ -274,13 +275,29 @@ export async function processResponsesStream<TApi extends Api>(
 ): Promise<void> {
 	let currentItem: ResponseReasoningItem | ResponseOutputMessage | ResponseFunctionToolCall | null = null;
 	let currentBlock: ThinkingContent | TextContent | (ToolCall & { partialJson: string }) | null = null;
-	const blocks = output.content;
-	const blockIndex = () => blocks.length - 1;
+	let currentContentIndex = -1;
+	let sawTerminalResponse = false;
+	const slots = new Map<
+		number,
+		{
+			item: ResponseReasoningItem | ResponseOutputMessage | ResponseFunctionToolCall;
+			block: ThinkingContent | TextContent | (ToolCall & { partialJson: string });
+			contentIndex: number;
+		}
+	>();
+	const blockIndex = () => currentContentIndex;
 
 	for await (const event of openaiStream) {
+		if ("output_index" in event) {
+			const slot = slots.get(event.output_index);
+			currentItem = slot?.item ?? null;
+			currentBlock = slot?.block ?? null;
+			currentContentIndex = slot?.contentIndex ?? output.content.length;
+		}
 		if (event.type === "response.created") {
 			output.responseId = event.response.id;
 		} else if (event.type === "response.output_item.added") {
+			currentContentIndex = output.content.length;
 			const item = event.item;
 			if (item.type === "reasoning") {
 				currentItem = item;
@@ -304,6 +321,13 @@ export async function processResponsesStream<TApi extends Api>(
 				output.content.push(currentBlock);
 				stream.push({ type: "toolcall_start", contentIndex: blockIndex(), partial: output });
 			}
+			if (currentItem && currentBlock) {
+				slots.set(event.output_index, {
+					item: currentItem,
+					block: currentBlock,
+					contentIndex: currentContentIndex,
+				});
+			}
 		} else if (event.type === "response.reasoning_summary_part.added") {
 			if (currentItem && currentItem.type === "reasoning") {
 				currentItem.summary = currentItem.summary || [];
@@ -420,6 +444,7 @@ export async function processResponsesStream<TApi extends Api>(
 				}
 			}
 		} else if (event.type === "response.output_item.done") {
+			slots.delete(event.output_index);
 			const item = event.item;
 
 			if (item.type === "reasoning" && currentBlock?.type === "thinking") {
@@ -445,10 +470,9 @@ export async function processResponsesStream<TApi extends Api>(
 				});
 				currentBlock = null;
 			} else if (item.type === "function_call") {
-				const args =
-					currentBlock?.type === "toolCall" && currentBlock.partialJson
-						? parseStreamingJson(currentBlock.partialJson)
-						: parseStreamingJson(item.arguments || "{}");
+				const args = parseStreamingJson(
+					item.arguments || (currentBlock?.type === "toolCall" ? currentBlock.partialJson : "") || "{}",
+				);
 
 				let toolCall: ToolCall;
 				if (currentBlock?.type === "toolCall") {
@@ -469,8 +493,21 @@ export async function processResponsesStream<TApi extends Api>(
 				currentBlock = null;
 				stream.push({ type: "toolcall_end", contentIndex: blockIndex(), toolCall, partial: output });
 			}
-		} else if (event.type === "response.completed") {
+		} else if (event.type === "response.completed" || event.type === "response.incomplete") {
+			sawTerminalResponse = true;
 			const response = event.response;
+			if (model.provider === "xai") {
+				for (const item of response.output ?? []) {
+					if (item.type !== "reasoning" || !item.encrypted_content) continue;
+					for (const block of output.content) {
+						if (block.type !== "thinking" || !block.thinkingSignature) continue;
+						const stored = JSON.parse(block.thinkingSignature) as ResponseReasoningItem;
+						if (stored.id === item.id && !stored.encrypted_content) {
+							block.thinkingSignature = JSON.stringify({ ...stored, encrypted_content: item.encrypted_content });
+						}
+					}
+				}
+			}
 			if (response?.id) {
 				output.responseId = response.id;
 			}
@@ -520,6 +557,9 @@ export async function processResponsesStream<TApi extends Api>(
 			});
 		}
 	}
+	if (model.provider === "xai" && !sawTerminalResponse) {
+		throw new StreamFailureError("xAI Responses stream ended before a terminal response event", { kind: "unknown" });
+	}
 }
 
 function mapStopReason(status: OpenAI.Responses.ResponseStatus | undefined): StopReason {

packages/ai/src/providers/openai-responses.ts

@@ -264,6 +264,7 @@ function buildParams(model: Model<"openai-responses">, context: Context, options
 				effort: (model.thinkingLevelMap?.off ?? "none") as NonNullable<typeof params.reasoning>["effort"],
 			};
 		}
+		if (model.provider === "xai") params.include = ["reasoning.encrypted_content"];
 	}
 
 	return params;

packages/ai/src/utils/oauth/index.ts

@@ -16,18 +16,20 @@ export {
 	refreshGitHubCopilotToken,
 } from "./github-copilot.js";
 export { loginOpenAICodex, openaiCodexOAuthProvider, refreshOpenAICodexToken } from "./openai-codex.js";
-
 export * from "./types.js";
+export { getXaiSubscriptionModel, loginXai, refreshXaiToken, xaiOAuthProvider } from "./xai.js";
 
 import { anthropicOAuthProvider } from "./anthropic.js";
 import { githubCopilotOAuthProvider } from "./github-copilot.js";
 import { openaiCodexOAuthProvider } from "./openai-codex.js";
 import type { OAuthCredentials, OAuthProviderId, OAuthProviderInfo, OAuthProviderInterface } from "./types.js";
+import { xaiOAuthProvider } from "./xai.js";
 
 const BUILT_IN_OAUTH_PROVIDERS: OAuthProviderInterface[] = [
 	anthropicOAuthProvider,
 	githubCopilotOAuthProvider,
 	openaiCodexOAuthProvider,
+	xaiOAuthProvider,
 ];
 
 const oauthProviderRegistry = new Map<string, OAuthProviderInterface>(