Ask the database for distinct edge connections instead of scanning edges

CONTEXT.md

@@ -62,6 +62,10 @@ A React-based web application that lets users visually explore graph databases w
 
 **Schema Sync**: The process that queries the database to discover vertex types, edge types, and their attributes. Required before a user can explore a new Connection. _Avoid_: Fetch, load
 
+**Edge Connection Discovery**: The process that finds which Edge Connections exist. Runs after Schema Sync completes and is a separate step: Schema Sync finds the Edge Types, Edge Connection Discovery finds how they link Vertex Types. Only the Schema View depends on it, so a failure degrades that view and leaves the rest of the app working. _Avoid_: Relationship Discovery (follows the Edge Connection entry), edge discovery
+
+**Complete** / **Sampled**: The two strategies Edge Connection Discovery can take. **Complete** reads every edge, so it finds every Edge Connection. **Sampled** caps the edges it reads per Edge Type, so it is bounded on a large graph but will miss an Edge Connection that occurs rarely. The strategy is chosen up front by comparing predicted cost, and a Complete scan is abandoned for Sampled if it proves too large for the database. Both strategies may split their work across several requests; neither split is a third strategy. _Avoid_: Full, exhaustive, partial, approximate, mode (for the pair)
+
 **Schema**: The discovered structure of a connected graph database — vertex types, edge types, their attributes, and how they connect. Populated by Schema Sync when a Connection is first used; not user-defined. _Avoid_: Model, structure
 
 **Exported Connection File**: The on-disk JSON format a user gets when they export a Connection (`saveConfigurationToFile`), and which import consumes. It bundles the connection config with a snapshot of the Schema (`lastUpdate` is an ISO string on disk). A single Zod schema in `parseConnectionFile.ts` is the source of truth: both the writer and the importer target the same inferred type (`ExportedConnectionFile`). The writer assigns a `Date` for `lastUpdate` and `JSON.stringify` serializes it to the ISO string; the parser coerces it back via `z.coerce.date()`. The schema is lenient (every level is a `looseObject`), so unknown and legacy fields — styling, `__inferred`/`__matches` on prefixes, attribute `dataType` — pass through untouched. It is intentionally decoupled from the in-memory configuration and from the IndexedDB storage shape, so the wire format can evolve independently. On import it is split — the connection lands in `configurationAtom`, the schema in `schemaAtom`. _Avoid_: Configuration file (the wire format is not the in-memory or persisted shape)
@@ -85,7 +89,7 @@ A React-based web application that lets users visually explore graph databases w
 - A **Schema** contains **Vertex Types**, **Edge Types**, and **Edge Connections**
 - A **Vertex** has one or more **Vertex Types** and zero or more **Properties**
 - An **Edge** connects exactly two **Vertices** (source → target), has one **Edge Type**, and zero or more **Properties**
-- An **Edge Connection** links a source **Vertex Type** to a target **Vertex Type** via an **Edge Type**
+- An **Edge Connection** links a source **Vertex Type** to a target **Vertex Type** via an **Edge Type**, and is found by **Edge Connection Discovery**
 - A **Session** belongs to a **Connection** and contains **Vertices** and **Edges**
 - **Neighbors** are **Vertices** one hop away from a given **Vertex**
 - **Styles** are scoped per **Vertex Type** (**Vertex Styles**) and **Edge Type** (**Edge Styles**)

docs/agents/schema.md

@@ -27,6 +27,8 @@ If the edge connection query fails, the schema records only that it failed, via
 
 `edgeConnectionNotice(schema, error)` (in `src/hooks/edgeConnectionNotice.ts`, next to `useSchemaSync`) checks failure first (the live query error or `lastEdgeConnectionSyncFail`) and only then `edgeConnections == null`, because partial connections added by exploration after a failure must still report the failure. `useEdgeConnectionNotice()` wraps it with `useMaybeActiveSchema()` and `useSchemaSync().edgeDiscoveryQuery` so the toolbar button, the sidebar details, and the connection detail panel share one resolution of the notice.
 
+The edge query's key includes the sorted edge types and the schema's `totalEdges`, so a schema refresh usually moves it to a new key. `refreshSchema` therefore fetches `edgeConnectionsQuery(refreshedSchema)` through the query client rather than calling the observer's `refetch()`, which would still target the pre-refresh key and cost a second request once the observer switched.
+
 `useCancelSchemaSync` cancels the edge query with `revert: false`. Reverting would restore its never-fetched state, and every observer that remounts under `SchemaDiscoveryBoundary` loads a query without data on mount (TanStack ignores `refetchOnMount` then), so the fetch would restart. Settled as a `CancelledError`, `retryOnMount: false` holds it, and `edgeConnectionNotice` reads the cancellation as not discovered rather than failed.
 
 ## Incremental Schema Growth

docs/guides/troubleshooting.md

@@ -125,7 +125,11 @@ If a request is cancelled instead, Graph Explorer shows a plain "Request cancell
 
 ### Out of Memory
 
-This can happen when your database is very large. Graph Explorer does its best to support larger databases and is always improving. Please [file an issue](https://github.com/aws/graph-explorer/issues/new/choose) if you encounter this situation.
+This can happen when your database is very large. Graph Explorer does its best to support larger databases and is always improving.
+
+For a Gremlin connection, this is often the database running out of memory while discovering edge connections for the Schema view. Graph Explorer samples each edge type instead of scanning every edge when a graph is too large, and if that still fails you can raise the query timeout in the database configuration, such as the DB cluster parameter group for Neptune, or use an instance with more memory.
+
+Otherwise, please [file an issue](https://github.com/aws/graph-explorer/issues/new/choose) if you encounter this situation.
 
 ### Proxy Server Cannot Be Reached
 

packages/graph-explorer/src/components/SchemaDiscoveryBoundary.integration.test.tsx

@@ -106,6 +106,9 @@ describe("SchemaDiscoveryBoundary against the real store", () => {
     state.activeSchema.edges = [];
     state.addTestableEdgeToGraph(edge);
     state.activeSchema.edgeConnections = [];
+    // Match the total the refresh reports, so the edge connection query keeps
+    // its key and only a loop could fetch it more than once.
+    state.activeSchema.totalEdges = 1;
 
     const store = getAppStore();
     state.applyTo(store);

packages/graph-explorer/src/connector/gremlin/fetchEdgeConnections/discoveryError.test.ts

@@ -0,0 +1,113 @@
+import { DatabaseTimeoutError, FetchTimeoutError, NetworkError } from "@/utils";
+
+import {
+  EdgeConnectionDiscoveryError,
+  type FailedDiscovery,
+  isTooBig,
+} from "./discoveryError";
+
+function attempt(overrides: Partial<FailedDiscovery> = {}): FailedDiscovery {
+  return {
+    strategy: "sampled",
+    requests: 1,
+    totalEdges: 10,
+    degraded: false,
+    cause: "database-limit",
+    ...overrides,
+  };
+}
+
+describe("isTooBig", () => {
+  it("is true for a FetchTimeoutError", () => {
+    expect(isTooBig(new FetchTimeoutError(1000, new Error("aborted")))).toBe(
+      true,
+    );
+  });
+
+  it("is true for a DatabaseTimeoutError", () => {
+    expect(
+      isTooBig(
+        new DatabaseTimeoutError(
+          "Query cannot be completed",
+          500,
+          {},
+          "TimeLimitExceededException",
+        ),
+      ),
+    ).toBe(true);
+  });
+
+  it("is true for a NetworkError carrying the memory limit code", () => {
+    expect(
+      isTooBig(
+        new NetworkError("Query cannot be completed", 500, {
+          code: "MemoryLimitExceededException",
+        }),
+      ),
+    ).toBe(true);
+  });
+
+  it("is true when the memory limit code is nested in a cause", () => {
+    expect(
+      isTooBig(
+        new NetworkError("Query cannot be completed", 500, {
+          cause: { code: "MemoryLimitExceededException" },
+        }),
+      ),
+    ).toBe(true);
+  });
+
+  it("is false for an unrelated NetworkError", () => {
+    expect(
+      isTooBig(
+        new NetworkError("Query cannot be completed", 500, {
+          code: "MalformedQueryException",
+        }),
+      ),
+    ).toBe(false);
+  });
+
+  it("is false for a user cancellation, so it never looks like a size problem", () => {
+    expect(isTooBig(new DOMException("Aborted", "AbortError"))).toBe(false);
+  });
+
+  it("is false for a plain error", () => {
+    expect(isTooBig(new Error("Network error"))).toBe(false);
+  });
+});
+
+describe("EdgeConnectionDiscoveryError recovery text", () => {
+  it("points only at the Fetch Timeout, not the parameter group, when a cheaper pass exhausted our own fetch timeout", () => {
+    const error = new EdgeConnectionDiscoveryError(
+      attempt({ cause: "fetch-timeout" }),
+      new Error("cause"),
+    );
+    expect(error.recovery).toContain("Fetch Timeout");
+    expect(error.recovery).toContain("advanced options");
+    expect(error.recovery).not.toContain("parameter group");
+    expect(error.recovery).toContain(
+      "the Schema view shows node types without the edge connections between them",
+    );
+  });
+
+  it("points at the database's own query timeout and the parameter group when a cheaper pass exhausted it", () => {
+    const error = new EdgeConnectionDiscoveryError(
+      attempt({ cause: "database-limit" }),
+      new Error("cause"),
+    );
+    expect(error.recovery).toContain("DB cluster parameter group");
+    expect(error.recovery).toContain(
+      "the Schema view shows node types without the edge connections between them",
+    );
+  });
+
+  it("includes the cause in the structured details, keyed apart from the JS cause", () => {
+    const jsCause = new Error("Query cannot be completed");
+    const error = new EdgeConnectionDiscoveryError(
+      attempt({ cause: "fetch-timeout" }),
+      jsCause,
+    );
+    expect(error.details).toMatchObject({ failureCause: "fetch-timeout" });
+    expect(error.cause).toBe(jsCause);
+  });
+});

packages/graph-explorer/src/connector/gremlin/fetchEdgeConnections/discoveryError.ts

@@ -0,0 +1,105 @@
+import { DatabaseTimeoutError, FetchTimeoutError, NetworkError } from "@/utils";
+
+import type { DiscoveryStrategy } from "./discoveryPlan";
+
+/** Neptune's code for a query that asked for more memory than the instance had. */
+const MEMORY_LIMIT_ERROR_CODE = "MemoryLimitExceededException";
+
+/** Whether the database gave up because one request asked for too much at once. */
+export function isTooBig(error: unknown): boolean {
+  return (
+    error instanceof FetchTimeoutError ||
+    error instanceof DatabaseTimeoutError ||
+    memoryLimitCode(error) !== undefined
+  );
+}
+
+/**
+ * The database's own memory-limit code, from either shape the body arrives in.
+ * Reading only the top level would miss a nested code and cost the degrade path
+ * its fast exit, leaving the user to wait out the request bound instead.
+ */
+function memoryLimitCode(error: unknown): string | undefined {
+  const data = error instanceof NetworkError ? error.data : undefined;
+  const code = data?.code ?? data?.cause?.code;
+  return code === MEMORY_LIMIT_ERROR_CODE ? code : undefined;
+}
+
+/**
+ * Which side gave up. A fetch timeout is the connection's own bound running out,
+ * fixed in the connection's settings; a database limit is the database itself
+ * refusing the request, fixed in the database's configuration.
+ */
+export type FailureCause = "fetch-timeout" | "database-limit";
+
+/** Classifies a size failure that `isTooBig` already confirmed. */
+export function causeOf(error: unknown): FailureCause {
+  return error instanceof FetchTimeoutError
+    ? "fetch-timeout"
+    : "database-limit";
+}
+
+/** What edge connection discovery had already tried when it gave up. */
+export type FailedDiscovery = {
+  strategy: DiscoveryStrategy;
+  requests: number;
+  totalEdges: number | undefined;
+  /** A complete scan was already abandoned as too large before this attempt. */
+  degraded: boolean;
+  /** Which side gave up: the connection's fetch timeout, or the database itself. */
+  cause: FailureCause;
+};
+
+/**
+ * Edge connection discovery has nothing cheaper left to try.
+ *
+ * Thrown only for a size failure, so the error reaches the user carrying the one
+ * thing the generic wording cannot give them: which recovery path is open. Other
+ * failures propagate untouched, because the existing display branches already
+ * read a refused connection or a bad URL correctly.
+ */
+export class EdgeConnectionDiscoveryError extends Error {
+  readonly attempt: FailedDiscovery;
+  /** What the user can do about it, in the order worth trying. */
+  readonly recovery: string;
+
+  constructor(attempt: FailedDiscovery, cause: unknown) {
+    super(describeFailure(attempt), { cause });
+    // A literal rather than the class name, because the production build
+    // minifies class names and the error details dialog shows this.
+    this.name = "EdgeConnectionDiscoveryError";
+    this.attempt = attempt;
+    this.recovery = describeRecovery(attempt);
+  }
+
+  /** Structured context for the error details dialog. */
+  get details() {
+    return {
+      strategy: this.attempt.strategy,
+      requests: this.attempt.requests,
+      totalEdges: this.attempt.totalEdges,
+      completeScanAbandoned: this.attempt.degraded,
+      // Named apart from `cause`, which `createErrorDetails` reserves for the
+      // serialized JS `Error.cause` and would otherwise overwrite this.
+      failureCause: this.attempt.cause,
+    };
+  }
+}
+
+/**
+ * Keyed on whether a complete scan was already abandoned, never on the
+ * strategy: either way discovery has run out of cheaper options.
+ */
+function describeFailure({ degraded }: FailedDiscovery): string {
+  if (degraded) {
+    return "The database could not discover edge connections either way. Scanning every edge was too large, and sampling each edge type failed as well.";
+  }
+  return "The database could not sample the edges of each edge type to discover edge connections.";
+}
+
+function describeRecovery({ cause }: FailedDiscovery): string {
+  if (cause === "fetch-timeout") {
+    return "Raise the Fetch Timeout in this connection's advanced options, or clear it, since this request may simply need longer than that allows. Until then, the Schema view shows node types without the edge connections between them.";
+  }
+  return "Raise the query timeout in the database configuration, such as the DB cluster parameter group for Neptune, or use an instance with more memory. Until then, the Schema view shows node types without the edge connections between them.";
+}