feat(speculation): add path scorer extension

Add the scorer seam from the speculation RFC, as a vendor-agnostic extension interface under submitqueue/extension/speculation/scorer/.

The scorer computes each speculation path's predicted-success score from the current state: the per-batch scores of the path's base batches (entity.Batch.Score) and which of those dependencies have resolved (landed or build-passed), plus optionally other signals. It is a prediction over live state, so the controller re-runs it on every respeculate right after reconciling status, and persists the result; the scorer owns only the formula.

This is the per-path scorer, distinct from the existing per-batch score stage (extension/scorer) that sets entity.Batch.Score — the path scorer consumes those to score whole paths. It is pure: it reads the tree and dependency batches, returns the tree with Score recomputed, and never reads storage/builds or writes status.

Follows the repo extension contract (conflict.Analyzer reference shape): Factory.For(Config) (Scorer, error) with Config carrying only QueueName. Includes README, gomock package, and Makefile mock-gen wiring. Interface only; concrete impls and controller wiring are deferred.

Author: behinddwalls

Makefile

@@ -364,7 +364,7 @@ local-stovepipe-stop: ## Stop the Stovepipe service
 
 mocks: ## Generate mock files using mockgen
 	@echo "Generating mocks..."
-	@$(BAZEL) run @rules_go//go -- generate ./submitqueue/extension/storage/... ./submitqueue/extension/buildrunner/... ./submitqueue/extension/changeprovider/... ./platform/extension/counter/... ./platform/extension/messagequeue/... ./submitqueue/extension/queueconfig/... ./submitqueue/extension/mergechecker/... ./submitqueue/extension/pusher/... ./submitqueue/extension/scorer/... ./submitqueue/extension/conflict/... ./submitqueue/extension/speculation/enumerator/... ./submitqueue/extension/speculation/dependencylimit/... ./platform/consumer/... ./submitqueue/core/changeset/... ./stovepipe/extension/storage/... ./stovepipe/extension/sourcecontrol/...
+	@$(BAZEL) run @rules_go//go -- generate ./submitqueue/extension/storage/... ./submitqueue/extension/buildrunner/... ./submitqueue/extension/changeprovider/... ./platform/extension/counter/... ./platform/extension/messagequeue/... ./submitqueue/extension/queueconfig/... ./submitqueue/extension/mergechecker/... ./submitqueue/extension/pusher/... ./submitqueue/extension/scorer/... ./submitqueue/extension/conflict/... ./submitqueue/extension/speculation/enumerator/... ./submitqueue/extension/speculation/dependencylimit/... ./submitqueue/extension/speculation/scorer/... ./platform/consumer/... ./submitqueue/core/changeset/... ./stovepipe/extension/storage/... ./stovepipe/extension/sourcecontrol/...
 	@echo "Mocks generated successfully!"
 
 proto: ## Generate protobuf files from .proto definitions

submitqueue/extension/speculation/scorer/BUILD.bazel

@@ -0,0 +1,9 @@
+load("@rules_go//go:def.bzl", "go_library")
+
+go_library(
+    name = "go_default_library",
+    srcs = ["scorer.go"],
+    importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/scorer",
+    visibility = ["//visibility:public"],
+    deps = ["//submitqueue/entity:go_default_library"],
+)

submitqueue/extension/speculation/scorer/README.md

@@ -0,0 +1,17 @@
+# Speculation Path Scorer
+
+Vendor-agnostic interface for scoring the paths in a batch's **speculation tree** — the predicted-success probability of each candidate bet, recomputed as the batch's world changes.
+
+See the [Speculation RFC](../../../../doc/rfc/submitqueue/speculation.md) for the end-to-end design and how scoring fits into the orchestrator pipeline.
+
+## Scorer
+
+A path's score is a **prediction**: *how likely is this bet to pay off, right now?* The scorer answers it from the current state — the per-batch success probabilities of a path's base batches (`entity.Batch.Score`, set by the score stage), which of those dependencies have already landed or had their build pass (resolved assumptions raise confidence), and optionally other signals such as how long the batch has waited or historical pass rates. The score is the common currency the [selector](../selector) and prioritizer both rank on, so keeping it current is what makes both act on the latest reality.
+
+Because it is a prediction over live state, the scorer is **re-run on every respeculate**, right after the controller reconciles path status — so when a dependency lands, its build passes, or a sibling path fails, the surviving paths' scores are recomputed before anything is selected or prioritized. The controller drives *when* to rescore (it is part of reconciliation) and persists the result; the scorer owns the *formula*.
+
+This is the per-**path** scorer, distinct from the per-**batch** [score stage](../../scorer), which sets `entity.Batch.Score`. The path scorer consumes those batch scores to score whole paths. It is a pure function of its inputs: it reads the tree and the dependency batches and returns scores only. It never reads storage or builds and never writes status — status is the controller's, fed back in on the tree; only `Score` is recomputed while path structure and status pass through unchanged.
+
+## Factory
+
+A per-queue factory returns the scorer for a queue, following the repo's extension contract. It is handed only the queue identity; scoring knobs and any extra signals are injected at construction by the integrator in the wiring layer, which resolves per-queue settings through `queueconfig`. Scoring itself stays config-free.

submitqueue/extension/speculation/scorer/fake/BUILD.bazel

@@ -0,0 +1,23 @@
+load("@rules_go//go:def.bzl", "go_library", "go_test")
+
+go_library(
+    name = "go_default_library",
+    srcs = ["fake.go"],
+    importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/scorer/fake",
+    visibility = ["//visibility:public"],
+    deps = [
+        "//submitqueue/entity:go_default_library",
+        "//submitqueue/extension/speculation/scorer:go_default_library",
+    ],
+)
+
+go_test(
+    name = "go_default_test",
+    srcs = ["fake_test.go"],
+    embed = [":go_default_library"],
+    deps = [
+        "//submitqueue/entity:go_default_library",
+        "@com_github_stretchr_testify//assert:go_default_library",
+        "@com_github_stretchr_testify//require:go_default_library",
+    ],
+)

submitqueue/extension/speculation/scorer/fake/fake.go

@@ -0,0 +1,70 @@
+// Copyright (c) 2025 Uber Technologies, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Package fake provides a programmable scorer.Scorer for tests and examples. It
+// returns the tree unchanged except that every path's Score is set to a single
+// configured value (0 unless SetScore is called), leaving path structure and
+// status intact. FailWith injects an error on every call. It is intended for
+// examples and tests only, never production.
+package fake
+
+import (
+	"context"
+
+	"github.com/uber/submitqueue/submitqueue/entity"
+	"github.com/uber/submitqueue/submitqueue/extension/speculation/scorer"
+)
+
+// Scorer is a programmable scorer.Scorer that stamps a constant score on every
+// path.
+type Scorer struct {
+	score float32
+	err   error
+}
+
+// New returns a fake Scorer that scores every path 0. Adjust with SetScore.
+func New() *Scorer {
+	return &Scorer{}
+}
+
+// SetScore sets the score stamped on every path by Score.
+func (s *Scorer) SetScore(score float32) *Scorer {
+	s.score = score
+	return s
+}
+
+// FailWith makes every Score call return err.
+func (s *Scorer) FailWith(err error) *Scorer {
+	s.err = err
+	return s
+}
+
+// Score returns a copy of the tree with every path's Score set to the configured
+// value; path structure and status are carried through unchanged. The input tree
+// is not mutated. The deps argument is ignored.
+func (s *Scorer) Score(_ context.Context, tree entity.SpeculationTree, _ []entity.Batch) (entity.SpeculationTree, error) {
+	if s.err != nil {
+		return entity.SpeculationTree{}, s.err
+	}
+	scored := tree
+	scored.Paths = make([]entity.SpeculationPathInfo, len(tree.Paths))
+	copy(scored.Paths, tree.Paths)
+	for i := range scored.Paths {
+		scored.Paths[i].Score = s.score
+	}
+	return scored, nil
+}
+
+// ensure the fake satisfies the interface.
+var _ scorer.Scorer = (*Scorer)(nil)

submitqueue/extension/speculation/scorer/fake/fake_test.go

@@ -0,0 +1,57 @@
+// Copyright (c) 2025 Uber Technologies, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package fake
+
+import (
+	"context"
+	"errors"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"github.com/uber/submitqueue/submitqueue/entity"
+)
+
+func tree() entity.SpeculationTree {
+	return entity.SpeculationTree{
+		BatchID: "q/batch/2",
+		Paths: []entity.SpeculationPathInfo{
+			{Path: entity.SpeculationPath{Head: "q/batch/2"}, Status: entity.SpeculationPathStatusCandidate},
+			{Path: entity.SpeculationPath{Base: []string{"q/batch/1"}, Head: "q/batch/2"}, Status: entity.SpeculationPathStatusBuilding},
+		},
+	}
+}
+
+func TestScore_StampsConstantAndPreservesStructure(t *testing.T) {
+	in := tree()
+	got, err := New().SetScore(0.5).Score(context.Background(), in, nil)
+	require.NoError(t, err)
+
+	for _, p := range got.Paths {
+		assert.Equal(t, float32(0.5), p.Score)
+	}
+	// structure and status carried through
+	assert.Equal(t, in.BatchID, got.BatchID)
+	assert.Equal(t, entity.SpeculationPathStatusBuilding, got.Paths[1].Status)
+	assert.Equal(t, []string{"q/batch/1"}, got.Paths[1].Path.Base)
+	// input not mutated
+	assert.Equal(t, float32(0), in.Paths[0].Score)
+}
+
+func TestScore_FailWith(t *testing.T) {
+	sentinel := errors.New("boom")
+	_, err := New().FailWith(sentinel).Score(context.Background(), tree(), nil)
+	require.ErrorIs(t, err, sentinel)
+}

submitqueue/extension/speculation/scorer/mock/BUILD.bazel

@@ -0,0 +1,13 @@
+load("@rules_go//go:def.bzl", "go_library")
+
+go_library(
+    name = "go_default_library",
+    srcs = ["scorer_mock.go"],
+    importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/scorer/mock",
+    visibility = ["//visibility:public"],
+    deps = [
+        "//submitqueue/entity:go_default_library",
+        "//submitqueue/extension/speculation/scorer:go_default_library",
+        "@org_uber_go_mock//gomock:go_default_library",
+    ],
+)

submitqueue/extension/speculation/scorer/mock/scorer_mock.go

@@ -0,0 +1,71 @@
+// Copyright (c) 2025 Uber Technologies, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package scorer
+
+//go:generate mockgen -source=scorer.go -destination=mock/scorer_mock.go -package=mock
+
+import (
+	"context"
+
+	"github.com/uber/submitqueue/submitqueue/entity"
+)
+
+// Scorer computes the predicted-success score of every path in a batch's
+// speculation tree.
+//
+// A path's score is a prediction — "how likely is this bet to pay off?" — and
+// predictions must move as evidence arrives. The scorer answers "how good is
+// each path right now" from the current state: the per-batch success
+// probabilities of a path's base batches (entity.Batch.Score, set by the score
+// stage), which of those dependencies have already landed or had their build
+// pass (resolved assumptions raise confidence), and optionally other signals
+// (how long the batch has waited, historical pass rates).
+//
+// The controller re-runs the scorer on every respeculate, right after it
+// reconciles path status — so when a dependency lands, its build passes, or a
+// sibling path fails, the surviving paths' scores are recomputed against the new
+// reality before anything is selected or prioritized. The controller drives
+// *when* to rescore and persists the result; the scorer owns the *formula*.
+//
+// This is the per-*path* scorer, distinct from the per-*batch* score stage
+// (extension/scorer), which sets entity.Batch.Score. The path scorer consumes
+// those batch scores to score whole paths.
+//
+// It is a pure function of its inputs: it reads the tree and the dependency
+// batches and returns scores only. It never reads storage or builds, and never
+// writes status — status is the controller's, fed back in on the tree.
+type Scorer interface {
+	// Score returns the speculation tree with each path's Score set to its
+	// freshly computed predicted-success value, given the current dependency
+	// batches (carrying their Batch.Score and current state). Path structure and
+	// controller-stamped Status are carried through unchanged; only Score is
+	// (re)computed. The combination formula is the implementation's concern.
+	Score(ctx context.Context, tree entity.SpeculationTree, deps []entity.Batch) (entity.SpeculationTree, error)
+}
+
+// Config carries the per-queue identity handed to a Factory. The system knows
+// only the queue name; everything an implementation needs (including scoring
+// knobs and extra signals) is injected at construction by the integrator.
+type Config struct {
+	// QueueName identifies the queue this Scorer serves.
+	QueueName string
+}
+
+// Factory builds the Scorer for a queue. Implementations are provided by
+// integrators (and tests) and inject whatever they need at construction.
+type Factory interface {
+	// For returns the Scorer for the given queue.
+	For(cfg Config) (Scorer, error)
+}