🗺 Annotation Configuration

[mikeldking]

Annotation queues provide an effective way to streamline and manage human annotation tasks by allowing groups of traces to be reviewed and annotated systematically. This feature will enhance the efficiency of tracking and analyzing feedback on specific traces in Phoenix.

Spike

• [annotation] annotation PRD
• [annotation] annotation configuration and queue ERD

Configuration

• [ENHANCEMENT]Allow for different eval colors in Phoenix #5842
• [annotation] [db] annotation configuration #6207
• [annotation] [db] project level annotation configuration #6343
• [annotation][db] remove constraint on annotation columns #6491
• [annotations][db] unset project annotation configs  #6803
• [annotations] add additional columns to annotations #6879
• [annotations] reserve "note"

UI

• [ui] add a config section to a project #6853
• [annotations] [ui] resize project table columns #6588
• [annotation] [ui] project level annotation configuration #6487
• [annotation] [ui] add the ability to page through traces (up / down) #6485
• [annotation] [ui] add the ability to persist the columns displayed on the project page #6486
• [annotation] [ui] split settings page into tabs #6682
• [annotation] [ui] build out annotation / feedback configuration UI #6801
• [annotation] [ui] hot keys on annotation details view
• [annotation] [ui] add the ability to associate annotations with a specific project #6802
• [ENHANCEMENT] more cleanly display metadata in the ui, add quick filters on the row #6125
• [ENHANCEMENT] allow saving filters in a "view" #6435
• [annotations] [ui] persist column selection on the traces tables under a specific project #6539
• [annotations] [ui] span status appears on the right of the table #6590
• [annotations] [ui] metadata column cell rendering #6541
• [annotations] [ui] metadata quick filtering from cell #6587
• [annotations] [ui] move slide in content to the header #6592
• [annotations] [ui] annotations slide in #6591
• [annotations] [ui] allow trace tree to get very compact #6593
• [annotations] [ui] notes UI on the side bar of a span #6917
• [annotations] [ui] slide-in for annotations #6920
• [annotation] [ui] annotation tooltip on the tables (spans) #6919
• [annotations] form elements for continuous, categorical, freeform #6918
• [ui] [annotations] show individual annotations on hover over aggregate annotation badge #6947
• [annotation][ui] Increase granularity of IsAdmin component #6959

API

• [annotation] add the ability to pull annotations with spans #6272
• [annotation] [gql] project annotation configs create, read, delete #6815
• [annotation] [rest] add identifier, metadata, ... to the annotation endpoints #6816
• [annotation] [gql] update span filters to work with new annotation summary data loader
• [annotation] [db] consider adding a backwards-compatible unique constraint
• [annotation] [rest] add patch endpoint
• [annotation] [rest] add delete endpoint
• [annotation] [gql] figure out bulk deletion semantics
• [annotation] [gql] note taking apis #6878
• [annotation] make the viewer the system #6916
• [annotations] filters on annotations (user_id, name include list, name exclude list, source) #6921
• [annotations] resolve user, source, ... on a span annotation
• [annotations] notes mutation and resolver
• [annotations] optional project_ids in the create annotation config mutation
• [annotations] annotations summary field on span #6948
• [annotations] modify filter dsl to be able to filter on aggregate metrics for annotations (averages) #6949

Python SDK

• [annotaiton] [python sdk] add the ability to pull spans with annotations #6271