Skip to content

About

Use an agent to look up datasets for your analysis

Resources

Stars

1 star

Watchers

0 watching

Forks

Latest commit

 

History

2 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 

Repository files navigation

Agentic Data Lookup

An agent that will accept a description of an analysis and use experiment knowledge bases to look up datasets for your analysis. It will work by looking up the nearest neighbors to your analsysis and retrieving from the research notes the datasets used.

Benchmarks

To begin with, we are collecting some benchmarks based on existing analyses and recording the actual datasets used. We will also record the Analysis ID so we can exclude those datasets from the benchmarking.

Each benchmark will be a subdirectory of the benchmarks/<experiment_name> directory. Name the subdirectory after the analysis ID and include two text files: prompt.txt: A prompt that describes the analysis. datasets.txt: A list of datasets used in the analysis.

We can have multiple prompts for each analysis to explore different ways of describing the analysis.

Evaluation

We will evaluate the agent's performance by comparing the datasets it retrieves with the actual datasets used in the analysis.

We will use MLFLow to track these questions along with the context used to answer them.

About

Use an agent to look up datasets for your analysis

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors