Skip to content

@METR METR

Change the repository type filter

All

    Repositories list

    28 repositories

    • vivaria

      Public
      Vivaria is METR's tool for running evaluations and conducting agent elicitation research.
      aielicitationai-evaluationevals
      TypeScript
      •
      MIT License
      •31•83•228•19•Updated Mar 22, 2025Mar 22, 2025

    • eval-analysis-public

      Public
      Public repository containing METR's DVC pipeline for eval data analysis
      Python
      •5•30•5•1•Updated Mar 21, 2025Mar 21, 2025

    • inspect_k8s_sandbox

      Public
      A Kubernetes sandbox environment for use with inspect_ai
      Python
      •
      MIT License
      •5•0•0•1•Updated Mar 21, 2025Mar 21, 2025

    • autonomy-evals-guide

      Public
      SCSS
      •
      MIT License
      •4•3•0•2•Updated Mar 20, 2025Mar 20, 2025

    • inspect_evals

      Public
      Collection of evals for Inspect AI
      Python
      •
      MIT License
      •101•0•0•0•Updated Mar 19, 2025Mar 19, 2025

    • task-protected-scoring

      Public
      Python
      •1•0•3•2•Updated Mar 18, 2025Mar 18, 2025

    • uplift_clone_hypothesis

      Public
      Hypothesis is a powerful, flexible, and easy to use library for property-based testing.
      Python
      •
      Other
      •597•0•3•1•Updated Mar 18, 2025Mar 18, 2025

    • hcast-public

      Public
      HTML
      •
      Other
      •1•7•0•0•Updated Mar 17, 2025Mar 17, 2025

    • public-tasks

      Public
      HTML
      •
      Other
      •9•87•1•2•Updated Mar 16, 2025Mar 16, 2025

    • task-assets

      Public
      Python
      •0•0•1•0•Updated Mar 8, 2025Mar 8, 2025

    • agent-prs-on-vivaria

      Public
      Vivaria is METR's tool for running evaluations and conducting agent elicitation research.
      TypeScript
      •
      MIT License
      •31•0•0•32•Updated Feb 19, 2025Feb 19, 2025

    • SWE-bench-fork

      Public
      [ICLR 2024] SWE-bench: Can Language Models Resolve Real-world Github Issues?
      Python
      •
      MIT License
      •452•0•0•0•Updated Feb 11, 2025Feb 11, 2025

    • KernelBenchFiltered

      Public
      Python
      •1•5•1•0•Updated Feb 11, 2025Feb 11, 2025

    • viv-task-dev

      Public
      Shell
      •1•0•6•0•Updated Feb 6, 2025Feb 6, 2025

    • inspect_ai

      Public
      Inspect: A framework for large language model evaluations
      Python
      •
      MIT License
      •196•0•0•0•Updated Feb 4, 2025Feb 4, 2025

    • task-standard

      Public
      METR Task Standard
      TypeScript
      •
      MIT License
      •32•146•6•3•Updated Feb 3, 2025Feb 3, 2025

    • RE-Bench

      Public
      Python
      •
      Other
      •6•66•3•1•Updated Jan 30, 2025Jan 30, 2025

    • agent-fork-of-eval-analysis-public

      Public archive
      For of eval-analysis-public repo intended for agent PRs
      Python
      •5•0•0•0•Updated Jan 28, 2025Jan 28, 2025

    • llm-foundry

      Public
      LLM training code for Databricks foundation models
      Python
      •
      Apache License 2.0
      •554•0•0•0•Updated Jan 28, 2025Jan 28, 2025

    • metr-task-boilerplate

      Public
      A Cookiecutter template for developing tasks according to the METR Task Standard
      evaluationssafetyllm
      TypeScript
      •0•1•0•0•Updated Jan 22, 2025Jan 22, 2025

    • task-artifacts

      Public
      Python
      •0•0•0•0•Updated Jan 21, 2025Jan 21, 2025

    • .github

      Public
      0•0•0•0•Updated Nov 24, 2024Nov 24, 2024

    • nanoGPT

      Public
      The simplest, fastest repository for training/finetuning medium-sized GPTs.
      Python
      •
      MIT License
      •6.6k•0•0•0•Updated Nov 22, 2024Nov 22, 2024

    • task-legacy-verifier

      Public
      Python
      •0•0•0•0•Updated Nov 8, 2024Nov 8, 2024

    • task-aux-vm-helpers

      Public
      Python
      •1•0•0•0•Updated Nov 2, 2024Nov 2, 2024

    • pyhooks

      Public archive
      A library that METR agents use to communicate with Vivaria.
      Python
      •1•0•1•0•Updated Sep 22, 2024Sep 22, 2024

    • vivaria-mentat

      Public archive
      Vivaria is METR's tool for running evaluations and conducting agent elicitation research.
      TypeScript
      •
      MIT License
      •31•0•1•1•Updated Sep 19, 2024Sep 19, 2024

    • task-template

      Public template
      TypeScript
      •6•9•2•3•Updated Aug 6, 2024Aug 6, 2024