Skip to content

Conversation

@dependabot
Copy link
Contributor

@dependabot dependabot bot commented on behalf of github Oct 28, 2025

Bumps promptfoo from 0.118.17 to 0.119.0.

Release notes

Sourced from promptfoo's releases.

0.119.0

What's Changed

Features

  • feat(webui): filter eval results by metric values with numeric operators (EQ, GT, LTE, etc.) by @​will-holley in #6011
  • feat(providers): 10-100x performance improvement for Python providers with persistent worker pools by @​mldangelo in #5968
  • feat(providers): add OpenAI Agents SDK integration with support for agents, tools, and handoffs by @​mldangelo in #6009
  • feat(providers): add function calling/tool support for Ollama by @​mldangelo in #5977
  • feat(providers): add support for Claude Haiku 4.5 by @​jameshiester in #5937
  • feat(redteam): add jailbreak:meta strategy with intelligent attack taxonomy learning by @​MrFlounder in #6021
  • feat(redteam): add COPPA plugin by @​typpo in #5997
  • feat(redteam): add GDPR preset mappings by @​typpo in #5986
  • feat(redteam): add modifiers support to iterative strategies by @​MrFlounder in #5972
  • feat(redteam): add authoritative markup injection strategy by @​typpo in #5961
  • feat(redteam): add wordplay plugin by @​typpo in #5889
  • feat(redteam): add Simba red team agent strategy by @​sklein12 in #5795
  • feat(redteam): add subcategory filtering to BeaverTails plugin by @​typpo (a70372f)
  • feat(redteam): include pluginId, strategyId, and sessionId in CSV exports by @​sklein12 in #6016
  • feat(webui): persist custom policy names by @​will-holley in #5990
  • feat(webui): show target responses for red team test cases by @​will-holley in #5869
  • feat(cli): log errors to file with console messages by @​sklein12 in #5992
  • feat(cli): show errors in eval progress bar by @​sklein12 in #5942
  • feat(cache): display latency measurements for cached responses by @​mldangelo in #5978

Fixes

  • fix(providers): restore runtime variable substitution in templates by @​mldangelo (5423f80)
  • fix(providers): improve Python provider reliability with automatic python3/python detection and better error handling by @​mldangelo in #6034
  • fix(providers): simulated-user and mischievous-user now respect system prompts in multi-turn conversations by @​mldangelo in #6020
  • fix(providers): improve MCP tool schema compatibility with OpenAI by @​mldangelo in #5965
  • fix(providers): properly store sessionId in metadata by @​sklein12 in #6016
  • fix(redteam): skip session management tests for stateless targets by @​faizanminhas in #5989
  • fix(redteam): improve Crescendo strategy accuracy by @​jameshiester in #5964
  • fix(redteam): reduce duplicate error messages for invalid strategy and plugin ids by @​typpo in #5954
  • fix(fetch): improve retry counter messages and error details by @​LizzHale in #6017, in #6019
  • fix(webui): pass extensions config when running evals by @​theLucasAntunes in #6006
  • fix(webui): fix visibility of reset config button in red team setup by @​will-holley in #5896
  • fix(webui): sync selected plugins to global config by @​will-holley in #5991
  • fix(webui): fix HTTP test agent by @​faizanminhas in #6033
  • fix(webui): reset strategy config dialog when switching strategies by @​sklein12 in #6035

Chores

Documentation

... (truncated)

Changelog

Sourced from promptfoo's changelog.

[0.119.0] - 2025-10-27

Added

  • feat(webui): filtering eval results by metric values w/ numeric operators (e.g. EQ, GT, LTE, etc.) (#6011)
  • feat(providers): add Python provider persistence for 10-100x performance improvement with persistent worker pools (#5968)
  • feat(providers): add OpenAI Agents SDK integration with support for agents, tools, handoffs, and OTLP tracing (#6009)
  • feat(providers): add function calling/tool support for Ollama chat provider (#5977)
  • feat(providers): add support for Claude Haiku 4.5 (#5937)
  • feat(redteam): add jailbreak:meta strategy with intelligent meta-agent that builds dynamic attack taxonomy and learns from full attempt history (#6021)
  • feat(redteam): add COPPA plugin (#5997)
  • feat(redteam): add GDPR preset mappings for red team testing (#5986)
  • feat(redteam): add modifiers support to iterative strategies (#5972)
  • feat(redteam): add authoritative markup injection strategy (#5961)
  • feat(redteam): add wordplay plugin (#5889)
  • feat(redteam): add pluginId, strategyId, sessionId, and sessionIds to metadata columns in CSV export (#6016)
  • feat(redteam): add subcategory filtering to BeaverTails plugin (a70372f)
  • feat(redteam): Add Simba Red Team Agent Strategy (#5795)
  • feat(webui): persist inline-defined custom policy names (#5990)
  • feat(webui): show target response to generated red team plugin test case (#5869)
  • feat(cli): log all errors in a log file and message to the console (#5992)
  • feat(cli): add errors to eval progress bar (#5942)
  • feat(cache): preserve and display latency measurements when provider responses are cached (#5978)

Changed

  • chore(internals): custom policy type def (#6037)
  • chore(changelog): organize and improve Unreleased section with consistent scoping and formatting (#6024)
  • chore(cli): show telemetryDisabled/telemetryDebug in promptfoo debug output (#6015)
  • chore(cli): improve error handling and error logging (#5930)
  • chore(cli): revert "feat: Improved error handling in CLI and error logging" (#5939)
  • chore(webui): add label column to prompts table (#6002)
  • chore(webui): gray out strategies requiring remote generation when disabled (#5985)
  • chore(webui): gray out remote plugins when remote generation is disabled (#5970)
  • chore(webui): improve test transform modal editor (#5962)
  • chore(webui): add readOnly prop to EvalOutputPromptDialog (#5952)
  • refactor(webui): organize red team plugins page into tabs with separate components (#5865)
  • chore(redteam): remove "LLM Risk Assessment" prefix (#6004)
  • chore(redteam): add top-level redteam telemetry events (#5951)
  • refactor(webui): reduce unnecessary API health requests (#5979)
  • chore(api): export GUARDRAIL_BLOCKED_REASON constant for external use (#5956)
  • chore(providers): add rendered request headers to http provider debug output (#5950)
  • refactor(transforms): refactor transform code to avoid 'require' (#5943)
  • refactor(transforms): refactor createRequest/ResponseTransform functions into separate module (#5925)
  • chore(examples): consolidate Ollama examples into unified directory (#5977)
  • chore(deps): move dependencies to optional instead of peer (#5948)
  • chore(deps): move natural to optional dependency (#5946)
  • chore(redteam): improve GOAT and Crescendo error logs with additional error details for easier debugging (#6036)

Fixed

... (truncated)

Commits
  • 804c511 chore: bump version 0.119.0 (#6039)
  • d1731e5 chore: Better error logging for goat and crescendo strategies (#6036)
  • 23381e0 chore(internals): Updates custom policy type (#6037)
  • 6da6a8f fix(providers): improve Python provider reliability and logging (#6034)
  • 619718b fix(webui): reset red team strategy config dialog state when switching strate...
  • e6b2565 fix: HTTP test agent (#6033)
  • 2679389 fix(app): test target (#6031)
  • ea4aadc feat: Simba, the most advanced Red Team Agent (#5795)
  • 2a2d9be revert(providers): eager template rendering that broke runtime variable subst...
  • a26ec96 feat(app): Eval results metric value filtering (#6011)
  • Additional commits viewable in compare view

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Bumps [promptfoo](https://github.com/promptfoo/promptfoo) from 0.118.17 to 0.119.0.
- [Release notes](https://github.com/promptfoo/promptfoo/releases)
- [Changelog](https://github.com/promptfoo/promptfoo/blob/main/CHANGELOG.md)
- [Commits](promptfoo/promptfoo@0.118.17...0.119.0)

---
updated-dependencies:
- dependency-name: promptfoo
  dependency-version: 0.119.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
@dependabot dependabot bot added dependencies Pull requests that update a dependency file javascript Pull requests that update Javascript code labels Oct 28, 2025
@dependabot @github
Copy link
Contributor Author

dependabot bot commented on behalf of github Oct 30, 2025

A newer version of promptfoo exists, but since this PR has been edited by someone other than Dependabot I haven't updated it. You'll get a PR for the updated version as normal once this PR is merged.

@mldangelo mldangelo merged commit 89d5446 into main Nov 3, 2025
3 checks passed
@dependabot dependabot bot deleted the dependabot/npm_and_yarn/promptfoo-0.119.0 branch November 3, 2025 08:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file javascript Pull requests that update Javascript code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants