Harden batch input validation and gem runtime surface#8
Merged
Conversation
csv moved from a default gem to a bundled gem in Ruby 3.4, so Bundler-managed consumers hit a LoadError on Report#export_csv without an explicit dependency. CodeFactory builder
Add validate_dataset! to evaluate_batch so malformed samples (non-hash, missing or nil :question/:answer, string or symbol keys) raise ArgumentError with the offending index before any API spend, making sequential and concurrent modes fail identically. Remove the redundant csv Gemfile entry now that it is a gemspec runtime dependency. Closes #5 CodeFactory builder
CodeFactory builder
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #5
What
csvruntime dependency (rubric_llm.gemspec):csvmoved from a default gem to a bundled gem in Ruby 3.4, so every Bundler-managed consumer hit aLoadErroronReport#export_csv. The gem now declares it; the redundantgem "csv"Gemfile entry and the "optional dependency" comment inreport.rbare removed.lib/rubric_llm.rb):evaluate_batchnow callsvalidate_dataset!before any evaluation. Non-hash samples, samples missing:question/:answer, and samples with nil values (string or symbol keys) raiseArgumentErrorwith the offending sample index, before any LLM call, so sequential and concurrent modes fail identically and without API spend.ruby_llmbumped to~> 1.16(separate commit).Descoped per issue
No
Thread#valuerefactor, noEvaluatorrescue broadening, no Ruby floor change.Tests
New
test/test_batch_validation.rb: non-hash rejection, missing-key rejection (both key styles), nil-value rejection, acceptance of complete symbol- and string-keyed samples, and malformed-sample-under-concurrency, each asserting zero judge calls where applicable.bundle exec rake: 141 runs, 328 assertions, 0 failures, 0 errors. RuboCop: 45 files, no offenses.