perf: poison non-HTTP flows to avoid repeated parse-fail-clear cycles#42
Merged
perf: poison non-HTTP flows to avoid repeated parse-fail-clear cycles#42
Conversation
Add per-direction poisoned flag to HttpFlowState. After the first parse error on a direction with no prior successful parse, mark it poisoned and skip all future buffering/parsing for that direction. - request_poisoned / response_poisoned bools on HttpFlowState - non_http_flows counter in HttpAnalyzer, surfaced in summarize() - Updated test to verify poisoned direction skips subsequent data Reduces parse_errors from 14 to 2 on http-full.cap fixture. Fixes #18
- Add POISON_THRESHOLD (3 errors) before poisoning to tolerate mid-stream joins where first segments are body data - Fix non_http_flows double-counting: use per-flow counted_as_non_http flag so counter increments once per flow, not once per direction - Add poisoned_bytes_skipped counter for observability of discarded data - Add tests: threshold behavior, direction independence, flow counter accuracy, poison cleared after flow close
There was a problem hiding this comment.
Pull request overview
This PR improves HTTP analyzer performance on non-HTTP TCP streams by “poisoning” a flow direction after repeated header-parse failures, preventing repeated parse-fail-clear cycles and adding summary counters for observability.
Changes:
- Add per-direction poisoning state and counters to
HttpFlowState(with aPOISON_THRESHOLDof 3). - Skip future data for poisoned directions while tracking
non_http_flowsandpoisoned_bytes_skipped. - Expand HTTP analyzer tests to cover poisoning threshold behavior, per-direction independence, per-flow counting, and reset on flow close.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
src/analyzer/http.rs |
Introduces per-direction poisoning logic and new summary metrics to avoid repeated parsing on non-HTTP traffic. |
tests/http_analyzer_tests.rs |
Adds/updates unit tests validating poisoning threshold, direction independence, per-flow counting, and cleanup on close. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Reset request_error_count/response_error_count to 0 on successful parse so poisoning is based on consecutive errors, not total errors - Add poisoned_bytes_skipped() accessor for test observability - Assert poisoned_bytes_skipped increments correctly in test
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
HttpFlowStateto skip non-HTTP TCP flows after repeated parse failuresPOISON_THRESHOLD(3 consecutive errors) to tolerate mid-stream joins where initial segments are body datanon_http_flowsper flow (not per direction) andpoisoned_bytes_skippedfor observabilityFixes #18
Test plan
test_parse_error_poisons_direction_after_threshold— 3 errors poison, 4th data skippedtest_single_error_does_not_poison— 1 error below threshold, next valid request parsestest_poison_request_does_not_affect_response— direction independencetest_non_http_flows_counts_per_flow_not_direction— counter accuracytest_poison_cleared_after_flow_close— poison doesn't persist across flow reusecargo fmtclean