Repository navigation
learn: retrospective learnings #589
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
peco-engineer-bot
wants to merge
1
commit into
main
Choose a base branch
from
ai/learning-distill-pr
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| # Learning journal 2026-11-21 | ||
|
|
||
| ### 2026-11-21: HTTP client — Custom retry RoundTrippers must drain and close every non-returned response | ||
| **Context:** PR #584 replaced go-retryablehttp with an internal retryTransport. The old path leaked connections on failed or retry-exhausted requests, which could exhaust a connector's pool. The fix drains up to 4 KiB and closes the body before each retry, closes the final response before returning the wrapped error, and calls CloseIdleConnections. | ||
| **Rule:** In any retrying http.RoundTripper, drain (bounded) and close the body of every response you will not return, including the last one on retry exhaustion, and return a nil response with the error, because net/http discards a response that comes back together with an error. | ||
|
|
||
| ### 2026-11-21: Go harness — Detect connection leaks with a body-counting transport wrapper | ||
| **Context:** PR #584 proved the leak fix by wrapping the base transport in a RoundTripper that counts responses and open bodies, with a sync.Once-guarded Close. Unit tests (retry exhaustion at -1/0/3 retries) and an E2E with an invalid token (repeated 403s) asserted zero open bodies. | ||
| **Rule:** To test HTTP resource leaks, inject a RoundTripper that wraps each response body with a once-guarded close counter, then assert open==0 after error, retry-exhaustion and rejected-auth paths, not just on success. | ||
|
|
||
| ### 2026-11-21: Reviewer — Check reimplemented-library review findings against main's real behavior | ||
| **Context:** In PR #584 a reviewer asked that an empty buffered body with a positive declared ContentLength be rejected. The author ran the same real httptest on origin/main and on the PR head. Both sent length 0: retryablehttp already used http.NoBody, and net/http ignores ContentLength for NoBody. The finding was withdrawn. | ||
| **Rule:** When a PR replaces a dependency with in-house code, run a real httptest on main and on the PR head before acting on behavior-changing review suggestions, and keep the existing behavior unless a regression is shown. | ||
|
|
||
| ### 2026-11-21: Arrow results and metadata — Take reported type names and value-decoder selection from different metadata keys | ||
| **Context:** In PR #583, GEOMETRY/GEOGRAPHY names now come from Spark:DataType:SqlName, which both Thrift and the kernel populate (e.g. GEOMETRY(0), GEOMETRY(ANY)). Binary-geo value decoding still uses the physical STRUCT type plus databricks.type_name. A reviewer repeatedly flagged a mismatch that only contradictory server metadata could cause, and it was deferred as out of scope. | ||
| **Rule:** Report DatabaseTypeName from Spark:DataType:SqlName so both backends agree, keep decoder selection on physical Arrow type plus databricks.type_name, and treat parameterized names (e.g. SRIDs) as a behavior change whose consumers must match the family before "(". |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🟡 Medium — L-0066, L-0067 and L-0070, plus the L-0041 (#577) and L-0050 (#574) updates, are distilled from a 2026-11-20 journal that isn't in the checkout, so they have no source entry and the journal is no longer the record they were built from. Restore
.claude/knowledge/journal/2026-11-20.mdin this PR, or drop those lessons and updates until it lands.🔀 Found by Claude Opus 5.5 · the judge couldn't confirm it