Skip to content

Fix multiple versions of msgpack being used #7399

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 22, 2025

Conversation

fxamacker
Copy link
Member

@fxamacker fxamacker commented May 12, 2025

Updates #7242

Currently, import statements are using different versions of msgpack:

  • github.com/vmihailenco/msgpack
  • github.com/vmihailenco/msgpack/v4

They cause go.mod to import different versions of msgpack:

  • msgpack (v4.0.4)
  • msgpack/v4 (v4.3.12)

Also, the msgpack version used by non-test code and test code sometimes doesn't match.

This PR replaces import statements using "msgpack" with import statements using "msgpack/v4". This change specifies a specific major version number and prevents different versions of msgpack being used by mistake.

I found this while getting familiar with storage code related to BadgerDB and Pebble (not looking for bugs).

Currently, import statements are using different versions
of msgpack:
- github.com/vmihailenco/msgpack
- github.com/vmihailenco/msgpack/v4

They cause go.mod to import different versions of msgpack:
- msgpack     (v4.0.4)
- msgpack/v4  (v4.3.12)

This commit replaces import statements using "msgpack" with
import statements using "msgpack/v4".  This change specifies
a specific major version number and prevents different versions
of msgpack being used by mistake.
@fxamacker fxamacker requested a review from a team May 12, 2025 12:18
@fxamacker fxamacker self-assigned this May 12, 2025
@fxamacker fxamacker requested a review from a team as a code owner May 12, 2025 12:18
@codecov-commenter
Copy link

codecov-commenter commented May 12, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 41.11%. Comparing base (926dbc0) to head (b51fba2).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #7399   +/-   ##
=======================================
  Coverage   41.10%   41.11%           
=======================================
  Files        2207     2207           
  Lines      193498   193498           
=======================================
+ Hits        79544    79550    +6     
+ Misses     107325   107321    -4     
+ Partials     6629     6627    -2     
Flag Coverage Δ
unittests 41.11% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Member

@zhangchiqing zhangchiqing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Faye

@@ -5,7 +5,7 @@ import (
"fmt"

"github.com/rs/zerolog"
"github.com/vmihailenco/msgpack"
"github.com/vmihailenco/msgpack/v4"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we double check the existing version and v4 are encoding into the same bytes data? Do we need some test case to verify?

Copy link
Member Author

@fxamacker fxamacker May 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we double check the existing version and v4 are encoding into the same bytes data? Do we need some test case to verify?

Good question.

I don't see tagged releases (on GitHub with release notes) for msgpack and their CHANGELOG.md doesn't show changes within v4 (no details about v4.x -> 4.y versions).

However, there seems to be a lot of bugfixes and performance/memory improvements between v4.0.4 and v4.3.12 based on git comparison.

Do we need some test case to verify?

This might be tricky (for someone unfamiliar with the code and the data used) because some code is calling Marshal on interface type, e.g. flow.Entity, or any type. Maybe people more familiar with the code and data can add some test cases.

BTW, I noticed in some cases non-test code used different msgpack version than the test code. So this PR made msgpack versions used by non-tests and tests consistent too.

@fxamacker fxamacker requested a review from a team May 12, 2025 17:57
@fxamacker fxamacker added this pull request to the merge queue May 22, 2025
Merged via the queue into master with commit 45ee9c6 May 22, 2025
57 checks passed
@fxamacker fxamacker deleted the fxamacker/cleanup-msgpack-import branch May 22, 2025 16:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants