Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: support authored blobs #3765

Merged
merged 13 commits into from
Aug 13, 2024
Merged

feat!: support authored blobs #3765

merged 13 commits into from
Aug 13, 2024

Conversation

cmwaters
Copy link
Contributor

@cmwaters cmwaters commented Aug 7, 2024

This PR uses the go-square to support authored blobs (using the v1 share version).

In this PR is the new block validation logic which ensures that all "authored" blobs have a signer that matches the signer of the PFB

TxSim now will randomly use blobs of share version 0 and 1

Copy link

github-actions bot commented Aug 7, 2024

PR Preview Action v1.4.7
🚀 Deployed preview to https://celestiaorg.github.io/celestia-app/pr-preview/pr-3765/
on branch gh-pages at 2024-08-07 15:23 UTC

@cmwaters cmwaters marked this pull request as ready for review August 8, 2024 09:16
@cmwaters cmwaters requested review from liamsi and a team as code owners August 8, 2024 09:16
@cmwaters cmwaters requested review from rootulp and staheri14 and removed request for a team August 8, 2024 09:16
Copy link
Contributor

coderabbitai bot commented Aug 8, 2024

Walkthrough

Walkthrough

The changes across the codebase primarily involve a transition from using constants and methods from the appconsts package to the newly integrated share package. This refactoring enhances clarity, maintainability, and consistency in blob handling and namespace management. Additionally, improvements in error handling and validations have been introduced, alongside new test cases that expand coverage for different blob versions, ensuring more robust functionality and integrity in processing transactions.

Changes

File(s) Change Summary
app/default_overrides.go Introduced share.ContinuationSparseShareContentSize in DefaultConsensusConfig, shifting from appconsts.
app/errors/insufficient_gas_price_test.go Updated blob creation in TestInsufficientMinGasPriceIntegration to use NewV0Blob, removing the size parameter for simplification.
app/errors/nonce_mismatch_test.go Replaced blob.NewBlob with blob.NewV0Blob, aligning with new blob creation methods.
app/test/check_tx_test.go Expanded TestCheckTx with new cases for v1 blobs, validating valid and invalid signers.
app/test/consistent_apphash_test.go Added validation to ensure transaction counts match during proposal preparation in executeTxs.
app/test/fuzz_abci_test.go Changed references from appconsts.ContinuationSparseShareContentSize to share.ContinuationSparseShareContentSize.
app/test/integration_test.go Adjusted parameters in share.NewBlob, replacing appconsts.ShareVersionZero with share.ShareVersionZero.
app/test/process_proposal_test.go Updated TestProcessProposal to use share.ShareVersionZero and added validations for blob transaction processing.
app/test/square_size_test.go Simplified calculation of maxBytes to use DefaultUpperBoundMaxBytes directly.
app/test/std_sdk_test.go Changed blob generation method to RandV0BlobsWithNamespace, reflecting new versioning.
pkg/appconsts/global_consts.go Removed several constants in favor of those from share, including DefaultShareVersion.
pkg/appconsts/initial_consts.go Updated DefaultMaxBytes and introduced DefaultUpperBoundMaxBytes, both now sourced from share.
pkg/da/data_availability_header_test.go Changed import to alias sh, updating references to share accordingly.
pkg/inclusion/nmt_caching_test.go Updated reference for ShareSize from appconsts to share.
pkg/user/signer.go Added validation for blobs in CreatePayForBlobs, ensuring only valid blobs are processed.
pkg/wrapper/nmt_wrapper.go Changed references from appconsts.NamespaceSize to share.NamespaceSize.
pkg/wrapper/nmt_wrapper_test.go Updated imports and references to remove appns, using share directly.
specs/src/specs/namespace.md Updated a URL reference in the Markdown documentation.
test/cmd/txsim/cli_test.go Simplified the setup function for consensus parameters by directly assigning appconsts.DefaultMaxBytes.
test/txsim/blob.go Enhanced BlobSequence to support multiple share versions, with updated methods for blob generation.
test/util/testnode/config.go Adjusted consensus parameter assignments to use appconsts.DefaultUpperBoundMaxBytes.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant Application
    participant BlobFactory
    participant Share

    User->>Application: Create Blob Request
    Application->>BlobFactory: Generate Blob with Signer
    BlobFactory->>Share: Create Blob (Version 0 or 1)
    Share-->>BlobFactory: Return Blob
    BlobFactory-->>Application: Return Created Blob
    Application-->>User: Respond with Blob Information
Loading

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

‼️ IMPORTANT
Auto-reply has been disabled for this repository in the CodeRabbit settings. The CodeRabbit bot will not respond to your replies unless it is explicitly tagged.

  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@cmwaters cmwaters self-assigned this Aug 8, 2024
@celestia-bot celestia-bot requested a review from a team August 8, 2024 09:18
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Outside diff range, codebase verification and nitpick comments (3)
test/txsim/blob.go (1)

24-27: Add comments for the new field shareVersions.

Consider adding comments to explain the purpose and usage of the shareVersions field for better maintainability and readability.

+ // shareVersions holds the possible share versions for blobs.
x/blob/client/cli/payforblob.go (2)

Line range hint 127-133: Handle errors consistently when broadcasting PFB.

Ensure that the error returned by broadcastPFB is handled consistently. Consider logging the error or providing additional context if needed.

if err != nil {
    fmt.Printf("Error broadcasting PFB: %v\n", err)
    return err
}

Line range hint 142-146: Ensure consistent error handling for parsed blobs.

The loop processing parsed blobs should handle errors consistently. Consider logging errors or adding context to error messages for better debugging.

if err != nil {
    fmt.Printf("Error processing parsed blob: %v\n", err)
    return err
}

Comment on lines +48 to +56
// WithShareVersion provides the option of fixing a predefined share version for
// all blobs else it will randomly select a share version for each blob.
func (s *BlobSequence) WithShareVersion(version uint8) *BlobSequence {
if version != share.ShareVersionZero && version != share.ShareVersionOne {
panic(fmt.Sprintf("invalid share version %d", version))
}
s.shareVersions = []uint8{version}
return s
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handle invalid share version more gracefully.

Instead of panicking, consider returning an error when an invalid share version is provided. This approach is more robust and user-friendly.

-		panic(fmt.Sprintf("invalid share version %d", version))
+		return fmt.Errorf("invalid share version %d", version)

Committable suggestion was skipped due to low confidence.

Copy link
Collaborator

@rootulp rootulp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My only blocking comment is: should this feature be gated behind a new app version? If yes, then the tests in this PR that currently pass with app version 2 should fail and should only pass if app version is 3.

d.Txs[0] = rawTx
d.Hash = calculateNewDataHash(t, d.Txs)
},
appVersion: appconsts.LatestVersion,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocking] I think this test case should fail b/c app version is 2. Should this feature be gated behind an app version bump?

Ref: https://github.com/celestiaorg/CIPs/blob/main/cips/cip-21.md#backwards-compatibility

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was waiting on this until we have more information regarding your v3 prototype. At the moment it's not gated but it should be

// ContinuationSparseShareContentSize is the number of bytes usable for data
// in a continuation sparse share of a sequence.
ContinuationSparseShareContentSize = ShareSize - NamespaceSize - ShareInfoBytes
DefaultShareVersion = share.ShareVersionZero
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[question] why default to the previous share version when a new one exists?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually don't know what is meant by the default here. I have left it as is because v1 is a new feature and I a) don't know if many people will use it and b) should be opt in anyway.

Maybe if this is just for tests we can change it but it would require a large refactor since the new version requires new arguments

specs/src/specs/namespace.md Outdated Show resolved Hide resolved
pkg/appconsts/global_consts.go Show resolved Hide resolved
test/txsim/blob.go Outdated Show resolved Hide resolved
test/util/testnode/config.go Outdated Show resolved Hide resolved
test/util/testnode/config.go Show resolved Hide resolved
@@ -22,16 +22,16 @@ import (
tmrand "github.com/tendermint/tendermint/libs/rand"
)

func TestNewBlob(t *testing.T) {
func TestNewV0Blob(t *testing.T) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[question] do we want a new test in this file for TestNewV1Blob?

@evan-forbes evan-forbes added WS: V3 3️⃣ item is directly relevant to the v3 hardfork authored blobs labels Aug 8, 2024
evan-forbes
evan-forbes previously approved these changes Aug 8, 2024
Copy link
Member

@evan-forbes evan-forbes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@evan-forbes
Copy link
Member

started to address the versioning thing in #3769

@celestia-bot celestia-bot requested review from evan-forbes and removed request for a team August 9, 2024 11:25
@cmwaters cmwaters requested a review from rootulp August 9, 2024 12:27
Copy link
Member

@evan-forbes evan-forbes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm good merging without the gate

#3771
#3772

@celestia-bot celestia-bot requested a review from a team August 12, 2024 13:50
Copy link
Collaborator

@rootulp rootulp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

[follow-up] maybe we should have a follow-up issue to ensure the authored blobs feature is only supported for app version 3. Even if we adopt the multiplexed app prototype, that shouldn't prevent us from adding defensive app version checks for new features (like authored blobs) in low level tests like x/blob/types/blob_tx_test.go.

@celestia-bot celestia-bot requested review from a team and rach-id and removed request for a team August 12, 2024 16:31
@cmwaters cmwaters merged commit e43786b into main Aug 13, 2024
32 of 34 checks passed
@cmwaters cmwaters deleted the cal/support-authored-blobs branch August 13, 2024 09:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
authored blobs WS: V3 3️⃣ item is directly relevant to the v3 hardfork
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants