New parser #158

spreston8 · 2025-09-16T18:45:43Z

Replace internal parser with parser from rholang-rs

…s; fix string leak

…n, restriction, drop_head, all as member functions.

…n from Rholang collection AST (normalizer support).

…alSpec

This reverts commit b66594d.

The PathMap crate depends on gxhash which requires AES and SSE2 CPU features. GitHub Actions runners support these features but Rust doesn't enable them by default for portability. This was causing the build to fail with: error: Gxhash requires aes and sse2 intrinsics

The new_parser branch adds circe-generic-extras auto-derivation for PathMap/Zipper types which requires heavy macro expansion. The node project uses forked compilation (Compile/fork := true) with -Xss8m stack, but the forked JVM was only getting 4GB heap from SBT_OPTS. This caused GC thrashing (56.9% time in GC) during node compilation, making it hang indefinitely. Increasing to 6GB provides enough memory for the macro expansion while leaving headroom on 7GB GitHub runners.

Increase SBT_OPTS stack from 2m to 8m to handle the circe-generic-extras macro expansion for PathMap types in the node project.

The forked JVM for Compile was causing 7+ minute compile times because it didn't inherit SBT_OPTS (including -Xss8m stack size). The forked JVM also incurs cold startup penalty. Now that SBT_OPTS has -Xss8m, the main SBT JVM can handle the circe-generic-extras macro expansion for PathMap types directly. This should restore node compile times to ~30 seconds.

The auto._ import triggered automatic implicit derivation for ALL types in scope, causing exponential macro expansion with recursive protobuf types (Par/Expr/Connective). This caused node compilation to take 2+ hours instead of 30 seconds. Changes: - Remove io.circe.generic.extras.auto._ import - Add explicit deriveConfiguredEncoder/Decoder for ExprInstance (sealed trait) - Add explicit encoders/decoders for new EPathMap and EZipper types

…rser

AndriiS-DevBrother

I don't have a full context and good knowledge of bnfc/tree-sitter differences

so just few minor comments from my side

AndriiS-DevBrother · 2026-01-02T13:53:10Z

rust-toolchain.toml

@@ -0,0 +1,2 @@
+[toolchain]
+channel = "nightly-2025-06-15"


why do we need this version?

AndriiS-DevBrother · 2026-01-02T14:01:08Z

rholang/src/main/tree_sitter/bindings/rust/lib.rs

@@ -1,53 +0,0 @@
-//! This crate provides Rholang language support for the [tree-sitter][] parsing library.


need to cleanup?

AndriiS-DevBrother · 2026-01-02T14:01:30Z

rholang/src/lib.rs

-            return std::ptr::null();
-        }
-    };
+    // let result = match Compiler::source_to_adt_with_normalizer_env(


AndriiS-DevBrother · 2026-01-02T14:09:35Z

casper/src/test/scala/coop/rchain/casper/util/rholang/InterpreterUtilTest.scala

  }

-  it should "return None for logs containing extra comm events" in withGenesis(genesisContext) {
+  it should "return None for logs containing extra comm events" ignore withGenesis(genesisContext) {


there is a good description for other ignored tests

can we document this one too? why gets it ignored label?

AndriiS-DevBrother · 2026-01-02T14:14:53Z

models/Cargo.toml

 crypto = { path = "../crypto" }
 rspace_plus_plus = { path = "../rspace++" }
 shared = { path = "../shared" }
+pathmap = { git = "https://github.com/Adam-Vandervorst/PathMap", branch = "cleanup_to_release" }


just curious, will we depends on a personal repo?

AndriiS-DevBrother · 2026-01-02T14:19:41Z

rholang/src/test/scala/coop/rchain/rholang/interpreter/RholangOnlyDispatcher.scala

@@ -49,8 +49,7 @@ class RholangOnlyDispatcher[M[_]](implicit s: Sync[M], reducer: Reduce[M])
              case ParBody(parWithRand) =>
                val env     = Dispatch.buildEnv(dataList)
                val randoms = parWithRand.randomState +: dataList.toVector.map(_.randomState)
-                // reducer.eval(parWithRand.body)(env, Blake2b512Random.merge(randoms))
-                s.unit
+                reducer.eval(parWithRand.body)(env, Blake2b512Random.merge(randoms))


AndriiS-DevBrother · 2026-01-02T14:23:57Z

casper/src/test/scala/coop/rchain/casper/util/rholang/RuntimeSpec.scala

      checkpoint <- runtime.createCheckpoint
      expectedHash = Blake2b256Hash.fromHex(
-        "10cce029738696f1e120a6bad4bdf3f18adca25ccf36133bd4916f607a6a50c0"
+        "1180006eedef0a38b891bd588e9aff630bcf6c83cd6b53db0375bb48f922f010"


is that expected? I dont see what input changed, but the expected output get changed

YuriiO-DevBrother

I saw a lot of changes related to Scala code. From this perspective If we still plan to modify Scala sources in the rust/dev and keep the hybrid node in working state I am thinking maybe we need to include CI checks for Scala. At least it should compile and unit tests should pass.

YuriiO-DevBrother · 2026-01-04T11:01:36Z

.github/workflows/build-test-and-deploy.yml


 env:
+  # Enable AES/SSE2 CPU features for gxhash dependency (required by PathMap crate)
+  # GitHub Actions runners support these features on x86_64


What about arm64 target? we are supporting both arm64 and amd64

YuriiO-DevBrother · 2026-01-04T11:04:18Z

node/Cargo.toml

 prost = { workspace = true }
 tonic-prost = { workspace = true }
 colored = "3.0"
+rholang-parser = { git = "https://github.com/F1R3FLY-io/rholang-rs", branch = "f1r3node_dependecies" }


typo in branch name. Is this correct?

spreston8 and others added 30 commits August 7, 2025 16:17

building with new parser in cargo.toml; need to move to nightly channel

63b74e2

created parallel set of new normalizers for new parser

9285f77

merge rust/dev in

ba1abd8

fix MakeMint.rho; interpolation; one rust-toolchain.toml

f7947b1

fix PoSTest.rho

280330e

checking in parallel set of types & errors for new source span and po…

db7fcab

…s; fix string leak

point rholang-rs to forked repo; fix failing tests

8406f66

Merge branch 'rust/dev' into new_parser

cedc563

remove old normalizers

6a4b9b9

remove old parser and related files

11672c9

merge rust/dev in - resolve conflicts; remove leftover old parser files

18c04f9

rename part 1

8eaf64a

rename part 2

3acc709

rename part 3

3aebcf0

rename part 4

3ab4801

rename part 5

24b515d

rename part 6

d1a84d7

update to use latest from rholang-rs

36f51c7

merge rust/dev in

2c25da6

refactor normalize test cases - remove all Box::Leak usage

b686dc8

update rholang-rs crate repo url & branch

3e507a6

Implements the initial PathMap struct and PathMapNode trie design.

76adf99

Implements S-string to byte path encoding logic for PathMap.

a17d3d0

Implements PathMap algebraic methods: union, intersection, subtractio…

349fb4f

…n, restriction, drop_head, all as member functions.

Integrates PathMap into collection normalization and adds constructio…

e16e791

…n from Rholang collection AST (normalizer support).

Adds tests.

b027b19

Fixes compilation.

a6e89b4

Adds Rholang tests. Updates Rholang branch reference.

71d1e70

Adds run-time support of PathMap into Rholang

f731fed

Makes a Rholang sample PathMap application print synchronously

2f181a8

HeorhiiC-DevBrother and others added 20 commits December 5, 2025 16:54

Fixes the RuntimeSpec test suite.

6c58e4d

Fixes the BlockRetrieverSpec test suite.

c37fea8

Fixes the MergeNumberChannelSpec test suite.

4ba0b6e

Removes hanging up while executing coop.rchain.rholang.interpreter.Ev…

3aff812

…alSpec

Fixes the test coop.rchain.rholang.interpreter.CryptoChannelsSpec

d97193d

Fixes the test coop.rchain.rspace.KryoRoundTripTest

95deb6e

Fixes coop.rchain.casper.batch1.MultiParentCasperDeploySpec

036e525

Fixes coop.rchain.casper.util.rholang.InterpreterUtilTest

053f42e

Fixes the test coop.rchain.casper.util.rholang.RuntimeManagerTest

da137cf

Fixes the test coop.rchain.rholang.StackSafetySpec

ba4e7d4

Minor fix.

509cb3a

merge rust/dev in

b2255ae

revert max heap size to 4g from 8g to see if it fixes CI

b66594d

Revert "revert max heap size to 4g from 8g to see if it fixes CI"

9344d2b

This reverts commit b66594d.

fix: Increase SBT stack to 8m for circe-generic-extras macro expansion

2338e1f

Increase SBT_OPTS stack from 2m to 8m to handle the circe-generic-extras macro expansion for PathMap types in the node project.

Merge branch 'rust/dev' of github.com:F1R3FLY-io/f1r3node into new_pa…

3e66b7d

…rser

spreston8 requested review from AndriiS-DevBrother, HeorhiiC-DevBrother, MarcinR-DevBrother, NazarY-DevBrother, SerhiiK-DevBrother, YuriiO-DevBrother and dylon December 31, 2025 00:45

spreston8 marked this pull request as ready for review December 31, 2025 00:46

AndriiS-DevBrother reviewed Jan 2, 2026

View reviewed changes

YuriiO-DevBrother reviewed Jan 5, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

New parser #158

New parser #158

Uh oh!

spreston8 commented Sep 16, 2025

Uh oh!

AndriiS-DevBrother left a comment

Uh oh!

AndriiS-DevBrother Jan 2, 2026

Uh oh!

AndriiS-DevBrother Jan 2, 2026

Uh oh!

AndriiS-DevBrother Jan 2, 2026

Uh oh!

AndriiS-DevBrother Jan 2, 2026

Uh oh!

AndriiS-DevBrother Jan 2, 2026

Uh oh!

AndriiS-DevBrother Jan 2, 2026

Uh oh!

AndriiS-DevBrother Jan 2, 2026

Uh oh!

YuriiO-DevBrother left a comment

Uh oh!

YuriiO-DevBrother Jan 4, 2026

Uh oh!

YuriiO-DevBrother Jan 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

		@@ -1,53 +0,0 @@
		//! This crate provides Rholang language support for the [tree-sitter][] parsing library.

New parser #158

Are you sure you want to change the base?

New parser #158

Uh oh!

Conversation

spreston8 commented Sep 16, 2025

Uh oh!

AndriiS-DevBrother left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

YuriiO-DevBrother left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants