DamlFuzz: Fuzzer for DAML#52
Conversation
Updated references from 'daml' to 'dpm' in the documentation for the DamlCov and DamlFuzz tools. Added a Go-to-Market strategy section outlining distribution, onboarding, and community evangelism plans. Signed-off-by: Martin Derka <martin@quantstamp.com>
Review Feedback — DamlCov & DamlFuzz (Coverage + Fuzzing Framework)Thank you for the submission and for contributing a proposal focused on improving testing, coverage, and security tooling for Daml on Canton. This is a strong and well-developed proposal that addresses a clear and widely recognized gap in the ecosystem. Coverage tooling and fuzzing are foundational in other smart contract ecosystems, and bringing these capabilities to Daml would significantly improve developer experience and security posture. That said, a few areas would benefit from clarification: 1. Evidence of ecosystem demandThe problem is well articulated, but it would help to clarify:
2. Scope and sequencingThe proposal spans multiple substantial components (coverage engine, fuzzing framework, DSL, engine, benchmarks, maintenance). It would be helpful to:
3. Adoption expectationsIt would be useful to define:
4. Integration with existing workflowsIt would help to clarify:
Overall, this is a high-quality proposal with strong alignment to ecosystem needs. Clarifying demand, sequencing, and adoption pathways would strengthen confidence in impact and delivery. |
Added responses to reviews regarding the test coverage and fuzzing framework proposal, detailing ecosystem demand, scope, adoption expectations, and integration with existing workflows. Signed-off-by: Martin Derka <martin@quantstamp.com>
|
Thank you for the review. I updated the proposal with the responses (see the appendix of the document). I also pasting them below: 1. Evidence of ecosystem demandQuantstamp has conducted multiple security audits of Daml and Canton Network applications. In each engagement, the absence of coverage tooling and automated invariant testing is a recurring friction point, both for our auditors and for the development teams we work with. We regularly report untested code, and we regularly report untested edge cases to Canton Network projects being audited. Audited projects also often seek guidance regarding testing from auditors, which we are unable to provide due to the tooling absence. The 2026 Canton Developer Experience Survey cited in the proposal, where 75% of respondents rated this category of tooling as "Important" or "Critical". We can obtain further evidence through direct testimonials from Canton Network projects to the committee. Both the proposed tools play an important role for agentic development as well. They provide data that agents can use to reason about their code quality, guide to to finding mishandled logic, and delivering more reliable code. The use of AI tooling during development is increasingly popular, making the tools even more valuable ecosystem addition. 2. Scope and sequencingThe delivery sequence already ensures independent value at each stage. The most critical component for immediate ecosystem value is DamlCov (Milestones 1–2). Milestone 2 delivers a complete, production-usable coverage tool. After Milestone 2, a team can run 3. Adoption expectationsThe team will consider the tools successful if they:
4. Integration with existing workflowsIntegration is intentionally low-friction. DamlCov requires no changes to how developers write tests: it wraps DamlFuzz similarly uses only standard Daml Script APIs. Campaigns are expressed in Daml Script and run via The tools will be executable locally, and their structured plain text output can be further used in CI/CD pipelines as is customary for other similar tools. |
monsieurleberre
left a comment
There was a problem hiding this comment.
🙋 Championing
I'd love to champion this one if I qualify — flagging that as a soft volunteer rather than an assumption, since I'm not sure where I sit on the formal champion criteria. Happy to take it on if it's mine to take.
👍 Thumbs ups
I really like this proposal. Coverage and fuzzing have been missing from the Daml toolchain for a long time, and this is a clean plan to fix both.
- Doing the instrumentation at the Daml-LF level makes sense.
- I like that DamlCov is usable on its own after M2 and DamlFuzz from M4 — each half stands up without the other.
- Pointing back to CIP-0013 as something invariant testing would have caught helps make the case land.
❓ Clarifying asks before vote
1. Coordination with DA / other key entities
Has anyone from Digital Asset (or another core Canton contributor) confirmed they aren't already working on filling this gap internally? It would be reassuring for the committee to know this isn't going to land alongside an unannounced parallel effort.
2. M5 — focus vs. deliverables
Milestone 5 (250,000 CC) lists its focus as:
pre-built properties for CIP standards, coverage-guided exploration, ecosystem validation
…but the Deliverables / Value Metrics line is just:
performance optimizations and new benchmarks
Could you reconcile those? Specifically, are the pre-built CIP-0056 invariants (supply conservation, non-negative balances, allocation lock exclusivity — the ones referenced in §3) an explicit M5 deliverable? Those feel like one of the highest-leverage outputs, because every CIP-0056 token implementation would inherit them just by adding DamlFuzz as a dependency.
3. Coverage from outside the Daml test runner
§2 mentions an alternative DamlCov instrumentation approach using:
a dedicated coverage contract that records probe hits as ledger state, queryable after test execution
If you go that route, does it open up coverage from any client driving the ledger — e.g., a service-layer integration test in Java / Node / C# exercising the contracts via the ledger API, with contract-level coverage observable from the service's own test suite? That would be a meaningful additional use case beyond dpm test-driven workflows.
Tied to that: does Daml's privacy model force the coverage contract to be added as an observer on essentially every instrumented template (or the probe-recording party as a divulgee everywhere)? If so, instrumentation isn't just inserting probes — it's a privacy-disclosure change to the source. How do you see that staying ergonomic? Options I can think of:
- a code-mod across the project,
- a wrapper template,
- a special "coverage party" with broad disclosure that's only present in test mode,
- …or something else?
🚀 On adoption
This is exactly the kind of tooling we'd reach for. Hand-authored Daml Script scenarios are what we have today, and the absence of branch-level coverage and automated invariant testing is felt on every project — both when reviewing our own contract logic and when onboarding others to a codebase.
We would be a early adopters of both DamlCov and DamlFuzz and happy to provide feedback as milestones land.
|
|
||
| Today, `daml test --show-coverage` reports only which templates were created and which choices were exercised. It tells developers nothing about which branches within a choice body were taken, which guard conditions were tested, or which error paths remain unexplored. Unit testing of Daml code relies on developers writing specific scenarios in Daml Script and asserting expected outcomes. There is no way to automatically generate random sequences of ledger operations, no way to define system-wide invariants, and no way to explore the contract state space beyond the scenarios a developer manually anticipates. Developers are used to tooling that is mature and readily available for languages and development frameworks in other ecosystems (c.f. the Foundry tools for Ethereum), and its absence provides an inferior experience for developers of applications on Canton Network. | ||
|
|
||
| The proposed test coverage tool will measure line, branch, and expression test coverage. The fuzzing tool will provision the first property-based testing and coverage-guided fuzzing framework for Daml smart contracts while being designed from the ground up for Daml's unique authorization model, UTXO-like state, and multi-party privacy semantics. Both tools will be developed as open-source, free to use, executable locally, will integrate with the current Daml tooling, and will provide reports usable in CI/CD pipelines. |
There was a problem hiding this comment.
@mderka please break this into two proposals and not a single one. You may find that there is great support for property-based testing but the code coverage is a lower priority.
There was a problem hiding this comment.
Opened #323 for coverage; this PR is now for the property-based testing only
- Removed test coverage (separate PR) - Added deliverable to M3 - Added total funding - Adjusted allocation to M3 - Clarified support
|
@hrischuk-da Thank you for the feedback. I separated the proposal into two parts. I will keep this PR dedicated to the fuzzer proposal to retain the discussion, and will open a new one (PR#323) for the test coverage tool. @monsieurleberre Thank you for the feedback, and the champion support. Please see the comment above for the proposal being separated into two parts. W.r.t. the fuzzer, I explicitly listed the properties for CIP-0059 in the deliverables. It was an omission and they are to be included. The rest of your feedback applies to DamlCov and is addressed in PR#323 (you're tagged there). We also received feedback that the M4 support milestone could use more clarifications, and that M3 funding allocation appears large compared to the other milestones. I clarified the deliverables and scope for M4, reduced the amount to reflect the reduction in scope (this proposal now contains the fuzzer only), and also reduced the funding allocation to M3. |
Development Fund Proposal Submission
Proposal file:
proposals/fuzzing-framework-damlfuzz.md
Summary
We propose developing a fuzzer for Daml smart contracts. The fuzzing tool will provision the a property-based testing and coverage-guided fuzzing framework for Daml smart contracts while being designed from the ground up for Daml's unique authorization model, UTXO-like state, and multi-party privacy semantics. It will be developed as open-source, free to use, executable locally, will integrate with the current Daml tooling, and will provide reports usable in CI/CD pipelines.
Checklist
/proposals/Notes for Reviewers
N/a