Skip to content

Commit eca8a89

Browse files
Ryan McKennacopybara-github
authored andcommitted
Clarify library scope and design philosophy in top-level README.md
PiperOrigin-RevId: 989871809
1 parent 0c3c170 commit eca8a89

2 files changed

Lines changed: 117 additions & 20 deletions

File tree

‎CONTRIBUTING.md‎

Lines changed: 24 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -89,20 +89,30 @@ for new standalone additions.
8989

9090
When reviewing contributions, we primarily look for two things:
9191

92-
### 1. Simplicity
93-
94-
Simpler contributions with smaller diffs are more likely to be accepted. For new
95-
mechanisms in particular:
96-
97-
- **Aim for a single file** with at most ~500 lines of code.
98-
- You don't need to use the same internal helper functions that existing
99-
mechanisms use. If your approach is novel, a self-contained implementation
100-
is perfectly fine.
101-
- However, your mechanism **should conform to the same API contract** as
102-
existing mechanisms (e.g., implementing `DPMechanism`, accepting the same
103-
calibration/configuration interface). The exception is mechanisms designed
104-
for a new data modality (e.g., relational data), where a new API surface may
105-
be necessary.
92+
### 1. Clean Mechanism Boundaries and Simplicity
93+
94+
For new mechanisms and subpackages, we look for a clean separation between a
95+
lightweight outer contract and a flexible internal implementation:
96+
97+
- **Implement `MechanismConfig` / `CalibratedMechanism`**: Conform to the two
98+
core abstractions in
99+
[`docs/mechanism_api.md`](docs/mechanism_api.md)
100+
(`MechanismConfig.configure` / `calibrate` returning a `CalibratedMechanism`
101+
with `dp_event` and `__call__`). This contract is lightweight and
102+
modality-agnostic (`domain` and `data` can be whatever types your modality
103+
requires) while ensuring consistent privacy calibration and accounting
104+
across the library.
105+
- **Curate the public API surface**: Be deliberate and thoughtful about what
106+
is exported in the public API and what the API contract is. Do not expose
107+
internal helper functions or implementation details to the top-level core
108+
library.
109+
- **Flexible internal implementation**: You do not need to use the same
110+
internal helper functions or data structures that existing mechanisms use.
111+
Because end users only depend on the curated outer API, your internal
112+
implementation details and code organization can be completely
113+
self-contained and change freely over time. Single-file mechanisms (~500
114+
lines) are welcome for focused algorithms, while larger modality-specific
115+
mechanisms can live in their own subpackage directory.
106116

107117
### 2. No New Heavy Dependencies
108118

‎README.md‎

Lines changed: 93 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
1-
# DPSynth: Differentially Private Synthetic Tabular Data
1+
# DPSynth: Differentially Private Synthetic Data Generation
22

33
[![Documentation](https://readthedocs.org/projects/dpsynth/badge/?version=latest)](https://dpsynth.readthedocs.io/en/latest/)
44

5-
DPSynth is a library for differentially private synthetic tabular data
6-
generation. Given a sensitive dataset of records defined w.r.t. a single-table
7-
schema, our library can generate a synthetic version of the dataset, preserving
8-
the structure and statistical properties of the source data while satisfying
9-
differential privacy.
5+
DPSynth is a library housing mechanisms for differentially private (DP)
6+
synthetic data generation across a wide range of data modalities. Given a
7+
sensitive dataset, our library can generate a synthetic version of the dataset,
8+
preserving the structure and statistical properties of the source data while
9+
satisfying differential privacy.
1010

1111
📖 **[Full documentation on ReadTheDocs](https://dpsynth.readthedocs.io/en/latest/)**
1212

@@ -30,6 +30,93 @@ differential privacy.
3030
> Your feedback directly shapes the library's direction. Thank you for your
3131
> patience as we build toward a stable release!
3232
33+
## Scope and Design Philosophy
34+
35+
DPSynth is a Python library for **differentially private synthetic data
36+
generation**. This scope is deliberately inclusive and does not mean just
37+
"tabular" data or "text" data. Methods for **any data modality** are in
38+
principle in scope for the library, and community contributions for new
39+
modalities and mechanisms are welcome.
40+
41+
Unlike [JAX Privacy](https://github.com/google-deepmind/jax_privacy), which
42+
focuses on lower-level DP-SGD ingredients, DPSynth implements a wide variety of
43+
DP mechanisms that naturally compose and build on top of each other:
44+
45+
* **Supported Modalities**:
46+
* **Discrete**: Low-level discrete, integer-encoded datasets via
47+
marginal-based graphical models
48+
([`DiscreteConfig`](dpsynth/discrete_mechanisms/discrete.py),
49+
[`discrete_mechanisms/`](dpsynth/discrete_mechanisms/README.md)).
50+
* **Tabular (Single-Table)**: Categorical, numerical, and open-set
51+
attributes ([`TabularConfig`](dpsynth/data_generation_v3.py)).
52+
* **Relational (Multi-Table)**: Multi-table databases with foreign-key
53+
hierarchies and referential integrity
54+
([`dpsynth.relational`](dpsynth/relational/synthesizer.py)).
55+
* **Nested**: Hierarchically structured and typed records with shared and
56+
per-type schemas
57+
([`dpsynth.experimental.nested`](dpsynth/experimental/nested.py)).
58+
* **Sequence**: Event sequences, trajectories, and user session logs.
59+
* **Text**: Unstructured text and prompt/response pairs via DP language
60+
model fine-tuning and conditional generation
61+
([`dpsynth.text`](dpsynth/text/dp_sft.py)).
62+
* **Why Co-Development Matters**: State-of-the-art mechanisms for text data
63+
(such as [ACTG](https://arxiv.org/abs/2510.18232) and
64+
[MAPLE](https://arxiv.org/abs/2603.19258)) depend directly on
65+
state-of-the-art primitives for tabular data (for example, privately
66+
synthesizing structured metadata or summary attributes first and generating
67+
text conditioned on those attributes). Similarly, novel mechanisms for new
68+
modalities (including [nested](dpsynth/experimental/nested.py),
69+
[relational](https://arxiv.org/abs/2503.22970), sequence, and text data)
70+
can often be defined by reducing the problem to a tabular synthetic data
71+
problem. Having a robust library of tabular primitives makes it easy for
72+
other modalities to leverage them directly.
73+
* **Built for Heterogeneous, Mixed-Modality Data**: Many real-world datasets
74+
are *mixed* (for example, combining structured tabular columns with
75+
free-form text). DPSynth has been designed *from the ground up* to support
76+
such heterogeneous use cases, from how the
77+
[`MechanismConfig` / `CalibratedMechanism`](docs/mechanism_api.md)
78+
API is curated to how tight privacy accounting and budget calibration are
79+
performed under the hood across composed sub-mechanisms.
80+
* **Library Code vs. Binaries and Infrastructure**: As a library, DPSynth
81+
consists of importable Python code and does not house deployment binaries,
82+
experiment orchestration, or surrounding serving infrastructure in the core
83+
package. Instead, we maintain a dedicated [`examples/`](examples/) directory
84+
with runnable scripts and binaries that demonstrate how to use the library.
85+
86+
### Mechanism Architecture and Requirements
87+
88+
To keep the library cohesive across diverse modalities without constraining how
89+
individual algorithms are built, mechanisms in DPSynth follow three core
90+
principles:
91+
92+
1. **Implement the `MechanismConfig` / `CalibratedMechanism` API**: Every
93+
mechanism is structured around two lightweight abstractions (detailed in
94+
[`docs/mechanism_api.md`](docs/mechanism_api.md)):
95+
* `MechanismConfig`: A serializable, frozen configuration recipe that
96+
defines `configure(domain, zcdp_rho=...)` and
97+
`calibrate(domain, epsilon=..., delta=...)` to bind a domain and privacy
98+
budget and return a `CalibratedMechanism`.
99+
* `CalibratedMechanism`: An immutable, runnable instance exposing
100+
`dp_event` (an exact characterization of privacy properties via
101+
`dp_accounting`) and `__call__(rng, data)` to execute on sensitive data.
102+
* Because this contract places no restrictions on the types of `domain` or
103+
`data`, it is lightweight, modality-agnostic, and a natural fit for any
104+
DP mechanism: it cleanly separates configuration from execution and
105+
enables automatic privacy calibration and composition across mechanisms.
106+
2. **Provide a Carefully Curated Public API**: Mechanisms should expose a
107+
deliberate, minimal public surface (typically the config class, calibrated
108+
mechanism class, and any required domain/schema types) with a clear API
109+
contract, rather than exporting internal helpers or intermediate utilities
110+
to the top-level library.
111+
3. **Complete Flexibility of Internal Implementation**: Because end users and
112+
downstream pipelines depend only on the curated outer API contract, a
113+
mechanism's internal implementation details, modeling abstractions, and file
114+
organization are completely flexible and can evolve freely over time without
115+
breaking callers. If you have a novel synthetic data mechanism for any
116+
modality, consider contributing it upstream by
117+
[opening an issue](https://github.com/google/dpsynth/issues) to discuss the
118+
design (see [`CONTRIBUTING.md`](CONTRIBUTING.md)).
119+
33120
## Two Code Paths
34121

35122
DPSynth contains **two independent implementations** of differentially private

0 commit comments

Comments
 (0)