Add loadAllOfSorted to TransactionManager#3136
Conversation
|
|
||
| private static final FluentLogger logger = FluentLogger.forEnclosingClass(); | ||
| private static final Retrier retrier = new Retrier(new SystemSleeper(), 6); | ||
| private static final Pattern VALID_SORT_FIELD_PATTERN = Pattern.compile("^[a-zA-Z0-9_.]+$"); |
There was a problem hiding this comment.
Add a comment explaining why this exists / what it does.
There was a problem hiding this comment.
Done. Added Javadoc explaining why regex validation is required to prevent JPQL/SQL injection in dynamic ORDER BY clauses.
| * <p>This creates an inner wrapping transaction for convenience, so you don't need to wrap it in | ||
| * a transaction at the call site. | ||
| */ | ||
| public static <T> ImmutableList<T> loadAllOfSorted(Class<T> clazz, String... sortFields) { |
There was a problem hiding this comment.
This is not necessary. You only have a few callsites in tests; it's fine to wrap them there with tm().
Also I don't think you even use this helper method anyway?!
There was a problem hiding this comment.
Done. Removed this unused helper method from DatabaseHelper.
51702fb to
6ed2ea7
Compare
6ed2ea7 to
3fad7b5
Compare
This commit implements the first stage of the Two-PR database schema deployment for registrar opt-in to the Expiry Access Period (XAP), along with updating GEMINI.md to mandate strict adherence to db/README.md for all future database schema modifications. Specifically, it: - Adds Flyway migration V224__add_registrar_expiry_access_period_enabled.sql, which adds the expiry_access_period_enabled boolean column (DEFAULT false NOT NULL) to the Registrar table. - Updates flyway.txt, nomulus.golden.sql, and ER diagrams to reflect the schema changes. - Updates GEMINI.md with explicit instructions for AI agents to always consult db/README.md, follow the mandatory Two-PR deployment split when altering database schemas, and run both :db:test and :core:sqlIntegrationTest whenever database schema or ORM mappings are touched. - Updates golden screenshot ConsoleScreenshotTest_globalRole_registrars_registrarsPage.png to reflect the new default database ordering after adding the column. (Note: A separate pull request, PR google#3136 / http://go/r3pr/3136, permanently fixes this underlying sort non-determinism across our persistence layer by enforcing deterministic alphabetical ordering in Registrar.loadAllSorted). TAG=agy CONV=f2488c74-8b4a-43f1-9c22-d1dddbdbb4e0 BUG=http://b/437398822
This commit implements the first stage of the Two-PR database schema deployment for registrar opt-in to the Expiry Access Period (XAP), along with updating GEMINI.md to mandate strict adherence to db/README.md for all future database schema modifications. Specifically, it: - Adds Flyway migration V224__add_registrar_expiry_access_period_enabled.sql, which adds the expiry_access_period_enabled boolean column (DEFAULT false NOT NULL) to the Registrar table. - Updates flyway.txt, nomulus.golden.sql, and ER diagrams to reflect the schema changes. - Updates GEMINI.md with explicit instructions for AI agents to always consult db/README.md, follow the mandatory Two-PR deployment split when altering database schemas, and run both :db:test and :core:sqlIntegrationTest whenever database schema or ORM mappings are touched. - Updates golden screenshot ConsoleScreenshotTest_globalRole_registrars_registrarsPage.png to reflect the new default database ordering after adding the column. (Note: A separate pull request, PR google#3136 / http://go/r3pr/3136, permanently fixes this underlying sort non-determinism across our persistence layer by enforcing deterministic alphabetical ordering in Registrar.loadAllSorted). TAG=agy CONV=f2488c74-8b4a-43f1-9c22-d1dddbdbb4e0 BUG=http://b/437398822
This commit implements the first stage of the Two-PR database schema deployment for registrar opt-in to the Expiry Access Period (XAP), along with updating GEMINI.md to mandate strict adherence to db/README.md for all future database schema modifications. Specifically, it: - Adds Flyway migration V224__add_registrar_expiry_access_period_enabled.sql, which adds the expiry_access_period_enabled boolean column (DEFAULT false NOT NULL) to the Registrar table. - Updates flyway.txt, nomulus.golden.sql, and ER diagrams to reflect the schema changes. - Updates GEMINI.md with explicit instructions for AI agents to always consult db/README.md, follow the mandatory Two-PR deployment split when altering database schemas, and run both :db:test and :core:sqlIntegrationTest whenever database schema or ORM mappings are touched. - Updates golden screenshot ConsoleScreenshotTest_globalRole_registrars_registrarsPage.png to reflect the new default database ordering after adding the column. (Note: A separate pull request, PR google#3136 / http://go/r3pr/3136, permanently fixes this underlying sort non-determinism across our persistence layer by enforcing deterministic alphabetical ordering in Registrar.loadAllSorted). TAG=agy CONV=f2488c74-8b4a-43f1-9c22-d1dddbdbb4e0 BUG=http://b/437398822
|
We may want to have an allow/forbidden list for clazz here, just in case someone uses it on domain or host. Code quote: Class<T> clazz |
weiminyu
left a comment
There was a problem hiding this comment.
@weiminyu reviewed 9 files and all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on CydeWeys).
CydeWeys
left a comment
There was a problem hiding this comment.
@CydeWeys made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on weiminyu).
core/src/main/java/google/registry/persistence/transaction/JpaTransactionManagerImpl.java line 556 at r2 (raw file):
Previously, weiminyu (Weimin Yu) wrote…
We may want to have an allow/forbidden list for clazz here, just in case someone uses it on domain or host.
loadAllOf() already exists though, and this particular footgun kind of falls into the umbrella category of "You can't really prevent people from writing bad code" (other than the normal code review process that would hopefully catch it). Someone could just as easily still call any of a number of other ways to load all entities from the database that would bypass this restriction on this one helper method, or they could even do something much more dangerous like deleting all entities of a given type.
CydeWeys
left a comment
There was a problem hiding this comment.
PTAL
@CydeWeys made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on weiminyu).
weiminyu
left a comment
There was a problem hiding this comment.
@weiminyu resolved 1 discussion.
Reviewable status:complete! all files reviewed, all discussions resolved (waiting on CydeWeys).
During our implementation of Expiry Access Period (XAP), adding a field to the Registrar entity altered database heap scan order when loadAllOf() was called without an explicit ORDER BY clause. This caused non-deterministic row shifting in the Angular console registrar table (/console/registrars), breaking golden image comparisons in ConsoleScreenshotTest. To permanently fix this root cause and guarantee deterministic UI rendering and test stability across schema evolutions, this commit introduces sorted database loading across our persistence layer. Specifically, this commit: - Adds loadAllOfSorted and loadAllOfSortedStream to TransactionManager, JpaTransactionManagerImpl, and DelegatingReplicaJpaTransactionManager. - Enforces strict allowlist regex validation (^[a-zA-Z0-9_.]+$) on property names in JpaTransactionManagerImpl before appending dynamic ORDER BY clauses, preventing JPQL/SQL injection since bind parameters cannot be used for schema identifiers. - Adds Registrar.loadAllSorted and updates RegistrarsAction to explicitly sort registrars alphabetically by registrarName and registrarId. - Updates golden screenshot ConsoleScreenshotTest_globalRole_registrars_registrarsPage.png to reflect the newly enforced deterministic alphabetical registrar ordering. - Adds comprehensive unit test coverage in JpaTransactionManagerImplTest verifying exact sorted entity retrieval across multiple fields and regex rejection of invalid field names. TAG=agy CONV=f2488c74-8b4a-43f1-9c22-d1dddbdbb4e0 BUG=http://b/531889856
3fad7b5 to
948fe7d
Compare
|
PTAL, had to sync to HEAD. |
During our implementation of Expiry Access Period (XAP), adding a field to the
Registrar entity altered database heap scan order when loadAllOf() was
called without an explicit ORDER BY clause. This caused non-deterministic
row shifting in the Angular console registrar table (/console/registrars),
breaking golden image comparisons in ConsoleScreenshotTest.
To permanently fix this root cause and guarantee deterministic UI rendering
and test stability across schema evolutions, this commit introduces sorted
database loading across our persistence layer.
Specifically, this commit:
JpaTransactionManagerImpl, and DelegatingReplicaJpaTransactionManager.
names in JpaTransactionManagerImpl before appending dynamic ORDER BY
clauses, preventing JPQL/SQL injection since bind parameters cannot be
used for schema identifiers.
sort registrars alphabetically by registrarName and registrarId.
ConsoleScreenshotTest_globalRole_registrars_registrarsPage.png to
reflect the newly enforced deterministic alphabetical registrar ordering.
verifying exact sorted entity retrieval across multiple fields and regex
rejection of invalid field names.
TAG=agy
CONV=f2488c74-8b4a-43f1-9c22-d1dddbdbb4e0
BUG=http://b/531889856
This change is