Skip to content

Improve handling of filter values containing "+" and "%" - #143

Merged
apdavison merged 1 commit into
HumanBrainProject:masterfrom
apdavison:remove-plus-workaround
Sep 15, 2026
Merged

apdavison merged 1 commit into
HumanBrainProject:masterfrom
apdavison:remove-plus-workaround

Conversation

@apdavison

Copy link
Copy Markdown
Member

In the master branch, get_filter_value() truncates string filter values at the first "+" (or raised ValueError if it appeared in the first three characters), to work around a KG bug affecting filter values sent as request parameters for stored queries.

This change removes the workaround, since fairgraph no longer uses stored queries, it writes fresh queries each time that include the filter values, and in this case filters containing "+" are handled correctly. This means that we can now match values such as "C++", email addresses with a "+" suffix, or timestamps with a UTC offset.

This also removes the exemptions for date/datetime properties and for Regex values, which existed only to bypass the workaround, and replaces the Regex-specific test (and the timezone-aware datetime test) with a single test that filter values containing "+" are passed through unchanged.

The underlying bug still affects filter values sent as request parameters: the KG decodes parameter values twice, once by Spring and again in DataQueryBuilder.createAqlForFilter() in marmotgraph-core, so a "+" is received as a space and a "%" causes an error or is decoded together with the following characters. The bug is absent from the v4 branch of marmotgraph-core. fairgraph no longer passes filter values as parameters, but KGClient.query() still accepts a filter dict, so it now raises ValueError if a filter value contains "+" or "%", rather than silently returning the wrong results. A live test checks the current KG behaviour, so that we will know when v4 has been deployed and this check can be removed.

In the master branch, get_filter_value() truncates string filter values at the first "+" (or raised ValueError if it appeared in the first three characters), to work around a KG bug affecting filter values sent as request parameters for stored queries.

This change removes the workaround since fairgraph no longer uses stored queries, it writes fresh queries each time that include the filter values, and in this case filters containing "+" are handled correctly. This means that we can now match values such as "C++", email addresses with a "+" suffix, or timestamps with a UTC offset.

This also removes the exemptions for date/datetime properties and for Regex values, which existed only to bypass the workaround, and replaces the Regex-specific test (and the timezone-aware datetime test) with a single test that filter values containing "+" are passed through unchanged.

The underlying bug still affects filter values sent as request parameters: the KG decodes parameter values twice, once by Spring and again in DataQueryBuilder.createAqlForFilter() in marmotgraph-core, so a "+" is received as a space and a "%" causes an error or is decoded together with the following characters. The bug is absent from the v4 branch of marmotgraph-core. fairgraph no longer passes filter values as parameters, but KGClient.query() still accepts a `filter` dict, so it now raises ValueError if a filter value contains "+" or "%", rather than silently returning the wrong results. A live test checks the current KG behaviour, so that we will know when v4 has been deployed and this check can be removed.
@apdavison apdavison added this to the 0.15 milestone Sep 15, 2026
@apdavison apdavison added the enhancement New feature or request label Sep 15, 2026
@apdavison
apdavison merged commit 71fecb5 into HumanBrainProject:master Sep 15, 2026
6 checks passed
@apdavison
apdavison deleted the remove-plus-workaround branch September 15, 2026 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant