Search results different if you use uppercase vs lowercase

[Pixney-William]

Bug Description

We have a client in Russia who has a website with English and Russian content.

When searching for : Киношкола which is the exact spelling as киношкола we get different search results.
Page titles that has Киношкола with the first letter in uppercase is found, but if you search for content with the first letter being in lowercase, no results are found.

How to Reproduce

Create a page with title :
Киношкола «Печенгафильм» в открытой студии «Арктик-ТВ» и радио «RECORD»

Then try search for it

Extra Detail

Environment

Statamic 3.0.45 Pro
Laravel 7.30.4
PHP 7.4.13
doublethreedigital/duplicator 1.2.3

Install method (choose one):

• Fresh install from statamic/statamic

[jasonvarga]

As a workaround, I would suggest using Algolia. https://statamic.dev/search#algolia

Case doesn't seem to matter when using the English alphabet.

[jasonvarga]

I took a quick look - adding u modifiers to these regexes seems to fix it, but I don't know what other effects that would have.

cms/src/Search/Comb/Comb.php

Lines 496 to 498 in 4df11ff

	'whole' => '#^'.$escaped_chunk.'$#i',
	'partial' => '#'.$escaped_chunk.'#i',
	'partial_from_start' => '#^'.$escaped_chunk.'#i',

[Pixney-William]

Thanks. The project budget is kind of an issue for Algolia implementation. But i guess in a worst case scenario it is something i have to go with.

I'll try to find if there are any solutions out there and get back.

[Pixney-William]

Also, i guess Algolia would work for the front end as a solution. But it would not work as a solution for search within the CMS admin area.

[jasonvarga]

Algolia has a free tier.
You can use it for the CP search. I wouldn't have suggested it otherwise.

[Pixney-William]

Ah, if it works for the cp then maybe thats a workaround.

[jeremyvienney]

@Pixney-William I get a workaround with customizing the search index.
You could add a transformers array https://statamic.dev/search#transforming-fields, for exemple I used this with lowercase and stripping accents to handle all situations :

use CodeInc\StripAccents\StripAccents;
...
...
'title' => function ($title) {
                    if ($title) {
                        $name2Search = $title . ' ' . StripAccents::strip($title) . ' ' .strtolower(StripAccents::strip($title));
                    }
                    return $title ? $name2Search : false;
                },

[jonassiewertsen]

We are running into the same issue.

Searching for Økotopen does return results, but økotopen does not return any.

[ryanmitchell]

This should have been fixed by #11363