Match failure with character class containing multiple Unicode codepoints

[ranvis]

Version: 10.48-DEV (2025-10-21) 3b91977
Built with cmake -DPCRE2_BUILD_PCRE2_32=ON on Windows

pcre2test -32

  re> /[\x{ff}\x{100}\x{8000}\x{8002}\x{8004}\x{8006}]/utf8_input
data> \x{100}
 0: \x{100}

  re> /[\x{ff}\x{100}\x{8000}\x{8002}\x{8004}\x{8006}\x{8008}]/utf8_input
data> \x{100}
No match

Expected behavior:
The second pattern should behave the same as the first, since the only difference
is the addition of another distinct codepoint \x{8008}.

Notes:
This issue is reproducible with PCRE2 10.47, 10.46 (called from PHP 8.5.0 RC3 offical Windows build) and 10.45
while 10.44 (called from PHP 8.4) is unaffected.

pcre2test -C
PCRE2 version 10.48-DEV 2025-10-21
Compiled with
  8-bit support
  32-bit support
  UTF and UCP support (Unicode version 16.0.0)
  No just-in-time compiler support
  Default newline sequence is LF
  \R matches all Unicode newlines
  \C is supported
  Internal link size
    Requested = 2
    Effective = 2
  Parentheses nest limit = 250
  Default heap limit = 20000000 kibibytes
  Default match limit = 10000000
  Default depth limit = 10000000
  pcre2test has libreadline support

[NWilson]

You are not using UTF-8 mode. The input is interpreted as single-byte encoding (eg Latin-1).

The UTF-8 for \x{100} is 0xC4 0x80, and the UTF-8 for \x{8008} is 0xE8 0x80 0x88. As you can see, both of these include 0x80.

So, if you aren't running PCRE2 in UTF mode, then the byte string does indeed match the character class.

---

Oh! Forget that... You're using 32-bit mode. OK, will need to re-think...

[ranvis]

It seems to me that the issue occurs during pattern compilation rather than matching.
When dumping with fullbincode, the compiled data differs depending on whether \x{100} is present.

  re> /[\x{ff}\x{100}\x{8000}\x{8002}\x{8004}\x{8006}]/utf8_input,fullbincode
------------------------------------------------------------------
  0  24 Bra
  2     [\xff\x{100}\x{8000}\x{8002}\x{8004}\x{8006}]
 24  24 Ket
 26     End
------------------------------------------------------------------

Adding \x{8008} to the pattern (which, I suppose, enables range lists) results in \x{100} disappearing from the compiled class:

  re> /[\x{ff}\x{100}\x{8000}\x{8002}\x{8004}\x{8006}\x{8008}]/utf8_input,fullbincode
------------------------------------------------------------------
  0  15 Bra
  2     [\xff\x{8000}\x{8002}\x{8004}\x{8006}\x{8008}]
 15  15 Ket
 17     End
------------------------------------------------------------------

Replacing \x{100} with \x{101} generates the expected code:

  re> /[\x{ff}\x{101}\x{8000}\x{8002}\x{8004}\x{8006}\x{8008}]/utf8_input,fullbincode
------------------------------------------------------------------
  0  15 Bra
  2     [\xff\x{101}\x{8000}\x{8002}\x{8004}\x{8006}\x{8008}]
 15  15 Ket
 17     End
------------------------------------------------------------------
data> \x{101}
 0: \x{101}

From what I can tell, it looks like the compiler incorrectly drops \x{100} when constructing range lists from certain codepoints.

[NWilson]

Thank you, that's a helpful investigation.

I will look into this further and fix it, when I have the time.

[zherczeg]

#841 should fix this

[ranvis]

Confirmed that the PR resolves the issue for both the reproducer and the original pattern I encountered.
Thank you!