Porting StreamDevice to libpcre2

[jodreyer]

Following a discussion on the EPICS mailing list, to keep track on the progress of the topic:

I faced today the fact that on my system (OpenSuSE Tumbleweed) there is no libpcre anymore. It got replaced completely by its successor libpcre2.
I came across that when trying to recompile my EPICS installation. It failed when building StreamDevice.

Now there are three options how to solve that:

1) Install the old libpcre 8.45 from source by hand in the system. Might still work for some time, but what when not? It has not been updated since 2021
and might contain bugs and security issues that will not be fixed.

2) Install libpcre as part of StreamDevice. It faces the same problems as above.

3) Port StreamDevice to libpcre2. That would make it future prove and more easy to maintain.

For me #1 is only a short term solution. For now only my OpenSuSE system is affected and I can build a package for that, but what when systems like Debian/Ubuntu arfe affected?
As I use E3 to deploy my EPICS installations now, I could setup an installation of libpcre for that, but it would require to fork the e3-StreamDevice wrapper from ESS
to add the pcre module dependency. I would prefer not to do that as it could make things incompatible at some time.
So this leaves #3 as the optimal solution an I would like to know if somebody has already looked into that or is actively working on it.

I took a quick look into that issue and found out that the API of libpcre2 has significantly changed.
E.g. pcre is replaced by pcre2_code. Any function that returns a pcre2_code structure has to be matched by a pcre2_free_code call
to prevent memory leaks. Some functions like pcre_infofull or pcre_exec have been replaced by others with different name and parameter list.

Jörn

[dirk-zimoch]

I took a quick look into that issue and found out that the API of libpcre2 has significantly changed.

Welcome to the club. 😉 That’s one of the reasons why I have not migrated yet.
I am still thinking of solution 2 too, in particular for cross architectures like VxWorks, RTEMS and Embedded Linux, where I cannot expect a pcre or pcre2 system library.

Aiming for the 8bit implementation seems good enough for the beginning. It is definitely no step backwards from the old pcre.

[jodreyer]

For VxWorks and RTEMS I agree that inclusion is the only option. For Embedded Linux, which is probably mostly build based on Yocto, I think that most projects build their own system. And then one could make pcre a requirement.

But this still does not solve the problem with future compilers /libc versions which might break the build.
For exampel I had already to add two more CPPFLAGS (-DPARENS_NEST_LIMIT=250 -DSUPPORT_PCRE32) to be able to build pcre.

I implemented all the changes necessary but do not have a device at hand currently to test it.

[dirk-zimoch]

I am trying to support both, pcre2 and the older pcre. Is not so difficult as I has expected but requires some preprocessor branches on the pcre version. I have decided to only support the 8 bit version of pcre2. Tests look fine so far. I am re-working the build system a bit in order to auto-detect the presence of pcre2 (and as a fallback pcre) on the build system. Still, for cross architectures, the user probably needs to specify that manually.

Also I am testing to build pcre2 or pcre from the sources inside StreamDevice for architectures where no system library is installed.
Updating all relevant documentation, too....

[daykin]

Here's a rough draft we're using at FRIB to at least get things that need stream to build on Debian 13 (which only ships with pcre2 now). all I can confirm is that it compiles, passes the tests and works with streamApp/regexp.cmd. Haven't done any more comprehensive testing.

https://github.com/daykin/StreamDevice/tree/pcre2

[dirk-zimoch]

Thanks. I will have a look but it may take a while until after summer vacation.

[wtup]

Hi, I'm interested in the status of this issue? Is the pcre2 port missing a review or does it need more extensive testing before merging?

[dirk-zimoch]

I have modified this to support both, pcre 1 and pcre2. But I am still working on an attempt to auto-detect which versions are available on the host system, so that the user does not need to bother but just install the required devel packages. Cross-compilation may be another can of worms, in particular for operating systems where no native pcre support is available (e.g. VxWorks). I am thinking of buildingpcre2 as a sub-module automatically.

[wtup]

Got it, thanks.
Are you working on a local branch? I cannot see any commits on this repo or your fork.

[dirk-zimoch]

Sure. I am working on the PSI internal branch. This is just a fork.

[dirk-zimoch]

I have now pushed my pcre2 branch (excluding local changes for auto-detection).

[wtup]

Thank you very much!

[mdavidsaver]

I am trying to support both, pcre2 and the older pcre. ...

Maybe a silly question, but in 2026 is it necessary to support both versions? pcre2 is not exactly a recent addition.

Here's a rough draft we're using at FRIB ...

I have now pushed my pcre2 branch ...

@dirk-zimoch @daykin Could you create (draft) pull requests? This would help to communicate which branch in which repository can be reviewed/tested.

[mdavidsaver]

This came up for me today when I realized that Debian 13 no longer packages pcre3 (the older one), which prevents me from building streamdevice or epics-extensions/ca-gateway#66.

[dirk-zimoch]

Could you create (draft) pull requests?

I was still working on auto-detection of pcre/pcre2. The tricky part is to find it for the cross architectures. But I got distracted by other stuff. Please refer to the pcre2 branch for the time being.

And yes, I have numerous old embedded Linux systems without pcre2.