[RFC] Add RFC 3986 and WHATWG compliant URL parsing support #14461

kocsismate · 2024-06-03T20:46:49Z

2nd take after the failed experiment with #11315

RFC: https://wiki.php.net/rfc/url_parsing_api

kocsismate · 2024-06-03T20:48:40Z

ext/standard/basic_functions.stub.php

@@ -3715,7 +3715,6 @@ function uniqid(string $prefix = "", bool $more_entropy = false): string {}

 /**
 * @return int|string|array<string, int|string>|null|false
- * @compile-time-eval


Removed for benchmarking purposes

ext/url/php_url.c

kocsismate · 2024-06-03T20:59:10Z

ext/url/php_url.c

+
+static void cleanup_parser(void)
+{
+	if (++URL_G(urls) % 500 == 0) {


Approach is copy-pasted from lexbor/lexbor#206

ext/url/php_url.c

ext/url/config.m4

kocsismate · 2024-11-18T21:44:05Z

ext/uri/php_uri.c

+			RETURN_THROWS();
+		}
+
+		zend_result result = this_internal_uri->handler->normalize_uri(this_uri);


@nielsdos I would like to ask your opinion about a problem I recently realized.

Context: I added a normalize_uri handler that is able to normalize the URI. It is used in multiple places (e.g. normalize(), toNormalizedString, equalsTo() methods). However, this would possibly cause some unexpected behavior when a built-in URI is extended in userland: if a child class overrides the normalize() method, they would legitimately assume that doing so will affect the rest of the relevant methods (toNormalizedString(), equalsTo()), but no, this won't be the case, since the internal handler is reused in fact... which PHP programmers have no control over. I can see two solutions so far:

make the built-in URI implementations final: there's no more problem with method overrides, but using composition instead of inheritance requires more work (implementing all the methods of the UriInterface), you can't just substitute an internal URI class with your own implementation etc.

the normalize() method itself should be called instead of the normalize_uri internal handler: this solves the issue, but it results in quite some performance overhead

Do you have any preference/insights about this question?

@kocsismate Interesting.

Here's an idea:
We can know upfront if the "normalize" method is overridden when we create the internal uri object. We can do this by checking whether the entry for "normalize" in the function table has type ZEND_USER_FUNCTION instead of ZEND_INTERNAL_FUNCTION. In that case, we know we should call the user handler, otherwise we should call the internal handler.

Create a helper function invoke_normalize_uri (name is just illustrative) that you use to invoke either the internal normalize handler or the user function. Then you can use invoke_normalize_uri everywhere without worrying. To make the invocation fast you can have a zend_function* field in the internal object that is NULL when the user did not override it, and points to the user function if the user did override it. Then you can use the Zend APIs to call that user function if necessary.

@nielsdos Thanks for the brilliant idea! :) That's really impressive. I'll have to think about in which cases a similar issue can happen, but if it's not too frequent, then your solution will perfectly mitigate it. I wouldn't like though if i.e. all the property reader functions would have to use this workaround, but hopefully it won't be the case.

TimWolla

Some comments. Did not look into any details, yet.

TimWolla · 2025-02-16T22:48:10Z

ext/standard/basic_functions.c

@@ -141,6 +141,7 @@ static void user_shutdown_function_dtor(zval *zv);
 static void user_tick_function_dtor(user_tick_function_entry *tick_function_entry);

 static const zend_module_dep standard_deps[] = { /* {{{ */
+	ZEND_MOD_REQUIRED("uri")


If uri is required, it should not require the --enable-uri flag to build it. It should always be built, like ext/random.

Ah yes, good catch!

TimWolla · 2025-02-16T22:49:11Z

ext/soap/tests/bugs/bug38067.phpt

@@ -32,13 +32,13 @@ class TestSoapClient extends SoapClient {

 $client = new TestSoapClient(__DIR__.'/bug38067.wsdl',
    array('encoding' => 'ISO-8859-1'));
-$str = 'test: �';
+$str = 'test: �';


Unintended change? Did you editor treat this file as UTF-8 instead of ISO-8859-1?

Yes, it's surely an editor issue.

TimWolla · 2025-02-16T22:50:15Z

ext/soap/soap.stub.php

@@ -604,7 +604,7 @@ public function __getLastRequestHeaders(): ?string {}
        public function __getLastResponseHeaders(): ?string {}

        /** @tentative-return-type */
-        public function __doRequest(string $request, string $location, string $action, int $version, bool $oneWay = false): ?string {}
+        public function __doRequest(string $request, string $location, string $action, int $version, bool $oneWay = false, ?string $uriParserClass = null): ?string {}


Given the number of tests that you changed, is this a breaking change? If so, it needs to be noted in the RFC.

yes, that's a good idea

TimWolla · 2025-02-16T22:50:38Z

EXTENSIONS

@@ -495,6 +495,12 @@ PRIMARY MAINTAINER:  Andrei Zmievski <[email protected]> (2002 - 2002)
 MAINTENANCE:         Maintained
 STATUS:              Working
 -------------------------------------------------------------------------------
+EXTENSION:           uri
+PRIMARY MAINTAINER   Máté Kocsis <[email protected]> (2024 - 2024)


TimWolla · 2025-02-16T22:51:43Z

ext/uri/config.m4

+    [Enable URI support])])
+
+if test "$PHP_URI" != "no"; then
+    PHP_LEXBOR_CFLAGS="-I$abs_srcdir/ext/dom/lexbor -DLEXBOR_STATIC -I$abs_srcdir/ext/uri/uriparser/include"


This effectively makes ext/dom a dependency of ext/uri. Should Lexbor be moved somewhere else?

It's just a WIP behavior from the early days of the PR. Once I manage to make the RFC approved, I'll create a lexbor "extension" (similarly to mysqlnd).

kocsismate added the Status: Requires RFC label Jun 3, 2024

github-actions bot added Category: Build System Category: Engine Extension: dom Extension: standard labels Jun 3, 2024

kocsismate force-pushed the ext-url2 branch from ee8cc69 to 0e29115 Compare June 3, 2024 20:57

kocsismate commented Jun 3, 2024

View reviewed changes

kocsismate force-pushed the ext-url2 branch 5 times, most recently from 823e4c6 to e65deb7 Compare June 6, 2024 08:23

kocsismate changed the title ~~Add ext/url based on Lexbor~~ [RFC] Add ext/url based on Lexbor Jun 12, 2024

nielsdos reviewed Jun 15, 2024

View reviewed changes

ext/url/php_url.c Outdated Show resolved Hide resolved

bukka reviewed Jul 8, 2024

View reviewed changes

ext/url/config.m4 Outdated Show resolved Hide resolved

kocsismate force-pushed the ext-url2 branch from f897c66 to dcb0299 Compare August 1, 2024 06:30

kocsismate changed the title ~~[RFC] Add ext/url based on Lexbor~~ [RFC] Add RFC 3986 and WHATWG compliant URL parsing support Aug 1, 2024

kocsismate force-pushed the ext-url2 branch 2 times, most recently from 0ccffa0 to 04d4e6e Compare August 13, 2024 12:30

github-actions bot added Extension: openssl Extension: intl Extension: com_dotnet Extension: curl Extension: date Extension: dba Extension: enchant Extension: ffi Extension: fileinfo Extension: filter Extension: ftp labels Aug 13, 2024

kocsismate force-pushed the ext-url2 branch 4 times, most recently from 13e8566 to 8e07430 Compare October 26, 2024 15:35

github-actions bot added the Extension: soap label Nov 11, 2024

kocsismate force-pushed the ext-url2 branch 3 times, most recently from d894dad to 8e21e67 Compare November 18, 2024 21:18

kocsismate commented Nov 18, 2024

View reviewed changes

kocsismate force-pushed the ext-url2 branch 4 times, most recently from d0b9a3d to b9293a2 Compare January 25, 2025 22:42

kocsismate added 11 commits February 6, 2025 08:51

Add RFC 3986 and WHATWG compliant URL parsing support

cf4daa9

Serialization

da878d2

Improve error handling

ea8b733

Lot of fixes and added support for equalsTo()

989e9d4

Add normalization support

4e9b49d

SOAP test fixes

b95e325

Fix some memory leaks

5d96e62

Some cleanups

9c16a83

Changes based on discussion

c8ac763

Removal of Uri\Uri

df943ee

A lot of fixes and API changes

354708e

kocsismate force-pushed the ext-url2 branch from b9293a2 to 20f2844 Compare February 6, 2025 07:52

Updates

0d2d2c3

kocsismate force-pushed the ext-url2 branch from 20f2844 to 0d2d2c3 Compare February 6, 2025 08:06

kocsismate added 2 commits February 10, 2025 22:34

Add new tests, path fixes

ed05911

Add more tests for verifying the behavior of withers

df8c742

TimWolla reviewed Feb 16, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RFC] Add RFC 3986 and WHATWG compliant URL parsing support #14461

[RFC] Add RFC 3986 and WHATWG compliant URL parsing support #14461

kocsismate commented Jun 3, 2024 •

edited

Loading

kocsismate Jun 3, 2024

kocsismate Jun 3, 2024

kocsismate Nov 18, 2024

nielsdos Nov 19, 2024 •

edited

Loading

kocsismate Nov 22, 2024

TimWolla left a comment

TimWolla Feb 16, 2025

kocsismate Feb 18, 2025

TimWolla Feb 16, 2025

kocsismate Feb 18, 2025

TimWolla Feb 16, 2025

kocsismate Feb 18, 2025

TimWolla Feb 16, 2025

TimWolla Feb 16, 2025

kocsismate Feb 18, 2025

[RFC] Add RFC 3986 and WHATWG compliant URL parsing support #14461

Are you sure you want to change the base?

[RFC] Add RFC 3986 and WHATWG compliant URL parsing support #14461

Conversation

kocsismate commented Jun 3, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nielsdos Nov 19, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TimWolla left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kocsismate commented Jun 3, 2024 •

edited

Loading

nielsdos Nov 19, 2024 •

edited

Loading