Max PKCS#11 id URI length of 100 characters?

[tdbhacks]

Hello,

Apologies if this has been asked already, I did a quick search online but couldn't find any references to "100 characters" or other PKCS#11 URI length limits, though I might have missed a doc somewhere.

Our PKCS#11 library (https://github.com/GoogleCloudPlatform/kms-integrations) uses relatively long key IDs, and I just ran into a surprising failure while trying to generate a self-signed certificate:

$ openssl req -new -x509 -days 3650 -subj '/CN=test/' -sha256 -engine pkcs11 -keyform engine -key pkcs11:id=some_id_longer_than_100_characters > my-test.crt

Engine "pkcs11" set.
The private key ID is not a valid PKCS#11 URI
The PKCS#11 URI format is defined by RFC7512
The private key ID is not a valid PKCS#11 URI
The PKCS#11 URI format is defined by RFC7512
The private key was not found at: pkcs11:id=some_id_longer_than_100_characters
PKCS11_get_private_key returned NULL
Could not read private key from org.openssl.engine:pkcs11:pkcs11:id=some_id_longer_than_100_characters
40D7D791067F0000:error:40000064:pkcs11 engine:ERR_ENG_error:invalid id:eng_back.c:400:
40D7D791067F0000:error:13000080:engine routines:ENGINE_load_private_key:failed loading private key:../crypto/engine/eng_pkey.c:79:
Aborted

"some_id_longer_than_100_characters" has been redacted, but the same behavior can also be reproduced with something simple such as 100 "a"s.

My questions:

• is this intended?
• is the limit documented anywhere?

I opened an issue in the OpenSSL repo as well, but they rightfully pointed me here. Thank you!

[richardkazuomiller]

Is anyone following this? I'd also like to know if this is intentional behavior or not? 😅

[mtrojnar]

I reviewed the current code of both OpenSSL and libp11 for this issue, but I failed to find a cause. Was anyone able to reproduce this issue? Which version of the libraries were tested? Has anyone tested it with a PKCS#11 module other than libkmsp11.so?

[tdbhacks]

Forgive me if this is silly and I'm missing something, but I think the libkmsp11.so library (or any library) might not even be needed to reproduce the issue. I tried unsetting PKCS11_MODULE_PATH to make sure it did not point to the library, and ran the command below:

openssl req -new -x509 -days 3650 -subj '/CN=test/' -sha256 -engine pkcs11   -keyform engine -key pkcs11:id=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa > ~/test.txt

which results in the same:

Engine "pkcs11" set.
The private key ID is not a valid PKCS#11 URI
The PKCS#11 URI format is defined by RFC7512
The private key ID is not a valid PKCS#11 URI
The PKCS#11 URI format is defined by RFC7512
The private key was not found at: pkcs11:id=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
PKCS11_get_private_key returned NULL
Could not find private key from org.openssl.engine:pkcs11:pkcs11:id=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
400619208E7F0000:error:40000064:pkcs11 engine:ERR_ENG_error:invalid id:eng_back.c:400:
400619208E7F0000:error:13000080:engine routines:ENGINE_load_private_key:failed loading private key:../crypto/engine/eng_pkey.c:79:

I think this suggests that the issue happens somewhere in the parsing code, before getting to the library, right?

Let me know if this is wrong, I also tried going through the underlying libp11 code without much luck, though the parsing code is fairly convoluted. Definitely interested in making sure this is not a library-side issue, the original author of our library is not on the team anymore so who knows if I missed a weird implementation quirk.

[richardkazuomiller]

@tdbhacks does the ID not need to be percent-encoded? As in %61%61%61 or something.

[tdbhacks]

@tdbhacks does the ID not need to be percent-encoded? As in %61%61%61 or something.

Not sure TBH, but IDs shorter than 100 chars work without percent-encoding so that would be interesting

[richardkazuomiller]

@tdbhacks I actually got a different result when running that without PKCS1_MODULE_PATH set.

$ openssl req -new -x509 -days 3650 -subj '/CN=test/' -sha256 -engine pkcs11   -keyform engine -key pkcs11:id=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa > ~/test.txt
Engine "pkcs11" set.
Failed to enumerate slots
Failed to enumerate slots
PKCS11_get_private_key returned NULL
Could not read private key from org.openssl.engine:pkcs11:pkcs11:id=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
409703C6267C0000:error:40000065:pkcs11 engine:ERR_ENG_error:object not found:eng_back.c:975:
409703C6267C0000:error:13000080:engine routines:ENGINE_load_private_key:failed loading private key:../crypto/engine/eng_pkey.c:79:

The above looks like it is failing successfully.

However, when I ran export PKCS11_MODULE_PATH="/path/to/libkmsp11/libkmsp11.so" I did get the not a valid PKCS#11 URI output. Are you positive the environment variable was not set?

[tdbhacks]

@richardkazuomiller did you use unset PKCS11_MODULE_PATH to clear it completely? That should trigger openssl to use its default pkcs#11 library, which gets me the invalid URI error.

If unsetting the env var with something like export PKCS11_MODULE_PATH="", then I don't see the error because I think openssl is just unable to get to the point where it parses the ID.

Double-check me on this:

• your "Failed to enumerate slots" error is here
• following back the trail, I think that's part of ctx_init_libp11_unlocked() which tries to enumerate the slots among other things. This function is called on line 600 of ctx_load_object()
• the RFC URI error is here
• backtracking just once to see where ctx_try_load_object() is called, I get to line 608 (or 615) of ctx_load_object()

So I think the enumeration error would show up before the URI error, and just indicates an earlier failure.

[richardkazuomiller]

@tdbhacks I have not added PKCS11_MODULE_PATH to my global environment variables anyway so there's nothing to unset actually. I ran env | grep PK to double check and got nothing back.

I think you're probably right that the Failed to enumerate slots error throws before the ID is parsed but I don't know how to get past that without ... well, giving it some slots to enumerate 😅. Do you have any other configuration that the library could be referencing to move past ctx_enumerate_slots_unlocked?

[dengert]

Is anyone following this? I'd also like to know if this is intentional behavior or not?

Yes but on vacation.
URI is defined here: https://www.rfc-editor.org/rfc/rfc7512.html

The error may be from any level just listing the URI but not saying what actually failed.
Some modules may expect the caller to provide more options.

One way to see PKCS11 calls and responses is to use OpenSC SPY
https://github.com/OpenSC/OpenSC/wiki/Using-OpenSC