[Python Functions] consumerProperties, poolMessages and messagePayloadProcessorSpec are ignored

[david-streamlio]

Master Issue: #26412

Search before reporting

• I searched in the issues and found nothing similar.

Found while auditing the three function runtimes against Function.proto for #26404, which tracks the equivalent gaps in the Go runtime. Filed separately because this is a different runtime and a different fix.

Motivation

The Python runtime applies receiverQueueSize, schemaProperties, serdeClassName, schemaType and cryptoSpec from ConsumerSpec, but not these three:

map<string, string> consumerProperties = 6;
bool poolMessages = 8;
MessagePayloadProcessorSpec messagePayloadProcessorSpec = 9;

grep -n "consumerProperties\|poolMessages\|messagePayloadProcessor" python_instance.py contextimpl.py returns nothing (Function_pb2.py excluded, since the generated protobuf contains every field name and makes a naive search report full support).

consumerProperties is the one most likely to be noticed: the properties are set on the subscription and read by broker-side tooling, so a function configured with them appears in pulsar-admin topics stats without them. poolMessages leaves a memory optimisation unavailable. messagePayloadProcessorSpec is a Java-centric extension point and may have no reasonable Python equivalent.

The Java runtime applies all three.

Solution

Pass them through in setup_consumer, where the existing per-field handling lives:

• consumerProperties → merge into the properties dict already passed to subscribe(). Worth deciding whether these merge with or override the runtime's own __pfn_* properties; the runtime's should win.
• poolMessages → confirm whether the Python client exposes an equivalent before promising it.
• messagePayloadProcessorSpec → likely out of scope; if so, saying that explicitly is better than continuing to ignore it silently.

Alternatives

Leaving messagePayloadProcessorSpec unimplemented is reasonable. Silently ignoring consumerProperties is not, since it is a plain string map with an obvious Python equivalent.

Anything else?

Verified against origin/master.

Are you willing to submit a PR?

• I'm willing to submit a PR!