Skip to content

[Go Functions] forwardSourceMessageProperty is not implemented, so source message properties are dropped #26408

Description

@david-streamlio

Master Issue: #26404

Search before reporting

  • I searched in the issues and found nothing similar.

Motivation

SinkSpec.forwardSourceMessageProperty tells the runtime to copy the input message's properties onto the output message. The Go runtime does not implement it — git grep -in "ForwardSourceMessageProperty" -- 'pulsar-function-go/**/*.go' returns nothing outside the generated pb package.

The Python runtime implements it in process_result and has tests pinning both directions (TestPropertiesForwarding.test_forwards_properties and test_do_not_forward_properties in test_python_instance.py), and the Java runtime implements it too.

The effect is that --forward-source-message-property is accepted, reported back by functions get, and produces output messages carrying none of the source properties. For a chain of functions this loses correlation ids, trace context and any other metadata a previous stage attached — and it does so silently, so the loss surfaces downstream as missing data rather than as a configuration error.

Solution

Copy the source message's properties onto the output message when the flag is set, in the Go equivalent of Python's process_result. Pulsar's own __pfn_input_topic__ and __pfn_input_msgid__ properties are set separately and should keep their current behaviour.

Worth confirming in review: whether the properties are copied before or after the runtime's own reserved keys, so a source message carrying a __pfn_* key cannot overwrite them. Python appends the runtime keys after the forwarded ones.

Alternatives

None worth taking. The Go client's ProducerMessage has a Properties map[string]string field, so this is a small change.

Anything else?

Verified against origin/master.

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/functiontype/bugThe PR fixed a bug or issue reported a bug

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions