sasl passthrough

[sawantmandar]

Can/how Kafka-proxy be enhanced with a change where the client passes SASL credentials to Kafka proxy and Kafka proxy just forwards the client provided SASL credentials to Kafka broker for authentication and authorization purposes?

[everesio]

this is already supported. if neither local sasl nor sasl by proxy is configured, sasl will passed to broker.

[sawantmandar]

Thank you for your response. It appears this works the way you described if the kafka proxy's client chooses SASL_PLAINTEXT as the security protocol. Its failing with following error with SASL_SSL

Reading data from local connection on 127.0.0.1:3904 from 127.0.0.1:51253 (xxxx.xx.xx.com:9093) had error: api key -19711 is invalid

Port 9093 is SSL listener port on the Kafka broker.

If security protocol setting is not set at all in the client code, it results in following

Reading data from xxxx.xx.xx.com:9093 had error: unexpected EOF

[Ladjack7]

The same issue, api key -19711 is invalid
security.protocol on broker side is SASL_SSL
@everesio is there any plan for the future releases to implicitly set security protocol setting?

[sarwarbhuiyan]

I've had the same experience, what are we missing from the client or proxy command?

[bitkill]

Same here, setting no sasl config in the proxy leads to no connection at all.

[hareacc]

@bitkill , I already spend more then 30 hours to setup this but due to combination of various configuration on client, cluster and proxy end, It was long to setup but your one comment "no connection at all without sasl config on proxy" end my struggle. Thanks,

[mrn3]

Is this issue still a problem? I am a bit confused by the discussion.

My situation is that I am able to use the kafka utilities to connect using a config like this:

client.id=event-hub
ssl.protocol=TLSv1.3   
ssl.enabled.protocols=TLSv1.3  
security.protocol=SASL_SSL 
sasl.mechanism=OAUTHBEARER 
sasl.jaas.config=org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required; 
sasl.login.callback.handler.class=com.adobe.core.pipeline.kafka.security.client.auth.ImsClientCallbackHandler  
sasl.ims.url=https://ims-na1.adobelogin.com
sasl.ims.secret.type=DIRECT
sasl.ims.secret.client.id=event-hub
sasl.ims.secret.client.secret=<redacted>
sasl.ims.secret.client.code=<redacted>
metric.reporters=org.apache.kafka.common.metrics.JmxReporter

And I can then run commands like:

./bin/kafka-console-consumer.sh --consumer.config config/consumer.properties --topic project-management_outbox_fullstate --bootstrap-server kafka-1-az1-or2-secure.prd.pipeline.adobedc.net:9097 --from-beginning

./bin/kafka-broker-api-versions.sh --bootstrap-server kafka-1-az1-or2-secure.prd.pipeline.adobedc.net:9097 --command-config config/consumer.properties

./bin/kafka-topics.sh --bootstrap-server kafka-1-az1-or2-secure.prd.pipeline.adobedc.net:9097 --command-config config/consumer.properties --list

./bin/kafka-get-offsets.sh --bootstrap-server kafka-1-az1-or2-secure.prd.pipeline.adobedc.net:9097 --command-config config/consumer.properties

However, I am not sure how to connect with kafka-proxy in the same way. Does it support passing through SASL parameters? Does it support a custom callback handler function (which I have to put a JAR file in the libs folder for)?

[DaspawnW]

Hi @everesio,

I can only confirm what the previous writers say. It seems that when the kafka brokers are configured in a way that sasl is not running over plaintext and it's configured to do only forwarding the requests seem to hang.

Even though I try to set some sasl-method or enable sasl (which doesn't work as it now requires sasl.username and sasl.password) the mechanism seems not to work. The only exception that I see is this one:

{"@level":"info","@message":"New connection for kafka-cluster-1-kafka-0.kafka-cluster-1-kafka-brokers.kafka-cluster-1.svc.cluster.local:9093","@timestamp":"2023-09-22T10:56:46Z"}
{"@level":"debug","@message":"Kafka request key 17, version 0, length 40","@timestamp":"2023-09-22T10:56:46Z"}
{"@level":"debug","@message":"Kafka response key 17, version 0, length 25","@timestamp":"2023-09-22T10:56:46Z"}
{"@level":"info","@message":"Reading data from kafka-cluster-1-kafka-0.kafka-cluster-1-kafka-brokers.kafka-cluster-1.svc.cluster.local:9093 had error: open request is missing","@timestamp":"2023-09-22T10:56:51Z"}

[ccrafael]

Hi all,
this configuration works for me to use SASL_PLAINTEXT in the client to connect with the proxy and SASL_SSL with the downstream and doing a passthrough of the secrets:

./kafka-proxy server --bootstrap-server-mapping=the-kafka-bootstrap-server:9092,localhost:19092 --tls-enable --dynamic-sequential-min-port=33000 --log-level=trace 

And this other conf works for me to use SASL_SSL in the client to connect with the proxy and SASL_SSL with the downstream and doing a passthrough of the secrets:

./kafka-proxy server --bootstrap-server-mapping=the-kafka-bootstrap-server:9092,localhost:19092 --tls-enable --dynamic-sequential-min-port=33000 --log-level=trace --proxy-listener-tls-enable --proxy-listener-cert-file kafka-proxy-selfsigned.crt --proxy-listener-key-file kafka-proxy-selfsigned.key