[Bug] Pulsar Client ignores producer schema #20586
Answered
by
BewareMyPower
anthonyikeda
asked this question in
Q&A
Search before asking
Version2.11.0 Minimal reproduce stepMethodConfigure Schema ValidationBy default, Pulsar does not validate schemas, this needs to be reconfigured: conf/standalone.conf isSchemaValidationEnforced=trueStart Pulsar in Standalone modeStarting in the $ bin/pulsar standaloneCreate our Tenant and NamespaceStarting in the $ bin/pulsar-admin tenants create my-tenant
$ bin/pulsar-admin tenants list
my-tenant
public
pulsar
$ bin/pulsar-admin namespaces create --tenant my-namespace
$ bin/pulsar-admin namespaces list my-tenant
my-tenant/my-namespaceCreate our Topic$ bin/pulsar-admin topics create persistent://my-tenant/my-namespace/basic-topic-3
$ bin/pulsar-admin topics list my-tenant/my-namespace
persistent://my-tenant/my-namespace/basic-topic-3Apply a schema to the topicProduct Schema for AVRO {
"type" : "record",
"namespace" : "com.pulsar.workshop",
"name" : "Product",
"fields" : [
{
"name" : "product_id",
"type" : "int"
},
{
"name" : "product_name",
"type" : "string"
},
{
"name" : "product_description",
"type" : [ "null","string" ]
}
]
}Apply the schema to the topictopic-schema.json {
"type" : "JSON",
"schema" : "{ \"type\" : \"record\", \"namespace\" : \"com.pulsar.workshop\", \"name\" : \"Product\", \"fields\" : [ { \"name\" : \"product_id\", \"type\" : \"int\" }, { \"name\" : \"product_name\", \"type\" : \"string\" }, {\"name\" : \"product_description\", \"type\" : [ \"null\",\"string\" ] } ] }"
}$ bin/pulsar-admin schemas upload -f topic-schema.json persistent://my-tenant/my-namespace/basic-topic-3Verify the schema$ bin/pulsar-admin schemas get persistent://my-tenant/my-namespace/basic-topic-3
{
"version": 0,
"schemaInfo": {
"name": "basic-topic",
"schema": {
"type": "record",
"namespace": "com.pulsar.workshop",
"name": "Product",
"fields": [
{
"name": "product_id",
"type": "int"
},
{
"name": "product_name",
"type": "string"
},
{
"name": "product_description",
"type": [
"null",
"string"
]
}
]
},
"type": "JSON",
"timestamp": 1680715759782,
"properties": {}
}
}Create a consumer to our Topic$ bin/pulsar-client consume persistent://my-tenant/my-namespace/basic-topic-3 -s 'my-subscription' -p Earliest -n 0
2023-04-05T09:37:36,229-0700 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://my-tenant/my-namespace/basic-topic][my-subscription] Subscribed to topic on localhost/127.0.0.1:6650 -- consumer: 0Produce Messagesapple.json {
"product_id" : 12,
"product_name" : "Apple",
"product_description" : "A red fruit"
}Standard produce expected to passPulsar Producer $ bin/pulsar-client produce persistent://my-tenant/my-namespace/basic-topic-3 -f apple.json -vs 'json:{ "type" : "record", "namespace" : "com.pulsar.workshop", "name" : "Product", "fields" : [ { "name" : "product_id", "type" : "int" }, { "name" : "product_name", "type" : "string" }, {"name" : "product_description", "type" : [ "null","string" ] } ] }'
2023-04-05T11:41:53,902-0700 [main] WARN io.netty.resolver.dns.DnsServerAddressStreamProviders - Can not find io.netty.resolver.dns.macos.MacOSDnsServerAddressStreamProvider in the classpath, fallback to system defaults. This may result in incorrect DNS resolutions on MacOS. Check whether you have a dependency on 'io.netty:netty-resolver-dns-native-macos'
2023-04-05T11:41:54,113-0700 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ConnectionPool - [[id: 0x86c56ff2, L:/127.0.0.1:51950 - R:localhost/127.0.0.1:6650]] Connected to server
2023-04-05T11:41:54,195-0700 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Starting Pulsar producer perf with config: {"topicName":"persistent://my-tenant/my-namespace/basic-topic-3","producerName":null,"sendTimeoutMs":30000,"blockIfQueueFull":false,"maxPendingMessages":1000,"maxPendingMessagesAcrossPartitions":50000,"messageRoutingMode":"RoundRobinPartition","hashingScheme":"JavaStringHash","cryptoFailureAction":"FAIL","batchingMaxPublishDelayMicros":1000,"batchingPartitionSwitchFrequencyByPublishDelay":10,"batchingMaxMessages":1000,"batchingMaxBytes":131072,"batchingEnabled":true,"chunkingEnabled":false,"chunkMaxMessageSize":-1,"compressionType":"NONE","initialSequenceId":null,"autoUpdatePartitions":true,"autoUpdatePartitionsIntervalSeconds":60,"multiSchema":true,"accessMode":"Shared","lazyStartPartitionedProducers":false,"properties":{},"initialSubscriptionName":null}
2023-04-05T11:41:54,213-0700 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Pulsar client config: {"serviceUrl":"pulsar://localhost:6650/","authPluginClassName":null,"authParams":null,"authParamMap":null,"operationTimeoutMs":30000,"lookupTimeoutMs":30000,"statsIntervalSeconds":60,"numIoThreads":1,"numListenerThreads":1,"connectionsPerBroker":1,"connectionMaxIdleSeconds":180,"useTcpNoDelay":true,"useTls":false,"tlsKeyFilePath":"","tlsCertificateFilePath":"","tlsTrustCertsFilePath":"","tlsAllowInsecureConnection":false,"tlsHostnameVerificationEnable":false,"concurrentLookupRequest":5000,"maxLookupRequest":50000,"maxLookupRedirects":20,"maxNumberOfRejectedRequestPerConnection":50,"keepAliveIntervalSeconds":30,"connectionTimeoutMs":10000,"requestTimeoutMs":60000,"initialBackoffIntervalNanos":100000000,"maxBackoffIntervalNanos":60000000000,"enableBusyWait":false,"listenerName":null,"useKeyStoreTls":false,"sslProvider":null,"tlsKeyStoreType":"JKS","tlsKeyStorePath":"","tlsKeyStorePassword":"*****","tlsTrustStoreType":"JKS","tlsTrustStorePath":"","tlsTrustStorePassword":"*****","tlsCiphers":[],"tlsProtocols":[],"memoryLimitBytes":0,"proxyServiceUrl":null,"proxyProtocol":null,"enableTransaction":false,"dnsLookupBindAddress":null,"dnsLookupBindPort":0,"socks5ProxyAddress":null,"socks5ProxyUsername":null,"socks5ProxyPassword":null}
2023-04-05T11:41:54,221-0700 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://my-tenant/my-namespace/basic-topic-3] [null] Creating producer on cnx [id: 0x86c56ff2, L:/127.0.0.1:51950 - R:localhost/127.0.0.1:6650]
2023-04-05T11:41:54,231-0700 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://my-tenant/my-namespace/basic-topic-3] [standalone-51-13] Created producer on cnx [id: 0x86c56ff2, L:/127.0.0.1:51950 - R:localhost/127.0.0.1:6650]
2023-04-05T11:41:54,269-0700 [main] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - [persistent://my-tenant/my-namespace/basic-topic-3] [standalone-51-13] Pending messages: 0 --- Publish throughput: 18.53 msg/s --- 0.01 Mbit/s --- Latency: med: 25.000 ms - 95pct: 25.000 ms - 99pct: 25.000 ms - 99.9pct: 25.000 ms - max: 25.000 ms --- BatchSize: med: 1.000 - 95pct: 1.000 - 99pct: 1.000 - 99.9pct: 1.000 - max: 1.000 --- MsgSize: med: 99.000 bytes - 95pct: 99.000 bytes - 99pct: 99.000 bytes - 99.9pct: 99.000 bytes - max: 99.000 bytes --- Ack received rate: 18.53 ack/s --- Failed messages: 0 --- Pending messages: 0
2023-04-05T11:41:54,272-0700 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://my-tenant/my-namespace/basic-topic-3] [standalone-51-13] Closed Producer
2023-04-05T11:41:54,273-0700 [main] INFO org.apache.pulsar.client.impl.PulsarClientImpl - Client closing. URL: pulsar://localhost:6650/
2023-04-05T11:41:54,278-0700 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ClientCnx - [id: 0x86c56ff2, L:/127.0.0.1:51950 ! R:localhost/127.0.0.1:6650] Disconnected
2023-04-05T11:41:56,359-0700 [main] INFO org.apache.pulsar.client.cli.PulsarClientTool - 1 messages successfully producedStandard produce expected to failPulsar Producer $ bin/pulsar-client produce persistent://my-tenant/my-namespace/basic-topic-3 -m '{"family" : "genus" }' -vs 'json:{ "type" : "record", "namespace" : "com.pulsar.workshop", "name" : "Product", "fields" : [ { "name" : "product_id", "type" : "int" }, { "name" : "product_name", "type" : "string" }, {"name" : "product_description", "type" : [ "null","string" ] } ] }'
2023-04-05T11:43:15,431-0700 [main] WARN io.netty.resolver.dns.DnsServerAddressStreamProviders - Can not find io.netty.resolver.dns.macos.MacOSDnsServerAddressStreamProvider in the classpath, fallback to system defaults. This may result in incorrect DNS resolutions on MacOS. Check whether you have a dependency on 'io.netty:netty-resolver-dns-native-macos'
2023-04-05T11:43:15,628-0700 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ConnectionPool - [[id: 0x7bf98cc5, L:/127.0.0.1:51959 - R:localhost/127.0.0.1:6650]] Connected to server
2023-04-05T11:43:15,705-0700 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Starting Pulsar producer perf with config: {"topicName":"persistent://my-tenant/my-namespace/basic-topic-3","producerName":null,"sendTimeoutMs":30000,"blockIfQueueFull":false,"maxPendingMessages":1000,"maxPendingMessagesAcrossPartitions":50000,"messageRoutingMode":"RoundRobinPartition","hashingScheme":"JavaStringHash","cryptoFailureAction":"FAIL","batchingMaxPublishDelayMicros":1000,"batchingPartitionSwitchFrequencyByPublishDelay":10,"batchingMaxMessages":1000,"batchingMaxBytes":131072,"batchingEnabled":true,"chunkingEnabled":false,"chunkMaxMessageSize":-1,"compressionType":"NONE","initialSequenceId":null,"autoUpdatePartitions":true,"autoUpdatePartitionsIntervalSeconds":60,"multiSchema":true,"accessMode":"Shared","lazyStartPartitionedProducers":false,"properties":{},"initialSubscriptionName":null}
2023-04-05T11:43:15,722-0700 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Pulsar client config: {"serviceUrl":"pulsar://localhost:6650/","authPluginClassName":null,"authParams":null,"authParamMap":null,"operationTimeoutMs":30000,"lookupTimeoutMs":30000,"statsIntervalSeconds":60,"numIoThreads":1,"numListenerThreads":1,"connectionsPerBroker":1,"connectionMaxIdleSeconds":180,"useTcpNoDelay":true,"useTls":false,"tlsKeyFilePath":"","tlsCertificateFilePath":"","tlsTrustCertsFilePath":"","tlsAllowInsecureConnection":false,"tlsHostnameVerificationEnable":false,"concurrentLookupRequest":5000,"maxLookupRequest":50000,"maxLookupRedirects":20,"maxNumberOfRejectedRequestPerConnection":50,"keepAliveIntervalSeconds":30,"connectionTimeoutMs":10000,"requestTimeoutMs":60000,"initialBackoffIntervalNanos":100000000,"maxBackoffIntervalNanos":60000000000,"enableBusyWait":false,"listenerName":null,"useKeyStoreTls":false,"sslProvider":null,"tlsKeyStoreType":"JKS","tlsKeyStorePath":"","tlsKeyStorePassword":"*****","tlsTrustStoreType":"JKS","tlsTrustStorePath":"","tlsTrustStorePassword":"*****","tlsCiphers":[],"tlsProtocols":[],"memoryLimitBytes":0,"proxyServiceUrl":null,"proxyProtocol":null,"enableTransaction":false,"dnsLookupBindAddress":null,"dnsLookupBindPort":0,"socks5ProxyAddress":null,"socks5ProxyUsername":null,"socks5ProxyPassword":null}
2023-04-05T11:43:15,729-0700 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://my-tenant/my-namespace/basic-topic-3] [null] Creating producer on cnx [id: 0x7bf98cc5, L:/127.0.0.1:51959 - R:localhost/127.0.0.1:6650]
2023-04-05T11:43:15,739-0700 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://my-tenant/my-namespace/basic-topic-3] [standalone-51-14] Created producer on cnx [id: 0x7bf98cc5, L:/127.0.0.1:51959 - R:localhost/127.0.0.1:6650]
2023-04-05T11:43:15,775-0700 [main] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - [persistent://my-tenant/my-namespace/basic-topic-3] [standalone-51-14] Pending messages: 0 --- Publish throughput: 19.44 msg/s --- 0.00 Mbit/s --- Latency: med: 25.000 ms - 95pct: 25.000 ms - 99pct: 25.000 ms - 99.9pct: 25.000 ms - max: 25.000 ms --- BatchSize: med: 1.000 - 95pct: 1.000 - 99pct: 1.000 - 99.9pct: 1.000 - max: 1.000 --- MsgSize: med: 21.000 bytes - 95pct: 21.000 bytes - 99pct: 21.000 bytes - 99.9pct: 21.000 bytes - max: 21.000 bytes --- Ack received rate: 19.44 ack/s --- Failed messages: 0 --- Pending messages: 0
2023-04-05T11:43:15,778-0700 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://my-tenant/my-namespace/basic-topic-3] [standalone-51-14] Closed Producer
2023-04-05T11:43:15,779-0700 [main] INFO org.apache.pulsar.client.impl.PulsarClientImpl - Client closing. URL: pulsar://localhost:6650/
2023-04-05T11:43:15,783-0700 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ClientCnx - [id: 0x7bf98cc5, L:/127.0.0.1:51959 ! R:localhost/127.0.0.1:6650] Disconnected
2023-04-05T11:43:17,857-0700 [main] INFO org.apache.pulsar.client.cli.PulsarClientTool - 1 messages successfully producedWhat did you expect to see?In the failed message produce, the data What did you see instead?The message was accepted and published to the consumers of the topic. Anything else?So far I've only tried this in standalone mode with the I've also tried configuring schema validation at the namespace level with no such luck: $ bin/pulsar-admin namespaces set-schema-validation-enforce --enable my-tenant/my-namespace
$ bin/pulsar-admin namespaces get-schema-validation-enforce my-tenant/my-namespace
true
$ bin/pulsar-admin topics set-schema-validation-enforce --enable persistent://my-tenant/my-namespace/basic-topic-3
$ bin/pulsar-admin topics get-schema-validation-enforce persistent://my-tenant/my-namespace/basic-topic-3
trueAre you willing to submit a PR?
|
Answered by
BewareMyPower
Apr 14, 2023
Replies: 4 comments
|
Am I getting my schemas mixed up? Tried using schema type AVRO but now getting: product.avsc {
"type" : "record",
"namespace" : "com.pulsar.workshop",
"name" : "Product",
"fields" : [
{
"name" : "product_id",
"type" : "int"
},
{
"name" : "product_name",
"type" : "string"
},
{
"name" : "product_description",
"type" : [ "null", "string" ]
}
]
}topic-schema.json {
"type" : "AVRO",
"schema" : "{ \"type\" : \"record\", \"namespace\" : \"com.pulsar.workshop\", \"name\" : \"Product\", \"fields\" : [ { \"name\" : \"product_id\", \"type\" : \"int\" }, { \"name\" : \"product_name\", \"type\" : \"string\" }, {\"name\" : \"product_description\", \"type\" : [ \"null\",\"string\" ] } ] }"
}2023-04-05T13:09:56,851-0700 [main] ERROR org.apache.pulsar.client.cli.PulsarClientTool - Error while producing messages
2023-04-05T13:09:56,851-0700 [main] ERROR org.apache.pulsar.client.cli.PulsarClientTool - java.lang.ArrayIndexOutOfBoundsException: Index 56 out of bounds for length 2
org.apache.pulsar.client.api.SchemaSerializationException: java.lang.ArrayIndexOutOfBoundsException: Index 56 out of bounds for length 2
at org.apache.pulsar.client.impl.schema.generic.GenericAvroReader.read(GenericAvroReader.java:91) ~[org.apache.pulsar-pulsar-client-original-2.11.0.jar:2.11.0]
at org.apache.pulsar.client.impl.schema.generic.GenericAvroReader.read(GenericAvroReader.java:41) ~[org.apache.pulsar-pulsar-client-original-2.11.0.jar:2.11.0]
at org.apache.pulsar.client.api.schema.SchemaReader.read(SchemaReader.java:40) ~[org.apache.pulsar-pulsar-client-api-2.11.0.jar:2.11.0]
at org.apache.pulsar.client.impl.schema.reader.AbstractMultiVersionReader.read(AbstractMultiVersionReader.java:61) ~[org.apache.pulsar-pulsar-client-original-2.11.0.jar:2.11.0]
at org.apache.pulsar.client.api.schema.SchemaReader.read(SchemaReader.java:40) ~[org.apache.pulsar-pulsar-client-api-2.11.0.jar:2.11.0]
at org.apache.pulsar.client.impl.schema.AbstractStructSchema.decode(AbstractStructSchema.java:66) ~[org.apache.pulsar-pulsar-client-original-2.11.0.jar:2.11.0]
at org.apache.pulsar.client.api.Schema.validate(Schema.java:63) ~[org.apache.pulsar-pulsar-client-api-2.11.0.jar:2.11.0]
at org.apache.pulsar.client.impl.schema.AutoProduceBytesSchema.encode(AutoProduceBytesSchema.java:80) ~[org.apache.pulsar-pulsar-client-original-2.11.0.jar:2.11.0]
at org.apache.pulsar.client.impl.schema.AutoProduceBytesSchema.encode(AutoProduceBytesSchema.java:32) ~[org.apache.pulsar-pulsar-client-original-2.11.0.jar:2.11.0]
at org.apache.pulsar.client.impl.TypedMessageBuilderImpl.value(TypedMessageBuilderImpl.java:173) ~[org.apache.pulsar-pulsar-client-original-2.11.0.jar:2.11.0]
at org.apache.pulsar.client.cli.CmdProduce.publish(CmdProduce.java:288) ~[org.apache.pulsar-pulsar-client-tools-2.11.0.jar:2.11.0]
at org.apache.pulsar.client.cli.CmdProduce.run(CmdProduce.java:239) ~[org.apache.pulsar-pulsar-client-tools-2.11.0.jar:2.11.0]
at org.apache.pulsar.client.cli.PulsarClientTool.run(PulsarClientTool.java:219) ~[org.apache.pulsar-pulsar-client-tools-2.11.0.jar:2.11.0]
at org.apache.pulsar.client.cli.PulsarClientTool.main(PulsarClientTool.java:259) ~[org.apache.pulsar-pulsar-client-tools-2.11.0.jar:2.11.0]
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 56 out of bounds for length 2
at org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:460) ~[org.apache.avro-avro-1.10.2.jar:1.10.2]
at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:283) ~[org.apache.avro-avro-1.10.2.jar:1.10.2]
at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:187) ~[org.apache.avro-avro-1.10.2.jar:1.10.2]
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:160) ~[org.apache.avro-avro-1.10.2.jar:1.10.2]
at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:259) ~[org.apache.avro-avro-1.10.2.jar:1.10.2]
at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:247) ~[org.apache.avro-avro-1.10.2.jar:1.10.2]
at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:179) ~[org.apache.avro-avro-1.10.2.jar:1.10.2]
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:160) ~[org.apache.avro-avro-1.10.2.jar:1.10.2]
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153) ~[org.apache.avro-avro-1.10.2.jar:1.10.2]
at org.apache.pulsar.client.impl.schema.generic.GenericAvroReader.read(GenericAvroReader.java:86) ~[org.apache.pulsar-pulsar-client-original-2.11.0.jar:2.11.0]
... 13 more
2023-04-05T13:09:56,859-0700 [main] INFO org.apache.pulsar.client.cli.PulsarClientTool - 0 messages successfully produced
|
0 replies
|
Attempted the same with a JSON Schema: product.schema.json {
"$schema" : "https://json-schema.org/draft/2020-12/schema",
"$id": "http://example.com/product.schema.json",
"title" : "Product",
"description" : "A product in the catalog",
"type" : "object",
"properties":{
"product_id" : {
"description" : "A unique identifier",
"type" : "integer"
},
"product_name" : {
"description" : "name of the product",
"type" : "string"
},
"product_description" : {
"description" : "A description of the product",
"type" : "string"
}
},
"required": [ "product_id", "product_name"]
}topic-schema-2.json {
"type" : "JSON",
"schema" : "{ \"$schema\" : \"https://json-schema.org/draft/2020-12/schema\", \"$id\": \"http://example.com/product.schema.json\", \"title\" : \"Product\", \"description\" : \"A product in the catalog\", \"type\" : \"object\", \"properties\":{\"product_id\" : {\"description\" : \"A unique identifier\", \"type\" : \"integer\" },\"product_name\" : {\"description\" : \"name of the product\", \"type\" : \"string\" }, \"product_description\" : { \"description\" : \"A description of the product\", \"type\" : \"string\" } },\"required\": [ \"product_id\", \"product_name\"]}"
}Upload schema: $ bin/pulsar-admin topics create persistent://my-tenant/my-namespace/basic-topic-4
$ bin/pulsar-admin schemas upload -f topic-schema-2.json persistent://my-tenant/my-namespace/basic-topic-4
2023-04-05T14:33:42,660-0700 [AsyncHttpClient-7-1] WARN org.apache.pulsar.client.admin.internal.BaseResource - [http://localhost:8080/admin/v2/schemas/my-tenant/my-namespace/basic-topic-4/schema] Failed to perform http post request: javax.ws.rs.ClientErrorException: HTTP 422 Invalid schema definition data for JSON schema
HTTP 422 Invalid schema definition data for JSON schema
Reason: HTTP 422 Invalid schema definition data for JSON schemaI think in this attempt it's about how to position a JSON schema vs an AVRO schema. Is JSON Schema meant to represent an AVRO schema definition written in JSON or actual JSON Schema (https://json-schema.org) ? |
0 replies
The schema validation only happens when the producer connects to the broker. When the producer sends a message with wrong format, the message could still be sent successfully. If you want to validate the schema when sending messages, you should create an It seems that the Pulsar CLI cannot create an BTW, |
0 replies
Answer selected by
tisonkun
|
The issue had no activity for 30 days, mark with Stale label. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The schema validation only happens when the producer connects to the broker. When the producer sends a message with wrong format, the message could still be sent successfully. If you want to validate the schema when sending messages, you should create an
AUTO_PRODUCEproducer. See https://pulsar.apache.org/docs/2.11.x/schema-understand/#auto-schemaIt seems that the Pulsar CLI cannot create an
AUTO_PRODUCEproducer currently.BTW,
isSchemaValidationEnforcedonly enforces the schema validation for bytes producer. i.e. a bytes producer cannot connect to a topic that already has schema, if users w…