Skip to content

segfault when trying to construct AvroProducer #769

Open
@stefansjs

Description

@stefansjs

Description

I get a SIGSEGV crashing my entire application when trying to construct an AvroProducer. I'm unclear if there is a problem with my configuration or not, but a segfault is not really acceptable. Is it possible to use cysignals sig_on() and sig_off to convert this to an exception?

How to reproduce

The following code produces the output below

    avro_schema_options = getConfigProperties('avro.*', section=section_name)
    # If there's any avro schema options provided, assume they're all provided and setup the avro producer
    default_key_schema = avro.load(PERF_KEY_SCHEMA_PATH)
    default_value_schema = avro.load(PERF_VALUE_SCHEMA_PATH)

    avro_producer_params = {key[len('avro.'):]: value for key, value in avro_schema_options.items()}

    self.logger.debug("Setting up AvroProducer with %r", avro_producer_params)

    try:
        producer = avro.AvroProducer(avro_producer_params,
                                     default_key_schema=default_key_schema,
                                     default_value_schema=default_value_schema)
    except:
        self.logger.warning("Failed to configure avro producer", exc_info=True)
    else:
        return producer

Setting up AvroProducer with {u'bootstrap.servers': u'gpu-test4.sf.smle.co:9092', u'schema.registry.url': u'http://gpu-test4.sf.smle.co:8081', u'queue.buffering.max.ms': 10}

0 signals.so 0x0000000103f4537b print_backtrace + 65
1 signals.so 0x0000000103f48c90 sigdie + 60
2 signals.so 0x0000000103f48c14 cysigs_signal_handler + 351
3 libsystem_platform.dylib 0x00007fff614efb5d sigtramp + 29
4 ??? 0x00007ffeeeaa0ac0 0x0 + 140732902542016
5 cimpl.so 0x0000000103f5bb1d Producer__len
_ + 13
6 Python 0x00007fff3f31ae28 _PyBuiltin_Init + 8463
7 Python 0x00007fff3f31ef74 PyEval_EvalFrameEx + 4753
8 Python 0x00007fff3f31dab2 PyEval_EvalCodeEx + 1555
9 Python 0x00007fff3f323d4c _PyEval_SliceIndex + 490
10 Python 0x00007fff3f31e498 PyEval_EvalFrameEx + 1973
11 Python 0x00007fff3f31dab2 PyEval_EvalCodeEx + 1555
12 Python 0x00007fff3f323d4c _PyEval_SliceIndex + 490
13 Python 0x00007fff3f31e498 PyEval_EvalFrameEx + 1973
14 Python 0x00007fff3f323cee _PyEval_SliceIndex + 396
15 Python 0x00007fff3f31e498 PyEval_EvalFrameEx + 1973
16 Python 0x00007fff3f323cee _PyEval_SliceIndex + 396
17 Python 0x00007fff3f31e498 PyEval_EvalFrameEx + 1973
18 Python 0x00007fff3f323cee _PyEval_SliceIndex + 396
19 Python 0x00007fff3f31e498 PyEval_EvalFrameEx + 1973
20 Python 0x00007fff3f31dab2 PyEval_EvalCodeEx + 1555
21 Python 0x00007fff3f2c52ea PyFunction_SetClosure + 779
22 Python 0x00007fff3f2a8078 PyObject_Call + 97
23 Python 0x00007fff3f2b2695 PyMethod_New + 1169
24 pydevd_cython_darwin_27_64.so 0x0000000101bafbd0 __pyx_pw_14_pydevd_bundle_26pydevd_cython_darwin_27_64_9PyDBFrame_5do_wait_suspend + 224
25 pydevd_cython_darwin_27_64.so 0x0000000101b9a69c __pyx_f_14_pydevd_bundle_26pydevd_cython_darwin_27_64_9PyDBFrame_trace_dispatch + 25660
26 pydevd_cython_darwin_27_64.so 0x0000000101bae8ae __pyx_pw_14_pydevd_bundle_26pydevd_cython_darwin_27_64_9PyDBFrame_21trace_dispatch + 142
27 pydevd_cython_darwin_27_64.so 0x0000000101bc5e8a pyx_pw_14_pydevd_bundle_26pydevd_cython_darwin_27_64_15SafeCallWrapper_3__call + 122
28 Python 0x00007fff3f2a8078 PyObject_Call + 97
29 Python 0x00007fff3f323710 PyEval_CallObjectWithKeywords + 159
30 Python 0x00007fff3f342f4b PySys_WriteStdout + 2646
31 Python 0x00007fff3f342fb3 PySys_WriteStdout + 2750
32 Python 0x00007fff3f31fc38 PyEval_EvalFrameEx + 8021
33 Python 0x00007fff3f31dab2 PyEval_EvalCodeEx + 1555
34 Python 0x00007fff3f2c52ea PyFunction_SetClosure + 779
35 Python 0x00007fff3f2a8078 PyObject_Call + 97
36 Python 0x00007fff3f2b2695 PyMethod_New + 1169
37 Python 0x00007fff3f2a8078 PyObject_Call + 97
38 Python 0x00007fff3f2ed603 _PyObject_SlotCompare + 5547
39 Python 0x00007fff3f2e8dcf _PyType_Lookup + 1372
40 Python 0x00007fff3f2a8078 PyObject_Call + 97
41 Python 0x00007fff3f31e9d3 PyEval_EvalFrameEx + 3312
42 Python 0x00007fff3f323cee _PyEval_SliceIndex + 396
43 Python 0x00007fff3f31e498 PyEval_EvalFrameEx + 1973
44 Python 0x00007fff3f323cee _PyEval_SliceIndex + 396
45 Python 0x00007fff3f31e498 PyEval_EvalFrameEx + 1973
46 Python 0x00007fff3f31dab2 PyEval_EvalCodeEx + 1555
47 Python 0x00007fff3f323d4c _PyEval_SliceIndex + 490
48 Python 0x00007fff3f31e498 PyEval_EvalFrameEx + 1973
49 Python 0x00007fff3f323cee _PyEval_SliceIndex + 396
50 Python 0x00007fff3f31e498 PyEval_EvalFrameEx + 1973
51 Python 0x00007fff3f31dab2 PyEval_EvalCodeEx + 1555
52 Python 0x00007fff3f31d499 PyEval_EvalCode + 32
53 Python 0x00007fff3f33c645 PyParser_ASTFromFile + 287
54 Python 0x00007fff3f33c6ec PyRun_FileExFlags + 130
55 Python 0x00007fff3f319ea0 _PyBuiltin_Init + 4487
56 Python 0x00007fff3f31e9e6 PyEval_EvalFrameEx + 3331
57 Python 0x00007fff3f323cee _PyEval_SliceIndex + 396
58 Python 0x00007fff3f31e498 PyEval_EvalFrameEx + 1973
59 Python 0x00007fff3f31dab2 PyEval_EvalCodeEx + 1555
60 Python 0x00007fff3f323d4c _PyEval_SliceIndex + 490
61 Python 0x00007fff3f31e498 PyEval_EvalFrameEx + 1973
62 Python 0x00007fff3f323cee _PyEval_SliceIndex + 396
63 Python 0x00007fff3f31e498 PyEval_EvalFrameEx + 1973
64 Python 0x00007fff3f31dab2 PyEval_EvalCodeEx + 1555
65 Python 0x00007fff3f31d499 PyEval_EvalCode + 32
66 Python 0x00007fff3f33c645 PyParser_ASTFromFile + 287
67 Python 0x00007fff3f33c6ec PyRun_FileExFlags + 130
68 Python 0x00007fff3f33c26b PyRun_SimpleFileExFlags + 718
69 Python 0x00007fff3f34dbfa Py_Main + 3174
70 libdyld.dylib 0x00007fff613043d5 start + 1

Unhandled SIGSEGV: A segmentation fault occurred.
This probably occurred because a compiled module has a bug
in it and is not properly wrapped with sig_on(), sig_off().
Python will now terminate.

Checklist

  • confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()): confluent-kafka[avro]==1.2.0, 16908288
  • Apache Kafka broker version:
  • Client configuration: {...}
  • Operating system: Mac OS 10.14.6, CentOS 7
  • Provide client logs (with 'debug': '..' as necessary)
  • Provide broker log excerpts
  • Critical issue

I don't know how to find the missing information. Please explain how to fill in the missing details.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugReporting an unexpected or problematic behavior of the codebasecomponent:schema-registryAny schema registry related isues rather than kafka isolated onesinvestigate furtherIt's unclear what the issue is at this time but there is enough interest to look into it

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions