RabbitMQ server not coming up in FIPS enabled cluster

[Vishnu-0809]

Describe the bug

We are installing RabbitMQ on FIPS enabled cluster with {crypto, [{fips_mode, false}]} in advanced.config
But the rabbitmq cluster is not coming up and peer discovery is failing.

RabbitMQ logs

2024-09-11 09:35:06.037098+00:00 [error] <0.410.0> ** State machine <0.410.0> terminating
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0> ** Last event = {{call,{<0.406.0>,#Ref<0.2222038192.1009778690.260325>}},
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                  {start,2250}}
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0> ** When server state  = [{data,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                           [{"State",
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                             {initial_hello,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                              {state,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                               {static_env,client,gen_tcp,tls_gen_connection,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                tcp,tcp_closed,tcp_error,tcp_passive,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                "kubernetes.default.svc.cluster.local",443,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                #Port<0.22>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                #Ref<0.2222038192.1009909761.258981>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                #Ref<0.2222038192.1009909761.258980>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                ssl_client_session_cache_db,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                {ssl_crl_cache,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 {{#Ref<0.2222038192.1009909761.258984>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   #Ref<0.2222038192.1009909761.258985>},
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                  []}},
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                {#Ref<0.2222038192.1009909761.258982>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 #Ref<0.2222038192.1009909761.258983>},
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                #Ref<0.2222038192.1009778690.260354>,undefined},
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                               "***",
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                               #{partial_chain => #Fun<ssl.4.1732737>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 session_tickets => disabled,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 crl_cache => {ssl_crl_cache,{internal,[]}},
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 log_level => notice,protocol => tls,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 max_handshake_size => 131072,dh => "***",
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 customize_hostname_check => [],
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 certs_keys => "***",srp_identity => "***",
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 signature_algs_cert =>
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                  [default,eddsa_ed25519,eddsa_ed448,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   ecdsa_secp521r1_sha512,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   ecdsa_secp384r1_sha384,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   ecdsa_secp256r1_sha256,rsa_pss_pss_sha512,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   rsa_pss_pss_sha384,rsa_pss_pss_sha256,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   rsa_pss_rsae_sha512,rsa_pss_rsae_sha384,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   rsa_pss_rsae_sha256,rsa_pkcs1_sha512,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   rsa_pkcs1_sha384,rsa_pkcs1_sha256,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   {sha512,ecdsa},
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   {sha384,ecdsa},
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   {sha256,ecdsa},
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   rsa_pkcs1_sha1],
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 server_name_indication =>
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                  "kubernetes.default.svc.cluster.local",
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 eccs =>
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                  {elliptic_curves,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   [{1,3,101,110},
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                    {1,3,101,111},
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                    {1,3,132,0,35},
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                    {1,3,132,0,34},
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                    {1,2,840,10045,3,1,7}]},
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 versions => [{3,4},{3,3}],
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 signature_algs =>
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                  [eddsa_ed25519,eddsa_ed448,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   ecdsa_secp521r1_sha512,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   ecdsa_secp384r1_sha384,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   ecdsa_secp256r1_sha256,rsa_pss_pss_sha512,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   rsa_pss_pss_sha384,rsa_pss_pss_sha256,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   rsa_pss_rsae_sha512,rsa_pss_rsae_sha384,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   rsa_pss_rsae_sha256,rsa_pkcs1_sha512,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   rsa_pkcs1_sha384,rsa_pkcs1_sha256,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   {sha512,ecdsa},
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   {sha384,ecdsa},
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   {sha256,ecdsa}],
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 verify => verify_peer,handshake => full,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 max_fragment_length => undefined,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 reuse_session => undefined,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 renegotiate_at => 268435456,cacerts => "***",
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 user_lookup_fun => undefined,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 fallback => false,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 key_update_at => 388736063997,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 ciphers =>
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                  [<<19,2>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<19,1>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<19,3>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<19,4>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<19,5>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<"À,">>,<<"À0">>,<<"À­">>,<<"À¯">>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<"À$">>,<<"À(">>,<<"̩"/utf8>>,<<"̨"/utf8>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<"À+">>,<<"À/">>,<<"À¬">>,<<"À®">>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<"À.">>,<<"À2">>,<<"À&">>,<<"À*">>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<"À-">>,<<"À1">>,<<"À#">>,<<"À'">>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<"À%">>,<<"À)">>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<0,159>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<0,163>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<0,107>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<0,106>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<0,158>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<0,162>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<"̪"/utf8>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<0,103>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<0,64>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<"À\n">>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<192,20>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<192,5>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<192,15>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<"À\t">>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<192,19>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<192,4>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<192,14>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<0,57>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<0,56>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<0,51>>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   <<0,50>>],
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 crl_check => false,early_data => undefined,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 alpn_advertised_protocols => undefined,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 psk_identity => "***",
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 cacertfile =>
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                  <<"/var/run/secrets/kubernetes.io/serviceaccount/..data/ca.crt">>,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 secure_renegotiate => true,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 use_ticket => undefined,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 reuse_sessions => true,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 supported_groups =>
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                  {supported_groups,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                   [x25519,x448,secp256r1,secp384r1]},
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 verify_fun => undefined},
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                               {socket_options,binary,0,0,0,false},
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                               "***","***",not_requested,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                               #{sender => <0.409.0>,active_n => 100,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                                 active_n_toggle => true},
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                               "***",undefined,"***","***","***",undefined,
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0>                               undefined,undefined}}}]}]
2024-09-11 09:35:06.037098+00:00 [error] <0.410.0> ** Reason for termination = error:{error,{"evp.c",136},"Can't make context"}
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0> ** Callback modules = [tls_connection]
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0> ** Callback mode = state_functions
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0> ** Stacktrace =
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0> **  [{crypto,generate_key,3,
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0>              [{file,"crypto.erl"},
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0>               {line,1690},
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0>               {error_info,#{erl_function_arg_num => undefined}}]},
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0>      {ssl_cipher,'-generate_client_shares/1-anonymous-0-',1,
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0>                  [{file,"ssl_cipher.erl"},{line,1174}]},
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0>      {lists,map,2,[{file,"lists.erl"},{line,1559}]},
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0>      {ssl_cipher,generate_client_shares,1,
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0>                  [{file,"ssl_cipher.erl"},{line,1176}]},
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0>      {ssl_gen_statem,initial_hello,3,[{file,"ssl_gen_statem.erl"},{line,493}]},
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0>      {gen_statem,loop_state_callback,11,[{file,"gen_statem.erl"},{line,1395}]},
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0>      {tls_connection,init,1,[{file,"tls_connection.erl"},{line,162}]},
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0>      {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,241}]}]
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0> ** Client <0.456.0> stacktrace
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0> ** [{gen,do_call,4,[{file,"gen.erl"},{line,240}]},
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0>     {gen_statem,call,3,[{file,"gen_statem.erl"},{line,923}]},
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0>     {ssl_gen_statem,call,2,[{file,"ssl_gen_statem.erl"},{line,1319}]},
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0>     {ssl_gen_statem,handshake,2,[{file,"ssl_gen_statem.erl"},{line,253}]},
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0>     {ssl_gen_statem,connect,8,[{file,"ssl_gen_statem.erl"},{line,222}]},
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0>     {ssl,connect,4,[{file,"ssl.erl"},{line,658}]},
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0>     {http_transport,connect,4,[{file,"http_transport.erl"},{line,82}]},
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0>     {httpc_handler,connect,4,[{file,"httpc_handler.erl"},{line,757}]}]
2024-09-11 09:35:10.399407+00:00 [error] <0.460.0>
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>   crasher:
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>     initial call: ssl_gen_statem:init/1
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>     pid: <0.460.0>
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>     registered_name: []
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>     exception error: {error,{"evp.c",136},"Can't make context"}
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>       in function  crypto:generate_key/3 (crypto.erl, line 1690)
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>       in call from ssl_cipher:'-generate_client_shares/1-anonymous-0-'/1 (ssl_cipher.erl, line 1174)
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>       in call from lists:map/2 (lists.erl, line 1559)
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>       in call from ssl_cipher:generate_client_shares/1 (ssl_cipher.erl, line 1176)
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>       in call from ssl_gen_statem:initial_hello/3 (ssl_gen_statem.erl, line 493)
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>       in call from gen_statem:loop_state_callback/11 (gen_statem.erl, line 1395)
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>       in call from tls_connection:init/1 (tls_connection.erl, line 162)
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>     ancestors: [<0.458.0>,tls_connection_sup,tls_sup,ssl_connection_sup,
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>                   ssl_sup,<0.98.0>]
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>     message_queue_len: 0
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>     messages: []
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>     links: [<0.458.0>]
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>     dictionary: [{ssl_manager,ssl_manager},
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>                   {log_level,notice},
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>                   {ssl_pem_cache,ssl_pem_cache}]
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>     trap_exit: true
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>     status: running
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>     heap_size: 46422
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>     stack_size: 28
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>     reductions: 130109
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>   neighbours:
2024-09-11 09:35:10.401964+00:00 [error] <0.460.0>
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>     supervisor: {<0.458.0>,tls_dyn_connection_sup}
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>     errorContext: child_terminated
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>     reason: {{error,{"evp.c",136},"Can't make context"},
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>              [{crypto,generate_key,3,
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>                       [{file,"crypto.erl"},
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>                        {line,1690},
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>                        {error_info,#{erl_function_arg_num => undefined}}]},
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>               {ssl_cipher,'-generate_client_shares/1-anonymous-0-',1,
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>                           [{file,"ssl_cipher.erl"},{line,1174}]},
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>               {lists,map,2,[{file,"lists.erl"},{line,1559}]},
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>               {ssl_cipher,generate_client_shares,1,
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>                           [{file,"ssl_cipher.erl"},{line,1176}]},
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>               {ssl_gen_statem,initial_hello,3,
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>                               [{file,"ssl_gen_statem.erl"},{line,493}]},
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>               {gen_statem,loop_state_callback,11,
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>                           [{file,"gen_statem.erl"},{line,1395}]},
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>               {tls_connection,init,1,[{file,"tls_connection.erl"},{line,162}]},
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>               {proc_lib,init_p_do_apply,3,
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>                         [{file,"proc_lib.erl"},{line,241}]}]}
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>     offender: [{pid,<0.460.0>},
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>                {id,receiver},
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>                {mfargs,{ssl_gen_statem,start_link,undefined}},
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>                {restart_type,temporary},
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>                {significant,true},
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>                {shutdown,5000},
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>                {child_type,worker}]
2024-09-11 09:35:10.403321+00:00 [error] <0.458.0>
2024-09-11 09:35:10.403535+00:00 [error] <0.255.0> Peer discovery: failed to query the list of nodes from the backend: {error,"{failed_connect,\n    [{to_address,{\"kubernetes.default.svc.cluster.local\",443}},\n     {inet,\n         [inet],\n         {eoptions,\n             {{{error,{\"evp.c\",136},\"Can't make context\"},\n               [{crypto,generate_key,3,\n                    [{file,\"crypto.erl\"},\n                     {line,1690},\n                     {error_info,#{erl_function_arg_num => undefined}}]},\n                {ssl_cipher,'-generate_client_shares/1-anonymous-0-',1,\n                    [{file,\"ssl_cipher.erl\"},{line,1174}]},\n                {lists,map,2,[{file,\"lists.erl\"},{line,1559}]},\n                {ssl_cipher,generate_client_shares,1,\n                    [{file,\"ssl_cipher.erl\"},{line,1176}]},\n                {ssl_gen_statem,initial_hello,3,\n                    [{file,\"ssl_gen_statem.erl\"},{line,493}]},\n                {gen_statem,loop_state_callback,11,\n                    [{file,\"gen_statem.erl\"},{line,1395}]},\n                {tls_connection,init,1,\n                    [{file,\"tls_connection.erl\"},{line,162}]},\n                {proc_lib,init_p_do_apply,3,\n                    [{file,\"proc_lib.erl\"},{line,241}]}]},\n              {gen_statem,call,[<0.455.0>,{start,2250},infinity]}}}}]}"}
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0> ** State machine <0.465.0> terminating
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0> ** Last event = {{call,{<0.461.0>,#Ref<0.2222038192.1010040833.2212>}},
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                  {start,2250}}
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0> ** When server state  = [{data,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                           [{"State",
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                             {initial_hello,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                              {state,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                               {static_env,client,gen_tcp,tls_gen_connection,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                tcp,tcp_closed,tcp_error,tcp_passive,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                "kubernetes.default.svc.cluster.local",443,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                #Port<0.38>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                #Ref<0.2222038192.1009909761.258981>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                #Ref<0.2222038192.1009909761.258980>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                ssl_client_session_cache_db,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                {ssl_crl_cache,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 {{#Ref<0.2222038192.1009909761.258984>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   #Ref<0.2222038192.1009909761.258985>},
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                  []}},
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                {#Ref<0.2222038192.1009909761.258982>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 #Ref<0.2222038192.1009909761.258983>},
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                #Ref<0.2222038192.1010040833.2215>,undefined},
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                               "***",
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                               #{partial_chain => #Fun<ssl.4.1732737>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 session_tickets => disabled,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 crl_cache => {ssl_crl_cache,{internal,[]}},
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 log_level => notice,protocol => tls,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 max_handshake_size => 131072,dh => "***",
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 customize_hostname_check => [],
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 certs_keys => "***",srp_identity => "***",
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 signature_algs_cert =>
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                  [default,eddsa_ed25519,eddsa_ed448,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   ecdsa_secp521r1_sha512,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   ecdsa_secp384r1_sha384,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   ecdsa_secp256r1_sha256,rsa_pss_pss_sha512,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   rsa_pss_pss_sha384,rsa_pss_pss_sha256,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   rsa_pss_rsae_sha512,rsa_pss_rsae_sha384,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   rsa_pss_rsae_sha256,rsa_pkcs1_sha512,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   rsa_pkcs1_sha384,rsa_pkcs1_sha256,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   {sha512,ecdsa},
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   {sha384,ecdsa},
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   {sha256,ecdsa},
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   rsa_pkcs1_sha1],
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 server_name_indication =>
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                  "kubernetes.default.svc.cluster.local",
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 eccs =>
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                  {elliptic_curves,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   [{1,3,101,110},
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                    {1,3,101,111},
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                    {1,3,132,0,35},
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                    {1,3,132,0,34},
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                    {1,2,840,10045,3,1,7}]},
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 versions => [{3,4},{3,3}],
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 signature_algs =>
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                  [eddsa_ed25519,eddsa_ed448,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   ecdsa_secp521r1_sha512,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   ecdsa_secp384r1_sha384,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   ecdsa_secp256r1_sha256,rsa_pss_pss_sha512,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   rsa_pss_pss_sha384,rsa_pss_pss_sha256,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   rsa_pss_rsae_sha512,rsa_pss_rsae_sha384,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   rsa_pss_rsae_sha256,rsa_pkcs1_sha512,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   rsa_pkcs1_sha384,rsa_pkcs1_sha256,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   {sha512,ecdsa},
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   {sha384,ecdsa},
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   {sha256,ecdsa}],
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 verify => verify_peer,handshake => full,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 max_fragment_length => undefined,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 reuse_session => undefined,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 renegotiate_at => 268435456,cacerts => "***",
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 user_lookup_fun => undefined,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 fallback => false,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 key_update_at => 388736063997,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 ciphers =>
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                  [<<19,2>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<19,1>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<19,3>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<19,4>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<19,5>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<"À,">>,<<"À0">>,<<"À­">>,<<"À¯">>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<"À$">>,<<"À(">>,<<"̩"/utf8>>,<<"̨"/utf8>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<"À+">>,<<"À/">>,<<"À¬">>,<<"À®">>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<"À.">>,<<"À2">>,<<"À&">>,<<"À*">>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<"À-">>,<<"À1">>,<<"À#">>,<<"À'">>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<"À%">>,<<"À)">>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<0,159>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<0,163>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<0,107>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<0,106>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<0,158>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<0,162>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<"̪"/utf8>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<0,103>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<0,64>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<"À\n">>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<192,20>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<192,5>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<192,15>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<"À\t">>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<192,19>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<192,4>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<192,14>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<0,57>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<0,56>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<0,51>>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   <<0,50>>],
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 crl_check => false,early_data => undefined,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 alpn_advertised_protocols => undefined,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 psk_identity => "***",
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 cacertfile =>
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                  <<"/var/run/secrets/kubernetes.io/serviceaccount/..data/ca.crt">>,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 secure_renegotiate => true,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 use_ticket => undefined,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 reuse_sessions => true,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 supported_groups =>
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                  {supported_groups,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                   [x25519,x448,secp256r1,secp384r1]},
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 verify_fun => undefined},
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                               {socket_options,binary,0,0,0,false},
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                               "***","***",not_requested,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                               #{sender => <0.464.0>,active_n => 100,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                                 active_n_toggle => true},
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                               "***",undefined,"***","***","***",undefined,
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0>                               undefined,undefined}}}]}]
2024-09-11 09:35:11.415382+00:00 [error] <0.465.0> ** Reason for termination = error:{error,{"evp.c",136},"Can't make context"`

openssl version used - OpenSSL 3.0.7 1 Nov 2022 (Library: OpenSSL 3.0.7 1 Nov 2022)
Erlang rpm used (26.2.5) - https://github.com/rabbitmq/erlang-rpm/releases?page=1

Expected behavior

As {crypto, [{fips_mode, false}]} is configured, where I expect that rabbitmq(non-fips) will come up

Additional context

No response

[Vishnu-0809]

Here it is mentioned that we dont have to compile erlang if we dont want to enable fips for rabbitmq alone on fips enabled env - erlang/otp#8797

[lukebakken]

Please familiarize yourself with GitHub's features for formatting large amounts of text, OR save logs to a file and ATTACH the file to a discussion. Pasting large amounts of text is lazy and difficult to read.

Team RabbitMQ's official community support policy states that we will not answer questions related to TLS or FIPS. Our expertise in those areas are reserved for RabbitMQ users who have paid for a support contract.

In your case, the very least you could have done is provide your full configuration files. There is little we can do to assist you.

Finally, I took the time to add {crypto, [{fips_mode, false}]} to my test project that builds a RabbitMQ cluster using TLS encrypted distributed Erlang, and it works fine. The issue is in your environment -

https://github.com/lukebakken/docker-rabbitmq-cluster/blob/rabbitmq-server-12278/rmq0/advanced.config#L3

[lukebakken]

This is where the Reason for termination = error:{error,{"evp.c",136},"Can't make context"} error originates:

https://github.com/erlang/otp/blob/OTP-26.2.5/lib/crypto/c_src/evp.c#L52-L53

You can see that it has something to do with Erlang's interaction with OpenSSL.

You state here:

So just wanted to know that if I am not doing that and using the erlang rpm I have mentioned above which is not fips supported, is there a way that I can use the default configuration (no fips) of erlang on the fips enabled env.

Then don't add any settings to rabbitmq.conf or advanced.config. RabbitMQ won't try to use FIPS.