-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] ReactorRejectedExecutionException when stopping the EventProcessorClient #44473
Comments
Thank you for your feedback. Tagging and routing to the team member best able to assist. |
Hi, I copy, pasted, and executed your code. It did not throw any exceptions. Do you have a DEBUG log from the beginning of your run to the end of it? Output I got:
|
Did you get a chance to run the same with the artifacts version what I provided (Added below for your reference)? I do not have the debug log but I can get one and share it here.
|
I created a new project and added the dependencies you listed. It does not run. Can you include a full project sample. What is the reason you're referencing azure-core, azure-messaging-eventhubs, azure-storage-blob explicitly? azure-messaging-eventhubs-checkpointstore-blob will pull them in transitively.
|
We expose API to our users who can access the azure storage blob storage account, azure datalake. Similarly we expose API to interact with azure event hub. Not all users are interested in event hub interaction. |
Please find the full project sample below. <?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>Evenhub_Error</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<maven.compiler.source>17</maven.compiler.source>
<maven.compiler.target>17</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<jakarta-jms.version>3.1.0</jakarta-jms.version>
<nimbus.version>9.48</nimbus.version>
<netty.version>4.1.118.Final</netty.version>
<netty-tcnative.version>2.0.70.Final</netty-tcnative.version>
<apache-httpclient5.version>5.4.2</apache-httpclient5.version>
<mssql-jdbc.version>12.8.1.jre11</mssql-jdbc.version>
<junit.version>5.12.0</junit.version>
<jackson.version>2.18.2</jackson.version>
<jackson.databind.version>2.18.2</jackson.databind.version>
</properties>
<dependencies>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-storage-blob</artifactId>
<version>12.29.0</version>
<exclusions>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-storage-blob-batch</artifactId>
<version>12.25.0</version>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-storage-common</artifactId>
<version>12.28.0</version>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-storage-file-datalake</artifactId>
<version>12.22.0</version>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-messaging-servicebus</artifactId>
<version>7.17.8</version>
</dependency>
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>azure-storage</artifactId>
<version>8.6.6</version>
<exclusions>
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-storage-file-share</artifactId>
<version>12.25.1</version>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-storage-queue</artifactId>
<version>12.24.0</version>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-messaging-eventhubs</artifactId>
<version>5.20.0</version>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-messaging-eventhubs-checkpointstore-blob</artifactId>
<version>1.20.3</version>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-core</artifactId>
<version>1.55.2</version>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-identity</artifactId>
<version>1.15.3</version>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-core-test</artifactId>
<version>1.26.2</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>jakarta.jms</groupId>
<artifactId>jakarta.jms-api</artifactId>
<version>${jakarta-jms.version}</version>
</dependency>
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>azure-servicebus</artifactId>
<version>3.6.7</version>
<exclusions>
<exclusion>
<groupId>net.minidev</groupId>
<artifactId>json-smart</artifactId>
</exclusion>
<exclusion>
<groupId>com.nimbusds</groupId>
<artifactId>nimbus-jose-jwt</artifactId>
</exclusion>
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</exclusion>
<exclusion>
<artifactId>okhttp</artifactId>
<groupId>com.squareup.okhttp3</groupId>
</exclusion>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.nimbusds</groupId>
<artifactId>nimbus-jose-jwt</artifactId>
<version>${nimbus.version}</version>
</dependency>
<dependency>
<groupId>org.apache.qpid</groupId>
<artifactId>qpid-jms-client</artifactId>
<version>2.7.0</version>
<exclusions>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
<version>${netty.version}</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-tcnative-boringssl-static</artifactId>
<version>${netty-tcnative.version}</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents.client5</groupId>
<artifactId>httpclient5</artifactId>
<version>${apache-httpclient5.version}</version>
</dependency>
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>msal4j</artifactId>
<version>1.19.0</version>
</dependency>
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<artifactId>mssql-jdbc</artifactId>
<version>${mssql-jdbc.version}</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>${junit.version}</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>${jackson.databind.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-csv</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-xml</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
<version>${jackson.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>flatten-maven-plugin</artifactId>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
</plugin>
<plugin>
<artifactId>maven-source-plugin</artifactId>
</plugin>
<plugin>
<artifactId>maven-jar-plugin</artifactId>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<forkCount>1</forkCount>
</configuration>
</plugin>
<plugin>
<artifactId>maven-javadoc-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
` |
Describe the bug
Bug: When stopping the EventProcessorClient with the latest version (5.20.0), it blows up with an exception ReactorRejectedExecutionException
Exception or Stack Trace
2025-02-25 09:16:31 ERROR GenericQuery:105 - LocalId:11akh4qxu3dzJUWmNNJM ERROR(128) Exception:reactor.core.Exceptions$ReactorRejectedExecutionException: Scheduler unavailable: Failed processing events
com.azure.core.exception.AzureException: reactor.core.Exceptions$ReactorRejectedExecutionException: Scheduler unavailable
at com.relx.rba.tardis.azure.eventhub.transport.EventHubPartitionProcessor.processError(EventHubPartitionProcessor.java:140) ~[classes/:?]
at com.azure.messaging.eventhubs.EventProcessorClientBuilder$1.processError(EventProcessorClientBuilder.java:909) ~[azure-messaging-eventhubs-5.20.0.jar:5.20.0]
at com.azure.messaging.eventhubs.PartitionPumpManager.handleError(PartitionPumpManager.java:380) ~[azure-messaging-eventhubs-5.20.0.jar:5.20.0]
at com.azure.messaging.eventhubs.PartitionPumpManager.lambda$startPartitionPump$5(PartitionPumpManager.java:223) ~[azure-messaging-eventhubs-5.20.0.jar:5.20.0]
at reactor.core.publisher.LambdaSubscriber.onError(LambdaSubscriber.java:149) ~[reactor-core-3.4.41.jar:3.4.41]
at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.trySchedule(FluxPublishOn.java:327) ~[reactor-core-3.4.41.jar:3.4.41]
at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.onNext(FluxPublishOn.java:237) ~[reactor-core-3.4.41.jar:3.4.41]
at reactor.core.publisher.FluxConcatMapNoPrefetch$FluxConcatMapNoPrefetchSubscriber.innerNext(FluxConcatMapNoPrefetch.java:258) ~[reactor-core-3.4.41.jar:3.4.41]
at reactor.core.publisher.FluxConcatMap$ConcatMapInner.onNext(FluxConcatMap.java:863) ~[reactor-core-3.4.41.jar:3.4.41]
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1839) ~[reactor-core-3.4.41.jar:3.4.41]
at reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onComplete(MonoCollectList.java:129) ~[reactor-core-3.4.41.jar:3.4.41]
at reactor.core.publisher.FluxWindowTimeout$InnerWindow.sendComplete(FluxWindowTimeout.java:1159) ~[reactor-core-3.4.41.jar:3.4.41]
at reactor.core.publisher.FluxWindowTimeout$WindowTimeoutWithBackpressureSubscriber.onComplete(FluxWindowTimeout.java:241) ~[reactor-core-3.4.41.jar:3.4.41]
at reactor.core.publisher.FluxPeek$PeekSubscriber.onComplete(FluxPeek.java:260) ~[reactor-core-3.4.41.jar:3.4.41]
at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onComplete(FluxDoFinally.java:128) ~[reactor-core-3.4.41.jar:3.4.41]
at reactor.core.publisher.FluxPeek$PeekSubscriber.onComplete(FluxPeek.java:260) ~[reactor-core-3.4.41.jar:3.4.41]
at reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:144) ~[reactor-core-3.4.41.jar:3.4.41]
at com.azure.core.amqp.implementation.MessageFlux$RecoverableReactorReceiver.terminateIfErrorOrCompletionSignaled(MessageFlux.java:601) ~[azure-core-amqp-2.9.12.jar:2.9.12]
at com.azure.core.amqp.implementation.MessageFlux$RecoverableReactorReceiver.drainLoop(MessageFlux.java:437) ~[azure-core-amqp-2.9.12.jar:2.9.12]
at com.azure.core.amqp.implementation.MessageFlux$RecoverableReactorReceiver.drain(MessageFlux.java:405) ~[azure-core-amqp-2.9.12.jar:2.9.12]
at com.azure.core.amqp.implementation.MessageFlux$ReactorReceiverMediator.onLinkComplete(MessageFlux.java:932) ~[azure-core-amqp-2.9.12.jar:2.9.12]
at com.azure.core.amqp.implementation.MessageFlux$ReactorReceiverMediator.lambda$onParentReady$3(MessageFlux.java:816) ~[azure-core-amqp-2.9.12.jar:2.9.12]
at reactor.core.publisher.LambdaSubscriber.onComplete(LambdaSubscriber.java:132) [reactor-core-3.4.41.jar:3.4.41]
at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.doComplete(FluxPublishOn.java:505) [reactor-core-3.4.41.jar:3.4.41]
at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.checkTerminated(FluxPublishOn.java:552) [reactor-core-3.4.41.jar:3.4.41]
at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.runAsync(FluxPublishOn.java:432) [reactor-core-3.4.41.jar:3.4.41]
at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.run(FluxPublishOn.java:527) [reactor-core-3.4.41.jar:3.4.41]
at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) [reactor-core-3.4.41.jar:3.4.41]
at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) [reactor-core-3.4.41.jar:3.4.41]
at java.base/java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264) [?:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java) [?:?]
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.base/java.lang.Thread.run(Thread.java:840) [?:?]
Caused by: reactor.core.Exceptions$ReactorRejectedExecutionException: Scheduler unavailable
at reactor.core.Exceptions.failWithRejected(Exceptions.java:271) ~[reactor-core-3.4.41.jar:3.4.41]
at reactor.core.publisher.Operators.onRejectedExecution(Operators.java:1024) ~[reactor-core-3.4.41.jar:3.4.41]
... 30 more
Caused by: java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@6e9afc06[Not completed, task = reactor.core.scheduler.WorkerTask@1a143d0e] rejected from BoundedScheduledExecutorService{IDLE, queued=0/10000, completed=3}
at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833) ~[?:?]
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) ~[?:?]
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:579) ~[?:?]
at reactor.core.scheduler.BoundedElasticScheduler$BoundedScheduledExecutorService.schedule(BoundedElasticScheduler.java:943) ~[reactor-core-3.4.41.jar:3.4.41]
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.submit(ScheduledThreadPoolExecutor.java:731) ~[?:?]
at reactor.core.scheduler.BoundedElasticScheduler$BoundedScheduledExecutorService.submit(BoundedElasticScheduler.java:1019) ~[reactor-core-3.4.41.jar:3.4.41]
at reactor.core.scheduler.Schedulers.workerSchedule(Schedulers.java:1439) ~[reactor-core-3.4.41.jar:3.4.41]
at reactor.core.scheduler.ExecutorServiceWorker.schedule(ExecutorServiceWorker.java:50) ~[reactor-core-3.4.41.jar:3.4.41]
at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.trySchedule(FluxPublishOn.java:312) ~[reactor-core-3.4.41.jar:3.4.41]
... 29 more
To Reproduce
Stop the EventProcessorClient. I have 2 partitions in the event hub.
Code Snippet
Below is the java code to reproduce this issue.
The text was updated successfully, but these errors were encountered: