Discussion on Zephyr's Poor Real-Time Performance

[lenghonglin]

Recently, the well-known embedded systems expert Jacob Beningo wrote the '2024 Real-Time Operating System Performance Report.' The report tested several commonly used RTOSes, including PX5, ThreadX, Zephyr, and FreeRTOS, to evaluate real-time performance through test cases such as coroutine scheduling, preemptive scheduling, memory allocation, message handling, and synchronization processing. Based on the results, Zephyr's real-time performance was nearly the worst. What could be the reason for this?

Beningo-RtosPerformance-2024-Report.pdf

[cocoeoli]

From this result, Zephyr still has a lot of space for performance optimization.

[heinwessels]

You should read the follow up article by the same person: https://www.beningo.com/is-rtos-performance-affected-by-an-mpu/#

Essentially, in the initial benchmark you posted Zephyr had a Memory Protection Unit enabled (MPU) enabled, which has an performance impact because it adds safety features.

Quote from the article:

Zephyr jumped from a relative performance against PX5 RTOS of 42% to 70.1%! That’s a 28% improvement in the benchmark and a nearly 67% improvement against itself! All because the MPU and a few additional checks were disabled!
You might argue that disabling the MPU in the benchmark is cheating, but since the other RTOSes in the study did not have them enabled, it really levels the playing field and makes it possible to compare the RTOSes more accurately.

[aniltirli]

As mentioned in that article Zephyr's defaults are really MPU, asserttions and error-checking enabled ? I am not sure about that, in my understanding from the documentation, i think you need to set those in config file to enable them, am i wrong ?

[nashif]

quick update as things improve in zephyr and some of the issues are being addressed. Here is the state of things:

++ with changes currently in review and pending merge.

[DBS06]

@nashif Hi, is it possible to have access to the test code which you are using?
I am doing right now a similar benchmarks for different RTOSes and I am curious to see the configuration and the test code.

[henrikbrixandersen]

@DBS06 Have you seen https://github.com/zephyrproject-rtos/zephyr/tree/main/tests/benchmarks/thread_metric ?

[DBS06]

@DBS06 Have you seen https://github.com/zephyrproject-rtos/zephyr/tree/main/tests/benchmarks/thread_metric ?

Thanks a lot!!!
Actually I am aware of zephyr/tests/benchmarks/ but I wasn't aware of this specific one. I started my benchmark-project 3 months ago and had to stop it because other things were more urgent.
So thanks a lot!

[nashif]

Update just before feature freeze of 4.1..... (c8f640a)

Few changes currently in review would improve things even more, stay tuned.

[romkell]

Also if that discussion is a bit dated:

Do we also have figures on the kernels worst case IRQ lock-out times?

ARM Cortex-M >= M3 do have an interrupt controller (NVIC) with ISR level lock register (BASEPRI) allowing for CONFIG_ZERO_LATENCY_IRQS.
ARM Cortex-M <= M1 and other architectures do not have that feature to avoid the kernel locking critical ISRs introducing some jitter to their timing.

Do we know those figures as clock cycles related to the gcc version and instruction set or probably simpler as time on certain SOC derivatives e.g. ARM Cortex-M0+ or others @ defined clock speeds and gcc version?