Repository navigation
[SYS] Re-enable serial logs on esp32c3_lolin_mini - #2429
MatthieuGrr wants to merge 1 commit into
Conversation
The silent log level was a workaround for the native USB CDC stall fixed in 1technophile#2428 (non-blocking HWCDC writes). With that fix, logging no longer delays message processing when no serial listener is connected. Signed-off-by: Matthieu <93468141+MatthieuGrr@users.noreply.github.com>
1technophile
left a comment
There was a problem hiding this comment.
Thanks for following up on #2428. Re-enabling logs on this env makes sense now that the USB-CDC writes are non-blocking, but a few things are worth checking first (details inline):
- With logs compiled in, the WebUI console capture (
addLog()) becomes reachable on this env, and it has an unlocked shared buffer that can overflow by 1–2 bytes under task preemption. - The soak in the description shows a 912 B minimum free heap. It would help to see the same soak on the current
LOG_LEVEL_SILENTbuild for comparison.
Outside the diff:
esp32c3_lolin_mini_with_serialis now effectively identical to the base env. Its extraCONFIG_ESP_CONSOLE_*-Dflags don't affect the precompiled Arduino libs, and-DCONFIG_ESP_CONSOLE_UART=ESP_CONSOLE_USB_SERIAL_JTAGredefines a boolean sdkconfig macro with an undefined token (redefinition warning; any#if CONFIG_ESP_CONSOLE_UARTthen evaluates to 0). I'd suggest deleting that env here, from bothenvironments.iniand.github/workflows/environments.json, so CI and the web installer don't ship two indistinguishable C3 images.
| ; Serial output currently causes a delay on each message, if no serial listener is connected. | ||
| ; As a default we disable serial output. Use the _with_serial environment if you | ||
| ; need serial output for debugging purposes. | ||
| '-DLOG_LEVEL=LOG_LEVEL_SILENT' ; disable serial logging. |
There was a problem hiding this comment.
With LOG_LEVEL_SILENT gone, every THEENGS_LOG_* call also goes through SerialWeb::write() → addLog() (main/webUI.cpp) once someone has opened the WebUI console and log_buffer is allocated. addLog() uses the global line (malloc'd ROW_LENGTH = 512) and lineIndex with no lock. If the loop task passes the lineIndex >= ROW_LENGTH - 2 check at index 509 and is then preempted by procBLETask (priority 2, same core on the single-core C3), which also logs, the loop task resumes and writes line[511] plus the terminator at line[512]. That is a heap overflow, which shows up later as Bad head or a panic.
The bug isn't C3-specific, but until now this env never ran that path because all logging compiled out. A mutex around addLog() (or a per-call local buffer) would fix it, either here or in a separate PR before this lands.
| ; Serial output currently causes a delay on each message, if no serial listener is connected. | ||
| ; As a default we disable serial output. Use the _with_serial environment if you | ||
| ; need serial output for debugging purposes. | ||
| '-DLOG_LEVEL=LOG_LEVEL_SILENT' ; disable serial logging. |
There was a problem hiding this comment.
Memory margin: the soak in the description reaches a 912 B minimum free heap with logging on. Both columns of the table come from the log-enabled env, so nothing shows how much of that comes from re-enabling logs versus what the current silent build reaches under the same ~260 msg/min load. Could you rerun the soak on the current env for comparison? With under 1 KB headroom, a transient WiFiClient, ArduinoJson or mbedTLS allocation during a BLE burst can fail and drop publishes or reboot the gateway.
Related: procBLETask is created with a fixed 8000 B stack (main/gatewayBT.cpp) and coreTask with 5120 B. Those margins were sized with logging compiled out. It would be worth logging uxTaskGetStackHighWaterMark() for both during the soak to confirm the added log formatting frames still fit.
| ; Serial output currently causes a delay on each message, if no serial listener is connected. | ||
| ; As a default we disable serial output. Use the _with_serial environment if you | ||
| ; need serial output for debugging purposes. | ||
| '-DLOG_LEVEL=LOG_LEVEL_SILENT' ; disable serial logging. |
There was a problem hiding this comment.
Minor, but relevant now that logs are visible: with SERIAL_TX_TIMEOUT_MS=0 from #2428, HWCDC::flush() doesn't wait for the TX ring to drain. In main.cpp, THEENGS_LOG_WARNING("Rebooting for reason code %d") is followed immediately by ESP.restart(), so that line (and any OTA or erase messages still in the ring) usually never reaches the host, even though it's the line that explains the reset. A short bounded drain before restart (e.g. wait until Serial.availableForWrite() recovers, capped at ~100 ms) would keep it.
| ; Serial output currently causes a delay on each message, if no serial listener is connected. | ||
| ; As a default we disable serial output. Use the _with_serial environment if you | ||
| ; need serial output for debugging purposes. | ||
| '-DLOG_LEVEL=LOG_LEVEL_SILENT' ; disable serial logging. |
There was a problem hiding this comment.
Flash headroom: with the log strings compiled back in, the image is 1,896,311 / 1,966,080 B (96.5%) of the min_spiffs app slot, leaving about 69 KB. That's fine today, but the next Theengs Decoder or NimBLE bump could push this env over (section .flash.text will not fit). It's worth knowing about, and maybe noting in the env comment.
Description:
Follow-up to #2428, as announced there: re-enables serial logging on
esp32c3_lolin_mini.The env was built with
LOG_LEVEL_SILENTandWM_DEBUG=0because serial output delayed every message when no listener was connected (native USB CDC). #2428 makes HWCDC writes non-blocking, so this workaround is no longer needed and the default log level applies again.esp32c3_lolin_mini_with_serialis left untouched, but it is now nearly identical toesp32c3_lolin_mini: it only re-sets the defaultLOG_LEVEL_NOTICEand adds twoCONFIG_ESP_CONSOLE_*defines. Happy to remove it (fromenvironments.iniandenvironments.json) in this PR or in a follow-up, as you prefer.Testing
Build:
esp32c3_lolin_minion this commit, no warnings, flash 96.5 % (1,896,311 / 1,966,080 bytes), RAM 17.5 %.ESP32-C3 board (not a Lolin, but same chip and native USB-Serial/JTAG port) flashed with this env, plugged into a Linux host, BLE scan
intervalset to 100 to load the loop. Same env (this PR'sesp32c3_lolin_mini+ MQTT presets) built twice: ondevelopment@ 9b09fcd (with #2428) and on 5e7d3cb (without it). Three phases each: port read continuously, port opened and never read, port closed.offline5 times over 17 min, SYStoMQTT queued then flushed at once; recovered without reboot, then ~43 msg/minWith #2428 the gateway ran 1 h 45 through all three phases without a single LWT
offline(and was still online after more than 2 h). Opening the port reboots the board once in both builds (DTR/RTS untouched), unrelated to this change.Checklist: