Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: migrate logging log -> tracing #1720

Draft
wants to merge 45 commits into
base: main
Choose a base branch
from

Conversation

CommanderStorm
Copy link
Collaborator

@CommanderStorm CommanderStorm commented Mar 4, 2025

Lets split this up into two PRs

  • one changing the interface, tests and dependencys
  • one adding additional configuration options to martin AND documenting them

Alternatively, we can keep this PR open and I am going to get back to this sometime on the weekend

Resolves #836

we should continue using the RUST_LOG env var for all the configuration, with the possible addition of CLI and Config params

  • RUST_LOG
  • custom env variables MARTIN_LOG_FORMAT, MARTIN_LOG_LEVEL, ...
  • deduplicating code
  • CLI
  • Config
  • Writing docs (🦖)

Copy link
Member

@nyurik nyurik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs a few cleanups, but this might be ok... We should probably run some perf tests (we have that using oha in the justfile iirc) - to see if there are any significant slowdowns due to the switch

@CommanderStorm CommanderStorm marked this pull request as draft March 6, 2025 02:52
@CommanderStorm
Copy link
Collaborator Author

CommanderStorm commented Mar 8, 2025

So I ran the performance benchmarks..
Seems like there is a supprising performance difference..

Will investigate where this is coming from.. I have some theories, such as my terminal not being the best at printing formatted output and such => will test them and report back and update the table below

experiment
baseline/log + with terminal output via ssh image image image
log + no terminal output image image image
tracing + with terminal output via ssh image image image
tracing + no terminal output image image image
tracing+ with terminal output via ssh + MARTIN_LOG_FORMAT=json image image image
tracing+ with terminal output via ssh + NO request logging via actix-tracing, but via actix-middleware + MARTIN_LOG_FORMAT=json image image image
tracing+ with terminal output via ssh + request logging via actix-tracing, but NOT via actix-middleware + MARTIN_LOG_FORMAT=json image image image
tracing + with terminal output via ssh + request logging via actix-tracing, but NOT via actix-middleware + MARTIN_LOG_FORMAT=json + interest cache in tracing_log image image image
tracing + no terminal output + request logging via actix-tracing, but NOT via actix-middleware + interest cache in tracing_log image image image

@CommanderStorm
Copy link
Collaborator Author

CommanderStorm commented Mar 8, 2025

Okay.. So I got performance up there again. Was a good call to do benchmarks 😅

I think there are severe performance issues with my ssh-terminal and rendering special characters => I am going to discount the initial and only consider MARTIN_LOG_FORMAT=json
I don't think on a server, this is reasonably a problem.

The rqs went down on some of these, but 170k -> 130k is likely not something that users will see in reality.
Will look further into what is causing the slowdown there

@CommanderStorm

This comment was marked as off-topic.

@CommanderStorm
Copy link
Collaborator Author

So apparently some of our dependencys log via tracing (switch on the trace level to see it.. that is a lot).
There is the expected change from not having so much semi-const stuff.. I think the ~relatively small change is fine..

=> I think performance is not a bottleneck anymore. Will investigate configurability next

@nyurik
Copy link
Member

nyurik commented Mar 8, 2025

WOW! This is a very thorough job you did there! Thank you!

Ideally, we should continue using the RUST_LOG env var for all the configuration, with the possible addition of CLI and Config params. If we need more values like output format (vs output filtering), that may go into separate params too.

When doing performance, stdout + term + ssh is almost always a bad choice - there might be locking and syscall issues affecting it. Best to redirect it to /dev/null to avoid additional uncertainty, or perhaps dump to a file (but that might also be less than stable).

#![doc = include_str!("../README.md")]

#[derive(Default)]
pub struct MartinObservability {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bit unsure about the API/naming here..

I tried to make it less "magic", but unsure if this is the best way to handle it.

@nyurik nyurik changed the title chore: migrate logging log -> tracing feat: migrate logging log -> tracing Mar 10, 2025
serde::Serialize,
serde::Deserialize,
)]
pub enum LogFormatOptions {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently the bare formatting option (added to support mbtiles as before) looks like this:

image

We could remove the target via adding this to the layer
.map_event_format(|e| e.with_target(false))

=> should mbtiles remain "as-is" or gain targets?

@CommanderStorm
Copy link
Collaborator Author

No clue where CI and my local device is different.
Will investigate further in the last part of the week.

image
image

@CommanderStorm
Copy link
Collaborator Author

@nyurik how would you want me to split this up?
I don't think this is currently a reviewable amount..
How about martin-observability-utils and the rest? (or strictly on the crate boundary)

CommanderStorm added a commit that referenced this pull request Mar 10, 2025
During #1720 I noticed that there are a few places in the CLI where a
few more line breaks and handling markdown better could help
readability.

Yes, this adds the `unstable-markdown` feature of clap, but given that
removal would only impact the CLI-Formatting, I think here the risk is
okay.
Tracking issue:
- clap-rs/clap#5900


Hidden rationale:
I have one place in #1720 where making it readable without markdown
formatting is nearly impossible

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
nyurik and others added 3 commits March 10, 2025 23:03
During maplibre#1720 I noticed that there are a few places in the CLI where a
few more line breaks and handling markdown better could help
readability.

Yes, this adds the `unstable-markdown` feature of clap, but given that
removal would only impact the CLI-Formatting, I think here the risk is
okay.
Tracking issue:
- clap-rs/clap#5900

Hidden rationale:
I have one place in maplibre#1720 where making it readable without markdown
formatting is nearly impossible

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…re#1727)

Bumps [axios](https://github.com/axios/axios) from 1.7.7 to 1.8.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/axios/axios/releases">axios's
releases</a>.</em></p>
<blockquote>
<h2>Release v1.8.2</h2>
<h2>Release notes:</h2>
<h3>Bug Fixes</h3>
<ul>
<li><strong>http-adapter:</strong> add allowAbsoluteUrls to path
building (<a
href="https://redirect.github.com/axios/axios/issues/6810">#6810</a>)
(<a
href="https://github.com/axios/axios/commit/fb8eec214ce7744b5ca787f2c3b8339b2f54b00f">fb8eec2</a>)</li>
</ul>
<h3>Contributors to this release</h3>
<ul>
<li><!-- raw HTML omitted --> <a href="https://github.com/lexcorp16"
title="+1/-1 ([#6810](axios/axios#6810)
)">Fasoro-Joseph Alexander</a></li>
</ul>
<h2>Release v1.8.1</h2>
<h2>Release notes:</h2>
<h3>Bug Fixes</h3>
<ul>
<li><strong>utils:</strong> move <code>generateString</code> to platform
utils to avoid importing crypto module into client builds; (<a
href="https://redirect.github.com/axios/axios/issues/6789">#6789</a>)
(<a
href="https://github.com/axios/axios/commit/36a5a620bec0b181451927f13ac85b9888b86cec">36a5a62</a>)</li>
</ul>
<h3>Contributors to this release</h3>
<ul>
<li><!-- raw HTML omitted --> <a
href="https://github.com/DigitalBrainJS" title="+51/-47
([#6789](axios/axios#6789) )">Dmitriy
Mozgovoy</a></li>
</ul>
<h2>Release v1.8.0</h2>
<h2>Release notes:</h2>
<h3>Bug Fixes</h3>
<ul>
<li><strong>examples:</strong> application crashed when navigating
examples in browser (<a
href="https://redirect.github.com/axios/axios/issues/5938">#5938</a>)
(<a
href="https://github.com/axios/axios/commit/1260ded634ec101dd5ed05d3b70f8e8f899dba6c">1260ded</a>)</li>
<li>missing word in SUPPORT_QUESTION.yml (<a
href="https://redirect.github.com/axios/axios/issues/6757">#6757</a>)
(<a
href="https://github.com/axios/axios/commit/1f890b13f2c25a016f3c84ae78efb769f244133e">1f890b1</a>)</li>
<li><strong>utils:</strong> replace getRandomValues with crypto module
(<a
href="https://redirect.github.com/axios/axios/issues/6788">#6788</a>)
(<a
href="https://github.com/axios/axios/commit/23a25af0688d1db2c396deb09229d2271cc24f6c">23a25af</a>)</li>
</ul>
<h3>Features</h3>
<ul>
<li>Add config for ignoring absolute URLs (<a
href="https://redirect.github.com/axios/axios/issues/5902">#5902</a>)
(<a
href="https://redirect.github.com/axios/axios/issues/6192">#6192</a>)
(<a
href="https://github.com/axios/axios/commit/32c7bcc0f233285ba27dec73a4b1e81fb7a219b3">32c7bcc</a>)</li>
</ul>
<h3>Reverts</h3>
<ul>
<li>Revert &quot;chore: expose fromDataToStream to be consumable (<a
href="https://redirect.github.com/axios/axios/issues/6731">#6731</a>)&quot;
(<a
href="https://redirect.github.com/axios/axios/issues/6732">#6732</a>)
(<a
href="https://github.com/axios/axios/commit/1317261125e9c419fe9f126867f64d28f9c1efda">1317261</a>),
closes <a
href="https://redirect.github.com/axios/axios/issues/6731">#6731</a> <a
href="https://redirect.github.com/axios/axios/issues/6732">#6732</a></li>
</ul>
<h3>BREAKING CHANGES</h3>
<ul>
<li>
<p>code relying on the above will now combine the URLs instead of prefer
request URL</p>
</li>
<li>
<p>feat: add config option for allowing absolute URLs</p>
</li>
<li>
<p>fix: add default value for allowAbsoluteUrls in buildFullPath</p>
</li>
<li>
<p>fix: typo in flow control when setting allowAbsoluteUrls</p>
</li>
</ul>
<h3>Contributors to this release</h3>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/axios/axios/blob/v1.x/CHANGELOG.md">axios's
changelog</a>.</em></p>
<blockquote>
<h2><a
href="https://github.com/axios/axios/compare/v1.8.1...v1.8.2">1.8.2</a>
(2025-03-07)</h2>
<h3>Bug Fixes</h3>
<ul>
<li><strong>http-adapter:</strong> add allowAbsoluteUrls to path
building (<a
href="https://redirect.github.com/axios/axios/issues/6810">#6810</a>)
(<a
href="https://github.com/axios/axios/commit/fb8eec214ce7744b5ca787f2c3b8339b2f54b00f">fb8eec2</a>)</li>
</ul>
<h3>Contributors to this release</h3>
<ul>
<li><!-- raw HTML omitted --> <a href="https://github.com/lexcorp16"
title="+1/-1 ([#6810](axios/axios#6810)
)">Fasoro-Joseph Alexander</a></li>
</ul>
<h2><a
href="https://github.com/axios/axios/compare/v1.8.0...v1.8.1">1.8.1</a>
(2025-02-26)</h2>
<h3>Bug Fixes</h3>
<ul>
<li><strong>utils:</strong> move <code>generateString</code> to platform
utils to avoid importing crypto module into client builds; (<a
href="https://redirect.github.com/axios/axios/issues/6789">#6789</a>)
(<a
href="https://github.com/axios/axios/commit/36a5a620bec0b181451927f13ac85b9888b86cec">36a5a62</a>)</li>
</ul>
<h3>Contributors to this release</h3>
<ul>
<li><!-- raw HTML omitted --> <a
href="https://github.com/DigitalBrainJS" title="+51/-47
([#6789](axios/axios#6789) )">Dmitriy
Mozgovoy</a></li>
</ul>
<h1><a
href="https://github.com/axios/axios/compare/v1.7.9...v1.8.0">1.8.0</a>
(2025-02-25)</h1>
<h3>Bug Fixes</h3>
<ul>
<li><strong>examples:</strong> application crashed when navigating
examples in browser (<a
href="https://redirect.github.com/axios/axios/issues/5938">#5938</a>)
(<a
href="https://github.com/axios/axios/commit/1260ded634ec101dd5ed05d3b70f8e8f899dba6c">1260ded</a>)</li>
<li>missing word in SUPPORT_QUESTION.yml (<a
href="https://redirect.github.com/axios/axios/issues/6757">#6757</a>)
(<a
href="https://github.com/axios/axios/commit/1f890b13f2c25a016f3c84ae78efb769f244133e">1f890b1</a>)</li>
<li><strong>utils:</strong> replace getRandomValues with crypto module
(<a
href="https://redirect.github.com/axios/axios/issues/6788">#6788</a>)
(<a
href="https://github.com/axios/axios/commit/23a25af0688d1db2c396deb09229d2271cc24f6c">23a25af</a>)</li>
</ul>
<h3>Features</h3>
<ul>
<li>Add config for ignoring absolute URLs (<a
href="https://redirect.github.com/axios/axios/issues/5902">#5902</a>)
(<a
href="https://redirect.github.com/axios/axios/issues/6192">#6192</a>)
(<a
href="https://github.com/axios/axios/commit/32c7bcc0f233285ba27dec73a4b1e81fb7a219b3">32c7bcc</a>)</li>
</ul>
<h3>Reverts</h3>
<ul>
<li>Revert &quot;chore: expose fromDataToStream to be consumable (<a
href="https://redirect.github.com/axios/axios/issues/6731">#6731</a>)&quot;
(<a
href="https://redirect.github.com/axios/axios/issues/6732">#6732</a>)
(<a
href="https://github.com/axios/axios/commit/1317261125e9c419fe9f126867f64d28f9c1efda">1317261</a>),
closes <a
href="https://redirect.github.com/axios/axios/issues/6731">#6731</a> <a
href="https://redirect.github.com/axios/axios/issues/6732">#6732</a></li>
</ul>
<h3>BREAKING CHANGES</h3>
<ul>
<li>
<p>code relying on the above will now combine the URLs instead of prefer
request URL</p>
</li>
<li>
<p>feat: add config option for allowing absolute URLs</p>
</li>
<li>
<p>fix: add default value for allowAbsoluteUrls in buildFullPath</p>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/axios/axios/commit/a9f7689b0c4b6d68c7f587c3aa376860da509d94"><code>a9f7689</code></a>
chore(release): v1.8.2 (<a
href="https://redirect.github.com/axios/axios/issues/6812">#6812</a>)</li>
<li><a
href="https://github.com/axios/axios/commit/fb8eec214ce7744b5ca787f2c3b8339b2f54b00f"><code>fb8eec2</code></a>
fix(http-adapter): add allowAbsoluteUrls to path building (<a
href="https://redirect.github.com/axios/axios/issues/6810">#6810</a>)</li>
<li><a
href="https://github.com/axios/axios/commit/98120457559e573024862e2925d56295a965ad7e"><code>9812045</code></a>
chore(sponsor): update sponsor block (<a
href="https://redirect.github.com/axios/axios/issues/6804">#6804</a>)</li>
<li><a
href="https://github.com/axios/axios/commit/72acf759373ef4e211d5299818d19e50e08c02f8"><code>72acf75</code></a>
chore(sponsor): update sponsor block (<a
href="https://redirect.github.com/axios/axios/issues/6794">#6794</a>)</li>
<li><a
href="https://github.com/axios/axios/commit/2e64afdff5c41e38284a6fb8312f2745072513a1"><code>2e64afd</code></a>
chore(release): v1.8.1 (<a
href="https://redirect.github.com/axios/axios/issues/6800">#6800</a>)</li>
<li><a
href="https://github.com/axios/axios/commit/36a5a620bec0b181451927f13ac85b9888b86cec"><code>36a5a62</code></a>
fix(utils): move <code>generateString</code> to platform utils to avoid
importing crypto...</li>
<li><a
href="https://github.com/axios/axios/commit/cceb7b1e154fbf294135c93d3f91921643bbe49f"><code>cceb7b1</code></a>
chore(release): v1.8.0 (<a
href="https://redirect.github.com/axios/axios/issues/6795">#6795</a>)</li>
<li><a
href="https://github.com/axios/axios/commit/23a25af0688d1db2c396deb09229d2271cc24f6c"><code>23a25af</code></a>
fix(utils): replace getRandomValues with crypto module (<a
href="https://redirect.github.com/axios/axios/issues/6788">#6788</a>)</li>
<li><a
href="https://github.com/axios/axios/commit/32c7bcc0f233285ba27dec73a4b1e81fb7a219b3"><code>32c7bcc</code></a>
feat: Add config for ignoring absolute URLs (<a
href="https://redirect.github.com/axios/axios/issues/5902">#5902</a>)
(<a
href="https://redirect.github.com/axios/axios/issues/6192">#6192</a>)</li>
<li><a
href="https://github.com/axios/axios/commit/4a3e26cf65bb040b7eb4577d5fd62199b0f3d017"><code>4a3e26c</code></a>
chore(config): adjust rollup config to preserve license header to
minified Ja...</li>
<li>Additional commits viewable in <a
href="https://github.com/axios/axios/compare/v1.7.7...v1.8.2">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=axios&package-manager=npm_and_yarn&previous-version=1.7.7&new-version=1.8.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/maplibre/martin/network/alerts).

</details>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
CommanderStorm added a commit that referenced this pull request Mar 11, 2025
During #1720 I noticed that these fields are missing
This PR adds them for completeness ^^
@nyurik
Copy link
Member

nyurik commented Mar 11, 2025

Good question, I wish i knew the best way to proceed. Thinking out loud:

  • if perf is similar, and all existing functionality is preserved, we can introduce some "log format" param into the ecosystem
  • the monitoring utility is clearly a separate PR, perhaps martin-monitor / martin-watch / martin-log / martin-* -- my guess is that this should be a purely CLI tool for any kind of debugging, perhaps with a nice UI with Ratatui? The only uncertainty is what happens when multiple martin instances are running?

What do you think?

@CommanderStorm
Copy link
Collaborator Author

CommanderStorm commented Mar 11, 2025

  • the monitoring utility is clearly a separate PR

I think there is some misunderstanding.
I am talking about the code pulled from the three crates to configure tracing.

There is no TUI involved and adding that would involve quite a bit more code than I think we should add for now (bigger issues and stuff ^^).

About naming:
The name of the crate was to enable tracing/metrics to be outsourced there too and the utils part is because the other -utils crate exists.

Thinking about it:
In the other crates in this repo, tracing/metrics are not relevant as they are CLI tools.
This is only relevant for martin => extracting this to martin-observability-utils is likely a bad choice..
Likely martin-tracing is a better choice, or should I stick with martin-tracing-utils?

@CommanderStorm
Copy link
Collaborator Author

Above I was talking about making this PR reviewable by splitting it up into smaller PRs.
Currently this is a +900 -300 lines chonker of a PR (even though a large part of this is metadata and trivial changes..). There is still a lot of implementation.

I will split this up into:

  • one PR laying the basic groundwork (martin-tracing/martin-tracing-utils) => looking over if the API/naming I came up with is okay..
  • one PR integrating it on a basic level (rip out env-logger, change request logging, add CLI/cofig file option)
  • one PR doing the rest (rip out log, ..) <-- adding docs pushed to here as this is would be nice to release soon-ish after documenting

Alternative would be
I will split this up into:

  • one PR laying the basic groundwork (martin-tracing/martin-tracing-utils)
  • one PR integrating this into martin/martin-cp
  • one PR integrating this into mbtiles

CommanderStorm and others added 6 commits March 11, 2025 14:39
Resolves maplibre#1475

One possible concern is that i32 is now used as i64 with sqlx. This
**might** impact `tile_xxh3_64_hash` value validity, but I don't see
conclusive evidence of that. Note that the tests were incorrectly dumped
as i32, thus had to be updated, but it does not seem to affect the
actual storage or validation...

---------

Co-authored-by: Frank Elsinga <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow jsonl-style logging for servers
2 participants