Skip to content

Conversation

@abarciauskas-bgse
Copy link
Contributor

No description provided.

@github-actions
Copy link

github-actions bot commented Jun 27, 2025

📚 Documentation preview will be available at: https://developmentseed.github.io/titiler-cmr/pr-previews/pr-66/

Status: ✅ Preview is ready!

github-actions bot pushed a commit that referenced this pull request Jun 27, 2025
github-actions bot pushed a commit that referenced this pull request Jun 27, 2025
github-actions bot pushed a commit that referenced this pull request Jun 27, 2025
github-actions bot pushed a commit that referenced this pull request Jun 27, 2025
@abarciauskas-bgse
Copy link
Contributor Author

@hrodmn this produces logs like:

Screenshot 2025-06-27 at 10 51 21 AM

What do you think? I did a brief review of https://docs.python.org/3/library/logging.config.html#logging-config-dictschema and everything in the example config seems reasonable, although I removed the additional "detailed" formatter.

github-actions bot pushed a commit that referenced this pull request Jun 27, 2025
@hrodmn
Copy link
Contributor

hrodmn commented Jun 27, 2025

What do you think? I did a brief review of https://docs.python.org/3/library/logging.config.html#logging-config-dictschema and everything in the example config seems reasonable, although I removed the additional "detailed" formatter.

I think it is possible to make all of the nested parameters more like actual JSON instead of just strings. The middleware should pass all of info via the extra arg and CloudWatch should just find it (I think). https://github.com/developmentseed/titiler/blob/5f04b4cf54ebecbe2e5b0b07290ecf0164af14b3/src/titiler/core/titiler/core/middleware.py#L134-L137

I would try disabling the custom config part and see what it looks like in CloudWatch.

github-actions bot pushed a commit that referenced this pull request Jun 27, 2025
@abarciauskas-bgse
Copy link
Contributor Author

@hrodmn I just had to sanity check myself but that is what I tried initially:

app.add_middleware(
    LoggerMiddleware,
    logger=logging.getLogger("titiler.requests"),
)

with no custom configDict and all I saw is the request received message (no extra=data), e.g.
Screenshot 2025-06-27 at 5 04 29 PM

But let me do some digging and see if I can figure out why that is...

github-actions bot pushed a commit that referenced this pull request Jun 28, 2025
github-actions bot pushed a commit that referenced this pull request Jun 28, 2025
@abarciauskas-bgse
Copy link
Contributor Author

Ok so my understanding is the extra field doesn't actually get logged when you include the LoggerMiddleware they are just available as additional data (or "context") to use in creating your own custom logging format.

The fourth keyword argument is extra which can be used to pass a dictionary which is used to populate the dict of the LogRecord created for the logging event with user-defined attributes. ref

This app already had a basicConfig for all logs. In the most recent commit I replaced that basicConfig to be a part of the logging.config.configDict mostly to reduce confusion, I think we could revert that change. But we can include both loggers in the configDict object which makes it a bit more clear that non-request logs will go to the the root logger which uses that default format and request logs will be logged using the LoggerMiddleware which includes all of the additional fields using the titiler.requests logger and the JSONFormatter.

github-actions bot pushed a commit that referenced this pull request Jun 28, 2025
@abarciauskas-bgse
Copy link
Contributor Author

Logs now appear like:

2025-06-28T01:12:44.234Z
{
    "timestamp": "2025-06-28 01:12:44,233",
    "level": "INFO",
    "logger": "titiler.requests",
    "message": "Request received: /tiles/WebMercatorQuad/0/0/0@3x GET",
    "method": "GET",
    "path": "/tiles/WebMercatorQuad/0/0/0@3x",
    "query_params": {
        "datetime": "2021-12-01T00:00:00.000Z/2021-12-01T23:59:59.999Z",
        "concept_id": "C2837626477-GES_DISC",
        "variable": "o3",
        "backend": "xarray",
        "colormap_name": "reds",
        "rescale": "20,70",
        "sel_method": "nearest",
        "reScale": "0,5000",
        "sel": "time=2021-12-01T00:00:00.000Z"
    },
    "path_params": {
        "tileMatrixSetId": "WebMercatorQuad",
        "z": "0",
        "x": "0",
        "y": "0",
        "scale": "3"
    },
    "headers": {
        "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
        "accept-encoding": "gzip, deflate, br, zstd",
        "accept-language": "en-US,en;q=0.5",
        "content-length": "0",
        "host": "v4jec6i5c0.execute-api.us-west-2.amazonaws.com",
        "priority": "u=0, i",
        "sec-fetch-dest": "document",
        "sec-fetch-mode": "navigate",
        "sec-fetch-site": "none",
        "sec-fetch-user": "?1",
        "upgrade-insecure-requests": "1",
        "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:139.0) Gecko/20100101 Firefox/139.0",
        "x-amzn-trace-id": "Root=1-685f4185-49cfc0c94df277865595483f",
        "x-forwarded-for": "174.21.145.29",
        "x-forwarded-port": "443",
        "x-forwarded-proto": "https"
    },
    "referer": null,
    "origin": null,
    "route": "/tiles/{tileMatrixSetId}/{z}/{x}/{y}@{scale}x"
}

@abarciauskas-bgse abarciauskas-bgse marked this pull request as ready for review June 28, 2025 01:16
github-actions bot pushed a commit that referenced this pull request Jun 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants