Why does DataDogProvider add_metric forceset a timestamp

[manzkel]

Hey,

we have been trying to send custom metrics with lambda powertools to datadog for a few days. While enhanced metrics were sent and received on Datadog, custom metrics did not show up.

After some debugging we found something peculiar in the DatadogProvider and would like to know if there is a reason behind the code and whether there are certain settings to be set to make it work.

Concretely, here

powertools-lambda-python/aws_lambda_powertools/metrics/provider/datadog/datadog.py

Line 117 in 85666bb

if not timestamp:

the setting of a timestamp is forced. Which in turn on datadog end forces the execution of this condition https://github.com/DataDog/datadog-lambda-python/blob/6e2e5d54b870bbe5d2b028e726fef1b5d8f70552/datadog_lambda/metric.py#L78

The extension does not support timestamps for distributions so we create a
a thread stats writer to submit metrics with timestamps to the API

Running this on our end did not succeed in sending / receiving custom metrics. Creating a custom Provider which does not override the timestamp however, allowed it to succeed.

Are there some settings we are missing to be able to use the Thread Stats Writer or is this an unwanted behavior?

Thank you!

[leandrodamascena]

Checking it now.

[leandrodamascena]

Hi @manzkel, I already have a branch working locally. We are releasing a version tomorrow, but I won't have time to include this change, so I hope to release it in the next few weeks.

BTW, can you confirm that change regarding DD_FLUSH_LOG was resolved?

[manzkel]

Any news on your fix? DD_FLUSH_LOG should be good :)

[manzkel]

Hi @leandrodamascena sorry to ping you again, but any updates on this?

[leandrodamascena]

Hey hey @manzkel! Really sorry for a long time, As I said: I already have a branch with some code working but didn't have time to test a little bit more and release.

I'll work to get this branch ready to push ASAP.

Thanks

[leandrodamascena]

Hey @manzkel thanks a lot for your huge patience on this topic! I was testing exhaustively this problem and I'm not sure if I'm reproducing the error.. I'm using this those layers in my Lambda with Python 3.12:

 Layers:
  - !Sub arn:aws:lambda:${AWS::Region}:464622532012:layer:Datadog-Python312:78
  - !Sub arn:aws:lambda:${AWS::Region}:464622532012:layer:Datadog-Extension:45

I tried lot of times sending data both with timestamp and without timestamp and in both cases I was able to create custom metrics + enhanced metrics in DataDog

If it's still possible, can you help me understand a little better what's happening on your side?

I'm converting this discussion into an issue.

[leandrodamascena]

Closing this discussion. It was converted into an issue #6961