If the evaluation window of the query is too short, the api might return an empty result which datadog-service treats as 0 and fails the evaluation

[vadasambar]

Problem

• Same as title
• This needs more research but the problem is, say the query is avg:system.load.1{*}.rollup(avg, $DURATION) and if the calculated $DURATION in the query is too short, Datadog API returns empty results. $DURATION is the time duration of test execution. If the tests run only for a short period of time, the evaluation window becomes short as well.
• I have noticed this happening with the system.load metric if the $DURATION is less than 10s.
• When the query results an empty response, datadog service treats it as a 0 and fails the SLO evaluation
• I think the issue occurs because the query $DURATION is shorter than time interval between two successive metric data exports

[vadasambar]

Might be related: #8