Parsing Http output

[pradeep-raja]

I can parse JSON responses using jq when I use the silent option with http. This works because it returns only the body of the response. However, when I disable the silent option, the output includes response code and headers, making it non-JSON and preventing me from parsing it with jq.

Is there any way or support in dagu for parsing status code and the response headers along with the JSON body?

Sample yaml file with silent true.

`steps:

• name: step1
  command: POST http://localhost:8080/test1
  output: DATA1
  executor:
  type: http
  config:
  timeout: 10
  headers:
  Authorization: "Bearer test token"
  silent: true # Making silent true I only get body
  query:
  key: "value"
• name: datastep
  executor:
  type: jq
  config:
  raw: true
  command: '.id' # I am able to parse since I get only response body which is json.
  script: |
  $DATA1
  output: ID
  depends:
  • step1`

[yohamta]

Hi,

Thanks for the detailed example. Can you please clarify a bit about how you want to handle the status code? One idea is to have a standard JSON schema like the example below. How does that sound?

{
  "status_code": 200,
  "body": {
    "some_field": "some_value"
  }
}

[pradeep-raja]

This is perfect. Can we also include headers?.

{ "status_code": 200, "body": { "some_field": "some_value" }, "headers" : { "some_header" : "header_value" } }

[yohamta]

Yes, it's possible. I have just filed an issue on this.

• Add json bool configuration option to HTTP executor #578

[pradeep-raja]

One potential issue is when the body itself is not in JSON format. In such cases, a possible solution could be to keep the entire body as text for non-JSON bodies.