More usable html reports : logging the requests and responses in the report and naming entries

[christopheblin]

Problem to solve

When you execute multiple tests, you can use --error-format long to have the response of the failed assertion

However, it is often useful to have the full context of the failure, mostly the body of the request

For ex, if you use a captured variable of a previous request as an input to the failing request, seeing only something like "expected status 200 but was 400" and the body with (hopefully) an error can be insuficient

So you have to find the logs of the test run in order to find the failing request

And sometimes, in order to understand why the payload was like it was, you also need to look at previous calls

Proposal

If the command contains --vey-verbose, then the html report should contain the request AND the response of all entries (or another option "--report-format long")

With this, a better UI would be needed (collapsed sub sections for certificate, request headers, request body, response header, response body for example)

As an additional hint, it would also be great to be able to name the entries

For ex,

GET https://acmecorp.net/info as Retrieve info

HTTP 200

would display "Retrieve info" isntead of "Entry 1"

That would allow to have a far more usable reports compared to

Drawabacks

The size of the report is going to be far more important which could be a problem in some circumstances (very long script with lots of HTTP requests)

[jcamiel]

Hi @christopheblin reviewing this issue, I made the link with:

• Issue include response bodies in the json output
• Discussion Can the response body be included when using the --json flag?

These issue/discussion were about including response bodies when using --json. It makes sense when using --json and I think it also make sense in the HTML report (even if there are no errors). My only fear is the size of the report and the "performance" hit of dumping all responses to disk.

For --json we're going to include it by default. Just need to decide if we dump response (and display it) in th HTML report by default or behind an option.

[jcamiel]

Hi @christopheblin regarding the size of the HTML report, which can be an issue, we proposed to intoduce a new option --max-body which will be the maximum number of bytes for a request / response body to be included in the HTML report. We could also reuse this option for --json:

--max-body <COUNT>
          Maximum number of bytes for exporting a body, -1 for unlimited size [default: 256]

If you want the bodies to be always saved:

$ hurl --max-body -1 --report-html /tmp/report.html *.hurl 

If you want the bodies to be never saved:

$ hurl --max-body 0 --report-html /tmp/report.html *.hurl 

Or we a fixed limitation:

$ hurl --max-body 128000 --report-html /tmp/report.html *.hurl 

[simonzkl]

Very interested in this as well. It would really help with debugging failing tests without having to pollute output with --very-verbose or trying to piece together requests manually with JSON reports. The --max-body approach to limit bloat seems like a fine way to do it.