Cannot use "attach" in an afterEach hook #1051
Comments
|
As per now, this is unfortunately intended behavior. Attachments is a Cucumber-term that must be associated with a step and |
|
For the way our tests run through Jenkins CI/CD, we can only have one instance of a workspace available for the newest job run. In this workspace, I can see any screenshots collected by Cypress for failed tests, but as soon as a new job begins, those are wiped out. By attaching these screenshots and additional log files to the scenario in Cucumber report, I'll have a constant history of what failed since those reports could be retained in Jenkins for a longer time. I've used other reporting tools for non-Cypress projects and we're able to add attachments post-scenario. I know this isn't the case for Cypress, but it feels like there is a discrepancy. Cucumber messages are appended linearly. While the scenario may have ended, the |
|
Screenshots of failed tests are automatically added to reports, as well as the error causing the test to fail. What else is there to add? |
|
Ok, so did a re-run and at first, screenshots were not being added at first due to a custom configuration, but I see it available now. I misspoke on the first part. However, in our suite's particular case, we prefer to track the Cypress logs using cypress-terminal-report as it contains specific datasets that we sometimes need to track within our tests. Since we only will create these files on test failure, it is important to add them to the report's last step if possible so we may see the full Cypress logs. |
|
I have a similar requirement and resolve this with the Cypress fail event i.e. Like this: In whatever your support file is: import { attach } from '@badeball/cypress-cucumber-preprocessor';
const attachLogs = () => {
// this will be specific to your setup...
const logPath = [urlPrefix(), 'logs', `${getFeatureFilename()}`].join('/');
// I'm using test/html because our cucumber reports end up in a third party service but you could attach the string rather than a URL
attach(
btoa(
`<strong>Feature file Cypress command log:</strong><ul><li><a href="${logPath}.txt">txt format</a></li><li><a href="${logPath}.json">JSON format</a></li></ul>`
),
'base64:text/html'
);
};
Cypress.on('fail', (e, runnable) => {
const { actual, expected, message } = e;
try {
//only add attachments if we are on the final attempt?
if (runnable._retries == runnable._currentRetry) {
attachLogs()
}
} catch (error) {
// Any failure here should not impact the test itself!
}
});If you did want to always attach logs, even when a test passes you could additionally utilise a common |
|
@MDG-JHowley, does calling |
|
@hammzj, said project appears to be able to log to file, which can be retained in Jenkins. This also feels a lot like a XY problem, which is a huge waste of someones' time. |
|
It seems like this is related similarly to #824, for which I reported the behavior of how Cucumber-js handles it. The same behavior exists for attachments in an After hook where it is possible to add them to a scenario as soon as it finishes. In this case, I'm using a Mocha hook rather than the Cucumber hook due to the above limitation. I understand this limitation, but this isn't the XY problem since the behavior of the preprocessor differs from the official Cucumber implementation; however, due to the issue with running After hooks on failures described in issue 824, this can be closed out as it seems that neither attempt will be accessible. |
@badeball Seems to work fine. I mean I'm actively using it to embed links to screenshots, videos and logs to cucumber reports. Incidentally I had some issues with the screenshots because you are embedding the image inline and in my workflow I need to upload the cucumber JSON file to a 3rd party service so this makes the filesize a bit large (the screenshots/videos are upload to hosted storage separately anyway). If it were possible to use attach in a node context if would be much simpler but your plugin doesn't support that. Instead I am using |
Current behavior
I am trying to attach screenshots and logs for the Cucumber report, but it fails in the
afterEachhook:Desired behavior
Attachments using
attachshould be allowed to occur in theafterEachhook, even if the test has completed.Test code to reproduce
Versions
Checklist
[email protected](package name has changed and it is no longer the most recent version, see #689).The text was updated successfully, but these errors were encountered: