After hooks should be able to mark the scenario as failed in Jenkins Cucumber report
While noticing #16148 (closed), it appeared that a failing @check_tor_leaks
hook does not mark the concerned scenario as failed in Cucumber’s JSON
output. This is only about the JSON output: on the stdout/stderr and
exit code front, the test suite and Cucumber bits behave as intended and
the scenario is correctly considered as failed.
The Cucumber JSON output only records per-step status, not per-scenario
status, and it does not take into account the After('product’,
’check_tor_leaks') cucumber hook that runs our code. This leads to
bad and misleading reporting by the Jenkins cucumber report plugin, that
relies on Cucumber’s JSON output. We should make it so that a failing
check_tor_leaks hook ensure the scenario is recorded as failed in
Cucumber’s JSON output.
Attachments
Related issues
- Related to #9424 (closed)
- Related to #16959
- Has duplicate #16768 (closed)
-
Blocked by #17080 (closed)
Original created by @bertagaz on 16150 (Redmine)