"I can view and print a PDF file" scenarios are fragile
I’ve seen “Scenario: I can view and print a PDF file stored in /usr/share” and “Scenario: I can view and print a PDF file stored in non-persistent /home/amnesia” fail on Jenkins with a screenshot that looks like it has succeeded, see e.g. the attached file (that comes from https://jenkins.tails.boum.org/job/test_Tails_ISO_experimental/117/).
Debug log:
Scenario: I can view and print a PDF file stored in non-persistent /home/amnesia # features/evince.feature:14
calling as root: echo 'hello?'
call returned: [0, "hello?\n", ""]
[log] CLICK on (1024,384)
calling as root: /sbin/ifconfig eth0 | grep -q 'inet addr'
call returned: [1, "", ""]
calling as root: date -s '@1451226453'
call returned: [0, "Sun Dec 27 14:27:33 UTC 2015\n", ""]
Given I have started Tails from DVD without network and logged in # features/step_definitions/snapshots.rb:201
calling as amnesia: cp "/usr/share/cups/data/default-testpage.pdf" "/home/amnesia"
call returned: [0, "", ""]
And I copy "/usr/share/cups/data/default-testpage.pdf" to "/home/amnesia" as user "amnesia" # features/step_definitions/common_steps.rb:633
calling as root: pidof -x -o '%PPID' gnome-terminal
call returned: [1, "", ""]
[log] CLICK on (51,16)
[log] CLICK on (58,431)
[log] CLICK on (264,470)
[log] TYPE "evince /home/amnesia/default-testpage.pdf
"
When I open "/home/amnesia/default-testpage.pdf" with Evince # features/step_definitions/evince.rb:1
Then I see "CupsTestPage.png" after at most 20 seconds # features/step_definitions/common_steps.rb:441
FindFailed: can not find CupsTestPage.png on the screen.
Line ?, in File ? (RuntimeError)
./features/step_definitions/common_steps.rb:446:in `rescue in block in <top (required)>'
./features/step_definitions/common_steps.rb:442:in `/^I (do not )?see "([^"]*)" after at most (\d+) seconds$/'
features/evince.feature:18:in `Then I see "CupsTestPage.png" after at most 20 seconds'
And I can print the current document to "/home/amnesia/output.pdf" # features/step_definitions/evince.rb:5
Scenario failed at time 00:19:35
Screenshot: https://jenkins.tails.boum.org/job/test_Tails_ISO_test-10774-fragile-mac-address-spoofing-fails-and-macchanger-returns-false/11/artifact/build-artifacts/00:19:35_I_can_view_and_print_a_PDF_file_stored_in_non-persistent__home_amnesia.png
Video: https://jenkins.tails.boum.org/job/test_Tails_ISO_test-10774-fragile-mac-address-spoofing-fails-and-macchanger-returns-false/11/artifact/build-artifacts/00:19:35_I_can_view_and_print_a_PDF_file_stored_in_non-persistent__home_amnesia.mkv
Note that this is a different problem from #10775 (closed).
Feature Branch: wip/test/10994-fragile-view-and-print-PDF
Attachments
Parent Task: #10288
Original created by @intrigeri on 10994 (Redmine)