"all notifications have disappeared" test suite step is fragile

I’ve seen My Additional Software list is configurable through a GUI or through notifications when I install or remove packages with APT or Synaptic fail because the I accept removing "sslh" from Additional Software step fails: no notification with a “Remove” button is visible on screen. That’s because the previous notification (“Additional software installed successfully”) was not hidden as planned.

As we can see in the attached video around 4:45, the click on the clock is somehow lost. That’s not a big problem in itself: if the rest of the code worked fine, we would retry, and there’s a good chance that this click would work eventually. However, despite this click not working and the calendar+notifications menu not hopening, for some reason, gnome_shell.child?('No Notifications', roleName: 'label') succeeds, even though there is a notification on screen, so we don’t try to close notifications by clicking GnomeCloseAllNotificationsButton.png (which is nowhere to be seen anyway so that would fail, but at least, it would fail closed loudly instead of failing open and breaking the following steps).

It looks like this bug is caused by #12191 and then adding showingOnly: true to gnome_shell.child?('No Notifications', roleName: 'label') should fix it.

Here are the relevant debug log bits:

01:59:07.841611204: call returned: [3, "â— tails-additional-software-install.service - Install Additional Software Packages\n   Loaded: loaded (/lib/systemd/system/tails-additional-software-install.service; static; vendor preset: enabled)\n   Active: activating (start) since Sat 2019-08-31 09:55:50 UTC; 2min 11s ago\n     Docs: https://tails.boum.org/contribute/design/persistence/\n Main PID: 9119 (python3)\n    Tasks: 3 (limit: 2349)\n   Memory: 487.2M\n   CGroup: /system.slice/tails-additional-software-install.service\n           â”œâ”€ 9119 python3 /usr/local/sbin/tails-additional-software install\n           â”œâ”€ 9324 apt-get --quiet --yes --no-remove --option DPkg::Options::=--force-confold install sslh\n           â”œâ”€32052 apt-get --quiet --yes --no-remove --option DPkg::Options::=--force-confold install sslh\n           â”œâ”€32053 sh -c /usr/local/sbin/tails-additional-software apt-post\n           â””â”€32054 python3 /usr/local/sbin/tails-additional-software apt-post\n\nAug 31 09:56:55 amnesia tails-additional-software[9119]: [INFO] Adding system-user for sslh daemon\nAug 31 09:56:55 amnesia groupadd[9692]: group added to /etc/group: name=sslh, GID=129\nAug 31 09:56:55 amnesia groupadd[9692]: group added to /etc/gshadow: name=sslh\nAug 31 09:56:55 amnesia groupadd[9692]: new group: name=sslh, GID=129\nAug 31 09:56:55 amnesia useradd[9696]: new user: name=sslh, UID=113, GID=129, home=/nonexistent, shell=/usr/sbin/nologin\nAug 31 09:56:56 amnesia chage[9702]: changed password expiry for sslh\nAug 31 09:56:56 amnesia tails-additional-software[9119]: [INFO] Created symlink /etc/systemd/system/multi-user.target.wants/sslh.service -> /lib/systemd/system/sslh.service.\nAug 31 09:56:57 amnesia tails-additional-software[9119]: [INFO] Processing triggers for systemd (241-5) ...\nAug 31 09:56:58 amnesia tails-additional-software[9119]: [INFO] Processing triggers for man-db (2.8.5-2) ...\nAug 31 09:57:37 amnesia tails-additional-software[9119]: [INFO] Processing triggers for libc-bin (2.28-10) ...\n", ""]
01:59:17.842001895: Remote shell: calling as root: systemctl status tails-additional-software-install.service
01:59:18.582309794: call returned: [0, "â— tails-additional-software-install.service - Install Additional Software Packages\n   Loaded: loaded (/lib/systemd/system/tails-additional-software-install.service; static; vendor preset: enabled)\n   Active: active (exited) since Sat 2019-08-31 09:58:04 UTC; 7s ago\n     Docs: https://tails.boum.org/contribute/design/persistence/\n  Process: 9119 ExecStart=/usr/local/sbin/tails-additional-software install (code=exited, status=0/SUCCESS)\n  Process: 32068 ExecStartPost=/usr/bin/install -m 0644 -D /dev/null /run/live-additional-software/installed (code=exited, status=0/SUCCESS)\n Main PID: 9119 (code=exited, status=0/SUCCESS)\n\nAug 31 09:57:37 amnesia tails-additional-software[9119]: [INFO] Processing triggers for libc-bin (2.28-10) ...\nAug 31 09:58:02 amnesia tails-additional-software[9119]: [INFO] [INFO] Examining package changes\nAug 31 09:58:03 amnesia tails-additional-software[9119]: [INFO] Installation completed successfully.\nAug 31 09:58:03 amnesia sudo[32059]:     root : TTY=unknown ; PWD=/ ; USER=amnesia ; ENV=DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus ; COMMAND=/usr/bin/gdbus call --session --dest org.freedesktop.Notifications --object-path /org/freedesktop/Notifications --method org.freedesktop.Notifications.CloseNotification 3\nAug 31 09:58:03 amnesia sudo[32059]: pam_unix(sudo:session): session opened for user amnesia by (uid=0)\nAug 31 09:58:04 amnesia sudo[32059]: pam_unix(sudo:session): session closed for user amnesia\nAug 31 09:58:04 amnesia sudo[32063]:     root : TTY=unknown ; PWD=/ ; USER=amnesia ; COMMAND=/usr/local/lib/tails-additional-software-notify Additional software installed successfully\nAug 31 09:58:04 amnesia sudo[32063]: pam_unix(sudo:session): session opened for user amnesia by (uid=0)\nAug 31 09:58:04 amnesia sudo[32063]: pam_unix(sudo:session): session closed for user amnesia\nAug 31 09:58:04 amnesia systemd[1]: Started Install Additional Software Packages.\n", ""]
01:59:18.582802378: Remote shell: calling as root: /usr/local/sbin/tor-has-bootstrapped
01:59:18.860709954: call returned: [0, "", ""]
01:59:18.861243701: Remote shell: calling as root: test -e /run/tordate/done
01:59:19.038555127: call returned: [0, "", ""]
01:59:19.038945154: Remote shell: calling as root: test -e /run/htpdate/success
01:59:19.258225573: call returned: [0, "", ""]
01:59:19.258966797: Remote shell: calling as root: test -e '/run/live-additional-software/doomed_to_fail'
01:59:19.465074061: call returned: [1, "", ""]
01:59:19.465338362: Remote shell: calling as root: test -e '/live/persistence/TailsData_unlocked/live-additional-software.conf'
01:59:19.621160925: call returned: [0, "", ""]
01:59:19.621456144: opening file /live/persistence/TailsData_unlocked/live-additional-software.conf in 'read' mode
01:59:19.667641995: read complete
01:59:19.668033661: Remote shell: calling as root: systemctl status tails-additional-software-upgrade.service
01:59:20.199165530: call returned: [3, "â— tails-additional-software-upgrade.service - Upgrade Additional Software Packages\n   Loaded: loaded (/lib/systemd/system/tails-additional-software-upgrade.service; static; vendor preset: enabled)\n   Active: activating (start) since Sat 2019-08-31 09:58:04 UTC; 9s ago\n     Docs: https://tails.boum.org/contribute/design/persistence/\n Main PID: 32069 (python3)\n    Tasks: 2 (limit: 2349)\n   Memory: 333.7M\n   CGroup: /system.slice/tails-additional-software-upgrade.service\n           â”œâ”€32069 python3 /usr/local/sbin/tails-additional-software upgrade\n           â””â”€32070 apt-get --quiet --yes update\n\nAug 31 09:58:04 amnesia systemd[1]: Starting Upgrade Additional Software Packages...\nAug 31 09:58:04 amnesia tails-additional-software[32069]: [INFO] Starting to upgrade additional software...\nAug 31 09:58:04 amnesia tails-additional-software[32069]: [INFO] Found additional packages list.\nAug 31 09:58:04 amnesia tails-additional-software[32069]: [INFO] Saving old APT lists...\nAug 31 09:58:10 amnesia tails-additional-software[32069]: [INFO] Hit:1 tor+http://deb.tails.boum.org devel InRelease\nAug 31 09:58:11 amnesia tails-additional-software[32069]: [INFO] Hit:2 tor+http://ftp.us.debian.org/debian buster InRelease\nAug 31 09:58:11 amnesia tails-additional-software[32069]: [INFO] Hit:3 tor+http://ftp.us.debian.org/debian bullseye InRelease\nAug 31 09:58:11 amnesia tails-additional-software[32069]: [INFO] Hit:4 tor+http://ftp.us.debian.org/debian sid InRelease\nAug 31 09:58:11 amnesia tails-additional-software[32069]: [INFO] Hit:5 tor+http://deb.torproject.org/torproject.org buster InRelease\n", ""]
01:59:30.199532571: Remote shell: calling as root: systemctl status tails-additional-software-upgrade.service
01:59:30.897191575: call returned: [0, "â— tails-additional-software-upgrade.service - Upgrade Additional Software Packages\n   Loaded: loaded (/lib/systemd/system/tails-additional-software-upgrade.service; static; vendor preset: enabled)\n   Active: active (exited) since Sat 2019-08-31 09:58:20 UTC; 4s ago\n     Docs: https://tails.boum.org/contribute/design/persistence/\n  Process: 32069 ExecStart=/usr/local/sbin/tails-additional-software upgrade (code=exited, status=0/SUCCESS)\n  Process: 32630 ExecStartPost=/usr/bin/install -m 0644 -D /dev/null /run/live-additional-software/upgraded (code=exited, status=0/SUCCESS)\n Main PID: 32069 (code=exited, status=0/SUCCESS)\n\nAug 31 09:58:19 amnesia tails-additional-software[32069]: [INFO] The following package was automatically installed and is no longer required:\nAug 31 09:58:19 amnesia tails-additional-software[32069]: [INFO]   libpcre2-posix0\nAug 31 09:58:19 amnesia tails-additional-software[32069]: [INFO] Use 'apt autoremove' to remove it.\nAug 31 09:58:19 amnesia tails-additional-software[32069]: [INFO] 0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.\nAug 31 09:58:19 amnesia tails-additional-software[32069]: [INFO] Installation completed successfully.\nAug 31 09:58:19 amnesia tails-additional-software[32069]: [INFO] The upgrade was successful.\nAug 31 09:58:19 amnesia tails-additional-software[32069]: [INFO] Reading package lists...\nAug 31 09:58:19 amnesia tails-additional-software[32069]: [INFO] Building dependency tree...\nAug 31 09:58:19 amnesia tails-additional-software[32069]: [INFO] Reading state information...\nAug 31 09:58:20 amnesia systemd[1]: Started Upgrade Additional Software Packages.\n", ""]
01:59:30.897820851: Remote shell: calling as root: systemctl is-system-running
01:59:31.131200260: call returned: [0, "running\n", ""]
01:59:31.131617915: executing Python as amnesia: 
    import dogtail.config
    import dogtail.tree
    import dogtail.predicate
    import dogtail.rawinput
    dogtail.config.logDebugToFile = False
    dogtail.config.logDebugToStdOut = False
    dogtail.config.blinkOnActions = True
    dogtail.config.searchShowingOnly = True
    node203 = dogtail.tree.root.application('gnome-shell')
01:59:33.308370657: execution complete
01:59:33.308699187: Sikuli: calling click(L(512,10)@S(0)[0,0 1024x768])...
01:59:34.034198055: executing Python as amnesia: node204 = node203.child('No Notifications', roleName='label')
01:59:34.052903660: [log] CLICK on L(512,10)@S(0)[0,0 1024x768]
01:59:35.860216217: execution complete
01:59:35.860552193: executing Python as amnesia: node205 = node203.child('No Notifications', roleName='label')
01:59:38.200205359: execution complete
01:59:38.242693614: [log]  TYPE "#ESC."
01:59:38.243085023: Remote shell: calling as root: test -e '/run/tails-upgrader/checked_upgrades'
01:59:38.497094785: call returned: [0, "", ""]
    And I start Tails from USB drive "__internal" and I login with persistence enabled and an administration password                                 # features/step_definitions/common_steps.rb:147

Feature Branch: test/17012-more-robust-notification-hiding+force-all-tests

Related issues

Related to #12191
Blocks #16209

Original created by @intrigeri on 17012 (Redmine)

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information