GitLab

This changes the way we block gpg from asking interactively for a
passphrase, ever. It's also a less hacky way to force this. This works
with gpg-2.0.26+gpgme-1.5.1, gpg-2.1.18+gpgme-1.8.0,
gpg-2.2.27+gpgme-1.14.0, and gpg-2.2.27+gpgme-1.15.1, which makes me
optimistic that it's universally working.

The previous solution brought problems for some platforms and specific
combinations of gnupg with gpgme (resulting in "GPGME::Error no such
file or directory").

Since we fixed the global regexp for email addresses in #483 list
names with spaces are already not anymore possible.
Adding some test case for potential regressions.

Do not rely on '127.0.0.1', but on 'localhost'.

dirmngr receives some extra care, so it's able to cope with it.

gpgconf --kill might hang indefinitely, therefore, rely on pkill.

Relax the expected output of gpg if refreshing keys without a keyserver
being available, as gpg might report various errors in such a situation.

The dependency on dirmngr will be dropped soon, anyway, so I deem this
acceptable as a workaround, for now.

Closes #472

Before, x-add-key failed if the mail included a signature in the body,
for example.

This commit drops the related code of is_armored_key?() completely: it's
buggy and will be removed anyway in Schleuder version 4.

Closes #470

Closes #467

If a part did not have a charset set, we falled back to US-ASCII
and then tried to encode the string into that. However, this only
worked if the part had actually only ascii charcters, but as soon
it contains non-ascii characters we failed. With this change we
are encoding such parts as utf-8, thus adopting the future.

This revisits the fixes for #409 (merged in !301), as when
trying to parse unencrypted, but signed utf-8 mails, they were
failing to be parsed and thus generated another error. More
in-depth later.

With this fix we are changing the approach:

  1. We switch to UTF-8 as default input
  2. If this is not a valid encoding, we try to convert the
     input to UTF-8
  3. If we are failing to convert, we scrub the input, so that
     at least what is proper UTF-8 will be passed on.

ASCII-8BIT is a BINARY encoding and the mail library will only
force the mail to have CRLF if it only contains ASCII conent.
While UTF-8 is not a BINARY encoding and thus will get CRLF if
it is a valid encoding. Getting CRLF is important for mails such
as the one in `signed_utf8.eml`, as otherwise the parts detection
will fail and the whole body will end up in the prologue, with no
parts.
Enforcing UTF-8 will still make some of our charset mails failing,
as they are - validly - not UTF-8. By using a new dependency
'charlock_holmes', we are able to detect the actual encoding and
thus try to convert it to UTF-8. If everything fails, we just
drop the invalid characters.

When we are unable to resend to one of the addresses in resend-cc-encrypted-only
we are not sending the email of the batch to any of the addreses.
Adding an additional info makes this more clear and visible to users.

Make it more clear what happens when resending an encrypted email
fails (due to missing or too many matching keys), but falling
back to unencrypted resend is allowed.

Relevant ticket: #365

This should help with filtering such messages, which is currently not
easy to do in a reliable way.

Handle incoming mails encrypted to an absent key, using symmetric
encryption or containing PGP-garbage in a more graceful manner: Don't
throw an exception, don't notify (and annoy) the admins, instead inform
the sender of the mail how to do better.

Closes #337

this should ensure we are able to parse most incoming
plain text mails in different charsets.

See:

   * schleuder/schleuder#409
   * schleuder/schleuder!301
   * https://ruby-doc.org/core-2.6.5/Encoding.html#class-Encoding-label-External+encoding

Especially:

  * schleuder/schleuder#409 (comment 176612)
  * https://idiosyncratic-ruby.com/56-us-ascii-8bit.html
  * https://github.com/mikel/mail/issues/1294#issuecomment-460855599

This should help users of Apple Mail, which jams this block and the body
together. Hopefully, this change makes it easier to dinstiguish both
parts from each other.

Closes #348

Closes #335

Relates ff335fc2
Relates !300

Previously, if the subject was unset, keywords were not recognized and
the original "protected headers" could leak.

Closes #431

Although mutt now supports protected headers, the content of a
message compiled by mutt is just a plain body, without wrapped
into further mime parts (contrary to other mailers). Also the
message does not contain a special marked protected headers
mime part.