schleuder issueshttps://0xacab.org/schleuder/schleuder/-/issues2020-03-22T21:57:46Zhttps://0xacab.org/schleuder/schleuder/-/issues/459Convert all text-parts to UTF-8?2020-03-22T21:57:46ZpazConvert all text-parts to UTF-8?If we would convert all incoming text-parts (or anything that has a charset) into UTF-8, we might have less trouble with the 'mail' gem. E.g. in https://github.com/mikel/mail/pull/738#issuecomment-303083226 the maintainers explain that t...If we would convert all incoming text-parts (or anything that has a charset) into UTF-8, we might have less trouble with the 'mail' gem. E.g. in https://github.com/mikel/mail/pull/738#issuecomment-303083226 the maintainers explain that they always prefer converting to UTF-8, regardless of what the original charset header says.
Actually, changing the charset-header to 'UTF-8' (instead of re-encoding the text) in [schleuder/mail/message.rb:254](https://0xacab.org/schleuder/schleuder/blob/master/lib/schleuder/mail/message.rb#L254) makes our test-suite green even with mail-gpg v.0.4.3 (see #455).
From a technical standpoint I don't see a problem changing all parts to UTF-8. I would even advocate for it, because it standardises on a modern solution.
But I can't really judge how many computers out there still don't know UTF-8 and would show broken texts.
Any opinion?https://0xacab.org/schleuder/schleuder/-/issues/461Provide script to sanitize real world example mails2020-03-19T22:03:17ZgeorgProvide script to sanitize real world example mailsFeeding our issue tracker with real world example mails takes time due to doing "data hygiene". I plan to write a script, which, if given a mail file, writes out a sanitized copy. Obviously, before publishing, people should still check t...Feeding our issue tracker with real world example mails takes time due to doing "data hygiene". I plan to write a script, which, if given a mail file, writes out a sanitized copy. Obviously, before publishing, people should still check the result. Probably, this takes a bit of time to do some iterations, to make it better. Lets start?georggeorghttps://0xacab.org/schleuder/schleuder/-/issues/463Add spec to check if invalid chars get dropped as expected during initial mai...2020-03-22T12:20:12ZgeorgAdd spec to check if invalid chars get dropped as expected during initial mail parsingWe've changed the way how we handle initial mail encoding parsing via !325. If all fails, we might drop invalid chars to at least be able to handle the rest of problematic mails. However, as of now, we didn't found a way to trigger such ...We've changed the way how we handle initial mail encoding parsing via !325. If all fails, we might drop invalid chars to at least be able to handle the rest of problematic mails. However, as of now, we didn't found a way to trigger such a case.Next Big Thinghttps://0xacab.org/schleuder/schleuder/-/issues/464Add keyword to configure `delivery_enabled`.2020-03-22T11:46:09ZpazAdd keyword to configure `delivery_enabled`.Currently there's no way to configure that using Email after subscribing. That should be possible.Currently there's no way to configure that using Email after subscribing. That should be possible.https://0xacab.org/schleuder/schleuder/-/issues/471List option to opt-out of check_keys mails2020-06-02T11:40:20ZgeorgList option to opt-out of check_keys mailsPeople told me that they would find it valuable to be able to not receive mails sent by `check_keys` for specific lists.People told me that they would find it valuable to be able to not receive mails sent by `check_keys` for specific lists.https://0xacab.org/schleuder/schleuder/-/issues/473Give advice to people in case their key expired (or is about to expire)2020-06-14T12:20:23ZgeorgGive advice to people in case their key expired (or is about to expire)It would probably help people if Schleuder would not only tell them, "your key expired", but also, "what to do to resolve this".It would probably help people if Schleuder would not only tell them, "your key expired", but also, "what to do to resolve this".https://0xacab.org/schleuder/schleuder/-/issues/474Inform user if unknown keyword was encountered and keyword-processing aborted2020-06-17T11:46:38ZgeorgInform user if unknown keyword was encountered and keyword-processing abortedThe current code correctly checks for unknown keywords and prepares an error message. However, this error message isn't passed back to the user.The current code correctly checks for unknown keywords and prepares an error message. However, this error message isn't passed back to the user.Next Big Thinghttps://0xacab.org/schleuder/schleuder/-/issues/475Tell users that mails needs to be validly signed to use keywords2020-06-17T13:35:56ZgeorgTell users that mails needs to be validly signed to use keywordsWe could think about if it would make sense, in terms of UX, to provide a helpful error message in case a mail wasn't signed, but it contained keywords.
Currently, that's not easy to do, because we only look for keywords if the mail was...We could think about if it would make sense, in terms of UX, to provide a helpful error message in case a mail wasn't signed, but it contained keywords.
Currently, that's not easy to do, because we only look for keywords if the mail was signed. I'm not sure how to tackle this, maybe via a "best effort" approach, via a superficial scan of the mail?
Any opinion?https://0xacab.org/schleuder/schleuder/-/issues/477CI: Add job to validate open api spec2020-06-24T13:51:07ZgeorgCI: Add job to validate open api specNext Big Thinggeorggeorghttps://0xacab.org/schleuder/schleuder/-/issues/482Please be more informative when you can't encrypt to a list-admin2021-01-18T23:47:57ZcasperPlease be more informative when you can't encrypt to a list-adminSchleuder 3.4.0-2+deb10u3 did sent this uninformative message (Subject: "Error"), when it couldn't find a matching key to one of the list-administrators.
```
No keys to encrypt to!
/usr/lib/ruby/vendor_ruby/mail/gpg/gpgme_helper.rb:17:...Schleuder 3.4.0-2+deb10u3 did sent this uninformative message (Subject: "Error"), when it couldn't find a matching key to one of the list-administrators.
```
No keys to encrypt to!
/usr/lib/ruby/vendor_ruby/mail/gpg/gpgme_helper.rb:17:in `encrypt'
/usr/lib/ruby/vendor_ruby/mail/gpg/encrypted_part.rb:23:in `initialize'
/usr/lib/ruby/vendor_ruby/schleuder/mail/encrypted_part.rb:10:in `initialize'
/usr/lib/ruby/vendor_ruby/mail/gpg.rb:43:in `new'
/usr/lib/ruby/vendor_ruby/mail/gpg.rb:43:in `block in encrypt'
/usr/lib/ruby/vendor_ruby/mail/gpg.rb:113:in `instance_eval'
/usr/lib/ruby/vendor_ruby/mail/gpg.rb:113:in `block in construct_mail'
/usr/lib/ruby/vendor_ruby/mail/message.rb:153:in `instance_eval'
/usr/lib/ruby/vendor_ruby/mail/message.rb:153:in `initialize'
/usr/lib/ruby/vendor_ruby/mail/mail.rb:51:in `new'
/usr/lib/ruby/vendor_ruby/mail/mail.rb:51:in `new'
/usr/lib/ruby/vendor_ruby/mail/gpg.rb:106:in `construct_mail'
/usr/lib/ruby/vendor_ruby/mail/gpg.rb:29:in `encrypt'
/usr/lib/ruby/vendor_ruby/schleuder/mail/gpg.rb:7:in `encrypt'
/usr/lib/ruby/vendor_ruby/mail/gpg/delivery_handler.rb:11:in `deliver_mail'
/usr/lib/ruby/vendor_ruby/mail/message.rb:260:in `deliver'
/usr/lib/ruby/vendor_ruby/schleuder/logger_notifications.rb:48:in `block in notify_admin'
/usr/lib/ruby/vendor_ruby/schleuder/logger_notifications.rb:29:in `each'
/usr/lib/ruby/vendor_ruby/schleuder/logger_notifications.rb:29:in `notify_admin'
/usr/lib/ruby/vendor_ruby/schleuder/logger_notifications.rb:13:in `error'
/usr/lib/ruby/vendor_ruby/schleuder/list.rb:356:in `rescue in block in send_to_subscriptions'
/usr/lib/ruby/vendor_ruby/schleuder/list.rb:347:in `block in send_to_subscriptions'
/usr/share/rubygems-integration/all/gems/activerecord-5.2.2.1/lib/active_record/relation/delegation.rb:71:in `each'
/usr/share/rubygems-integration/all/gems/activerecord-5.2.2.1/lib/active_record/relation/delegation.rb:71:in `each'
/usr/lib/ruby/vendor_ruby/schleuder/list.rb:346:in `send_to_subscriptions'
/usr/lib/ruby/vendor_ruby/schleuder/runner.rb:77:in `run'
/usr/lib/ruby/vendor_ruby/schleuder/cli.rb:36:in `work'
/usr/lib/ruby/vendor_ruby/thor/command.rb:27:in `run'
/usr/lib/ruby/vendor_ruby/thor/invocation.rb:126:in `invoke_command'
/usr/lib/ruby/vendor_ruby/thor.rb:369:in `dispatch'
/usr/lib/ruby/vendor_ruby/thor/base.rb:444:in `start'
/usr/bin/schleuder:13:in `<main>'
```
To debug, I made everyone a simple user but me. Once the problematic account was a simple user, schleuder would send:
```
The following error occurred while sending a message to user@example.org: No keys to encrypt to!
```https://0xacab.org/schleuder/schleuder/-/issues/485Improve helpfulness of error messages in response to keywords2021-02-03T11:37:07ZpazImprove helpfulness of error messages in response to keywordsE.g. If a resend fails due to a string not being a valid email-address.E.g. If a resend fails due to a string not being a valid email-address.Next Big Thinghttps://0xacab.org/schleuder/schleuder/-/issues/489Feature Request: (Add option to) publish list-key to web key directory2021-05-25T22:30:55Zlu xFeature Request: (Add option to) publish list-key to web key directory**Problem:**
Since many mailclients are moving from SKS to verified keyservers and WKD, it has become rather cumbersome to publish a whole list key.
For example, when publishing a key to keys.openpgp.org, you would need to verify the ...**Problem:**
Since many mailclients are moving from SKS to verified keyservers and WKD, it has become rather cumbersome to publish a whole list key.
For example, when publishing a key to keys.openpgp.org, you would need to verify the identity for its sub-identites too (list-request or list-owner), and for an encrypted-only list you would need to fish the verification link out of the attachment of the bounce-notification
(I am aware that you can write to list-sendkey for the public key, but many schleuder users are either not aware or generally alienated by the idea of managing a subscription via mail commands)
**Solution**
It would be great if schleuder could publish it's list keys via web key directory.
`GnuPG >= 2.2.14` [includes a web key service](https://wiki.gnupg.org/WKS), so this might be possible without adding additional dependencies to schleuder.
I do not know ruby (yet) and my programming experience is almost none, but if you approve the idea I could a look at the codebase and see if I can do a contribution (..and maybe you already know a convenient way to do this and can point me to the direction :smile: )https://0xacab.org/schleuder/schleuder/-/issues/492Running db:init on schleuder install might not be suitable for external DB ad...2021-11-07T20:01:07ZngRunning db:init on schleuder install might not be suitable for external DB adapters with less privileged usersIf you have your schleuder DB in a DBMS (e.g. postgresql) you likely have a user that is not privileged to create the database.
However, on `schleuder install` we blindly do `db:create` which fails.
Related code snippets:
* https://0x...If you have your schleuder DB in a DBMS (e.g. postgresql) you likely have a user that is not privileged to create the database.
However, on `schleuder install` we blindly do `db:create` which fails.
Related code snippets:
* https://0xacab.org/schleuder/schleuder/-/blob/18c1e07a5c414e3ac7d42a495b95603f1a5da837/lib/schleuder/cli.rb#L116
* https://0xacab.org/schleuder/schleuder/-/commit/2ee7f06015a82d5a88e98e8859e15efcb98a7a78
Not sure if we should point it out , be more clever or just document it. Or everybody (except me) runs out of sqlite.https://0xacab.org/schleuder/schleuder/-/issues/493Bulk delete all unused key2021-07-07T11:42:36ZngBulk delete all unused keyit would be great if there was an option to delete all unused keys from a lists's keyring. So it can be cleaned up.it would be great if there was an option to delete all unused keys from a lists's keyring. So it can be cleaned up.https://0xacab.org/schleuder/schleuder/-/issues/494Delete subscription & key together2021-07-07T11:42:38ZngDelete subscription & key togetherHave an option to delete a subscription together with the key. Atm it is a two step process to delete a subscription and than its key. Which is a bit tediousHave an option to delete a subscription together with the key. Atm it is a two step process to delete a subscription and than its key. Which is a bit tedioushttps://0xacab.org/schleuder/schleuder/-/issues/497Consider setting Mail-Followup-To header when Reply-To is set to original sender2021-06-01T19:57:48Zabc defConsider setting Mail-Followup-To header when Reply-To is set to original senderWhen REPLY-TO is set to the original sender, users have complained that when they hit "Reply-All" (Reply-To-All) in their clients, they expect to send a mail to the original sender AND the list (`FROM(list)+REPLY-TO(original sender)+TO(r...When REPLY-TO is set to the original sender, users have complained that when they hit "Reply-All" (Reply-To-All) in their clients, they expect to send a mail to the original sender AND the list (`FROM(list)+REPLY-TO(original sender)+TO(recipient)+CC`). Instead, the TO header is set to `REPLY-TO(original sender)+TO(recipient)+CC` and thus not including the list.
I investigated this and it seems that most mail clients [follow this procedure](https://wiki.mozilla.org/Thunderbird:Help_Documentation:Mail-Followup-To_and_Mail-Reply-To) for "Reply All":
```
if MAIL-FOLLOWUP-TO in received mail then use MAIL-FOLLOWUP-TO as TO
else if MAIL-REPLY-TO in received mail then use MAIL-REPLY-TO + TO + CC as TO
else if REPLY-TO in received mail then use REPLY-TO + TO + CC as TO # this
else use FROM + TO + CC as TO
```
This means, when schleuder is configured to set REPLY-TO to the original sender, the clients will set TO to `REPLY-TO (original sender) + TO (recipient) + CC`, which does NOT include the mailing list.
Thus I would suggest to add a MAIL-FOLLOWUP-TO header **when REPLY-TO is set to the original sender**. The MAIL-FOLLOWUP-TO header then includes:
- list
- original sender
- CC
If you don't see any problems, I would implement this and open a MR. Please let me know :)https://0xacab.org/schleuder/schleuder/-/issues/498undefined method `has_content_type?' for nil:NilClass2022-04-11T21:47:51Zngundefined method `has_content_type?' for nil:NilClassthis looks similar to #458 but it was seen on a 3.5.3 schleuder installation.
```
undefined method `has_content_type?' for nil:NilClass
/opt/schleuder/bundler/ruby/2.7.0/gems/mail-gpg-0.4.2/lib/mail/gpg/sign_part.rb:22:in `verify_signat...this looks similar to #458 but it was seen on a 3.5.3 schleuder installation.
```
undefined method `has_content_type?' for nil:NilClass
/opt/schleuder/bundler/ruby/2.7.0/gems/mail-gpg-0.4.2/lib/mail/gpg/sign_part.rb:22:in `verify_signature'
/opt/schleuder/bundler/ruby/2.7.0/gems/mail-gpg-0.4.2/lib/mail/gpg/mime_signed_message.rb:9:in `setup'
/opt/schleuder/bundler/ruby/2.7.0/gems/mail-gpg-0.4.2/lib/mail/gpg.rb:144:in `verify'
/opt/schleuder/bundler/ruby/2.7.0/gems/mail-gpg-0.4.2/lib/mail/gpg/message_patch.rb:91:in `verify'
/opt/schleuder/lib/schleuder/mail/message.rb:31:in `setup'
/opt/schleuder/lib/schleuder/runner.rb:38:in `run'
/opt/schleuder/lib/schleuder/cli.rb:36:in `work'
/opt/schleuder/bundler/ruby/2.7.0/gems/thor-0.20.3/lib/thor/command.rb:27:in `run'
/opt/schleuder/bundler/ruby/2.7.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in `invoke_command'
/opt/schleuder/bundler/ruby/2.7.0/gems/thor-0.20.3/lib/thor.rb:387:in `dispatch'
/opt/schleuder/bundler/ruby/2.7.0/gems/thor-0.20.3/lib/thor/base.rb:466:in `start'
/opt/schleuder/bin/schleuder:13:in `<top (required)>'
/opt/schleuder/bundler/ruby/2.7.0/bin/schleuder:23:in `load'
/opt/schleuder/bundler/ruby/2.7.0/bin/schleuder:23:in `<top (required)>'
/opt/rh/rh-ruby27/root/usr/share/gems/gems/bundler-2.1.4/lib/bundler/cli/exec.rb:63:in `load'
/opt/rh/rh-ruby27/root/usr/share/gems/gems/bundler-2.1.4/lib/bundler/cli/exec.rb:63:in `kernel_load'
/opt/rh/rh-ruby27/root/usr/share/gems/gems/bundler-2.1.4/lib/bundler/cli/exec.rb:28:in `run'
/opt/rh/rh-ruby27/root/usr/share/gems/gems/bundler-2.1.4/lib/bundler/cli.rb:476:in `exec'
/opt/rh/rh-ruby27/root/usr/share/gems/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/opt/rh/rh-ruby27/root/usr/share/gems/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/opt/rh/rh-ruby27/root/usr/share/gems/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor.rb:399:in `dispatch'
/opt/rh/rh-ruby27/root/usr/share/gems/gems/bundler-2.1.4/lib/bundler/cli.rb:30:in `dispatch'
/opt/rh/rh-ruby27/root/usr/share/gems/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/base.rb:476:in `start'
/opt/rh/rh-ruby27/root/usr/share/gems/gems/bundler-2.1.4/lib/bundler/cli.rb:24:in `start'
/opt/rh/rh-ruby27/root/usr/share/gems/gems/bundler-2.1.4/libexec/bundle:46:in `block in <top (required)>'
/opt/rh/rh-ruby27/root/usr/share/gems/gems/bundler-2.1.4/lib/bundler/friendly_errors.rb:123:in `with_friendly_errors'
/opt/rh/rh-ruby27/root/usr/share/gems/gems/bundler-2.1.4/libexec/bundle:34:in `<top (required)>'
/opt/rh/rh-ruby27/root/usr/bin/bundle:23:in `load'
/opt/rh/rh-ruby27/root/usr/bin/bundle:23:in `<main>'
```
Original email is available. Will need to validate it with schleuder 4 and then also whether I can easily reproduce it.ngnghttps://0xacab.org/schleuder/schleuder/-/issues/499Keeping original email's TO and CC2021-06-08T17:25:54Zabc defKeeping original email's TO and CCAs discovered during testing of !374 and commented in #497, outlook does not seem to use `MAIL-FOLLOWUP-TO`.
"Keeping" the original email's `TO` and `CC` could solve this.
So an email sent to the schleuder list `schleuderlist@abc.de` l...As discovered during testing of !374 and commented in #497, outlook does not seem to use `MAIL-FOLLOWUP-TO`.
"Keeping" the original email's `TO` and `CC` could solve this.
So an email sent to the schleuder list `schleuderlist@abc.de` like this:
```
From: sender@xyz.de
To: schleuderlist@abc.de, person1@def.de
CC: ccperson@ghi@de
```
could result in schleuder sending to its recipient `recipient@jkl.de` an email like this:
```
From: schleuderlist@abc.de
To: schleuderlist@abc.de, person1@def.de
CC: ccperson@ghi@de
Reply-To: sender@xyz.de
```
This means, the envelope should then only contain `recipient@jkl.de`, but the `TO` header contains all the original `TO`.
- Do you see any problems (e.g. DMARC, other protocol stuff) when implemented like this?
- Would this make sense as a global configurable option in your opinion? Or would you rather have it only when reply-to is set to original sender?https://0xacab.org/schleuder/schleuder/-/issues/500Tighten umask2021-05-31T06:37:32ZpazTighten umaskWe currently use a umask of `027`. This is due to historical reasons, when some people preferred a (pre-database) setup in which each list ran as a different unix-user while a "meta user" needed access for maintenance work.
These days w...We currently use a umask of `027`. This is due to historical reasons, when some people preferred a (pre-database) setup in which each list ran as a different unix-user while a "meta user" needed access for maintenance work.
These days we are not aware of anyone still running such a setup, and if someone does it could still work without group readable list directories.
Using a umask of `077` on the other hand has two obviuos advantages:
* a generally reduces chance to accidentally reveal private keys,
* we can get rid of the flag `--no-permission-warning` for gpg when accessing it through the shell. Gpg's warnings might be debateable, but bluntly silencing them isn't a good way, either.pazpazhttps://0xacab.org/schleuder/schleuder/-/issues/504Decide how to handle keys with one or more blanks within the mail addr part o...2021-07-14T14:11:05ZgeorgDecide how to handle keys with one or more blanks within the mail addr part of a uidSchleuder is currently not capable to find the correct key, if the lookup searches for a mail addr, e.g. if resending messages, if the key in question contains one or more blanks within the mail addr part of a uid.
The spec defines ...Schleuder is currently not capable to find the correct key, if the lookup searches for a mail addr, e.g. if resending messages, if the key in question contains one or more blanks within the mail addr part of a uid.
The spec defines this part as an UTF-8 string, which, it seems, leaves room for interpretation.
Different implementations handle this differently: some do accept this, others do not. I just learnt that it's possible to create such keys via Thunderbird, while it's not via `gpg`.
We could either disallow such keys to be added to the keyring, although that might seem drastic, and would do 'harm' if people use such keys only for subscribers, e.g. resending is not of a concern.
Personally, I would like to get both (subscription vs. resending) 'in sync': Allowing such keys in general, or disallowing them; I'm leaning towards the later. If so, this might be a breaking change.
Any opinions wrt this topic?