diff --git a/atheme-7.md b/atheme-7.md new file mode 100644 index 0000000000000000000000000000000000000000..53efb1e5dee5f140702eb2b50d13fa16f54af583 --- /dev/null +++ b/atheme-7.md @@ -0,0 +1,16 @@ +Atheme services were upgraded from 6.x to 7.0. + +* new functionalities: +** warn before clones limit is hit (4/5) +** identified users do not count towards the clones limit, up to double the limit +* new modules: +** Bad email address blocking (@nickserv/badmail@) +** LISTGROUPS command (@nickserv/listgroups@) +** RESTRICT command (@nickserv/restrict@) +** CLONE command (@chanserv/clone@) +** SET NEVERGROUP command (@nickserv/set_nevergroup@) +** SET NEVEROP command (@nickserv/set_neverop@) +** SET NOGREET command (@nickserv/set_nogreet@) +** SYNC command (and automatic ACL syncing, @chanserv/sync@) +** SET command for OperServ to modify temporarily config options (@operserv/set@) +** StatServ (adds basic stats and split tracking, main, netsplit and server enabled, channel disabled) diff --git a/certfp-support-fixed-servers-upgraded.md b/certfp-support-fixed-servers-upgraded.md new file mode 100644 index 0000000000000000000000000000000000000000..5911afe7ac53e4367b6021d77795ed94cd524b8d --- /dev/null +++ b/certfp-support-fixed-servers-upgraded.md @@ -0,0 +1 @@ +Some server upgrades were performed today to fix vulnerabilities in the atheme-services code. We also worked with upstream to fix a long standing issue with CertFP that kept certain servers from allowing proper certification-based authentication. diff --git a/certificate-authentication-requires.md b/certificate-authentication-requires.md new file mode 100644 index 0000000000000000000000000000000000000000..597d3a5b7d0d887f5a61a7de6bd9d1f83ccdf446 --- /dev/null +++ b/certificate-authentication-requires.md @@ -0,0 +1,143 @@ +The [Charybdis 4.1 upgrade->https://we.riseup.net/ircd/charybdis-4-1-upgrade] imposed some changes on the TLS configuration of the servers which change the way public certificates are fingerprinted across the networks. This was changed to fix inter-server authentication but also affects @CERTFP@ authentication with @NickServ@ which means users authenticating with this mechanism now will need to add a new fingerprint to be able to authenticate in the future. + +h1. How to add your new fingerprint + +Because the servers are not currently setup using the new mechanism, you will need to compute the certificate by hand. This can be done with the @certtool@ command that is part of the [GnuTLS->https://www.gnutls.org/] distribution (on Debian, it is in the @gnutls-bin@ package), for example with: + +<code> +$ certtool --pubkey-info --load-privkey .irssi/certs/nickserv.privkey | grep sha256 | sed 's/.*sha256:/SPKI:SHA2-256:/' SPKI:SHA2-256:b89943391739daadf89e191575307b590b5db40f1b6a52cbe39f2ea62cb16637 +</code> + +Note that only the more recent versions of GnuTLS give out SHA256 fingerprints. The above works on Debian stable with GnuTLS 3.5. If you are running older releases of GnuTLS (for example 3.4 only gives out a SHA1 checksum), you might need to use OpenSSL instead, for example: + +<code>openssl pkey -inform PEM -pubout -outform DER < .irssi/certs/nickserv.privkey | sha256sum</code> + +This fingerprint can then be added to the configuration through @NickServ@ with: + +<code> +/msg NickServ cert add SPKI:SHA2-256:b89943391739daadf89e191575307b590b5db40f1b6a52cbe39f2ea62cb16637 +</code> + +And you're done! Alternatively, you can just wait for the authentication to fail and then authenticate with your password and use this to add the new certificate: + +<code> +/msg NickServ cert add +</code> + +The above will naturally not work until we have performed the migration to the new fingerprint mechanism. + +The certificate fingerprint mechanism migration will occur in two weeks from publication of this notice, before the certification expiry date (2018-07-03). It can be done without upgrading to 4.1 so it's not because a server is running version 4.1 that it uses the new mechanism, nor that a 3.5 server means it can't use the new mechanism either. + +h1. How to configure CERTFP authentication + +Even if you do not currently use CERTFP to authenticate with the IRC server, you might want to consider adding that to your configuration anyways. It makes it possible to avoid storing cleartext passwords in your IRC configuration files and is generally considered more secure against bruteforce attack than password-based authentication. Even more so if you do not store the password in your configuration files: then you don't have to remember the password and can use a much stronger one. + +We have [previously->https://we.riseup.net/ircd/guerin+certfp] suggested following the [OFTC guide->https://www.oftc.net/NickServ/CertFP/] to configure CERTFP, but when the new changes become live in two weeks, the configuration will be even simpler. All you need to do is generate a private key and add it to your IRC client's configuration. This procedure here is an example using @certtool@ and @irssi@. + +First generate a RSA private key: + +<code>(umask 077; certtool --generate-privkey --rsa | sed -n '/^-----BEGIN/,$p' > .irssi/certs/nickserv.privkey)</code> + +Then generate the fingerprint to send to @NickServ@: + +<code>certtool --pubkey-info --load-privkey .irssi/certs/nickserv.privkey | grep sha256 | sed 's/.*sha256:/SPKI:SHA2-256:/'</code> + +or with OpenSSL: + +<code>openssl pkey -inform PEM -pubout -outform DER < .irssi/certs/nickserv.privkey | sha256sum</code> + +Send it to @NickServ@: + +<code>/msg NickServ cert add SPKI:SHA2-256:<...></code> + +And add the certificate to your configuration, for example: + +<code> + { + address = "irc.indymedia.org"; + chatnet = "IMC"; + port = "6697"; + use_ssl = "yes"; + ssl_cert = "~/.irssi/certs/nickserv.privkey"; + ssl_verify = "yes"; + autoconnect = "yes"; + }, +</code> + +The latter configuration and the path to the certificate file will change according to your IRC client configuration. + +h1. Problems? + +If you have any problems with this, come and chat with us in the @#ircd@ channel or in private with an oper (see @/stats p@ for a list), as usual. + +update, 2018-06-20: + +``` +-----BEGIN PGP SIGNED MESSAGE----- +Hash: SHA256 + +I, anarcat, certify there is a configuration change required on the +CERTFP configuration of clients of the irc.indymedia.org network, to +be performed between 2018-06-20 and 2018-07-03. + +The change consists of changing the fingerprint from the public +certificate to the RSA public key. The resulting fingerprint should be +extracted with: + +certtool --pubkey-info --load-privkey .irssi/certs/nickserv.privkey | grep sha256 | sed 's/.*sha256:/SPKI:SHA2-256:/' + +This message should be signed with an OpenPGP key that you can verify +through Debian's keyrings. + +More details are available in this announcement: + +https://we.riseup.net/ircd/certificate-authentication-requires+467154 +-----BEGIN PGP SIGNATURE----- + +iQEzBAEBCAAdFiEEexZCBNCWcjsBljWrPqHd3bJh2XsFAlsqY0UACgkQPqHd3bJh +2XsFJQf+NUsqwbx0So4acoPE6ZEYUbR9LTv+gfkfYlrhEmzz1rTtFLtCqFA3Yphd +FXha7iT6hUJTY0Z5Led+dr9qBZNfBUzCWAoehSb5ZA8QKgaGHyo9fE1ZkXjXIHDF ++8lClp+oV9J7LZS+U09dsANruwUacarDWYP/dSyuy0i7Q9CSr/Svx6UuQEO0NGI/ +Qd6dZ+wra412c3wjEh/1bmS63+CTzWzB9en6wy2S4yfb1pfgKprlzRsd9J2sRXj0 +poqlu6S9eCDoy6owbKxfIIUTE5/cPnA1pWV0V2qnkZfYzVaHIIzj7w7091Ep8wAx +8Lica9I9dVLIBqCr44TKCjBwL0veoA== +=VXuk +-----END PGP SIGNATURE----- +``` + +update, 2018-06-22: + +For irc.koumbit.net: + +``` + * Certification info: +Subject: + CN=irc.indymedia.org +Issuer: + C=US + O=Let's Encrypt + CN=Let's Encrypt Authority X3 +Public key algorithm: rsaEncryption (2048 bits) + Sign algorithm sha256WithRSAEncryption + Valid since Apr 4 15:38:01 2018 GM to Jul 3 15:38:01 2018 GM +* Cipher info: + Version: TLSv1.2, cipher ECDHE-RSA-AES256-GCM-SHA384 (256 bits) +``` + +This server uses only GnuTLS (Version TLSv1.2). You may consider with using or add this configuration for irc client: + +``` + { + address = "irc.indymedia.org"; + chatnet = "IMC"; + port = "6697"; + use_tls = "yes"; + tls_cert = "~/.irssi/certs/nickserv.privkey"; + tls_verify = "yes"; + autoconnect = "yes"; + }, +``` + +update, 2018-07-20: + +The `certfp_method` was changed today successfully. The older keys can be removed from nickserv, but it’s harmless to keep them there as well. diff --git a/charybdis-4.1-upgrade b/charybdis-4.1-upgrade new file mode 100644 index 0000000000000000000000000000000000000000..beacee73e5e63f7ab75a5de1a65b1668bf97f907 --- /dev/null +++ b/charybdis-4.1-upgrade @@ -0,0 +1,189 @@ +I am in the process of performing the charybdis 4.1 upgrade and here are my notes. + +[[toc]] + +h1. Upgrade procedure + +The Debian package was updated to 4.1 by Unit 193, a recent contributor to the IRCD Debian packages -- he also helped with the recent @atheme-services@ [upload->https://tracker.debian.org/news/966548/accepted-atheme-services-729-2-source-amd64-into-unstable-unstable/]. The @charybdis-4.1@ package was rebuilt as a backport for stretch and uploaded to the least busy server at the time (@chat0@). + +h2. CERTFP problems + +The server rebooted fine but failed to relink, with this error message: + +<code>/var/log/charybdis/ircd.log:2018/6/19 12.18 "sharedconf/connects.conf", line 10: Ignoring connect block for che.indymedia.org -- no fingerprint provided for SSL connection.</code> + +The fix was to add the fingerprint of the remote server in the @connect {}@ block. The fingerprint can be extracted from the certificate file with: + +<code>certtool -i < /etc/letsencrypt/live/irc.indymedia.org/chain.pem | grep -A 1 Fingerprint -m 1</code> + +Unfortunately, this means the fingerprint is based on the *certificate* (as opposed to the private key) which changes every time the certificate is renewed. That is a pain in the bottom, because that changes regularly, and even more because we're using Let's encrypt. + +This mechanism is defined by the [certfp_method->https://github.com/charybdis-ircd/charybdis/blob/22b34124836da7e9f53901af64231294efd4e2f8/doc/reference.conf#L1417-L1431] and it can be changed to @spki_sha256@ to use the fingerprint of the private key instead. The problem is that this also changes the CERTFP authentication from clients to nickserv, so we need our users to add those fingerprints. + +To find that fingerprint, you use: + +<code>( openssl x509 -pubkey -noout | openssl pkey -pubin -outform DER | sha256sum ) < /etc/letsencrypt/live/irc.indymedia.org/chain.pem</code> + +This is also how it works for clients although we've established a simpler, @certtool@-based procedure for them, in [the announcement->https://we.riseup.net/ircd/certificate-authentication-requires+467154]. There are 23 users affected: + +<code> +root@chat1:~# grep MCFP /var/lib/atheme/services.db | awk '{ print $2 }' | sort -u | wc -l +23 +</code> + +... This is less than 10% of the userbase: + +<code> +root@chat1:~# grep ^MU /var/lib/atheme/services.db | wc -l +338 +</code> + +The nice thing with the CERTFP change is that users do not need to generate a self-signed cert anymore, which greatly simplifies the configuration. The announcement will be sent to users invidually, using the following message: + +<code>Hi! In about two weeks, a change will be made to this IRC server's configuration which requires you to add a new fingerprint to your NickServ configuration. More information and proof available here: https://we.riseup.net/ircd/certificate-authentication-requires+467154</code> + +h2. CERTFP rotation + +The above procedure involved upgrading @certbot@ on the main renewal host (@chat1.koumbit.net@): + +<code> +certbot certonly --webroot --webroot-path /var/www/html --reuse-key -d irc.indymedia.org +</code> + +The certs were then redistributed around the cluster and charybdis reloaded. Then the @certfp_method@ was changed to be based on the private key: + +<code> +commit d90fec53ecba740f7b2fb91ead4631a962943083 +Author: root <root@chat0.koumbit.net> +Date: Mon Jul 2 12:01:51 2018 -0400 + + switch to SPKI + +diff --git a/common.conf b/common.conf +index 44b0a75..a5933ce 100644 +--- a/common.conf ++++ b/common.conf +@@ -142,8 +142,8 @@ general { + reject_ban_time = 1 minute; + reject_after_count = 3; + reject_duration = 5 minutes; +- certfp_method = sha1; +- #certfp_method = spki_sha256; ++ #certfp_method = sha1; ++ certfp_method = spki_sha256; + }; + + modules { +diff --git a/connects.conf b/connects.conf +index 79b3e71..f98463a 100644 +--- a/connects.conf ++++ b/connects.conf +@@ -6,6 +6,7 @@ connect "che.indymedia.org" { + hub_mask = "*"; + class = "server"; + flags = ssl, topicburst, autoconn; ++ fingerprint = "SPKI:SHA2-256:8ad5a5348902566d3c5bf859f006b901a620b6a833f5ffaac992d6418ad8c0aa"; + }; + + connect "chat0.koumbit.net" { +@@ -16,6 +17,7 @@ connect "chat0.koumbit.net" { + hub_mask = "*"; + class = "server"; + flags = ssl, topicburst, autoconn; ++ fingerprint = "SPKI:SHA2-256:8ad5a5348902566d3c5bf859f006b901a620b6a833f5ffaac992d6418ad8c0aa"; + }; + + connect "chat1.koumbit.net" { +@@ -31,9 +33,9 @@ connect "chat1.koumbit.net" { + # Fingerprint: + # sha1:4eede5e01df9b58526855c0d6fa22d4306b15d3e + # sha256:d706b0aeaba48e8e8a5094be86b34062ed3bafb5aea439f98eff141682b51686 +- fingerprint = "4eede5e01df9b58526855c0d6fa22d4306b15d3e"; ++ #fingerprint = "4eede5e01df9b58526855c0d6fa22d4306b15d3e"; + # certtool --pubkey-info --load-privkey /etc/letsencrypt/live/irc.indymedia.org/privkey.pem | grep sha256 | sed 's/.*sha256:/SPKI:SHA2-256:/' +- #fingerprint = "SPKI:SHA2-256:8ad5a5348902566d3c5bf859f006b901a620b6a833f5ffaac992d6418ad8c0aa"; ++ fingerprint = "SPKI:SHA2-256:8ad5a5348902566d3c5bf859f006b901a620b6a833f5ffaac992d6418ad8c0aa"; + }; + + connect "services.indymedia.org" { +</code> + +The above config was distributed everywhere and yay, it works. It will need to change again when we fix the renewal process: it would be preferable to have different private keys for each host, but for now this is good enough. + +h2. AUTHD problems + +The new server otherwise seems to work correctly, although a weird bug was introduced during the upgrade (or before?) where connections would hang after this line: + +<code>16:27:56 -!- Irssi: Looking up irc.indymedia.org +16:27:56 -!- Irssi: Connecting to irc.indymedia.org [216.46.7.99] port 6697 +16:27:56 -!- Irssi: Certificate Chain: +16:27:56 -!- Irssi: Subject: CN: irc.indymedia.org +16:27:56 -!- Irssi: Issuer: C: US, O: Let's Encrypt, CN: Let's Encrypt Authority X3 +16:27:56 -!- Irssi: Subject: C: US, O: Let's Encrypt, CN: Let's Encrypt Authority X3 +16:27:56 -!- Irssi: Issuer: O: Digital Signature Trust Co., CN: DST Root CA X3 +16:27:56 -!- Irssi: Protocol: TLSv1.2 (256 bit, ECDHE-RSA-AES256-GCM-SHA384) +16:27:56 -!- Irssi: EDH Key: 521 bit ECDH: secp521r1 +16:27:56 -!- Irssi: Public Key: 2048 bit RSA, valid from Apr 4 15:38:01 2018 GMT to Jul 3 15:38:01 2018 GMT +16:27:56 -!- Irssi: Public Key Fingerprint: + D1:28:F9:B0:4B:92:E0:88:A8:81:1E:84:CA:8E:0C:B0:FC:1E:65:CA:1A:DF:0E:B1:10:2A:AF:D2:83:81:EF:13 (SHA256) +16:27:56 -!- Irssi: Certificate Fingerprint: + 2F:F8:29:36:5B:70:84:05:35:17:0E:9D:D5:3D:65:21:B1:AC:4B:EB:D8:88:DE:FC:0A:CA:71:12:68:0A:BA:16 (SHA256) +16:27:56 -!- Irssi: Connection to irc.indymedia.org established +16:27:56 !chat0.koumbit.net *** Ident disabled, not checking ident +16:27:56 !chat0.koumbit.net *** Looking up your hostname... +16:27:56 !chat0.koumbit.net *** Couldn't look up your hostname +</code> + +and eventually abort after a ~60 seconds timeout: + +<code> +16:28:56 -!- ERROR Closing Link: 127.0.0.1 (Connection timed out) +</code> + +The "fix" was the following diff: + +<code> +commit 184b02d99f3e2272dde3b10a4d8f0636a25baae3 +Author: root <root@chat0.koumbit.net> +Date: Tue Jun 19 17:53:27 2018 -0400 + + do not disable_auth, it breaks connexions somehow + +diff --git a/common.conf b/common.conf +index 7ec4fdb..21294b1 100644 +--- a/common.conf ++++ b/common.conf +@@ -129,7 +129,7 @@ general { + short_motd = no; + ping_cookie = yes; + connect_timeout = 30 seconds; +- disable_auth = yes; ++ disable_auth = no; + no_oper_flood = yes; + max_targets = 4; + client_flood = 20; +</code> + +It is unclear why that stopped working considering it worked before the 4.1 *and* that downgrading to 3.5 did *not* fix the issue. + +h2. certbot changes + +To keep our sanity, we'll want to avoid regenerating the private keys of each server on renewal, which requires a change in the @certbot@ client. This has been implemented as the @--reuse-key@ flag in [PR #5901->https://github.com/certbot/certbot/pull/5901] which was shipped in @certbot 0.25@. That version was accepted in Debian unstable on 2018-06-12 and migrated to testing on 2018-06-14. It is therefore a candidate to a stretch backport and I have asked one of the certbot maintainers (@hlieberman@) if when the backport would be updated. + +Until then, we can either work on a backport ourselves or wait. + +h1. Next steps + +4.1 was tested on @chat0@ and might still be running at the time of writing. An upgrade should be possible without breakage, but will break when certificates are rotated again. The above @certtool@ command can be used to extract the new fingerprint and reconfigure the ircd to match it, but we need to rotate to SPKI so that we don't have to change the config all the time. We also need to make sure certbot does *not* regenerate the private part of the key when we renew the certificate, something that is supported but it is unclear how to do this with certbot. + +h2. Checklist + +1. fix the 4.1 upgrade and chat0 (done! was a problem with authd) +2. confirm SPKI CERTFP works (done! confirmed in server-to-server between chat0 and chat1 and with the anarcat account with client-side CERTFP) +3. send announcement of the change to the 23 users (done! a PRIVMSG was sent to each of the 23 users. out of those, 3 were away and 5 were offline. a MEMOSERV message was sent to those 8 users, out of those 2 were refusing messages. a GLOBAL notice was also sent.) +4. upload the 4.1 debian package to unstable (done) +5. fix certbot so it does not regenerate the private key on renewal (done! backport complete) +6. wait two weeks (done!) +7. upgrade certbot to backport everywhere, test a renewal to see if the privkeys are kept (done, it works) +8. switch all servers @certfp_method@ to SPKI, send announcements (done) +9. in the meantime, progressively upgrade all servers to 4.1 while the users rebalance across the cluster (postponed) diff --git a/dealing-with-abuse.md b/dealing-with-abuse.md new file mode 100644 index 0000000000000000000000000000000000000000..1720e35ade382ff8e4b8e0a02a9b35ff7542a532 --- /dev/null +++ b/dealing-with-abuse.md @@ -0,0 +1,118 @@ +The network is hooked into [dronebl.org->https://dronebl.org/] which keeps away some spammers, but obviously not all. Sometimes it's necessary to kick people out of the network when they don't play by the rule. We don't have formal rules on how to handle abuse, but in general we don't directly meddle in channel politics, except when users try to circumvent channel bans. Obviously, we sometimes have to resolve conflicts between channel operators or add channel operators when there are no operators at all. + +[[toc]] + +h1. Channel operator resources + +Very often, channel operators do not need network operators to deal with abuse, but might not know all the tricks they have at their disposal, so it's useful to note them here again. A lot of this comes from the [channel modes documentation->https://charybdis.readthedocs.io/en/release-3.5/cmodes.html]. Basic settings include: + +* @+n@ and @+t@: no messages from users outside the channel and topic modifiable only by channel operators +* @+s@: secret channel. it will not be visible in the server listings or @/WHOIS@ for a user. surprisingly effective +* @+r@: only registered users can join. very effective, but might not be possible on public support channels as registration is a big barrier for entry on IRC +* @+S@: SSL-only, surprisingly effective: some bots do not bother connecting over TLS +* @+b $HOST@: ban the user from the channel. +* @+q $HOST@: like ban, but the user is not kicked out and is just silenced. useful for trolls as they don't know they are censored. +* @+i@: make channel invite only +* @+O@: make channel IRC operator only +* @+m@ or @+z@: moderated channels + +All those require users to be operators in the channel. A network operator can get channel operator rights in any channel with this: + +<code> +/MSG OperServ MODE #channel +o user +</code> + +See the other [channel modes->https://charybdis.readthedocs.io/en/latest/cmodes.html] you can use in the charybdis documentation, although that list is incomplete: the authoritative list is visible by typing @/QUOTE HELP CMODE@ in your IRC client. + +h1. Private message spam + +Users susceptible to PM spam can set the @+R@ mode on their nick to ignore messages from unregistered users. + +<code> +/mode +R user +</code> + +h1. Killing users + +The basic way to ban a user network wide is to set a @AKILL@ "line", which blocks a user based on its IP. The simplest way to do this is to talk with @OperServ@ which deploys the ban on all servers for us, with a command like this: + +<code> +/MSG OperServ AKILL ADD *@*.compute.amazonaws.com !T 7d weird freenode spam +</code> + +In this extreme case, we blocked all of Amazon from connecting to the servers because of some spam coming from multiple AWS servers. A less brutal case is to block a single user: + +<code> +/MSG OperServ AKILL ADD baduser !T 7d harassing users and avoiding channel bans +</code> + +Notice how we set a temporary ban (@!T@) of limited duration (seven days, @7d@) and specify a reason (@harassing users and avoiding channel bans@). It's nice to put a "reason" that will make sense to other operators (including yourself) in the future when we review those bans. + +A permanent ban can be set with @!P@. + +You can also look for users joining the network with the @+c@ server notice masks, for example: + +<code> +/MODE user +s +c +</code> + +There are more such flags that can be useful, but too many to detail here, see [the full list of flags->https://charybdis.readthedocs.io/en/release-3.5/umodes.html#meanings-of-server-notice-masks] for more information. + +h1. Host/nick pattern-matching + +Sometimes the above is not enough and users return under a different IP address. In that case, it might be useful to match certain patterns like username or nickname, which can be done with the @RWATCH@ command. For example, this bans the nick @psycl0n@ from the network completely, regardless of the hostname: + +<code> +/MSG OperServ RWATCH ADD /^psycl0n!/iSK troll, death threats +</code> + +Some explanation: + +* @/^psycl0n!/@ is a regular expression. +* @/@ marks the beginning and end of the regex, like in Perl +* @^@ is the beginning of the host string, like in Perl +* @!@ is the hostmask seperator (which separates the IRC name from the hostname) +* @i@ means case-insensitive, like in Perl +* @S@ means to show the nick in the @#services@ channel +* and, finally, @K@ means to kill the darn user + +Those patterns should be tested with @RMATCH@ first to make sure only the right user is banned. + +h1. Killing users joining a channel + +Botnet frequently try to grab a hold on a network by rejoining always a control channel. Once found, this channel can be blocked by first registering it with @ChanServ@ then telling @OperServ@ to automatically kill users joining a certain channel: + +<code> +/MSG ChanServ REGISTER #channel +/MSG OpeServ KLINECHAN #channel ON reason +</code> + +Anyone that joins @#channel@ will be temporarily killed on the network, which is very effective. Already present users will not be killed, but might if they disconnect and reconnect. + +h1. Other approaches + +There's a [irssi script->https://dronebl.org/downloads/dronebl.pl] to interoperate directly with DroneBL. It can list and add entries to the database which can be useful for other networks, but wasn't tested by the staff here (yet?). + +Freenode uses a [Linmoria->https://github.com/ProgVal/Limnoria] bot called [Sigyn->https://github.com/freenode/Sigyn] for some of that management. It can kill users based on regular expressions on things said in channels, on repetition and all sorts of neat stuff. + +Atheme has a bot to censor "bad words" called [cs_badwords.c->https://github.com/atheme/atheme-contrib-modules] that could be useful to channel operators as well. It should be available in our version of atheme and might just be a matter of enabling the module and configuring it... + +h1. Server-wide announces + +When trouble comes, it's good to warn users something is going on. The typical way to do this is to send a Global Notice. This can be done by talking with the @Global@ service: + +<code> +/msg Global This is a test. Do not try this at home. +/msg Global SEND +</code> + +If services are down, however, this will not work and you'll need to issue a notice by hand, using a special @/notice@ command, targeting all server domains: + +<code> +/notice $$*.koumbit.net This is a test. Do not try this at home. +/notice $$*.indymedia.org This is a test. Do not try this at home. +</code> + +Note how you need to cycle through all the domain names of the servers in the network (see the list with @/links@ or @/map@). + +In both cases, you obviously need to be a network operator on the servers. diff --git a/getting-a-cloak.md b/getting-a-cloak.md new file mode 100644 index 0000000000000000000000000000000000000000..cddf6215c1d82438dc7326517689d9a84dfa6415 --- /dev/null +++ b/getting-a-cloak.md @@ -0,0 +1,21 @@ +A "cloak" is a string of text that will show to other people on the network if you are logged in to your user instead of your hostname. + +*Requirements (e.g. you need to do this before getting a cloak)* + +# if not already done, register with nickserv: <pre>/msg nickserv register <newirconlypassword></pre> (you need to replace <newirconlypassword> by a good password you made up) +# login to your user. You may want to configure your client to do this every time you login to the network. most clients have mechanisms to perform the login for you. You can also login manually by communicating with nickserv: <pre>/msg nickserv identify <nickname> <yourircpassword></pre> (again, you need to replace <nickname> by your actual nickname, and <yourircpassword> with the password you set for your account). + +*Grab the generic cloak by yourself* + +There is one cloak pattern that you can configure for your login by yourself. The generic cloak is @$account.unaffiliated@. To get it, you can talk with the @hostserv@ service: + +# <pre>/msg hostserv take $account.unaffiliated</pre> -- please note that the @$account@ part should *not* be replaced by your account name. The @hostserv@ service will substitute it automatically for you, and it is part of the cloak pattern. + +*To get a cloak of your choosing* + +If the generic cloak doesn't suit your taste or needs, you can request one of your choosing: + +# decide what string of text should be your cloak. when your cloak is set, you will show up on the network as <nickname>@<yourcloak> -- only when logged in, however. +** Cloak names can contain numbers, letters, dots and hyphens. They must start and end with either a digit or a letter. +# once logged in, say in the support channel (@#ircd@): I would like a cloak: <the cloak name> +# wait, we will add it when we can diff --git a/guerin-certfp.md b/guerin-certfp.md new file mode 100644 index 0000000000000000000000000000000000000000..0d20caf01ae6dc117f4a949836bed3f179c41b87 --- /dev/null +++ b/guerin-certfp.md @@ -0,0 +1 @@ +the venerable guerin server that has served as the previous services host is now officially retired. it has been removed from rotation a while ago and the IRC daemon has been shutdown today. the machine will be decommissionned shortly. also, certfp authentication has been enabled. instructions should be the same than the excellent OFTC documentation: https://www.oftc.net/NickServ/CertFP/ diff --git a/how-to-connect-through-tor-hidden-service.md b/how-to-connect-through-tor-hidden-service.md new file mode 100644 index 0000000000000000000000000000000000000000..ea8c74ea31a19e6e3daad9e444794214954cb64c --- /dev/null +++ b/how-to-connect-through-tor-hidden-service.md @@ -0,0 +1,9 @@ +You can connect to Indymedia IRC server through a Tor Hidden Service, instead of just using Tor to connect to the normal @irc.indymedia.org@ address (which is also possible, by the way). + +The hidden service has the address @akeyxc6hie26nlfylwiuyuf3a4tdwt4os7wiz3fsafijpvbgrkrzx2qd.onion@. + +If your tor software is too old and can't connect to the previous address, you can use the following address: @h7gf2ha3hefoj5ls.onion@ -- however, if it is possible to upgrade your tor software to a more recent version we strongly recommend that you do this and use the first address mentioned above. + +You should connect to it *without SSL* on *port 6667*. The SSL/TLS is already present in Tor, so it's not necessary here. + +Warning: SSL/TLS connections are currently allowed through the hidden service (only the "old" v2 onion service), however the service will present to you a certificate that is only valid for host name @irc.indymedia.org@, thus not for the onion host name. Because of this we recommend not to use SSL/TLS with the hidden service since it's impossible to trust the certificate in this situation. diff --git a/outage-20170202.md b/outage-20170202.md new file mode 100644 index 0000000000000000000000000000000000000000..b074b93567b0e95d99b146196cdd023e08e4079c --- /dev/null +++ b/outage-20170202.md @@ -0,0 +1,41 @@ +Starting yesterday (Thursday March 2nd, 2017) at around 20:00EST (UTC-0500), the InIRC network witnessed a major netsplit, as one of the servers (@che.indymedia.org@) couldn't link back with the other two. This was triggered by a reboot performed by Koumbit to do security upgrades. When trying to restore the network in proper order, I discovered various faults related to a configuration file refactoring I performed two days earlier but that didn't take effect until that reboot. + +It is still unclear the exact cause(s) of the netsplits. The ircd (Charybdis) is not very well maintained anymore and the main development branch ("Charybdis 4") has actually been [abandoned upstream->https://github.com/charybdis-ircd/charybdis/commit/40ac82e87820d9941bdd38571d0390122008e5d3]. It could also be that the Charybdis upgrades performed on Feb. 28th were the cause of some of the problems. + +The following day, starting at around 10:00EST, many different solutions were tried, but in the end the solution was to revert the configuration files to how they were before the refactoring. This was done using @git checkout@ in @/etc@ which caused massive permission problems that had to be fixed (with the magic @.etckeeper@ file) to restore proper operation of the SSH server. That allowed two of the three servers (@che@ and @chat1@) to be relinked, but we still couldn't link all servers: when @chat0@ joined, it kicked @chat1@ out. The "hub" configuration was then changed to make @chat1.koumbit.net@ the hub and the other two nodes only "leaf" servers (@hub=no@). Eventually, @chat0.koumbit.net@ was promoted to @hub=yes@ as well, without obvious adverse effects, @che@ was left at a @hub=no@ configuration. The network was relinked correctly at around 12:30EST. + +One of the side-effects of the Charybdis upgrade is that TLSv1.0 is deprecated. If you are running on older platforms, you may run into problems negociating TLS connexions. Make sure your client support at least TLSv1.1. We have had reports of issues with stunnel on Debian 6 "wheezy" and a hardcoded @sslVersion = TLSv1@ setting, in particular. + +A positive effect of the upgrades is that CertFP should now work on all servers. + +Remaining issues: + 1. some stunnel users had problems connecting + 2. some users disconnect with the server link error we had before (@Read error: Error in the pull function.@) or another TLS error (@Read error: The TLS connection was non-properly terminated.@) + 3. we are not sure servers could survive a full relink + 4. configurations are not in sync. ircd.conf still varies slightly between servers + +Solutions: + 1. upgrade your shit. @sslVersion = TLSv1@ means @TLSv1.0@ AKA RFC 2246, released in 1999. That's 18 years ago. v1.1 was published in 2006, and is supported in OpenSSL 1.0.1, which *is* shipped in Wheezy, so just upgrade your stuff. + 2. it seems this is mostly happening with users that already had connectivity issues due mostly to unreliable connexions. it's just a new error message because upstream bumped up error reporting + 3. relink everything with @ping_time=20s@ and be more patient with relinking. also, upstream proposed testing mbedtls (backport available in jessie) or we could try to cherry-pick some patches (build *without* aaf6039, 65b9b1d, and 8d0153f, or build from 2afd965 with @gnutls.c@ from 3.5.5) to test if there's a linking regression + 4. we'd like to have our ircd.conf more uniform. we could complete that merge one server at a time, progressively. + +A detailed chatlog of the operations is attached for future reference and details. + +update, 2017-07-20: + +an update on this: I have managed to build charybdis against mbedtls, and in a jessie backport too. directives are here: + +we.riseup.net/ircd/charybdis-maintenance + +the package is in my home on chat0 and che. + +the two servers link well now, but can’t link against chat1, presumably because of the bug in gnutls. + +we need to figure out how to migrate over the new servers now, since we can’t link them anymore… i guess we need to flip the switch? + +comments? + +update, 2017-09-26: + +final update on this: the servers were all upgraded to latest charybdis with mbedtls recently, and the network is all cross-linked again, whee!