samizdat issueshttps://0xacab.org/rysiek/samizdat/-/issues2020-10-05T17:55:30Zhttps://0xacab.org/rysiek/samizdat/-/issues/23More advanced method of creating the array of plugins, with checks implemented2020-10-05T17:55:30ZMichał "rysiek" WoźniakMore advanced method of creating the array of plugins, with checks implementedWe are getting to a point where there are very specific requirements on plugins ("*must have `fetch()`; can have `stash()` and `publish()`; must have `unstash()` if has `stash()`*", etc), but we're still using very naive way of adding pl...We are getting to a point where there are very specific requirements on plugins ("*must have `fetch()`; can have `stash()` and `publish()`; must have `unstash()` if has `stash()`*", etc), but we're still using very naive way of adding plugins to the `SamizdatPlugins` array.
We need a more sane way of dealing with it, that would also perform some basic checks.https://0xacab.org/rysiek/samizdat/-/issues/74Censorship-circumvention for the config2020-10-05T17:53:41ZMichał "rysiek" WoźniakCensorship-circumvention for the configCurrently, all config options are kept in `config.js` and imported directly in `service-worker.js`, which means it's considered a part of the Service Worker by the browser and only allowed to update via direct HTTPS `fetch()`.
We need t...Currently, all config options are kept in `config.js` and imported directly in `service-worker.js`, which means it's considered a part of the Service Worker by the browser and only allowed to update via direct HTTPS `fetch()`.
We need to be able to fetch updated config settings (by fetching `config.js` or via some other means) and update the Service Worker configuration using it. This probably requires going through the cache, and storing *specific config options* there.
Not all config options can be meaningfully changed this way, however. For example, the set of JS files (and thus, plugins) that is loaded by the Service Worker cannot change without a direct HTTPS `fetch()` (which is a limitation imposed by the API specs); if implemented right, their order could be different, however. This could create a situation where a new config requires loading of files that have not been loaded and breaks Samizdat for someone for whom the original site is blocked.
tl;dr this is trickyMichał "rysiek" WoźniakMichał "rysiek" Woźniakhttps://0xacab.org/rysiek/samizdat/-/issues/72Allow loading the same plugin multiple times with different config2020-10-04T00:48:07ZMichał "rysiek" WoźniakAllow loading the same plugin multiple times with different configWe need a way to load the same plugin multiple times with different configs. For instance, we might want to try using a few different IPNS identities, or different HTTPS gateways, etc.
This is also relevant for #70.We need a way to load the same plugin multiple times with different configs. For instance, we might want to try using a few different IPNS identities, or different HTTPS gateways, etc.
This is also relevant for #70.Michał "rysiek" WoźniakMichał "rysiek" Woźniakhttps://0xacab.org/rysiek/samizdat/-/issues/73Make it possible to explicitly configure each plugin when loading2020-09-29T22:37:11ZMichał "rysiek" WoźniakMake it possible to explicitly configure each plugin when loadingCurrently a plugin file gets loaded and a plugin immediately instantiated, with a hard-coded `SamizdatConfig` path (based on plugin name) used for config. This needs to change.
This is relevant for #72Currently a plugin file gets loaded and a plugin immediately instantiated, with a hard-coded `SamizdatConfig` path (based on plugin name) used for config. This needs to change.
This is relevant for #72Michał "rysiek" WoźniakMichał "rysiek" Woźniakhttps://0xacab.org/rysiek/samizdat/-/issues/2Consider implementing an IPNS+IPFS plugin2020-09-18T00:08:22ZMichał "rysiek" WoźniakConsider implementing an IPNS+IPFS pluginThis seems... saner than Gun? https://github.com/ipfs/js-ipnsThis seems... saner than Gun? https://github.com/ipfs/js-ipnshttps://0xacab.org/rysiek/samizdat/-/issues/63Pure Gun transport plugin?2020-09-16T23:18:08ZMichał "rysiek" WoźniakPure Gun transport plugin?Seems like it should be possible to have a pure Gun transport plugin:
>>>
Michał Woźniak
@rysiekpl_gitlab
Sep 15 22:43
>> @rysiekpl_gitlab I've synced even video files as big as 30MB, and part of the goal of the new version is to see ho...Seems like it should be possible to have a pure Gun transport plugin:
>>>
Michał Woźniak
@rysiekpl_gitlab
Sep 15 22:43
>> @rysiekpl_gitlab I've synced even video files as big as 30MB, and part of the goal of the new version is to see how far I can push this limit ( @QVDev at one point was near live-streaming video across 4G) if you wanna help with the Infinity Scroll test... once that is passing I want to apply it to video.
>
> Right. When you say "synced", how exactly? I assume not using put(), right? Is there some API I am missing?
gunchatbridge
@gunchatbridge
22:06
> [D] marknadal: <@!105969993933422592>
> @rysiekpl_gitlab yes using .put( 😛 not saying it was the best idea but works to some extent and I'm trying to make it better over time. https://github.com/amark/gun/blob/master/examples/basic/upload.html
>>>
Not sure how useful that would be. Gun still does not have a public node network, and is still rather... unstable.https://0xacab.org/rysiek/samizdat/-/issues/4Consider implementing WebTorrent alongside IPFS2020-09-15T23:50:39ZMichał "rysiek" WoźniakConsider implementing WebTorrent alongside IPFSThis seems usable:
- https://webtorrent.io/docs
- https://github.com/webtorrent/webtorrent
Importantly, it is possible to fetch a particular file from a torrent. So, Gun could store the `magnet:` link of the latest torrent, and WebTor...This seems usable:
- https://webtorrent.io/docs
- https://github.com/webtorrent/webtorrent
Importantly, it is possible to fetch a particular file from a torrent. So, Gun could store the `magnet:` link of the latest torrent, and WebTorrent would be able to fetch only the files relevant to a given pageview.https://0xacab.org/rysiek/samizdat/-/issues/49Enable limiting Samizdat to a particular set of locations2020-07-08T16:54:13ZMichał "rysiek" WoźniakEnable limiting Samizdat to a particular set of locationsCurrently once deployed Samizdat will handle all sub-locations of wherever is deployed.
For example, if it's deployed at `example.com/`, it will handle all locations on that domain; if it's deployed at `example.com/samizdated/`, it will...Currently once deployed Samizdat will handle all sub-locations of wherever is deployed.
For example, if it's deployed at `example.com/`, it will handle all locations on that domain; if it's deployed at `example.com/samizdated/`, it will only handle sub-locations of that (ut not, for example, `example.com/non-samizdated/`).
It would be good to be able to configure which sub-locations should be handled by Samizdat. This could be done as a configurable regex in `service-worker.js`Michał "rysiek" WoźniakMichał "rysiek" Woźniakhttps://0xacab.org/rysiek/samizdat/-/issues/5[Design Discussion] Switch to keeping a single IPFS directory address in Gun?2020-07-07T09:39:56ZMichał "rysiek" Woźniak[Design Discussion] Switch to keeping a single IPFS directory address in Gun?Apparently IPFS is pretty smart about modified resources:
```
root@628dd289a8e6:/tmp# /node_modules/go-ipfs/bin/ipfs --api=/ip4/$( getent hosts ipfs | cut -d ' ' -f 1 )/tcp/5001 add -r ipfstest/
added QmNRxNyFAN3aW61qsyRUhZnKMPNAoHy91kt...Apparently IPFS is pretty smart about modified resources:
```
root@628dd289a8e6:/tmp# /node_modules/go-ipfs/bin/ipfs --api=/ip4/$( getent hosts ipfs | cut -d ' ' -f 1 )/tcp/5001 add -r ipfstest/
added QmNRxNyFAN3aW61qsyRUhZnKMPNAoHy91ktLtLfgEzJJm8 ipfstest/test1
added QmZNhZXeejz1nebssQVoLZptgYcyseaLPyTgNTvDo6YmYt ipfstest/test2
added Qmdb1FxJVBXK9rffNxMzPefczQeGByiui42eoEn4HzXUxc ipfstest/testdir1/test3
added QmXHm5fQ6jH6UbFGhrYpV4UYJSBisrkyt69oB61U9iXhnw ipfstest/testdir2/test4
added QmbasMx71y9VqF7Gu5cSR1LSUVjzVsZ6kv8Q84fk1wornH ipfstest/testdir2/test5
added QmQ21HkVtgHj6Q5fkM2JNsHxvUsyBc2QXoP4YsNXsFuKc3 ipfstest/testdir1
added QmazcfxeFukkyano7wtHUMzV5WiongVNfkj7oioD84uMbf ipfstest/testdir2
added QmR5aQS6eTUKm4YCCqtxgrtvGwopxHycHZZgCPoKNKmRcc ipfstest
145 B / 145 B [=====================================================] 100.00%
root@628dd289a8e6:/tmp# date > ipfstest/testdir2/test5
root@628dd289a8e6:/tmp# date > ipfstest/testdir2/test6
root@628dd289a8e6:/tmp# date > ipfstest/test2
root@628dd289a8e6:/tmp# /node_modules/go-ipfs/bin/ipfs --api=/ip4/$( getent hosts ipfs | cut -d ' ' -f 1 )/tcp/5001 add -r ipfstest/
added QmNRxNyFAN3aW61qsyRUhZnKMPNAoHy91ktLtLfgEzJJm8 ipfstest/test1
added QmQbrS4tGuaqJxzJ1ZQBjgJVYqSaaM74v1CvJ1mK87S5TV ipfstest/test2
added Qmdb1FxJVBXK9rffNxMzPefczQeGByiui42eoEn4HzXUxc ipfstest/testdir1/test3
added QmXHm5fQ6jH6UbFGhrYpV4UYJSBisrkyt69oB61U9iXhnw ipfstest/testdir2/test4
added QmUsijQuChQJqq1381XQp7va1aBAf1GebWSCLht756nAY4 ipfstest/testdir2/test5
added QmcYwvzgsy3ceJv4KAKRW4PXmUbuJEWTVsgu8upU4n88j4 ipfstest/testdir2/test6
added QmQ21HkVtgHj6Q5fkM2JNsHxvUsyBc2QXoP4YsNXsFuKc3 ipfstest/testdir1
added QmPftYxJeAwZgzKFffieHnfPXvn81ksYeoaSxTiNnccNXa ipfstest/testdir2
added QmY1SRpgHw83j5RQnyxYXUgDWVT1SrpYCQ5XVb1aLn5RVe ipfstest
174 B / 174 B [======================================================] 100.00%
```
Notice how the IPFS addresses of files that have not been modified do not change, but the address of the whole directory changes. So, a directory is not added as a single blob, but as a collection of independent files.
That means that we could reasonably only keep **one** IPFS address in Gun (the latest version of the whole directory) and rely on IPFS to fetch handle resolving each file. That would simplify things on the Gun side (no need for an entry for each file). Impact on reliability and speed is unclear.
It seems this is how IPNS works, in fact.https://0xacab.org/rysiek/samizdat/-/issues/45Check-out dweb-transports2020-02-12T18:23:31ZMichał "rysiek" WoźniakCheck-out dweb-transportsBught be useful: https://github.com/internetarchive/dweb-transportsBught be useful: https://github.com/internetarchive/dweb-transportsMichał "rysiek" WoźniakMichał "rysiek" Woźniakhttps://0xacab.org/rysiek/samizdat/-/issues/7Consider implementing/using Dat2020-02-02T21:33:37ZMichał "rysiek" WoźniakConsider implementing/using DatDat's interesting and relevant:
- https://github.com/datproject/sdk
- https://hacks.mozilla.org/2018/08/dweb-serving-the-web-from-the-browser-with-beaker/
Dat's interesting and relevant:
- https://github.com/datproject/sdk
- https://hacks.mozilla.org/2018/08/dweb-serving-the-web-from-the-browser-with-beaker/
https://0xacab.org/rysiek/samizdat/-/issues/9Consider implementing a fetcher hitting WayBack machine2019-12-10T12:49:33ZMichał "rysiek" WoźniakConsider implementing a fetcher hitting WayBack machineSince we're already handling fetches ourselves in the ServiceWorker, we might as well try to get stuff from the WaybackMachine if it's there are network issues.Since we're already handling fetches ourselves in the ServiceWorker, we might as well try to get stuff from the WaybackMachine if it's there are network issues.https://0xacab.org/rysiek/samizdat/-/issues/28Consider using Snowflake as a transport plugin?2019-12-10T12:47:49ZMichał "rysiek" WoźniakConsider using Snowflake as a transport plugin?Might be relevant: https://trac.torproject.org/projects/tor/wiki/doc/SnowflakeMight be relevant: https://trac.torproject.org/projects/tor/wiki/doc/Snowflake