tired: using CDNs for your fediverse instance

wired: turning the fediverse into a global CDN itself

with @cwebber's datashards proposal, it's possible. we plan to implement datashards in Pleroma 2.0 to accomplish this.

@kaniini @cwebber is this the "sharing bandwidth rather than rehosting everything" solution or something?

@abby @cwebber

datashards in combination with the Kademlia-based DHT work we are already doing for Pleroma 2.0 will allow for a distributed CDN. this is a combination of bandwidth sharing, and also selective mirroring of encrypted chunks.

the DHT provides the ability to know what nodes have what data shards, the data shards themselves provide a cryptographically verifiable source of truth

@kaniini @abby @cwebber
May I suggest talking with the bittorrent people before implementing a DHT ?

My opinion of DHTs at this point is they basically only work as long as there are no bad actors.


@cjd @kaniini @abby Could you elaborate on why? I have my own suspicion of your answer: for content-addressed content, I think they're probably often fine, but if it's a "varying bucket" full of content, it's easier to intentionally drop content from it.

BTW, for the mutable datashards component (which I think Pleroma probably wants for moveable actor profiles) my suspicion is that a gossip network may be more useful than a DHT for informing about updates. Not sure about immutable chunks tho

@cwebber @kaniini @abby
I wrote a modified version of kademlia (it had an additional constraint) and I also ended up knowing a few of the bittorrent people. My opinion of bittorrent DHT is it's mostly a mouse that roars.
When you put data in the DHT, it gets pulled back to a real tracker by DHT scrapers and the same DHT scrapers are in there answering your requests. BUT it's Just Decentralized Enough to be opaque to the lawyers. @astro what's your opinion?

@cjd I'm not aware of trackers scraping the DHT, but it would be a nice idea to enhance connectivity.

I don't think law enforcement services constrain themselves to just HTTP/UDP trackers. They use (modified) standard clients.

BTW, setting the private bit in torrents disables DHT tracking.

@astro I thought you did a talk back in the day about DHT vulns.

@cjd My personal opinion: the BitTorrent DHT is a joyful playground.

Sign in to participate in the conversation

Octodon is a nice general purpose instance. more