There was some talk of this issue in the recent fediverse inefficiencies thread. I'm hopeful that in the future we'll have a decentralized solution for file hosting but for now I deeply believe that users should pay for their own file hosting.

you are viewing a single comment's thread
view the rest of the comments

[–] sosodev@lemmy.world 2 points 2 days ago* (last edited 2 days ago) (1 children)

Are the images duplicated when shared? My understanding is that only a link to the file is replicated across servers and duplication comes from users manually uploading the same file to another server.

My website does not do any deduplication at this time.

[–] Deebster@lemmy.ml 4 points 2 days ago* (last edited 2 days ago) (1 children)

Yes, for example go to https://infosec.exchange/explore

I see the top post as https://infosec.exchange/@nocontexttrek@mastodon.social/113433063621462027 and the image is https://media.infosec.exchange/infosec.exchange/cache/media_attachments/files/113/433/063/582/671/258/original/71da3801e4e4f08c.png

The link is to the original on https://files.mastodon.social/media_attachments/files/113/433/062/676/773/993/original/f828afef5cc7ed1c.png but when you click image the javascript loads a modal with the local cached version (same image as the thumbnail that infosec.exchange loads.

There's lots of different codebases across the fediverse so perhaps some hotlink, but local copies is the default.

[–] sosodev@lemmy.world 5 points 2 days ago* (last edited 2 days ago) (1 children)

The Lemmy server config indicates that is an optional setting to improve user privacy so requests don't ever hit the original server from the client. Those cached files are only temporary and will be deleted after some time. So it's not really full blown duplication.

The default setting is to only generate the thumbnails and store those locally (indefinitely?) but even that can be turned off. I checked and it appears that lemmy.world has the thumbnail generation disabled so all images from other instances just link to the original on that instance.

[–] Deebster@lemmy.ml 2 points 2 days ago (1 children)

Ok, so Lemmy doesn't cause the same amount of duplication, but I'd still argue that dedupe is valuable: it saves on hosting costs (your costs, in this case) and users will get a small advantage in having slightly higher cache hits.

[–] sosodev@lemmy.world 3 points 2 days ago

For sure, I'll add it to the list. :)