this post was submitted on 29 Oct 2023
31 points (83.0% liked)

Asklemmy

43343 readers
895 users here now

A loosely moderated place to ask open-ended questions

Search asklemmy ๐Ÿ”

If your post meets the following criteria, it's welcome here!

  1. Open-ended question
  2. Not offensive: at this point, we do not have the bandwidth to moderate overtly political discussions. Assume best intent and be excellent to each other.
  3. Not regarding using or support for Lemmy: context, see the list of support communities and tools for finding communities below
  4. Not ad nauseam inducing: please make sure it is a question that would be new to most members
  5. An actual topic of discussion

Looking for support?

Looking for a community?

~Icon~ ~by~ ~@Double_A@discuss.tchncs.de~

founded 5 years ago
MODERATORS
 

Not just speaking technically but economically. As user number mount up their post is just going to add up. I know instaces may distribute user, but server space isn't going to go down for anyone. It just seems like built to fall.

you are viewing a single comment's thread
view the rest of the comments
[โ€“] squaresinger@feddit.de 21 points 10 months ago (1 children)

Lemmy is setup quite ok to scale users and traffic, since the users are (theoretically) distributed over many instances. In reality it doesn't work out perfectly, since people generally are more likely to join the biggest instances, so there's quite an imbalance there.

But what's worse is content replication. As soon as an user requests to look at a community for the first time, the whole community will get completely replicated on that user's instance. So any decently sized instance will pretty much replicate almost all communities, at least all that have content.

There is no scaling or storage balancing mechanic here. Even if no user ever touches that replicated community again, it will continue to be replicated, will fetch all new posts/comments and store them in the instance.

There is also currently no workaround to this (like there is for users/traffic, which you can just tell to join a different instance).

So if Lemmy ever gets to the point where gigabytes of data gets posted every day (which is only about 1000 pictures a day) storage demand will get so high, that hosting an instance will be seriously costly, which will probably lead to instances without any kind of cash flow shutting down, which will in turn lead to more users and thus traffic on the remaining instances.

I guess, that's one of the biggest technical (and conceptual) roadblocks that need to be addressed if Lemmy ever grows that big.

[โ€“] arthur@lemmy.zip 4 points 10 months ago (1 children)

As the project evolves, there will be fixes for that kind of problem, it is not an immediate problem.

[โ€“] squaresinger@feddit.de 5 points 10 months ago

Hopefully. Because right now this is not a simple bug but engrained in the core concept of how Lemmy works.