this post was submitted on 23 Apr 2024
550 points (97.1% liked)
Technology
60079 readers
3372 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
[Warning: "ideas guy" tier babble]
It's somewhat clear that search engines are too prone to go to shit, either due to malice or something worse (like stupidity).
Based on that, I wonder if a user-run, free-as-speech and open source decentralised search system wouldn't work. Roughly in the spirit of torrents - where anyone can use the system but if you're using it you're expected to contribute with it too.
You just described the categories pages many search engines had before Google. Or proto Web 2.0 bookmark sharing sites like del.icio.us. Sites like Metafilter also existed as a kind of Internet index before everyone was adding reddit.com to their Googling. It's a laudable idea, but these systems all seem to fall prey to market manipulation in much the same way that SEO helped kill Google.
It's interesting that you mention MetaFilter, because they're literally in the process of transitioning fully to a non-profit organization.
https://metatalk.metafilter.com/26430/MeFi-Nonprofit-Update-March-26-2024
They're the only aggregator that still isn't flooded with ads and has pretty decent moderation policies.
There's absolutely a reason I linked to the discussion over there: because it's quality, and it's the first place I saw the article pop up.
Wow, that's really neat.
Thanks for letting me know about MetaFilter and its transition to NPO. This really seems like a great move for the site.
I've heard of the site before, but haven't had the chance to try it before. Guess a bit late is better than never, right? :D
I was thinking on something slightly different. It would be automatic; a bit more like "federated Google" and less like old style indexing sites. It's something like this:
It would be vulnerable to SEO, but less so than Google - because SEO tailored to the algorithm being used by one server won't necessarily work well for another server.
Please, however, note that this is "ideas guy" tier. I wouldn't be surprised if it's unviable, for some reason that I don't know.
I think you could do it in Lemmy itself combined with RSS feeds. The mods would curate a list of RSS feeds, and use the keywords to pick the ones for a bot to automatically post (which means if a programming blog did a post about windsurfing, it wouldn't show up as long as the meta keywords didn't match). Mods could take suggestions each week for feeds to add or remove.
There was (is?) the yacy project which used a distributed index, and the individual nodes would contribute to the index.
A hybrid of original Yahoo! and Google is probably the best option. Sites submit themselves, they get reviewed, and an algorithm catalogs the contents. So curation and automatic indexing together.
https://en.m.wikipedia.org/wiki/YaCy
That exists, it's called Searx, and in my experience it isn't great
No that still uses traditional search engines in the background
Searx is a meta-engine, as bdonvr mentioned.
since i changed the instances i use, it works great for me
One part of this (which isn't really covered in the article) is that Google historically had a give-and-take relationship with people gaming search engine results. SEO has been a thing for a long time, and it's impossible to make it go away. However, Google used to punish sites that took it too far. It wasn't necessarily ideal, but it worked well enough to keep egregious spam out of the top level results, and companies could still direct users to their site when they had something they were actually looking for. SEO consulting companies sprang up who knew Google's rules well, and that arguably meant a bunch of grifters being overpaid, but at least the results stayed relevant.
Google seems to have given up on enforcing many of those rules.
There's also some minor discussion in the MeFi thread about "federated search" as well.
Self-hosted search also seems like a strong possibility.
The problem that I see with self-hosting is that it isn't a practical reality for most people, due to different tech expertises and machine capabilities. Instead I think that a better system would allow you to simply install some software, and contribute as much as you can while you use it.
I'm not informed on MetaFilter. From your other comment it seems that it's also an indexing site (besides being a community - from their "About" page). Is this correct?
Yes, it's got a really old-school layout, because it's been around since 1999. To me, the fact that they've managed to avoid advertising for 25 years while having the main indexing site as well as things like Ask MetaFilter, IRL meetups, and even a jobs board, it means they've been pretty darn good at managing their finances and figuring out how to support the site long-term without ads. They're also in the process of becoming an actual non-profit organization. They pay their moderators a living wage, because it's a job. That's... pretty amazing.
The comment section takes a bit to get used to, because it's just chronological order of comments, no sorted threads. Very, very old school web ethos. However, if you can get used to it, some really amazing discussion can happen in there.
One of the benefits of the ways MeFi posts work is often you have users doing massive amounts of research and providing literally mountains of links and analysis, you can get pretty lost in the weeds on some posts.
It's been the source of high quality discussions for a long time and there's some really interesting professionals on there who have been staples of the community for a long time. Think hackernews and how many people it has from the industry, but instead of it all being tech people (MeFi has it own share of techies) it is thoughtful and sometimes expert opinion from a large variety of disciplines, as well as first person accounts from people of all walks of life.
It's also where I first found this link (The Man Who Killed Google Search) and decided to post it here.
Seems like something the public library system should be doing. That and hosting websites for the community not for profit but as a public service.
While I'm on wish list tangent, post offices should be municipal banks and be a free email domain provider.