this post was submitted on 04 Mar 2025
608 points (98.9% liked)
Technology
64074 readers
5893 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Thanks for sharing!
Thanks for reading and commenting!
During my first (shitty) job as a dev outta school, they had me writing scrapers. I was actually able to subvert it pretty easily using this package that doesn't appear to be maintained anymore https://github.com/VeNoMouS/cloudscraper
Was pretty surprised to learn that, at the time, they were only checking if JS was enabled, especially since CF is the gold standard for this sort of stuff. I'm sure this has changed?
Given that the last updates to this repo were five years ago, I'm not too sure if it's still valid. I don't follow Cloudflare bypasses but I am fairly certain there are more successful frameworks and services now. The landscape is evolving quickly. We are seeing a proliferation of "bot as a service", captcha passing farms, dedicated browsers for botting, newsletters, substacks, Discord servers, you name it. Then there are the methods you don't readily find much talk on like custom modified Chrome browsers. It's fascinating how much effort is being funneled into this field.
Oh i can definitely see custom browsers being useful in that area. I remember the JavaScript navigator properties were always such a PITA as there was nothing you could really do to get around what they exposed
There's a whole world of tools you can use that do that for you now. It's easier than ever. To me it's concerning. The level of automation, coupled with a halfway decent LLM, can give you the ability to summon hordes of fake humans to social media. I can't help but think it's why X and Reddit don't use any kind of anti-bot solution.