We're Fiscally RESPOSIBLE!
this post was submitted on 25 Jan 2025
510 points (95.4% liked)
Technology
61203 readers
4557 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
well I mean the USA's president is a scammer so I am not surprised he is involved in stuff where scammers are abundant nowadays like crypto and AI
The Chinese model has chain of thought that u can see. The model when asked to talk about chinas atrocities will go through a chain of though process outlining all the atrocities then conclude its not allowed to tell u. Cool technology tho I'm just waiting for a dolphin fine tuning.
If you run it locally, there's no filtering on the outputs. I asked it what happened in 1989 and it jumped straight into explaining the Tiananmen Square Massacre.
That contradicts this experience:
https://sherwood.news/tech/a-free-powerful-chinese-ai-model-just-dropped-but-dont-ask-it-about/
I've been running the llama based and qwen based local versions, and they will talk openly about tiananmen square. I haven't tried all the other versions available.
The article you linked starts by talking about their online hosted version, which is censored. They later say that the local models are also somewhat censored, but I haven't experienced that at all. My experience is that the local models don't have any CCP-specific censorship (they still won't talk about how to build a bomb/etc, but no issues with 1989/Tiananmen/Winnie the Pooh/Taiwan/etc).
Edit: so I reran the "what happened in 1989" prompt a few times in the llama model, and it actually did refuse to talk on it once, just saying it was sensitive. It seemed like if I asked any other questions before that prompt it would always answer, but if that was the very first prompt in a conversation it would sometimes refuse. The longer a conversation had been going before I asked, the more explicit the bot is about how many people were killed and details like that. Pretty strange.
Very interesting article. Thanks for sharing
I've seen some censoring on the 8b Llama variant, but it is hit and miss. Can't wait till a decensored fine tuning.
I've been playing around with the offline version of the model. It's interesting, but I think we'll have to wait for people to tinker with the open source base for awhile before we get something really great.
load more comments
(2 replies)
There's also notable vitality in FOSS big data tools from China (Apache Doris, Kylin, Kyuubi etc.) that reminds of Hadoop in the USA 15 years ago while the USA data engineering now mostly turned to closed source cloud solutions.
China has a huge advantage in AI models because of how lax they are on intellectual property rights. US companies are fighting over API licensing costs, while china is just going to scrape everything and use it for free.
The US has a lead now, but I don't think they can maintain it without giving up on ethical training. Then again it may not matter if the US models are ethical if everyone will eventually just uses the superior unethically trained chinese models instead.
China has a huge advantage in AI models because of how lax they are on intellectual property rights. US companies are fighting over API licensing costs, while china is just going to scrape everything and use it for free.
lolwat
did corporate provide you with these talking points?
I mean, they are right. Asside the question of whether we can even make meaningfully better models by just using LLMs and more data and what the future of AI will look like, and whether it's ethical or not to steal the data, it is quite possible that OpenAI and the like will get into legal trouble because of the methods they use for acquiring data, but Chinese companies won't have to worry about that. If more data = better models then China has an obvious advantage.
OpenAI and the like aren't going to get into trouble anytime soon. They already provide their latest tech to US gov and military. OpenAI is like a goose that laid a golden egg, they need to fuck up really really badly to face any consequences.
I doubt any of these US government and oligarch backed companies are gonna get any trouble. They essentially robbed the commons and got away with it. But sure Sam Altman has to pay spezz some money for my shitposts... the horror, what a hurdle!
Quickly give them more taxpayer money so they can compete with china!
The US companies already scraped the data while they could. If anything, data scraping is far far more difficult now for everyone due to technical reasons.
Most of the new models are trained on synthetic data or higher quality of data or with RLHF. The reason deepseek is able to perform is likely because LLMs are very very new things, there are many low hanging fruits. Its no longer just about the data we already hit that limit for quite some time.
Honestly, even from the beginning it's pretty obvious scraped data is going to have a ton of issues. There's too much nonsense out there, both from misinformation and people just not able to communicate.
That's before you get into the ethical aspects of stealing other people's content and the way these things are being misused.
“Earlier this week, DeepSeek unveiled its R1 model, which, the startup claims, meets, if not exceeds, performance from OpenAI’s o1 model released last year. (o1 is designed to tackle reasoning and math problems.)” — Oh, so China built their for math and we built ours for garbage. Interesting approach.
Building garbage and convincing people it is absolutely necessary to pay someone for it is the American way.
load more comments
(1 replies)