this post was submitted on 21 Jul 2023
78 points (98.8% liked)
Technology
58306 readers
3209 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
LLMs choose words based on probabilities, i.e. given the word "blue", it will have a list of words and probabilities that those words should follow "blue". So "sky" would be a high probability, "car" might also be quite high, as well as a long list of other words. The LLM chooses the words not by selecting whatever has the highest probability, but with a degree of randomness. This has been found to make the text sound more natural.
To watermark, you essentially make this randomness happen in a predefined way, at least for cases where many different words could fit. So (to use a flawed example), you might make it so that "blue" is followed by "car" rather than "sky". You do this throughout the text, and in a way that doesn't affect the meaning of the text. It is then possible to write a simple algorithm to detect whether this text was written by an AI, because of the probability of different words appearing in particular sequences. Because its spread throughout the text, it's quite difficult to remove the watermark completely (although not impossible).
Here's an article that explains it better than I can: https://www.kdnuggets.com/2023/03/watermarking-help-mitigate-potential-risks-llms.html