this post was submitted on 20 Jul 2023
12 points (100.0% liked)

Technology

55964 readers
4627 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS
 

Over just a few months, ChatGPT went from accurately answering a simple math problem 98% of the time to just 2%, study finds::ChatGPT went from answering a simple math correctly 98% of the time to just 2%, over the course of a few months.

top 11 comments
sorted by: hot top controversial new old
[–] impiri@lemmy.world 3 points 1 year ago

Have we considered the possibility that math has just gotten more difficult over the past few months?

[–] meeeeetch@lemmy.world 1 points 1 year ago (1 children)

Ah fuck, it's been scraping the Facebook comments under every math problem with parentheses that was posted for 'engagement'

[–] Matt_Shatt@lemmy.world 0 points 1 year ago (1 children)

The masses of people there who never learned PEMDAS (or BEDMAS depending on your region) is depressing.

[–] orclev@lemmy.world 0 points 1 year ago* (last edited 1 year ago) (1 children)

Pretty much all of those rely on the fact that PEMDAS is ambiguous with actual usage. The reason why is it doesn't differentiate between explicit multiplication and implicit multiplication by placement. E.G. in actual usage "a*b" and "ab" are treated with two different precedence. Most of the time it doesn't matter but when you introduce division it does. "a*b/c*d" and "ab/cd" are generally treated very differently in practice, while PEMDAS says they're equivalent.

[–] 0ops@lemm.ee 0 points 1 year ago (1 children)

I see your point. When those expressions are poorly handwritten it can be ambiguous. But as I read it typed out it's ambiguous only if PEMDAS isn't strictly followed. So I guess you could say that it might be linguistically ambiguous, but it's not logically ambiguous. Enter those two expressions in a calculator and you'll get the same answer.

[–] orclev@lemmy.world 1 points 1 year ago* (last edited 1 year ago) (1 children)

You actually won't. A good graphing calculator will treat "ab/cd" as "(a*b)/(c*d)" but "a*b/c*d" as "((a*b)/c)*d" (or sometimes as "a*(b/c)*d") and actual usage by engineers and mathematicians aligns with the former not the later. You actually can't enter the expression in a non graphing calculator typically because it won't support implicit multiplication or variables. While you can write any formula using PEMDAS does that really matter when the majority of professionals don't?

Actual usage typically goes parentheses, then exponents, then implicit multiplication, then explicit multiplication and division, then addition and subtraction. PEI(MD)(AS) if you will.

[–] 0ops@lemm.ee 0 points 1 year ago (1 children)

Interesting, I decided to try it with a few calculators I had laying around (TI-83 plus, TI-30XIIS, and Casio fx-115ES plus), and I found that the TI's obeyed the order of operations, while the Casio behaved as you describe. I hardly use the Casio, so I guess that I've been blissfully unaware that usage does differ. TIL. I don't think I've ever used or heard of a calculator that supports parentheses but not implicit multiplication though. Honestly though, the only time I see (AB)/(CD) written as AB/CD in clear text (or handwritten with the dividend and divisor vertically level with each other visually) is in derivatives, but that doesn't even count because dt and dx are really only one variable represented by two characters. I'm only a math minor undergrad though who's only used TI's so maybe I'm just naive lol

[–] orclev@lemmy.world 1 points 1 year ago

Or you take HPs approach and just sidestep the entire debate by using reverse polish notation in your calculators. From a technical standpoint RPN is really great, but I still find it a little mind bending to try to convert to/from on the fly in my head so I'm not sure I could ever really use a RPN calculator regularly.

[–] xantoxis@lemmy.one 1 points 1 year ago (1 children)

Why is "98%" supposed to sound good? We made a computer that can't do math good

[–] dojan@lemmy.world 1 points 1 year ago* (last edited 1 year ago) (1 children)

It’s a language model, text prediction. It doesn’t do any counting or reasoning about the preceding text, just completes it with what seems like the most logical conclusion.

So if enough of the internet had said 1+1=12 it would repeat in kind.

[–] kromem@lemmy.world 1 points 1 year ago

Not quite.

Legal Othello board moves by themselves don't say anything about the board size or rules.

And yet when Harvard/MIT researchers fed them into a toy GPT model, they found that the neural network best able to predict outputting legal moves had built an internal representation of the board state and rules.

Too many people commenting on this topic as armchair experts are confusing training with what results from the training.

Training on completing text doesn't mean the end result can't understand aspects that feed into the original generation of that text, and given a fair bit of research so far, the opposite is almost certainly the case to some degree.