mastodon - Link to source

"While LLMs offer immediate convenience, our findings highlight potential cognitive costs. Over four months, LLM users consistently underperformed at neural, linguistic, and behavioral levels. These results raise concerns about the long-term educational implications of LLM reliance and underscore the need for deeper inquiry into AI's role in learning."

media.mit.edu/publications/you…

#MOLE #AI #LLM #cognition

Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task – MIT Media Lab

This study explores the neural and behavioral consequences of LLM-assisted essay writing. Participants were divided into three groups: LLM, Search Engine, and …
MIT Media Lab
#AI #cognition #LLM #mole
in reply to Strypey

friendica (DFRN) - Link to source

Nate

A bit of a pet peeve, but that study rubs me the wrong way. Not the results, but that we had to have a study to confirm the results.

It'd be like having a study where one group fixes their car for three months, and a control group pays a mechanic to fix their car instead. Of course the people fixing their car are going to be more mechanically proficient in the end, and if you measure brain activity, fixing a car will show more brain activity than sitting in the waiting room of a garage.

/end rant

in reply to Nate

mastodon - Link to source

Strypey

@nate
> if you measure brain activity, fixing a car will show more brain activity than sitting in the waiting room of a garage

Sure. But if you swap the groups over, you'd expect that the group now fixing their own car would show the same level of brain activity as the previous group, who are now waiting for the mechanic. What the study shows is that the erosion of brain activity during LLM use persists, even once they're asked to write their own essay unaided.

@Nate
in reply to Strypey

friendica (DFRN) - Link to source

Nate

IIRC, that was at the end of three months when they swapped out who was doing what (been a bit since I went through it). Practice makes perfect - you take somebody who's been working on their car for three months, and somebody who outsourced it to a mechanic for three months, and even doing the same task I'd assume the former would have much more activity in their neural tracts than somebody doing it for the first time. The brain (at least mine anyway - maybe that's why I'm so lazy) doesn't like doing things it doesn't have to. If somebody's been practicing a skill, and somebody's been outsourcing a task, I can't imagine any other outcome to be expected. That was part of what rubbed me the wrong way, even if my mini above wasn't very complete.
in reply to Nate

mastodon - Link to source

Strypey

I still seems to me like you're missing the point, but I'm not sure how to explain why I think so more clearly. Maybe read the full paper (again)?

Maybe the problem is not your logic, but your premises. Particularly the idea that having a mechanic fix your car would lead to a persistent, reduced level of cognitive activity, in the way that using LLMs to write essays seems to. I don't see why it would.

Are you aware of any evidence that supports this proposition?

This entry was edited (2 days ago)
⇧