Being honest about generative AI

 The New York Times this weekend covers students 'fending off accusations of cheating' using AI  [gift link there]:

 In interviews, high school, college and graduate students described persistent anxiety about being accused of using A.I. on work they had completed themselves — and facing potentially devastating academic consequences.

In response, many students have imposed methods of self-surveillance that they say feel more like self-preservation. Some record their screens for hours at a time as they do their schoolwork. Others make a point of composing class papers using only word processors that track their keystrokes closely enough to produce a detailed edit history.

The next time Ms. Burrell had to submit an assignment for the class in which she had been accused of using A.I., she uploaded a 93-minute YouTube video documenting her writing process. It was annoying, she said, but necessary for her peace of mind.

Of course, as we learned in April, also in the Times: 

We have the integrity to note that what students are doing is cheating, but if teachers outsource their intellectual work--and yes, creating lesson plans and quizzes is intellectual work--then somehow that is not? 
Give me a break.

I appreciate the discussion that Marcus Luther at "The Broken Copier" had with his sophomores on this topic.

I spent part of last week with school business officials, which of course is another front in the "AI can do it!" battle. As I shared earlier this month, the latest testing has demonstrated no more than a 50% accuracy rate on finances.

The interview of Colin Fraser that Benjamin Riley shared on 'Cognitive Resonance' perhaps demonstrates why: 

...what you hope for is that the model is able to take a prompt such as “what is two plus two?” and find a symbolic representation which corresponds to two plus two, and then carry out the addition, two plus two equals four. The end. That’s what you hope happens.

But whether or not that is what’s happening is just a really complicated question. It’s very rare nowadays that a production model is not going to tell you two plus two is four, and in fact they can go pretty high in terms of addition.

Yet this actually breaks down once you get to a certain size of number. I've tested this out pretty extensively. Even the best production models start to break down around 40 digits of addition. And that’s a big number to add, for sure. But at the same time, if you know how to add, you can do it. Even a kid can add 40 digit numbers—it'll take them a long time, but they can do it.

If a model gives me an answer to a 40 digit sum and it’s wrong, that means it wasn’t adding.

When I tell you that I find the prospective of generative AI in school finance terrifying, I am not exaggerating.