AI Judgement And The Shake And Bake School Paper

There’s a certain kind of writing assignment that has never gone out of style in some classrooms; the rise of AI gives teachers a chance to reconsider inflicting it on students ever again and turn their attention to more important things.

The shake and bake assignment, sometimes called a research paper, asks students to collect some sources about a topic and to mush them together. Get some sources, shake them up, and bake the result into submittable format. The final product may suffer from misunderstandings, mistranscriptions from the source, or simple made up baloney that the student has added to pad out the length, but you can hand it in.

What was the supposed to get out of the assignment? Practice citing sources? Deeper understanding of the topic? Skill in disguising obvious plagiarism? The educational value of the shake and bake was never great, but technology has reduced it further, because AI software can shake and bake like nobody’s business.

The problem is not simply that the AI can do the assignment for students. The issues run deeper.

Google’s Gemini is only the most recent example to find itself in trouble. Gemini is the next logical step in truncating the research process, moving from “Google, find me a bunch of sources about widgets” to “Google, find me a bunch of sources about widgets and then sum up what they all say.” Search, shake, bake, and hand over the result.

It’s not just that this takes an assignment that had minimal educational value to begin with and sucks the last edu-molecules from it. It’s not even that Gemini, like most AI programs, has a tendency to just make stuff up. It’s the problems with judgment.

All research involves judgment. Even the most sloppy tenth grade plagiarist looks at the three or four sources they’ve located and judges, “This is the best one to crib from to meet the requirements of this assignment.” The serious researcher has to make hundreds of judgment calls. Is this source reliable and trustworthy? Which pieces of information best fit the story I’m choosing to tell, and how do I keep that story in tune with what the best sources are telling me? How do I decide which sources to use and which to pass over?

All research involves judgment. And that includes research conducted by AI.

Gemini drew a ton of attention because of the charge that it is woke. It generated historical images tweaked for diversity, resulting in Black Wehrmacht officers. It argued that comparing the impact of Hitler and Elon Muck tweets was “difficult to say definitively.” It’s a reminder that it has been almost exactly eight years since Microsoft had to shut its chatbot Tay down because it had become a racist Nazi jerk.

AI isn’t really woke or racist, because it has neither intelligence or judgement as we understand the terms. What it has is algorithms that mimic the use of judgment.

One of the biggest mistakes we indulge with AI (and many other pieces of software) is the notion that the computer somehow exercises objective judgment, that it makes decisions on some pure, unbiased basis. We don’t even need to argue if such a thing is possible, because that’s not what software does. Software exercises whatever judgment creators and managers have “taught” it to exercise, and it does so with the mindless devotion of a machine that does not actually understand what it’s doing.

Rick Hess wrote about this flap, and his fear is worth underlinng:

I’m not fretting here about AI-powered cheating or other abuses of the technology. I’m concerned that, when used precisely as intended, AI will erode the breadth of thought that students are meant to encounter and cast doubt on the need to verify what they’re being told or question current conventions.

That is exactly it, and it goes far beyond simply having AI write that report on widgets for you. Every spot on the internet inserts its own judgment between the material it provides and the users who ask for it, and that judgment is increasingly clouded by commerce (tech writer Cory Doctorow has a word for the process). A critical 21st century skill is using personal judgment to filter and evaluate the results delivered by machine judgment.

Searching for answers cannot be passive or simple. Question, verify, and explore are better actions to pursue than shake and bake.