When AI Becomes Judge (Part 2): The Double Standard: When AI Judges Students

This will likely be the last thing I write as a university student—at least for a while. Over the past two years, alongside my work in Generative AI, I have been researching the rise of what I call the "Digital Leviathan": the creeping normalization of algorithmic authority over human judgment and the harmful societal effects that result from it.

To some extent, this is a logical continuation of my BA thesis in law, which concerned the mandatory data retention of telecommunications companies. In one week, I defend my Master's thesis, Beyond Fragmentation: A Life-Value Alternative for AI Governance. I will share more of that soon, but first, let me finish what I started.

The lesson from Bifröst is that we cannot trust algorithms to be judges. The lesson from Turnitin is that we cannot trust institutions to protect us from these algorithms.

In Part 1, I discussed the incident at Bifröst University where administrators, according to news reports, used AI to judge the professional career of their staff. We saw the outrage. We saw the "2/10" rating. We saw the system reject the idea that a "black box" should determine professional worth.

But while we were busy being outraged on behalf of the professors, we forgot about the students.

Every day, in almost every university in Iceland, student assignments are "judged" according to the exact same black box that we just declared unethical for staff.

We call it Turnitin.

The Double Standard

If it is administrative malpractice to use Claude to judge a professor's contribution, why does a different rule apply to students? Both systems are:

Opaque: No one knows exactly how the algorithm processes the data or what weight different factors have.
Extractive: They rely on data harvested without the meaningful consent of authors.
Hallucinations and Statistical Guesses: Unlike traditional plagiarism protection, which is based on facts, the AI scanner relies on probability assessments produced by the system, which administrators often treat as infallible truth.

One truth, one guess

It is important to distinguish between two different "features" within Turnitin.

On one hand, there is the Similarity Report, which compares text to a database and finds direct matches—that is a fact check.

On the other hand, there is AI Writing Detection, the new tool meant to catch AI. That tool does not compare text; it looks for statistical patterns in sentence structure. This is not proof, but a prediction. Research and testing repeatedly show that these scanners produce "false positives," where honest students are given a failing grade by an oracle that cannot explain its conclusion. When universities use this probability assessment as a basis for disciplinary sanctions, the legal security of students is sacrificed for the convenience of technology.

The difference? The professors at Bifröst had a union and an ethics committee. The students check a box that says: "I Agree."

Is there not a certain irony in Icelandic students signing away the rights to their work by agreeing to a contract in English?

The Rent Model of Education

Turnitin was acquired for $1.75 billion. That value does not come from their code. It comes from the billions of essays and assignments they have sucked up from students.

When you submit an essay, you are not just getting a grade. You are performing unpaid data labor. You are training the AI that will be sold back to your school next year—at a higher price. This is the money-sequence in action: your intellectual property converted into their capital. Your tuition pays for the privilege of feeding the machine.

The "Trap"

Most students don't read the End-User License Agreement (EULA). I did.

"...You hereby grant to Turnitin... a non-exclusive, royalty-free, perpetual, worldwide, irrevocable license to use such papers..."

Translation: You pay tuition to create the work. They keep it forever—for free.

The Solution: "Do not store"

There is a way out. Turnitin has a setting called "Do not store the submitted papers." If a professor enables this option, your paper is scanned but not saved in the database. It is checked for plagiarism but never added to the permanent repository.

Why isn't this the default setting?

Because of what I call Responsibility Fog. The collection setting is the default. Few likely question it. No one "owns" the decision. And by the time you realize what happened, your work is already someone else's asset.

How to Opt Out

If you are a student, you have rights—under Article 17 of the GDPR and under Turnitin's own EULA—to demand your data be deleted after assessment.

Do not accept the Benevolent Cage. Use these templates or create your own; you do not need to follow prerequisites to reclaim what is yours.

For Submission: The Statement

Attach this or a similar text to your assignment:

I am submitting this assignment to Turnitin to comply with this school's requirements. However, I do not consent to the permanent storage of my intellectual property in a commercial database. I will be filing a formal request for deletion immediately following assessment.

For Deletion: The Demand

Send this to your instructor after grading:

Subject: Formal Request for Deletion from Turnitin Repository
Please forward this to the School's Turnitin Administrator:
I submitted the assignment [Title] (Paper ID: [ID]) for assessment. Now that grading is complete, I am exercising my right under the Turnitin EULA ("License by You") and GDPR Article 17 to have this submission permanently deleted from the repository.
Please confirm by email when this action is complete.

Conclusion

The only lesson left is this: we must protect ourselves.

Further Reading: