Archives
Rethinking generative AI in the classroom – I finished up the last little bits of administrative work at the end of last week, which means that yet another semester is in the books. Both of my classes—my graduate seminar and my undergrad class—went well, and I had a good group of students in both. That being said, there was a familiar specter that darkened my desk with its shadow when I was marking final essays for my undergrad class: the specter of generative AI.
There were two essays in particular that were very obviously written by AI, and I contacted both students to ask them about this. Neither of the students is a native speaker of English (my undergraduate class is conducted in English, for the record), and they both told me that they had written their essays in their native languages and then had AI translate them into English. They both, however, insisted that they had done the research and written the original essays themselves. I have no reason to doubt them on this; they’re both good students and worked hard throughout the semester. They simply availed themselves of a tool to help them produce what they thought would be a better finished product.
Were the essays really better, though? It is impossible to say, of course, because I had nothing to compare them against. But I do know that both essays reeked of the artificial, jargon-laden writing that AI models tend to associate with “academic” writing; one of the students even said that they had asked the AI to make their essay sound more “academic.” The other student did not mention doing this, but I suspect they did, as their essay on Korean art was absolutely buried in things that were “hyper-” this or “hyper-” that, people being “atomised” and having “phenomenological experiences,” etc. I had to wonder if the student even understood the language that the AI had produced, because for the most part it was a post-modern obfuscatory word salad. Take “phenomenological experiences,” for example. In philosophy, phenomenology refers to a focus on how people experience things (as opposed to a focus on the things themselves), so this phrase is a tautology at best.
But my aim here today is not to point out how AI writing is devoid of creativity and often nonsense when it attempts to mimic “academic” writing. No, that’s just shooting fish in a barrel. Instead, I want to talk a bit about the use of AI among students and what I am thinking of doing about it. I’m still in the middle of working through all of this, so what follows may come out a little scattered and under-baked.
My policy toward AI so far has been, I will admit, somewhat lackadaisical. I have told students that they can use AI for proofreading purposes and to clean up their language, but I have been clear that they should not have AI write their assignments for them. Why the partial leniency? Well, I have students from around the world, and for many of them, English is not their native language. I know what it’s like to have to write assignments in a second or foreign language, and not allowing students to avail themselves of a readily available tool always seemed a little harsh, not to mention quixotic.
After reading the two essays mentioned above, though, I am beginning to have second thoughts. My initial, knee-jerk reaction was to enact a complete AI ban for classroom assignments. Then I stopped to think about it for a moment. Would such a ban be practical? In other words, is it enforceable? And, perhaps more importantly, would such a ban be fair? Generative AI—specifically here we are talking about LLMs, or large-language models like ChatGPT—is a part of our lives now, for better or for worse. Out in the real world, students will be asking ChatGPT or Gemini or whatever to do all sort of things for them; is it fair to ask them not to do this in class?
If you are a reasonable individual who did not grow up with genAI, you’re probably already a few steps ahead of me here, and the conclusion I came to after giving the matter a little thought will likely not surprise you. To cut to the chase, the answer I came up with was: Yes... duh. When I learned basic arithmetic in primary school, I was not allowed to use a calculator. In the real world, I could use a calculator whenever I wanted or needed to. In class, though, I had to memorize multiplication tables and work out long division by hand. Why, though, when a computer could just do it for me? After all, how many of us, once we finish school, don’t just reach for a calculator when we need to calculate something beyond the most basic of problems? Nonetheless, we teach children these basic skills because we believe there is value in doing so. There are situations when you do need to do mental arithmetic, and it might not be practical or possible to whip out a calculator. More importantly, though, practicing these skills lays the groundwork for learning other, more important skills.
The art of writing is probably even more important than arithmetic. I am a little biased, being a humanities person, but I do think that learning how to write properly is about more than just being able to write an essay. Knowing how to write properly means knowing how to think properly—how to logically organize your thoughts, how to present your ideas in such a way that you can communicate your message to your intended audience, etc.
This is, in fact, what started me down this rabbit hole. It wasn’t the fact that these two students used AI to translate essays into English from their native languages. After all, I have been doing this for a while. No, what gave me pause was something that one of the students wrote when they were describing their process and use of AI. They said that they asked the AI to translate their essay into “academic English,” but when they read the final product they felt that the essay didn’t flow properly, so they asked the AI to organize the text more logically. That is what set alarm bells ringing in my head and started me thinking about all of this. To this student, asking an AI to translate their essay and asking an AI to organize their ideas so they flowed more logically were not fundamentally different things; if you are going to do one, why not do the other as well?
Once I had started thinking about what students gained from the use of AI and what they lost, the issue of using AI for translation started to look a little different, too. I thought back to my own experience as a student here in Korea. Back then, we didn’t have LLMs, and even if we had, I wouldn’t have used them. I wrote all of my Korean assignments myself, even though it required a lot more time and effort of me than it did of my Korean classmates. Of course, I was a graduate student, not an undergrad. And, as HJ pointed out when I mentioned this, not everyone is me. But the time and effort I put into writing all of those assignments paid dividends far beyond whatever grades I might have received—it laid the groundwork for my entire academic career. So, yes, my situation and the situation my undergrad exchange students find themselves in might be different, but I have been coming around to the idea that using AI to translate or even just proofread assignments is robbing students of the opportunity to learn and practice valuable skills.
Despite what my students might sometimes think, my goal is not to make life as hard for them as I can. My goal is, in fact, to help them learn something, and you don’t actually learn anything by having AI do a task for you. As such, I think a ban on the usage of genAI for assignments is both fair and reasonable. It may indeed make life more difficult for students, but the path of true learning is never an easy one. For my part, I would rather read something written in my students’ true voices than some generic slop spit out by AI. I had a Korean student in this same class this semester whose English was not great, but they still managed to get their ideas across and produced what I thought was a very interesting essay. The struggles with the language were there, but I can guarantee that this student benefited more than the students who simply had AI translate their essays.
So, a ban on genAI usage might be fair and reasonable, but an important question remains: Is it practical? Could I feasibly enforce such a ban, or would it turn into an arms race and end up being more trouble than it’s worth? This is where we need to get into the practicalities of my assignments. Right now, students write brief “response essays” throughout the semester, responding to our readings and what we discuss in class. Then, at the end of the semester, they write a longer essay on a topic of their choosing. All of these assignments are submitted electronically, via our school system. This, of course, opens the door for the use of genAI in writing assignments.
I mentioned this problem to my friend Kevin over dinner last week (I’m not dropping a link here because he is on the eve of making his blog private), and he suggested bringing back the bluebook exam; he also later sent me a link to an interesting (student-written) article arguing for exactly that. When I first started teaching this class over a decade ago (yikes!), I actually did have students write in-person essays for their exams. But then we were encouraged to transition to an online system, and I got with the times. This was all well before genAI became what it is today, of course. Even after I moved to online submissions for the final essay, though, I still had quizzes sprinkled throughout the semester. These consisted of five short-answer questions; for each answer I would award full credit (two points), half credit (one point), or no credit (zero points), for a total of ten possible points.
Some years ago, though, I decided to switch to the response essays (again, well before the advent of LLMs). The reasons for the change were multiple. For one, in-class quizzes require time that might be spent on other forms of instruction. Even though the questions and expected answers were short, as I mentioned above many of my students are not native speakers of English, so I always made sure to allow a full fifteen minutes at the end of class for the students to complete the quiz; this comes out to one-fifth of the total class time.
Then there is the problem of what to do if a student misses class on a quiz day. Giving them a zero for the quiz would be rather harsh; students can be sick or have unavoidable absences, after all. It would also be difficult to organize make-up quizzes, not to mention possibly not fair once the quiz questions are out in the wild. What I ended up doing was allowing students to make up for one missed quiz per semester by writing a short essay on the module covered by the quiz. Since all of the quiz days were known from the beginning of the semester, I figured that was sufficient grace to cover unavoidable absences; if a student missed a second quiz, they were on their own.
Finally, grading the quizzes was a surprisingly time-consuming task, especially when you have around fifty students in a class. Since I had to determine whether an answer deserved full or only partial credit, it required more time and thought than that required for, say, a simple multiple-choice quiz (which are mostly useless, as far as I’m concerned). Each specific instance might not take an incredibly long time, but it all adds up. Add to that the fact that I recorded statistics on how each individual question performed to determine whether they were too easy, too hard, or just right. I would then use that information to either modify the questions or drop them from my question pool entirely. Yes, this is me being a little too thorough, perhaps, but I am who I am, and I’m not going to change at this point.
I don’t remember exactly when I got the idea to just make the short make-up essays the actual assignment, but it did indeed reduce my workload. It also allowed for more time in class. But once genAI came along, I realized that students would be tempted to use LLMs to improve their writing. As I mentioned above, initially I was open to the idea of AI-assisted writing. Now, as I rethink that, I’m starting to wonder if I should go back to the old-fashioned in-class quizzes as well. I would be slightly increasing my workload and taking time out of class, so it would be a reversion in that sense, but perhaps it is worth it for what it will achieve in terms of encouraging students to write for themselves.
Another option, though, has occurred to me in the process of writing this entry—perhaps there is a way of combining these approaches. That is, what if I have students continue to write brief response essays, but they do so in class instead? I would also make it an open-book assignment, since I am less concerned about them memorizing materials and more concerned with what they think about and how they engage with those materials. There are a number of things to consider here, such as how much time this would require in class. I am still mulling it over; I have the summer to think about, so I’m not coming to any decisions yet. Whatever the case, I am leaning toward having some form of in-class evaluation for these assignments.
The final essay, though, is not suited to a bluebook-style exam. This is something that I want them to be able to think about, research, refine, etc. This does mean I will still have to contend with the possibility of genAI usage, but if I do have the in-class evaluations I mentioned above, I will at least have writing samples to compare to the final essays. The most important thing, though, no matter what I decide to do, is that the students receive clear and unambiguous guidance from me on the use (or, more precisely, the non-use) of genAI in assignments. I need to make it clear to them that the process is more important than the product, and that they can still write a good essay even if their English isn’t perfect. I always upload PDF files that contain guidelines for their various assignments; I will have to add guidelines on genAI to that.
Will this approach be foolproof? No, it won’t; I don’t think there is such a thing as a foolproof approach here. My experience, though, is that students generally want to do the right thing. Both of the students I questioned this semester were very upfront about their use of AI, and the student who used AI to organize their essay also said that they took full responsibility for the end result. In fact, I’m not sure if I’ve ever had a student who openly rebelled against the idea of doing the right thing. I once had a student who plagiarized their final essay and failed the class as a result, but in discussions with them later I realized that they had not been properly educated in their home country on the evils of plagiarism. This student vowed to me that they would improve, and a year later they enrolled in the class again and passed with a good grade.
Are there students out there who are hellbent on cheating, who will do whatever they can to get around the rules and make sure they have to do as little real work as possible? I’m sure there are. I don’t think I get too many of these students, though, if I’ve ever gotten any at all. Perhaps my faith in humanity is misplaced, but I believe that the vast majority of students will do their best to do the right thing as long as you let them know what that is. They want to do well, and they want to learn.
And this leads us to something I remember reading a lot about when LLMs first became an issue in academia a few years back: a reevaluation of the transactional nature of modern education. Many scholars were writing about how education had become a mere transactional experience in which students exchanged a certain amount of work for an agreed-upon set of credentials. If you did the work required of you, you got your degree; if you didn’t do the work, you didn’t get your degree. That makes sense if you view the university as a mere means to an end—a degree-producing factory, if you will. But if you view the university as an environment that should foster learning and growth, it doesn’t make quite as much sense.
The truth is, though, that university education has indeed largely become transactional. And this is not something that has happened in the past few years—it’s been happening for quite some time now. Part of the problem lies in how we evaluate students. Practicality demands quantitative evaluation so that we can sift the wheat from the chaff. You need to get a certain score on a standardized test to be considered for an institution in the first place, and then once you are in that institution you need to continue to meet quantitative goals to be awarded your certification at the end.
This leads us to Goodhart’s Law (which, ironically, I often hear mentioned these days in the context of genAI itself): “When a measure becomes a target, it ceases to be a good measure.” In the university context, what this means is that we have forgotten that the measure (the quantitative evaluations) were supposed to be measuring something and did not have any practical value in and of themselves. That something was what the student was learning, which was ostensibly the whole point of a university education in the first place. But once those measures became targets, we lost sight of the process and started focusing solely on the results. And if a certain tool can give you satisfactory (if not spectacular) results, why not reduce your workload by using that tool to achieve your end? In other words, we’ve created an educational environment that not only allows for the possibility of students using AI to do their work, it practically encourages them to do so!
Ideally, then, we would change the nature of education so that it focuses on the process of learning as opposed to the results of quantitative evaluation. That would require a fundamental shift in the way we think about education, and that’s obviously not something I’m going to be able to do all by myself. All I can do is work within the existing system to encourage students to value human effort and improvement—to let them know that it’s not just about where you end up, it’s also about how you get there. In a society where the tech companies are trying to shove AI down our throats at every opportunity, that might seem like a hopeless battle. But I believe that most students not only want to do the right thing, they want to believe that what they are doing now—that is, getting a university education—means something. That it’s more than a job where you work X amount of hours and get Y amount of dollars. That it is a time of growing, learning, and experiencing that will shape them into the people they will eventually become.
Idealistic? Probably. But I don’t think you get into education unless you’re an idealist at heart. Maybe, somewhere along the way, the system beats you down enough that you lose that idealism, and then it’s probably time to get out of the game. I think I still have at least a little idealism left in me, though, so I’ll keep pushing forward.