A Student's Guide to Writing with ChatGPT

I teach at the university level, and I just expect more from my students. Instead of implementing data structures like we did when I was in school, something ChatGPT is very good at; my students are building systems, something ChatGPT has more trouble with.

Instead of paper exams asking students "find the bug" or "implement a short function", they get a takehome exam where they have to write tests, integrate their project into a CI pipeline, use version control, and implement a dropbox-like system in Rust, which we expect to have a good deal of functionality and accompanying documentation.

I tell them go ahead and use whatever they want. It's easier than policing their tools. If they can put it together, and it works, and they can explain it back to me, then I'm satisfied. Even if they use ChatGPT it'll take a great deal of work and knowledge to get running.

If ChatGPT suddenly is able to put a project like that together, then I'll ask for even more.

Wouldn't it be unfair towards the students who want to learn without LLMs?

You rock. This is such a great perspective.

Why does that matter? LLMs are going to be increasingly important tools, so it's valuable for educators to help students understand how to use them well. If you choose to exclude modern tools in your teaching to avoid disadvantaging those who don't want to use them, you disadvantage all the students who do want to use them.

To put it another way, modern high school level math classes disadvantage students who want to learn without using a calculator, but it would be quite odd to suggest that we should exclude calculators from math curricula as a result.

Many high school classes are taught in such a way that your calculator rarely helps you.

> but it would be quite odd to suggest that we should exclude calculators from math curricula as a result.

That wouldn't be odd at all. Calculators have no place in a math class. You're there to learn how to do math, not how to get a calculator to do math for you.

I disagree. I see an LLM as less calculator and more as cheating. I think there's a lot of value in creating something entirely yourself without having an LLM spit out a mean solution for you to start from.

LLMs have their place and maybe even somewhere in schools but the more you automate the hard parts of tasks, the less people value the struggle of actually learning something.

I also teach in a university. There are two concepts: teaching with the AI, and teaching against it. At first, I want my students to gain a strong grasp of the basics, so I teach “against” it - warnings for cheating, etc. This semester, I’m also teaching “with” it. Write an algorithm that finds the cheapest way to build roads to every one of a set of cities, given costs for each street segment. I tell them to test it. Test it well. Then analyze its running time. What technique did it pick? What are the problems with this technique? Are there any others? What input would cause it to break? If I assumed (some different condition), would this change the answer?

Students today will be practitioners tomorrow, and those that know how to work with AI will be more effective than those who do not.

> If ChatGPT suddenly is able to put a project like that together, then I'll ask for even more.

Is having a paid subscription with a company that potentially tracks and records every keystroke a requirement for future courses?

No they won't. It takes 10 min to be "effective" with an "AI", it takes 10 years to be effective with TAOCP.

GPT-4o-mini costs $0.15 per million input tokens and $0.6 per million output tokens. I'm sure most schools have the budget to allocate many millions of tokens to each student without a sweat.

The people who become truly effective with AI, i.e., the folks who write truly good code with it, make truly beautiful art, spend closer to effectively 10 years of man-hours than 10 mins with it.

Using AI is a skill too. People who use it every day quickly realize how poor they are at using it vs the very skilled when they compare themselves. Ever compared your own quality AI art vs the top rated stuff on Civit.AI? Pretty sure your stuff will be garbage, and the community will agree.

Math class has no place without calculators. You're there to learn how to do math in the real world, not how to do math in a contrived world where we pretend that the ability to do calculations isn't ubiquitous. There are almost certainly more calculator capable devices on earth than people today. Ludditism is the human death drive expressed in a particularly toxic fashion.

Calculators in early math classes, such as algebra, would be 100% detrimental to learning. Getting an intuitive understanding of addition and multiplication is invaluable and can only be obtained through repetition. Once you reach higher levels of math, the actual numbers become irrelevant so a calculator is fine. But for anything below that, you need to do it by hand to get any value.

When speaking of Math class, are you ignoring everything up to pre-calculus or do you think everything from addition flashcards, times tables, and long division is useless? I'd argue those exercises are invaluable. Seeing two numbers and just knowing the sum is always faster than plugging into a calculator.

FWIW I teach upper level courses.

I see LLMs as almost sufficiently advanced compilers. You could say the same thing about gcc or even standard libraries. "Why back in my day we wrote our own hash maps while walking uphill both ways! Kids these days just import a lib and they don't learn anything!"

They are still learning, just at a higher level of abstraction.

They're allowed to use whatever tools they want. But they have to meet higher standards in my classroom because more is going to be expected of them when they graduate. What would be unfair is if I don't prepare them for the expectations they're going to have to meet.

Yeah! Computer science students can do more "science" with the LLM. Before they spend all their time just writing and debugging. Instructors are happy if students can just write code that compiles.

When every student can write code that compiles, then you can ask them to write good code. Fast code. Robust code. Measure it, characterize it, compare it.

I don't know how that can be true. People were making very beautiful art with SD less than a year after it hit the scene. Sure, I think you need more than 10 minutes, but the time required is closer to that than it is to 10 years.

This is the same fallacy that people make when they learn a new language, so they pick up anki spend a ton of time on it and most burn out, some don't, but neither see any real benefits greater than if they just spent that time on learning the language. The fallacy comes from the fact the goal of learning isn't to finish problems quickly, but to understand what is trying to be said or taught.

For example you claim that addition flashcards and times tables are invaluable, but you don't specify a base, in base 2 you have 4 addition flashcards, in base 100 you have 10,000, clearly understanding addition isn't related to the base, but flashcards increase as base increases, thus there is a relation, implying of course that understanding addition isn't related to the number of addition flashcards you understand. Oh but of course they aren't invaluable in understanding addition, they are invaluable in understanding concepts that use addition, cause ... why exactly? You saved 1 second finishing the problem that you may have understood before you completed that addition step? You didn't have to "context switch" by using a calculator? Students who don't know the sum often give unused name and go back at the end of the problem and solve it later. This behavior is of course discouraged since students can't understand variables until much later if ever and not knowing something you were taught represents the failure of the student and thus the teacher, school, government and society.

Infinitely better is learning from someone who speaks the language. A 30 minute solo tutoring session once a week for a month, in a no distraction environment (aside from a snack), even just working through homework, is more than enough for most students to go from Fs to As for multiple years.

Personally I have dyscalculia and to this day I need to add on my fingers. Still, I ended up with degrees in physics and computer engineering. I don't think those things you mention are useless, but they never worked for me so I don't view them as invaluable.

Incredible username. And as a current math student, I agree with you completely, for the simple fact that I can do proofs far easier than I can do arithmetic. Students like me who are fine at math generally but who are not great at arithmetic in particular really suffer in our current environment that rejects the use of machine assistance.

My high school math classes were mostly about solving problems. The most important was learning the formulas and the steps of the solution. The calculator was mostly a time saver for the actual computation. And once I move to university, almost all the numbers were replaced by letters.

In the same sense that there are many ways of thinking left behind by modern CS curricula – as it is now, the way we teach CS is unfair towards students who want to learn flowcharting, hand-assemble and hand-optimize software, etc. They're very worthy things to master and very zen to do, but sadly not a crucial skill anymore.

University is supposed to be about dedicating one's life to learning and ultimately gaining brand new insights into the world. It's not supposed to be about training people to produce stuff in the exact same way everyone already produces stuff. Do you think this approach will help them come up with new stuff?

Well I don't agree with your premise on what University is supposed to be. There's a lot one has to learn about how things have been done before one can even conceive of whether or not an idea is new.

Today we stand on the shoulders of giants to create things previous generations could not, but we still have to climb up to their shoulders in order to see where to go. Without that perspective, people spend a lot of cycles doing things that have already been done, making mistakes that have already been made. There's value in gaining that knowledge yourself through trial and error but it takes much longer than a 4 year program if that's the way you want to learn.

My role is that of a ladder. People are free to do whatever they want, create whatever they want once they get to the top.

And anyway, we graduate students who go on to create new things every year. So proof is in the puddin.