A Student's Guide to Writing with ChatGPT

Lots of interesting debates in this thread. I think it is worth placing writing/coding tasks into two buckets. Are you producing? Or are you learning?

For example, I have zero qualms about relying on AI at work to write progress reports and code up some scripts. I know I can do it myself but why would I? I spent many years in college learning to read and write and code. AI makes me at least 2x more efficient at my job. It seems irrational not to use it. Like a farmer who tills his land by hand rather than relying on a tractor because it builds character or something. But there is something to be said about atrophy. If you don't use it, you lose it. I wonder if my coding skill will deteriorate in the years to come...

On the other hand, if you are a student trying to learn something new, relying on AI requires walking a fine line. You don't want to over-rely on AI because a certain degree of "productive struggle" is essential for learning something deeply. At the same time, if you under-rely on AI, you drastically decrease the rate at which you can learn new things.

In the old days, people were fit because of physical labor. Now people are fit because they go to the gym. I wonder if there will be an analog for intellectual work. Will people be going to "mental" gyms in the future?

Our internal metrics show a decrease in productivity when more inexperienced developers use AI, and an increase when experienced developers with 10+ years use it. We see a decrease in code quality across experience levels which needs to be rectified across all experience levels but even with the time spent refactoring it’s still an increase in productivity. I think I should note that we don’t use these metrics for employee review in any way. The reason we have them is because they come with the DORA (eu regulation) compliance tool we use to monitor code-quality. They won’t be used for employee measurement while I work here. I don’t manage people, but I was brought in to help IT transition from startup to enterprise so set the direction with management confidence.

I’m a little worried about developers turning to LLMs instead of official documentation as the first thing they do. I still view LLMs as mostly being fancy auto-complete with some automation capabilities. I don’t think they are very good at teaching you things. Maybe they are better than Google programming, but the disadvantage LLMs have seem to be that our employees tend to trust the LLMs more than they would trust what they found on Google. I don’t see an issue with people using LLMs on fields they aren’t too experienced with yet however. We’ve already seen people start using different models to refine their answers, we’ve also seen an increase in internal libraries and automation in place of external tools. Which is what we want, again because we’re under some heavy EU regulations where even “safe” external dependencies are a bureaucratic nightmare.

I really do wonder what it’ll do to general education though. Seeing how terrible and great these tools can be from a field I’m an expert in.