Define policy forbidding use of AI code generators

I've started seeing this at work with coworkers using LLMs to generate code reviews. They submit comments which are way above their skill level which almost trick you in to thinking they are correct since only a very skilled developer would make these suggestions. And then ultimately you end up wasting tons of time proving how these suggestions are wrong. Spending far more time than the person pasting the suggestions spent to generate them.

Yep 100%, it is something I have also observed. Frankly has been frustrating to the point I spun up a quick one off html site to rant/get my thoughts out. https://jaysthoughts.com/aithoughts1

funny enough I had coworkers who similarly had a hold of the jargon but without any substance. They would always turn out to be time sinks for others doing the useful work. AI imitating that type of drag on the workplace is kinda funny ngl.

By far the largest review-effort PRs of my career have been in the past year, due to mid-sized LLM-built features. Multiple rounds of other signoffs saying "lgtm" with only minor style comments only for me to finally read it and see that no, it is not even remotely acceptable and we have several uses built by the same team that would fail immediately if it was merged, to say nothing of the thousands of other users that might also be affected. Stuff the reviewers have experience with and didn't think about because they got stuck in the "looks plausible" rut, rather than "is correct".

So it goes back for changes. It returns the next day with complete rewrites of large chunks. More "lgtm" from others. More incredibly obvious flaws, race conditions, the works.

And then round three repeats mistakes that came up in round one, because LLMs don't learn.

This is not a future style of work that I look forward to participating in.

I'm not really in the field any longer, but one of my favorite things to do with LLMs is ask for code reviews. I usually end up learning something new. And a good 30-50% of the suggestions are useful. Which actually isn't skillful enough to give it a title of "code reviewer", so I certainly wouldn't foist the suggestions on someone else.

People keep telling LLM will improve efficiency, but your comment has proved it's the otherwise.

It look like LLM is not good for cooperation, because the nature of LLM is randomness.

I think a future with LLM coding requires much more tests, both testing happy and bad flows.

Probabilistic patterns stringed together are something different from an end-to-end intention driven solidly linked chain of thought that is with pylons grounded in relevant context at critical points.

Just some feedback: your site is hard to read on mobile devices because of the sidebar.

It also needs proper guideline enforcement. If an engineer produces poorly tested and unreviewed code, then the buck stops with them. This is a human problem more than it is a tool problem.

Thank you, I'll get that fixed.

Edit: Mobile should be fixed now

I think the issue is with people taking mental shortcuts and thus no longer properly thinking about design decisions and the bigger picture in terms of concepts of the software.