I can understand why you would feel this way.
As someone that enjoys the problem solving, creativity, learning, and sense of ownership that comes out of building software, I would feel less fulfillment if I hand over everything (or most tasks) to an LLM. It's like being a painter but not painting.
Hence I choose not to do that.
LLMs can be good if used responsibly and when it makes sense. It should not be at the cost of taking away something that matters to you.
I don't know your current situation. Maybe you are doing this at your workplace. Maybe you are under pressure of using LLMs. I get it. But you have a choice.
Now that you have come to this realization, you'll know what to do. Find the right balance.