Law doesn't seek to punish but to rehabilitate. Act of taking freedom away from the criminal is violent enough. Treating them badly is just a sign of unfair/poor society that cannot maintain (afford to keep) it's promise to be civil to all citizens.
Must justice include punishment? If someone hits me, I'd much rather they take responsibility, apologise, and work on themselves to become a better person, than simply get locked up for a while.
Those are not the only two options that they can choose from. And being hit is a very minimal example. Would you say the same for rape? No punishment required; just apologise and work on yourself?
You skipped the “take responsibility”. In case of a more serious crime a simple apology is indeed not enough. But it should still be possible to proceed with something more productive for everyone than putting the perpetrator behind bars. As you say, there are many options.