That's unfortunately not universally true. This is most obvious when considering the death penalty.
Norway exemplifies a rehabilitative justice model and it is effective, evidenced by low recidivism rates.
"First-world" is Cold War terminology meaning Western countries and their allies, as opposed to second-world Warsaw Pact states and their allies, versus third-world non-aligned states. This would include death penalty states like Pakistan and Iran, who at one point were British dominions.
If we instead mean "developed countries" (as defined by the IMF), then 4 out of 60 developed countries have not abolished the death penalty: they are the United States, Japan, Singapore and Taiwan.
The other 49 states continuing to use the death penalty (including China) are not considered "developed countries" by the IMF.