I have this issue with my Echo next to my bed. Even on vol 1 it’s too loud for me.
So, instead of using the Spotify alexa skill, I have to use my phone and connect it via bluetooth, so I can turn down the volume on my phone in addition to using the quietest volume on the Echo.