I think the iPos shuffle (usb stick form) is still the best mp3 player I ever had – it was small, pluggable without extra cords, and battery lasted a really long time. It didn't have a screen to browse music but that was part of the idea – just let the shuffle do its thing. Even this relatively simple concept has not been replicated in the hardware market.
People will say it's not a hardware problem but a software/drm issue. I think that's a real shame. I wish there were a good, inexpensive, portable device that would just play my music.