M5stack.com makes a whole bunch of ESP32 modules with display, enclosures, batteries, etc already integrated, and they're cheap and robust. You can get one with a built-in keyboard for $30.
For the more budget concious - you can get ESP32 modules with USB C, Wifi/BT and a few IO pins from AliExpress for $4. These things are amazingly cheap.
Agree, but M5stack devices also start at $5 and come with a case. Add $10 or 15 and you can get a device with a screen, accelerometer, buttons, microphone, and more. No soldering required.