I use an extension called "Bar Breaker" that hides these when you scroll away from the top/bottom of the page.[0] More people should know about it.
[0] https://addons.mozilla.org/en-US/firefox/addon/bar-breaker/
I'm happy to learn something new about other people's preferences, though. If people prefer scrolling to the top, so be it!
EDIT: It occurs to me that this could be a preference setting. A few of the websites that have let me have my way, I've started generating CSS from a Django template and adding configuration options to let users set variables like colors--with really positive feedback from disabled users. At a fundamental level, I think the solution to accessibility is often configurability, because people with different disabilities often need different, mutually incompatible accommodations.
There can be a logic to keeping the header at the top like a menu bar, and I applaud you if you take an approach that focuses on value to the user. Though I'd still say most sites that use this approach, don't have a strong need for it, nor do they consider smaller viewports except for portrait mobile.
Configuration is great, though it quickly runs into discoverability issues. However it is the only way to solve some things - like you pointed out with colors. I know people who rely on high contrast colors and others that reduce contrast as much as they effectively can.