Should programming languages use cryptographic rngs like a ChaCha20 based one in their standard libraries to stop accidental use of non cryptographic rngs for cryptographic purposes? But that comes at the cost of speed
The challenge is things that don't _obviously_ need cryptographically secure generators. For example, do you need a secure generator for the seed of a hash table, or a sorting algorithm? (For those that do use a seed). Some will argue that yes, this is important. Until a few years ago, the hash tables used static hash algorithms without any randomization, but "hash flooding" changed that. I think that nowadays, still many hash table implementations don't use secure generators.
Then, there's secure and insecure hash functions. Secure hash functions like SHA-256 are (compared to non-secure functions) specially slow for short keys. There are "somewhat" secure hash function algorithms like SipHash that can be used for this purpose.
What methods, what CPU? Is that using chacha20 a couple bytes at a time? If you generate your random bytes in medium size blocks you'll probably see a much smaller difference.