An old post by Scott Alexander (16+ years, mind blown) discusses this, long before the term "aphantasia" became a thing [1]. There was a debate about what "imagination" actually means already in the late 1800s; some people were absolutely certain that it was just a metaphor and nobody actually "sees" things in their mind; others were vehement that mental images are just as real as those perceived with our eyes. The controversy was resolved by Francis Galton, who did some rigorous interviewing and showed that it really does vary a lot from person to person.
[1] https://www.lesswrong.com/posts/baTWMegR42PAsH9qJ/generalizi...
Some children don't see any differentiation between their imagination and reality, so it's a matter of paying attention to how others' behave to know what to do.
Because you can't trust that the reality that you're in is shared by the people around you.