It was like pulling teeth. Folks in education are highly skeptical of these kinds of things unless you can show hard evidence of their efficacy (which is fair, we shouldn’t be afraid of having our methods evaluated). But as much as this kind of simulation-based learning feels like it should be better, in practice it’s difficult to actually demonstrate that it is. If you’re lucky, you get to train some teachers ahead of time, do an A/B test, get back the results, and it’s a non-statistically-significant mess. In the end, my PhET efforts got crunched in the gears as the curriculum updates I was building them into got cancelled for other budgetary reasons.
I still believe these kinds of tools must be good for something, it feels ridiculous to think they aren’t. But one hazard I have definitely observed: People who already know the concepts being taught tend to love these things for the elegant way they demonstrate the principles, but actual learners who don’t know the concepts yet don’t always feel the same way.
Evidence based really isn’t that good of a methodology when it comes to human behaviour
This is the sentence scientists should be repeating over and over again.
In the years I was an active member of the skeptics organization, the first argument provided by the astrologists, homeopaths, telepaths etc. was "you do not have an open mind and cannot get beyond your science". To what I replied that if someone shows me something that cannot be explained by science, I will immediately switch to that in my PhD because, you know, Nobel prize. 30 years later and without a Nobel prize, here I am still waiting :)
Scientists would go wild if there was something that big nit explained by science (I mean that there are plenty of things we do not know for many reasons, but macroscopic events wild be insane to witness. The closest I can think of was cold fusion.)
I helped those students who were building physical circuits, helping them to remedy their missteps. Then those students became group leaders once all students moved on to building physical circuits. Each group would always have some difficulty, but having "experienced" group leaders meant there were far fewer problems for me to solve.
A key understanding was that there were always discrepancies between the theoretical results of the PhET and the actual results from the physical circuits. The main source of these discrepancies was simply explained as the extra resistance provided by the wires. Evaluation was accomplished by the students building different circuits that I drew on the whiteboard and writing a report that included a photo of their group with their circuit(s).