I think this general idea is going to be the key to really making LLMs widely useful for solving real problems.
I’ve been playing with using GPT-4 together with the Wolfram Alpha plugin, and the combo of the two can reliably solve difficult quantitative problems that neither can individually by working together, much like a human using a calculator.