←back to thread

340 points agomez314 | 1 comments | | HN request time: 0.396s | source
Show context
jeswin ◴[] No.35246507[source]
In an example [1] referenced in the article, ChatGPT (maybe v3.5) did a poor job of answering the following question.

    Boll is a new company founded by three recent MBAs. The company plans to buy used automobiles at auctions and then sell them at four times the purchase price direct to consumers (i.e., a car that is purchased for $1,000 would be sold at $4000).  Based on an analysis of similar companies, the owners of the company expect to turn their inventory 1.5 times per year. To achieve an annual revenue of $3,000,000, how much inventory, expressed in dollars, should the company expect its operation to require?

How does GPT4 fare?

------

Let's break down the problem step by step.

The company wants to achieve an annual revenue of $3,000,000. They plan to sell cars at 4 times the purchase price. They expect to turn their inventory 1.5 times per year. First, we need to find out how much revenue they will generate per inventory turnover. To do this, we'll divide the annual revenue by the expected number of inventory turnovers per year:

Annual revenue / Inventory turnover rate = Revenue per inventory turnover $3,000,000 / 1.5 = $2,000,000

Now, we know that they plan to sell the cars at 4 times the purchase price. So, we need to find out the purchase price of the cars that will generate $2,000,000 in revenue per inventory turnover:

Revenue per inventory turnover / Markup factor = Purchase price of cars $2,000,000 / 4 = $500,000

Therefore, Boll should expect its operation to require $500,000 worth of inventory to achieve an annual revenue of $3,000,000.

------

That V4 is able to do this is in itself just as important as everything said in the article. We live in interesting times.

[1]: https://aiguide.substack.com/p/did-chatgpt-really-pass-gradu...

replies(3): >>35247183 #>>35247621 #>>35249854 #
1. the_af ◴[] No.35247621[source]
How do we know Mitchell's counterexample isn't in GPT-4's training set?

To truly test GPT-4 you must be sure to give it a problem which is not in its training set and which differently worded enough from anything it can recall. A variation that a human would understand but GPT-4 wouldn't.

I bet this can still be done, it's just that this particular example is now tainted.