Even with boring tech that's been in the training set for ages (rails), you can get some pretty funny hallucinations: https://bengarcia.dev/making-o1-o3-and-sonnet-3-7-hallucinat... (fortunately this one was the very non-dangerous kind, making it very obvious; though I wonder how many non-obvious hallucinations entered the training set by the same process)