Better results if you… tip the AI, offer it physical touch, you need to say the words “go slow and take a deep breath first”…
It’s a subjective system without control testing. Humans are definitely going to apply religion, dogma, and ritual to it.
I'm not saying I've proven it or anything, but it doesn't sound far-fetched that a thing that generates new text based on previous text, would be affected by the previous text, even minor details like using ALL CAPS or just lowercase, since those are different tokens for the LLM.
I've noticed the same thing with what exact words you use. State a problem as a lay/random person, using none of the domain words for things, and you get a worse response compared to if you used industry jargon. It kind of makes sense to me considering how they work internally, but happy to be proven otherwise if you're sitting on evidence either way :)
I’m not saying introducing noise isn’t a valid option, just doing it in ‘X’ or ‘y’ method as dogma is straight bullshit.
- Threatening or tipping a model generally has no significant effect on benchmark performance.
- Prompt variations can significantly affect performance on a per-question level. However, it is hard to know in advance whether a particular prompting approach will help or harm the LLM's ability to answer any particular question.
Now… for fun. Look up “best prompting” or “the perfect prompt” on YouTube. Thousands of videos “tips” and “expect recommendations” that are bordering the arcane.
The issue is that you can’t know if you are positively or negatively effecting because there is no real control.
And the effect could switch between prompts.
"You are a world-class developer in <platform>..." type of crap.
… right. Now you are on the same page. Maybe adding fluff helps, maybe it hurts. You have no idea of knowing before or after the prompt.
Show me research that says over thousands of benchmarks that pole riding you AI before the request gives better responses.
It’s placebo.