←back to thread

Guide to Fine-Tuning LLMs

(arxiv.org)
145 points ignoramous | 1 comments | 22 Oct 24 04:41 UTC | HN request time: 0s | source
Show context
kcorbitt ◴[22 Oct 24 05:21 UTC] No.41911422[source]▶
I saw this when it was making the rounds on X a few days ago. Fair warning: it seems like at least some sections are AI-generated, and there isn't much insight to be gained from reading the actual sections compared to eg. reading the relevant category pages on Huggingface.
replies(4): >>41911596 #>>41911605 #>>41912253 #>>41912270 #
worstspotgain ◴[22 Oct 24 08:12 UTC] No.41912253[source]▶
Glancing at the authors' names, it's possible that none of them are native English speakers. Any chance that the sections you're referring to were just AI-polished rather than AI-generated?
replies(1): >>41912649 #
qeternity ◴[22 Oct 24 09:33 UTC] No.41912649[source]▶
No, this paper was edited yesterday. The original (you can verify on arxiv) contained this incredible section: "6.10 Optimised Routing and Pruning Operations (ORPO)"

The actual ORPO paper is "Odds Ratio Preference Optimisation" and it has nothing to do with pruning. This goes way beyond native language preference.

replies(2): >>41913061 #>>41913443 #
1. cubefox ◴[22 Oct 24 10:56 UTC] No.41913061[source]▶
Wow, so significant parts of the paper could still be LLM confabulation.