←back to thread

Heretic: Automatic censorship removal for language models

(github.com)
745 points melded | 1 comments | 16 Nov 25 15:00 UTC | HN request time: 0.235s | source
Show context
embedding-shape ◴[16 Nov 25 16:14 UTC] No.45946159[source]▶
Optuna is a generally useful project, that I'm surprised isn't used in more places in the ecosystem. The ability to do what they're doing here, incrementally find the best hyperparameter to use can really make a large difference in how quickly you can move past having to fine-tune those values. Basically any time you aren't sure about the perfect value, throw Optuna on it with a quick script, and make it go for a broad search first, then narrow it down, and you can let the computer figure out the best values.

Nicely done to pair that with something as fun as censorship removal, currently in the process on running it on gpt-oss-120b, eager to see the results :) I'm glad that someone seems to be starting to take the whole "lobotimization" that happens with the other processes seriously.

replies(3): >>45946248 #>>45946393 #>>45946623 #
1. p-e-w ◴[16 Nov 25 17:11 UTC] No.45946623[source]▶
Please let me know if you encounter any problems with the 120b! I'm really interested in how well it will work. When presented with the Pareto front at the end, I recommend choosing a configuration with a KL divergence below 1, even if the refusal rate seems high. The gpt-oss models are trained to do an internal monologue about refusing in the CoT, so the actual refusal rate is often substantially lower because Heretic's refusal classifier gets confused by the trigger words.