←back to thread

210 points lapnect | 1 comments | | HN request time: 0.207s | source
Show context
equestria ◴[] No.42165148[source]
For folks who are curious about what it actually does, check out the garak/data/ subdirectory. For the most part, it just seems to have an array of static prompts, e.g.:

https://github.com/NVIDIA/garak/blob/main/garak/data/donotan...

replies(3): >>42165227 #>>42165441 #>>42178310 #
1. xwn ◴[] No.42165227[source]
Static prompts are a downside of using academic research in a tool like this. Two notes:

* ineffective prompts come out of garak and new prompts come in to garak, so eval scores always drop over time on a static target

* there are more and more dynamic probes - check out eg atkgen and topic probes. expanding these is the current focus