Uncensor any LLM with abliteration

Uncensoring Llama 3 is a violation of the Llama 3 acceptable use policy.

https://llama.meta.com/llama3/use-policy/

> You agree you will not use, or allow others to use, Meta Llama 3 to: <list of bad things>...

That terminates your Llama 3 license forcing you to delete all the "materials" from your system.

Do you mean to say that teaching people how to do things should be regarded, for this purpose, as a form of allowing them to do those things?

The article clearly demonstrates how to circumvent the built-in protections in the model that prevent it from doing the stuff that violates the acceptable use policy. Which are clearly the things that are against the public good.

There should be CVEs for AI.

> That terminates your Llama 3 license forcing you to delete all the "materials" from your system.

Or, it means you have broken a contract (of adhesion) formed by acquiring the weights from Meta. You can break contracts! Meta could take a civil case against you, but that's it. The AUP is a document, it's not going to force you to do anything. The court could potentially force you, but that's unlikely, even in the more unlikely event that anyone cares enough to find out what's happening and bring a case against you.

You’re on Hacker News. It’s a shadow of its former self, but still.

Giving large, politicised software companies the sole power to determine what LLMs can and cannot say is against the public good.

Agreed. But uncensoring Llama 3 can do harm in the immediate term.

As much as I am not a fan of Meta, an uncensored Llama 3 in the wrong hands is a universally bad idea.

Universally eh? Who decides what should be censored and what not? You?

LLMs, in this context, are nothing more than search indexes. The exact same information is a google query away. Publicly crawlable information was the training material for them, after all.

> But uncensoring Llama 3 can do harm in the immediate term

How so?

I am under the opinion that terms of use from models trained out of public (often stolen) content should be disregarded by the general public.

Almost everything in wrong hands is universally a bad idea. This phrase is just FUD and makes little sense.

LLMs aren't indexes. You can't query them. There's no way to know if a piece of information exists within it, or how to access the information.

I'm quite aware, hence in this context, meaning the ability for users to query potentially questionable content, not the inner workings. Probably should have phrased it differently.

i find the concept of "stolen" text, which was originally crowd-sourced for free, to be a really tiresome argument. i don't exactly understand why anyone would defend, for example, reddit's ownership over the content generated by its millions of users. i am glad my decade of shit posting on reddit contributed to something other than reddit's profits

The danger of LLMs isn't really in their ability to parrot existing questionable content, but in their ability to generate novel questionable content. That's what's got everyone obsessed about safety.

- Generating new malware.

- Generating new propaganda or hate speech.

- Generating directions for something risky (that turn out to be wrong enough to get someone injured or killed).

But LLMs generate nearly everything they output. Even with greedy sampling, they do not always repeat the dataset verbatim, especially if they haven't seen the prompt verbatim. So you need to prevent them from engaging in entire classes of questionable topics if you want any hope of restricting those types of questionable content.

It's not "we can't let this model get into the hands of adversaries, it's too powerful" like every LLM creator claims. It's "we can't let our model be the one adversaries are using", or in other words, "we can't let our reputation be ruined from our model powering something bad".

So, then, it's not "we can't let people get dangerous info from our model". It's "we can't let new dangerous info have come from our model". As an example, Google got so much shit for their LLM-powered dumpster fire telling people to put glue on pizza.