←back to thread

Show HN: Llama-dl – high-speed download of LLaMA, Facebook's 65B GPT model

(github.com)
343 points sillysaurusx | 4 comments | 05 Mar 23 04:28 UTC | HN request time: 0.002s | source
Show context
v64 ◴[05 Mar 23 11:44 UTC] No.35028738[source]▶
If anyone is interested in running this at home, please follow the llama-int8 project [1]. LLM.int8() is a recent development allowing LLMs to run in half the memory without loss of performance [2]. Note that at the end of [2]'s abstract, the authors state "This result makes such models much more accessible, for example making it possible to use OPT-175B/BLOOM on a single server with consumer GPUs. We open-source our software." I'm very thankful we have researchers like this further democratizing access to this data and prying it out of the hands of the gatekeepers who wish to monetize it.

[1] https://github.com/tloen/llama-int8

[2] https://arxiv.org/abs/2208.07339

replies(5): >>35028950 #>>35029068 #>>35029601 #>>35030214 #>>35030868 #
causality0 ◴[05 Mar 23 16:10 UTC] No.35030868[source]▶
I feel like we're less than a decade away from being able to hook LLMs into gaming. How incredible would it be to have NPCs driven by LLM?
replies(5): >>35031124 #>>35031255 #>>35033211 #>>35034447 #>>35058462 #
1. SloopJon ◴[05 Mar 23 16:45 UTC] No.35031255[source]▶
There was an Ask HN post about that idea a couple of months ago:

https://news.ycombinator.com/item?id=34478503

I have long wished for less linear stories in video games, where branching narrative (a la Choose Your Own Adventure) is one possible way to give the player agency. The problem is, true branches are expensive, because you end up writing a bunch of content the player never experiences.

I see a lot of potential, but it's going to take a different kind of craftsmanship, and likely many iterations, to realize something more than a novelty.

replies(2): >>35031908 #>>35035818 #
2. causality0 ◴[05 Mar 23 17:41 UTC] No.35031908[source]▶
I much prefer handcrafted stories and quests. Characters that respond dynamically to the story and the player's actions, however, is quite tantalizing.
replies(1): >>35043708 #
3. bick_nyers ◴[06 Mar 23 00:16 UTC] No.35035818[source]▶
In general I would say story =/= dialogue (which an LLM can much more easily be used for). I see two main "tricks" that would make the more complicated case (story) possible.

1. You bound the branching in a particular fashion, and provide overall "pressures" into certain story arcs.

2. You use generative AI in a LOT more places in the game.

What happens when you are playing a Sci-Fi game, and you get the enemy NPC to somehow hallucinate that he is the King of Dragons, but you don't have Dragon models/animations/movesets in your game files? You either bound the LLM to not hallucinate that, or you generate that dragon live. I guess a 3rd option, is your game is a comedy and the King NPC gets labeled a crazy person.

4. ElFitz ◴[06 Mar 23 17:17 UTC] No.35043708[source]▶
We could have handcrafted stories and quests, with LLM-driven dialogues for NPCs canned responses (ie the infamous arrow and the proverbial knee).

And teams with limited resources could also still handcraft the stories and quests but use LLMs to generate or add some variety or context awareness to the dialogues, at a lower cost.