(github.com)

186 points darkolorin | 2 comments | 15 Jul 25 11:29 UTC | HN request time: 0.625s | source

We wrote our inference engine on Rust, it is faster than llama cpp in all of the use cases. Your feedback is very welcomed. Written from scratch with idea that you can add support of any kernel and platform.

1. greggh ◴[15 Jul 25 15:20 UTC] No.44572085[source]▶

>>44570048 (OP) #

"trymirai", every time I hear the word Mirai I think of the large IOT DDoS botnet. Maybe it's just me though.

replies(1): >>44574563 #

2. fnord77 ◴[15 Jul 25 18:51 UTC] No.44574563[source]▶

>>44572085 (TP) #

I think of the goofy Toyota fuel cell car. I think a grand total of about 6 have been sold (leased) in california

↑

Show HN: We made our own inference engine for Apple Silicon