Teal – A statically-typed dialect of Lua

I've been diving into Lua (a little late to this party, but turns out it's a perfect language to rewrite some commandline scripts I had that were getting unwieldy in Bash, especially with LLM assistance!) and it's really something of an eye-opener.

LuaJITted Lua code runs at 80% (on average, sometimes faster!) of the compiled C version of the same algorithm, typically. Lua is embedded in a surprisingly massive number of products: https://en.wikipedia.org/wiki/List_of_applications_using_Lua The startup time of a script is in nanoseconds. An "echo" written in Lua runs faster than the native echo implementation.

The only warts so far are 1-based indexing (you get used to it), and the fact that LuaJIT is stuck at Lua 5.1 while Lua itself is up to 5.3 or 5.4 and has added some niceties... with Lua proper running slower. And no real standard library to speak of (although some would argue that's a feature; there are a few options and different flavors out there if that's what you need, though- Such as functional-flavored ones...)

Anyway, there's nothing else like it out there. Especially with its relative simplicity.

There are also some neat languages that compile to (transpile to?) Lua, and deserve more attention, such as YueScript https://yuescript.org/, which is a still actively-updated enhanced dialect of MoonScript https://moonscript.org/ (described as "Coffeescript for Lua", although it hasn't been updated in 10 years) although neither of these are typed. HOWEVER... there IS this: TypescriptToLua https://typescripttolua.github.io/, which takes advantage of ALL the existing TypeScript tooling, it just outputs Lua instead of JS!

It is true that LuaJIT is stuck at 5.1, but you could write any performance critical sections in C/C++ and call it from Lua.

Lack of LuaJIT for 5.1+ isn't that big of a deal for desktop apps. The embedded world is still stuck in 5.1, but for them, the benefits of the latest Lua is marginal.

and if you use luajit ffi, those calls actually get called just as fast as from a c program

Any recommendations going from bash to lua to watch out for except indexing?

And despite it being stuck at 5.1, it still implements features from other versions. For example, there is the "LJ_52" macro, so you can compile "table.pack" and "table.unpack" into LuaJIT, which I do, because I use both at times.

As someone else have pointed it out, they are cherry picked: https://luajit.org/extensions.html.

> LuaJITted Lua code runs at 80% (on average, sometimes faster!) of the compiled C version of the same algorithm, typically

Cannot confirm this. It might be true on selected micro benchmarks. Here are the results of the Are-we-fast-yet benchmark suite, which includes a decent set of benchmarks challenging CPU, cache and memory access: https://github.com/rochus-keller/Oberon/blob/master/testcase....

On average, the C and C++ implementations are five times faster than LuaJIT.

> There are also some neat languages that compile to (transpile to?) Lua

Here is a comprehensive list: https://github.com/hengestone/lua-languages. Lanuages like Oberon or Luon directly compile to LuaJIT bytecode (i.e. not to Lua).

The code in Are-we-fast-yet has been heavily optimized. It might still be true that naive LuaJIT can run almost as fast as Naive C.

I haven’t found many downsides yet. I was already starting to rewrite some things in Awk (which I was drawn to for similar reasons- fast script startup time, simple easy language with good defaults), but Awk (while still also awesome) isn’t really designed for stuff beyond a certain size (no signal handling unless you use a fork, for example)

LuaJIT is missing bignums, ints that aren’t floats, bit operations, and native utf8 handling, but it can pretty easily be extended with libraries, ffi and metatabling. (I actually made a working bignum library that integrates with gmp, but it has a memory leak somewhere and it’s a rabbit hole/bikeshedding project at this point…)

LLM assistance helps hugely and I really like YueScript’s syntax additions. You can point any LLM at a syntax describing webpage and it will pretty much write that language for you…

There is also the compat53 library which polyfills most of the missing parts. The Teal compiler has --gen-target and --gen-compat flags which adapts the generated Lua code for different Lua versions, and allows using the compat53 library behind the scenes if desired, so you can get a mostly Lua-5.3+ experience over LuaJIT using Teal.

Have a look at the results and the code; there are benchmarks in the suite where the (ideomatic) C/C++ implementation is "only" twice as fast as the corresponding (idiomatic) Lua implementation, but on average (geomean of all factors) it's about five times as fast. The guidelines of the benchmark are pretty strict to enable fair comparisons (see https://github.com/smarr/are-we-fast-yet/blob/master/docs/gu...).

Another one of the biggest uses of Lua outside the hyperscaler-type software like nginx or redis: Roblox. Soooo many kids run games on Roblox every day!

Roblox not only runs entirely on Lua, but they've been working on their own type inference version of Lua named Luau, and open sourced it, and it's still in very active development.

https://github.com/luau-lang/luau

I do think you're better off using ruby, but if you insist.

Lots of default functionality missing so you MUST have these packages: inspect, luaposix, lrexlib-pcre, lrexlib-posix, lpeg, luastd/stdlib, luasocket, luahttp, luasec, luacheck, penlight

* luajit is unnecessary in almost all cases, you don't need the speed.

* use lsp or luacheck whenever you write something, entr -c luacheck file on everything.

* patterns are not regex which means they do not support lookups, backtracking or |, so you must install lrexlib-pcre or lrexlib-posix (frankly I never need pcre so I stick to lrexlib-gnu or lrexlib-posix).

* overload _ENV so it auto requires unknown things, I have a lua wrapper that does this and it makes it a joy not having all of my scripts with a bunch of require"posix" on all of them

* install inspect to inspect tables

* os.execute and io.popen only accepts strings as parameters which means you should overload it and make a function that accepts tables as well.

* 5.4 is still lacking support for many libraries, 5.3 has most of the libraries.

* assignments default to the global environment so you have to use local keyword or set _ENV to error on assignment (or better yet, don't care, just local _ENV = mymodule)

Overall, Lua is just a mixture of C with a pascal syntax and garbage collection (and also tables which is a weird data structure)

I looked at the docs. This seems to be comparing against Lua, which is why I specifically said LuaJIT

This is quite a distinction to be made. Can you clarify?

Directly from their guidelines page:

    Lua
    
    We write code compatible with Lua 5.1, 5.2 and 5.3.
    Smalltalk/Ruby symbols are represented as normal strings.
    We use Lua 1-based array and the length operator #.
    We use single object when a class is not required.
    Bitwise operators with various Lua versions is a nightmare.
    We use luacheck as a linter.

If they are writing code compatible with Lua 5.2 or 5.3, then that cannot be LuaJIT, which is ONLY compatible with Lua 5.1. (Unless they mean that they JUST write 5.1 code, which due to backwards compatibility is runnable on 5.2 and 5.3? It's unclear from here.)

It's essentially written in Lua 5.1 with specific alternative implementations of mandelbrot and hashindextable for Lua 5.3 selectable by the test runner. But this doesn't matter much because my reference is LuaJIT. I have also compared different LuaJIT and also PUC Lua implementations, see http://software.rochus-keller.ch/are-we-fast-yet_LuaJIT_2017... and http://software.rochus-keller.ch/are-we-fast-yet_Lua_results....

This actually gave me an idea that I'm going to build out

For bignums, have you tried this?

https://web.tecgraf.puc-rio.br/~lhf/ftp/lua/#limath

luajit does have integers (try 5ull+6), bit operations via the bit library (absorbed from lua5.2)

How does imath compare to gmp?

Ruby has a startup time and is slow. (Although compiled Crystal might be an idea...)

I am specifically looking for a language that:

1) is a scripting language (don't have to worry about compiling it)

2) starts up fast (so if I use it in a loop or a sequence of pipes, it won't necessarily be the bottleneck)

3) runs fast

4) is easy to work with and is either ridiculously simple or well-designed (Bash has so many footguns...), and scales up to medium-size code (anything that would take more than 1 file to implement should be promoted to another language)

Awk fit this. So did D, actually. Lua certainly does.

> luajit is unnecessary

Why not use it if it's there, it's faster, and you don't need features beyond 5.1?

> patterns are not regex

They're almost regex, they're faster than regex, and for 90% of use cases, you don't need full regex

> tables which is a weird data structure

Isn't a table basically just a JavaScript object, where metatables are the prototype object? Sort of, at least...

In lua? It works for one.

> Awk fit this. So did D, actually. Lua certainly does.

I do like awk and it definitely should be used more, but I stopped using it because of modules. gawk added it, but its strange having a script end in .awk and then its actually .gawk, and not working with mawk and so on.

So to me, modules are paramount, Lua focuses on embeddability and being an extension language means that writing to _G is fine. But as a system language you should never write to _G. this means that you may find a module for lua, but in reality its not lua, its a module for a program, usually a game.

Frankly, I think lua should not even be offered in your package manager and instead some sort of wrapper that works on POSIX systems should be used, batteries included and sane defaults as well. No more installing luaposix for basic functionality (if bash has it, this should as well). I and you are disparaging bash and claim lua is fast, but at the end of the day, bash has many builtins like test,cd,setenv while lua does not.

Also can D be used as a scripting language? I'm genuinely amazed and would give it a try.

> Why not use it if it's there, it's faster, and you don't need features beyond 5.1?

Sure, but I doubt anybody will need the performance unless of course you do.

FWIW there is a killer feature that I consider essential that everybody needs and that is <close>. I no longer have to worry about closing files, and I could probably use it for something else.

> They're almost regex, they're faster than regex, and for 90% of use cases, you don't need full regex

You may not need full regex (PCRE) but you usually need EREs specifically {} and | are used a whole lot.

> Isn't a table basically just a JavaScript object, where metatables are the prototype object? Sort of, at least...

I'd say JS is a weird language as well.

Mind you, I'm not disparaging lua, I do use it, but these are the foot guns. (Bundling is also a mess)

> but turns out it's a perfect language to rewrite some commandline scripts I had that were getting unwieldy in Bash

IMO this is the perfect use case for Perl. I mean, it's why Perl was invented, but to this day it remains incredibly good at doing this specific task. It's incredibly easy to write and can be very easy to read, and it's much more robust than bash. But best of all - and the reason I think Perl still has a place in the modern world - it's available on practically every computer on Earth. And it will all work. It's so backwards-compatible that 25 year old scripts will run just fine. It's so portable that nothing even comes close. Also, it's shockingly fast. Faster than you would think. Regex is really powerful and it's been the template for so many regex implementations, but definitely be careful of runaway time complexity.

Anyway, Lua is great too. Really nice for embedding into applications ala VBA. Great for config.

Good point about Perl. Why don't I consider Perl? I don't know.

Don’t worry, nobody considers Perl