ML needs a new programming language – Interview with Chris Lattner

Julia could be a great language for ML. It needs more mindshare and developer attention though

What's the current state of time to first plot and executable size? Last time it was several seconds to get a 200 MB hello world. I'm sure they are moving in the right direction the only questions is are they there yet?

What makes Julia "great" for ML?

I would use the term "potentially great" rather than plain "great"

but all the normal marketing words: in my opinion it is fast, expressive, and has particularly good APIs for array manipulation

with juliac.jl and --trim, hello world is now 1MB and compiles in a second.

more realistic examples of compiling a Julia package into .so: https://indico.cern.ch/event/1515852/contributions/6599313/a...

improving, slowly. 5 steps forward 3 steps back.

1.9 and 1.10 made huge gains in package precompilation and native code caching. then attentions shifted and there were some regressions in compile times due to unrelated things in 1.11 and the upcoming 1.12. but at the same time, 1.12 will contain an experimental new feature `--trim` as well as some further standardization around entry points to run packages as programs, which is a big step towards generating self-contained small binaries. also nearly all efforts in improving tooling are focused on providing static analysis and helping developers make their program more easily compilable.

it's also important a bit to distinguish between a few similar but related needs. most of what I just described applies to generating binaries for arbitrary programs. but for the example you stated "time to first plot" of existing packages, this is already much improved in 1.10 and users (aka non-package-developers) should see sub-second TTFP, and TTFX for most packages they use that have been updated to use the precompilation goodies in recent versions

julia> @time begin

       using Plots

       display(plot(rand(8)))

       end

  1.074321 seconds

On Julia 1.12 (currently at release candidate stage), <1mb hello world is possible with juliac (although juliac in 1.12 is still marked experimental)

Interesting. I am experimenting with different ML ecosystems and wasn't really considering Julia at all but I put it on the list now.

Glad to hear. I've found it's a very welcoming community.

I'll warn you that Julia's ML ecosystem has the most competitive advantage on "weird" types of ML, involving lots of custom gradients and kernels, integration with other pieces of a simulation or diffeq, etc.

if you just want to throw some tensors around and train a MLP, you'll certainly end up finding more rough edges than you might in PyTorch

Built-in autodifferentiation and amazing libraries built around it, plus tons of cutting edge applied math libraries that interoperate automatically, thanks to Julia's well conceived approach to the expression problem (multiple dispatch). Aside from that, the language itself is like a refined python so it should be pretty friendly off the bat to ML devs.

What Julia needs though: wayyyy more thorough tooling to support auto generated docs, well integrated with package management tooling and into the web package management ecosystem. Julia attracts really cutting edge research and researchers writing code. They often don't have time to write docs and that shouldn't really matter.

Julia could definitely use some work in the areas discussed in this podcast, not so much the high level interfaces but the low level ones. That's really hard though!

I don't understand why in the world someone would go from one dynamically typed language to another. Even the kernels example cited below is "eh, the types are whatever you want them to be" https://cuda.juliagpu.org/stable/tutorials/introduction/#Wri...

Then again, I am also open to the fact that I'm jammed up by the production use of dynamically typed languages, and maybe the "for ML" part means "I code in Jupyter notebooks" and thus give no shits about whether person #2 can understand what's happening

If I wanted to get into research ML, I'd pick Julia no doubt. It allows both conventional ML techniques where we throw tons of parameters at the problem, but additionally a more nimble style where we can train over ordinary functions.

Combine that with all the cutting edge applied math packages often being automatically compatible with the autodiff and GPU array backends, even if the library authors didn't think about that... it's a recipe for a lot of interesting possibilities.

It's very important that readers, writers, maintainers, etc. of code are able to easily understand what that code is doing.

explicit and strict types on arguments to functions is one way, but certainly not the only way, nor probably the best way to effect that

I would actually be curious to hear your perspective on the "best way" that isn't typechecking. I literally cannot comprehend why someone would write such a thing

I readily admit that I am biased in that I believe that having a computer check that every reference to every relationship does what it promises, all the time

first and foremost great documentation & design docs cannot be surpassed as a tool to explain and understand code. and that is entirely language agnostic.

more generally, the most important bits of a particular function to understand is

* what should it be called with

* what should it return

* what side effects might it have

and the "what" here refers to properties in a general sense. types are a good shortcut to signify certain named collections of properties (e.g., the `Int` type has arithmetic properties). but there are other ways to express traits, preconditions, postconditions, etc. besides types

> It needs more mindshare and developer attention though

That is the problem. Julia could not compete against Python's mindshare.

A competitor to Python needs to be 100% compatible with its ecosystem.

I mean documentation can be wrong whereas types can't, so it seems like it's strictly a worse tool if your goal is to understand what's actually going on and not what someone said was going on at some point in the past.

> whereas types can't

they sure can...

I recently looked into making Julia binaries, and it's not at all a good process. They say it's supported, but it's not exactly as easy as "cargo build" to get a Julia binary out. And the build process involves creating this minimal version of Julia you're expected to ship with your binary, so build times were terrible. I don't know if that gets amortized though.

As far as the executable size, it was only 85kb in my test, a bouncing balls simulation. However, it required 300MB of Julia libraries to be shipped with it. About 2/3 of that is in libjulia-codegen.dll, libLLVM-16jl.dll. So you're shipping this chunky runtime and their LLVM backend. If you're willing to pay for that, you can ship a Julia executable. It's a better story than what Python offers, but it's not great if you want small, self-contained executables.

note that as a few other commenters have pointed out, this situation will improve greatly in 1.12 (although still many rough edges)

Can they? How does that work?

It's a low level language with a high level interface. In theory, GC aside, you should be able to write code as performant as C++ without having to actually write C++. It's also homoiconic and the compiler is part of the language’s API, so you can do neat things with macros that let more or less you temporarily turn it into a different language.

In practice, the Julia package ecosystem is weak and generally correctness is not a high priority. But the language is great, if you're willing to do a lot of the work yourself.

types can be (and almost always are) overly restrictive, preventing otherwise valid code from running. they can also be under powered, and not expressing necessary invariants for the algorithm.

Yeah, that's what I've been hearing about Julia for about 10 years now: "situation will improve greatly in the next version, but still many rough edges remain".

I've been involved in a few programming language projects, so I'm sympathetic as to how much work goes into one and how long they can take.

At the same time, it makes me wary of Julia, because it highlights that progress is very slow. I think Julia is trying to be too much at once. It's hard enough to be a dynamic, interactive language, but they also want to claim to be performant and compiled. That's a lot of complexity for a small team to handle and deliver on.

  but there are other ways to express traits, preconditions, postconditions, etc. besides types

You can also put that in the type system, and expressive languages do. Its just a compiler limitations when we can't.

I mean, even in C++ with concepts we can do most of that. And C++ doesn't have the most expressive type system.

I didn't make any claims about that, just that they can't be wrong. And by that, I didn't mean you can't choose the wrong types, just that once you've chosen types that compile they can't be incompatible or other than what they are.

That being said, I've always found the argument that types can be overly restrictive and prevent otherwise valid code from running unconvincing. I've yet to see dynamic code that benefits from this alleged advantage.

Nearly universally the properly typed code for the same thing is better, more reliable and easier for new people to understand and modify. So sure, you can avoid all of this if the types are really what bother you, but it feels a bit like saying "there are stunts I can pull off if I'm not wearing a seatbelt that I just can't physically manage if I am."

If doing stunts is your thing, knock yourself out, but I'd rather wear a seatbelt and be more confident I'm going to get to my destination in one piece.

Different things have been improving for the past 10 years. TTFX is in a good spot now, multithreading is in a pretty good spot now, GC issues for most people are basically solved. AOT compilation was pretty much the last big item that a lot of people wanted, and that has recently (within the last year) gotten to the point where the code exists, it's merged, and it will be releasing soon (and then spending the next year or two getting improved)

That has been the situation with many programming languages until they finally explode, including Python, which is 34 years old, and also was largely ignored for a decade until Zope in 2000's.

Yes, my experience has been that it is great if you need to do something particularly weird, but less smooth to do something ordinary.