Most active commenters
  • jampekka(3)

←back to thread

I don't like NumPy

(dynomight.net)
480 points MinimalAction | 33 comments | | HN request time: 1.238s | source | bottom
1. ChrisRackauckas ◴[] No.43998345[source]
One of the reasons why I started using Julia was because the NumPy syntax was so difficult. Going from MATLAB to NumPy I felt like I suddenly became a mediocre programmer, spending less time on math and more time on "performance engineering" of just trying to figure out how to use NumPy right. Then when I went to Julia it made sense to vectorize when it felt good and write a loop when it felt good. Because both are fast, focus on what makes the code easiest to read an understand. This blog post encapsulates exactly that experience and feeling.

Also treating things like `np.linalg.solve` as a black box that is the fastest thing in the world and you could never any better so please mangle code to call it correctly... that's just wrong. There's many reasons to build problem specific linear algebra kernels, and that's something that is inaccessible without going deeper. But that's a different story.

replies(4): >>43998533 #>>43998779 #>>43999127 #>>43999507 #
2. Bostonian ◴[] No.43998533[source]
"Then when I went to Julia it made sense to vectorize when it felt good and write a loop when it felt good. Because both are fast, focus on what makes the code easiest to read an understand."

This is true of modern Fortran also.

replies(3): >>43998567 #>>43998703 #>>44006520 #
3. sundarurfriend ◴[] No.43998567[source]
The amount of work that's been put into Fortran by the LFortran [1] folks and others in recent years [2] is nothing short of absurdly impressive!

[1] https://lfortran.org/

[2] "in recent years" because that's when I became aware of this stuff, not to say there wasn't effort before then

replies(1): >>43998719 #
4. ChrisRackauckas ◴[] No.43998703[source]
Indeed, I also like modern Fortran.
5. pklausler ◴[] No.43998719{3}[source]
If you’re going to give praise, include the GNU Fortran developers, who have really set the standard for open source production quality Fortran compiler development.
replies(1): >>44000156 #
6. jampekka ◴[] No.43998779[source]
> Going from MATLAB to NumPy I felt like I suddenly became a mediocre programmer, spending less time on math and more time on "performance engineering" of just trying to figure out how to use NumPy right.

Matlab is about as slow without readily vectorized operations as Python.

Slowness of Python is a huge pain point, and Julia has a clear advantage here. Sadly Julia is practically just unusable beyond quite niche purposes.

Python does now have quite serviceable jit hacks that let one escape vectorization tricks, but they are still hacks and performant Python alternative would be very welcome. Sadly there aren't any.

replies(6): >>43999029 #>>43999037 #>>43999826 #>>44001248 #>>44007445 #>>44007523 #
7. AnotherGoodName ◴[] No.43999029[source]
One of the things i suspect will happen very soon is that all languages will become as fast as each other and your reason to use one over the other for performance reasons might not exist. I know this sounds optimistic and wishful thinking but hear me out here;

Modern AIs are literal translation engines. That's their origin. The context that lets them do what they do was originally there to allow good translations. It doesn't matter if the translation is programming language output, or actual language output. I can today ask an AI to rewrite a chunk of Matlab code into Rust and it works! There's even code generators that will utilize the GPU where sensible.

We're really not that far off that we can write code in any language and transparently behind the scenes have it actually running on a more performant backend when needed. Ideally you keep the Matlab/Python frontend and the translation will be on the intermediate layers in a two way fashion so step through/debug works as expected.

Based on playing with the above steps manually with good results we're almost at the stage of just needing some glue to make it all work. Write in Matlab/Python, and run as fast as any LLVM backed language.

replies(2): >>43999282 #>>44013351 #
8. Zambyte ◴[] No.43999037[source]
> Sadly Julia is practically just unusable beyond quite niche purposes.

Why? Just the ecosystem?

replies(2): >>43999389 #>>44013645 #
9. abdullahkhalids ◴[] No.43999127[source]
The reason is quite simple. Julia is a language designed for scientific computation. Numpy is a library frankenstein-grafted onto a language that isn't really designed for scientific computation.

We can only hope that Julia somehow wins and those of forced to work in python because of network effects can be freed.

replies(4): >>44001532 #>>44003075 #>>44003296 #>>44006465 #
10. codethief ◴[] No.43999282{3}[source]
Sounds like transpiling Typescript to JavaScript, except that you translate to a vastly different language, thereby making you, the programmer, largely unable to reason about the performance characteristics of your code ("Can the transpiler optimize this Python loop or nah?"), and you also throw in the indeterminism of LLMs. Oooof, I'm not sure I'd want that.
replies(1): >>43999681 #
11. jampekka ◴[] No.43999389{3}[source]
Mainly the lack of practical support for non-REPL/Notebook usage and interoperability.
replies(1): >>44013377 #
12. amluto ◴[] No.43999507[source]
Is MATLAB materially different? Loops are slow (how slow depends on the version), and the fastest thing in the world is the utter perfection of the black box called '\'.
replies(1): >>44000914 #
13. AnotherGoodName ◴[] No.43999681{4}[source]
When you're using Python the type of performance characteristics you're usually thinking of are "can i call an external library that runs this on the GPU" or "is this log(n) or n^2 complexity?" and such language translations shouldn't change those parts.

As in the language is regularly benchmarked at 10-100x slower than other languages. It's not a languages for caring about subtle performance characteristics outside "am i calling the external library correctly or not?" (as in this article) or "is this algorithm exponential time?".

14. bandrami ◴[] No.43999826[source]
R
replies(1): >>43999979 #
15. jampekka ◴[] No.43999979{3}[source]
R is for sure not a Python replacement. And it's even slower than Python.
replies(2): >>44000870 #>>44006239 #
16. sundarurfriend ◴[] No.44000156{4}[source]
That sounds great, kudos to them too!

The only times I set foot in the Fortran world is when some Julia-related development [1] [2] happens, and then I peruse some surrounding links and related projects - so I have very little familiarity with GNU Fortran myself.

[1] https://fortran-lang.discourse.group/t/update-on-prima-a-jul... [2] https://discourse.julialang.org/t/ann-julia-back-end-for-lfo...

17. moregrist ◴[] No.44000914[source]
The last time I seriously used Matlab, over 10 years ago, they had implemented JIT compilation and often straightforward loops were a lot faster than trying to vectorize. And definitely less error-prone.

Iirc, ‘\’ was just shorthand for solving a system of equations, which you could mentally translate into the appropriate LAPACK function without loss of precision. You did have to be a little more careful about making extra copies than in Fortran or C (or even Python). But nothing was magic.

18. tkuraku ◴[] No.44001248[source]
I've found matlabs's JIT to be about on par with numba for loops.
19. glial ◴[] No.44001532[source]
I've been hoping this for 15 years, but so far I still use Python for everything.
20. davedx ◴[] No.44003075[source]
This has me wondering, if not that, then what is python designed for?
replies(3): >>44010593 #>>44013165 #>>44021839 #
21. lenkite ◴[] No.44003296[source]
Nah, because "Python is second-best at everything" and has more libraries than the Galactic Federation, that will never happen.
22. bandrami ◴[] No.44006239{4}[source]
R has significant performance advantages over Python if you are keeping it in its lane. But it's a DSL and not a general-purpose language.
23. waffletower ◴[] No.44006465[source]
Julia really needs to generalize and clarify its deployment story before it could possibly take off. It was built with a promise of generality but its tethered packaging is an albatross.
24. waffletower ◴[] No.44006520[source]
I like drag racing too, and modern Fortran is really competitive on the track. But I think I need a car that is street legal for my needs.
25. ◴[] No.44007445[source]
26. leephillips ◴[] No.44007523[source]
“quite niche purposes”

I don’t understand this. Julia is a general-purpose language that I increasingly use to replace bash scripts. I’ve also happily used it for web scraping tasks¹, interactive web² projects³, generation of illustrations and animations, and, yes, scientific calculations. Aside from the language itself, which is fun to program in, its brilliant package system massively facilitates composing libraries into a solution.

1 https://lee-phillips.org/amazonJuliaBookRanks

2 https://www.admin-magazine.com/Archive/2025/85

3 https://lee-phillips.org/pluckit

27. xypage ◴[] No.44010593{3}[source]
To be general purpose with a very large package ecosystem so that you can get just about anything started pretty quickly. It is relatively easy to do things that aren't performance critical with python, which is great if you want an MVP to grow off of or if you're messing around with something and want to make a little flask server for it, or maybe run some image recognition, or a little anything. It's really just insanely flexible and puts a lot of the cognitive load in libraries so you can get straight to doing stuff, again, often at the cost of performance.
28. lolc ◴[] No.44013165{3}[source]
The best description I heard of Python is "glue for C libs".
29. lolc ◴[] No.44013351{3}[source]
There are enough ambiguities between language libs, even different versions of the same lib, that transpilation will always depend on verification.

It'll work for toy examples. For nontrivial code the generated transpilation will flame out in a different way on every run.

30. joshjob42 ◴[] No.44013377{4}[source]
Interop with what? It has a very good FFI for C, PyCall, RCall, etc. work quite well.

It is pretty REPL focused, but what is the thing you want that you don't have? Like better debugger support in VSCode or something?

replies(1): >>44020008 #
31. Zambyte ◴[] No.44013645{3}[source]
Just happened to find this article, seems to do a good job of explaining why.

https://news.ycombinator.com/item?id=44013455

32. ◴[] No.44020008{5}[source]
33. derriz ◴[] No.44021839{3}[source]
Originally a scripting language - from what I recall. An alternative to Perl or Tcl.