Every time I want to rewrite a shell function in python, I always hesitate due to the slow startup.
Every time I want to rewrite a shell function in python, I always hesitate due to the slow startup.
$ time A=1 B=1 python -c "import os; print(int(os.getenv('A'))+int(os.getenv('B')))"
2
real 0m0.068s
user 0m0.029s
sys 0m0.026s
So a fast SSD will help, and somewhat surprisingly putting it inside docker helps (in an HPC context, not so sure it’s implications here as we’re talking about a short scripts.)
But the context here is to port shell scripts to Python, I’m not sure how huge amounts of imports matters.
And it is probably an intrinsic problem of the language, unless we start talking about compiling (and somehow statically) not interpreting the Python program, whichever implementation of the language probably won’t help the situation.
Lastly, if high startup costs of the script becomes relevant, perhaps it is orchestrating wrong. This is an infamous problem of Julia, and their practice is then just keep the Julia instance alive and use it as “the shell”. Similarly, you can do so in Python. Ie rather than calling a script from the shell acting on millions of things, write a wrapper script that start the Python instance once. Memory leak could be a problem if it or its dependencies are not well written but even in that case you have ways to deal with that.
Never question the modern developers ability to import 1500 heavy libraries to accomplish something that only takes 10 lines of code.