Most active commenters
  • pxc(3)

←back to thread

Fun with uv and PEP 723

(www.cottongeeks.com)
618 points deepakjois | 16 comments | | HN request time: 1.589s | source | bottom
Show context
ACAVJW4H ◴[] No.44370555[source]
finally feels like Python scripts can Just Work™ without a virtualenv scavenger hunt.

Now if only someone could do the same for shell scripts. Packaging, dependency management, and reproducibility in shell land are still stuck in the Stone Ages. Right now it’s still curl | bash and hope for the best, or a README with 12 manual steps and three missing dependencies.

Sure, there’s Nix... if you’ve already transcended time, space, and the Nix manual. Docker? Great, if downloading a Linux distro to run sed sounds reasonable.

There’s got to be a middle ground simple, declarative, and built for humans.

replies(16): >>44370574 #>>44370603 #>>44370609 #>>44370667 #>>44370757 #>>44370943 #>>44371081 #>>44371402 #>>44372258 #>>44372278 #>>44372472 #>>44372942 #>>44373775 #>>44374090 #>>44374399 #>>44376192 #
1. bigstrat2003 ◴[] No.44370667[source]
> Packaging, dependency management, and reproducibility in shell land are still stuck in the Stone Ages.

IMO it should stay that way, because any script that needs those things is way past the point where shell is a reasonable choice. Shell scripts should be small, 20 lines or so. The language just plain sucks too much to make it worth using for anything bigger.

replies(4): >>44372677 #>>44374216 #>>44374396 #>>44375053 #
2. pxc ◴[] No.44372677[source]
When you solve the dependency management issue for shell scripts, you can also use newer language features because you can ship a newer interpreter the same way you ship whatever external dependencies you have. You don't have to limit yourself to what is POSIX, etc. Depending on how you solve it, you may even be able to switch to a newer shell with a nicer language. (And doing so may solve it for you; since PowerShell, newer shells often come with a dependency management layer.)

> any script that needs those things

It's not really a matter of needing those things, necessarily. Once you have them, you're welcome to write scripts in a cleaner, more convenient way. For instance, all of my shell scripts used by colleagues at work just use GNU coreutils regardless of what platform they're on. Instead of worrying about differences in how sed behaves with certain flags, on different platforms, I simply write everything for GNU sed and it Just Works™. Do those scripts need such a thing? Not necessarily. Is it nicer to write free of constraints like that? Yes!

Same thing for just choosing commands with nicer interfaces, or more unified syntax... Use p7zip for handling all your archives so there's only one interface to think about. Make heavy use of `jq` (a great language) for dealing with structured data. Don't worry about reading input from a file and then writing back to it in the same pipeline; just throw in `sponge` from moreutils.

> The language just plain sucks too much

There really isn't anything better for invoking external programs. Everything else is way clunkier. Maybe that's okay, but when I've rewritten large-ish shell scripts in other languages, I often found myself annoyed with the new language. What used to be a 20-line shell script can easily end up being 400 lines in a "real" language.

I kind of agree with you, of course. POSIX-ish shells have too much syntax and at the same time not enough power. But what I really want is a better shell language, not to use some interpreted non-shell language in their place.

replies(2): >>44372862 #>>44373420 #
3. MatmaRex ◴[] No.44372862[source]
Broke: Dependency management used for shell scripts

Woke: Dependency management used for installing an interpreter for a better programming language to write your script in it

Bespoke: Dependency management used for installing your script

4. m2f2 ◴[] No.44373420[source]
Nice, if only you could count on having it installed on your fleet, and your fleet is 100pct Linux, no AIX, no HPUX, no SOLARIS, no SUSE on IBM Power....

Been there, tried to, got a huge slap in the face.

replies(1): >>44374117 #
5. kstrauser ◴[] No.44374117{3}[source]
Been there, done that. I am so glad I don’t have to deal with all that insanity anymore. In the build farm I was responsible for, I was always happy to work on the Linux and BSD boxes. AIX and HPUX made me want to throw things. At least the Itanium junk acted like a normal server, just a painfully slow one.

I will never voluntarily run a bunch of non-Linux/BSD servers again.

replies(1): >>44381293 #
6. xavdid ◴[] No.44374216[source]
My rule of thumb is that as soon as I write a conditional, it's time to upgrade bash to Python/Node/etc. I shouldn't have to search for the nuances of `if` statements every time I need to write them.
replies(3): >>44379432 #>>44380173 #>>44384656 #
7. johnisgood ◴[] No.44374396[source]
I have a couple of projects consisting of around >1k lines of Bash. :) Not to bloat, but it is pretty easy to read and maintain. It is complete as well. I tested all of its functionalities and it just works(tm). Were it another language, it may have been more than just around 1k LOC, however, or more difficult to maintain. I call some external programs a lot, so I stick'd to a shell script.
8. maccard ◴[] No.44375053[source]
Unfortunately there’s basically no guarantee that even the simplest scripts work.

    #!/bin/bash
    make $1
Has multiple possible problems with it.
9. w0m ◴[] No.44379432[source]
Historically; my rule of thumb is as soon as I can't see the ~entire script without scrolling - time to rewrite in Python/ansible. I Think about the rewrite, but it usually takes awhile to do it (if ever)
10. pxc ◴[] No.44380173[source]
What nuances are there to if statements, exactly?

An if statement in, for instance bash, just runs any command and then runs one of two blocks of code based on the exit status of that command. If the exit status is truthy, it runs what follows the `then`. If it's falsey, it rhns what follows the `else`. (`elsif` is admittedly gratuitous syntax— it would be better if it were just implemented as an if inside an else statement.) This seems quite similar to other programming languages and like not very much to remember.

I'll admit that one thing I do in my shell scripts is avoid "fake syntax"— I never use `[` or `[[` because these obscure the real structure of the statements for the sake of cuteness. I just write `test`, which makes clear that it's just an ordinary command, ans also signals to someone who isn't sure what it's doing that they can find out just by running `man test`, `help test`, `info test`, etc., from the same shell.

I also agree that if statements and if expressions should be kept few and simple. But in some ways it's actually easier to do this in shell languages than in many others! Chaining && and/or || can often get you through a substantial script without any if statements at all, let alone nested ones.

replies(2): >>44382934 #>>44396166 #
11. lenerdenator ◴[] No.44381293{4}[source]
I honestly don't get why there are still a bunch of non-Linux/BSD servers, at least if the goal is to do UNIX-y stuff.

I haven't touched AIX or HPUX in probably a decade and I thought they were a weird idea back then: proprietary UNIX? Is it still 1993?

replies(1): >>44382244 #
12. kstrauser ◴[] No.44382244{5}[source]
At the time (10 years ago) I worked for a company with enormous customers who had all kinds of different deployment targets. I bet that list is a lot shorter today.

I hope so, for their sake. shudder

13. xavdid ◴[] No.44382934{3}[source]
I mean, there are 3 equally valid ways to write an if statement: `test`, `[`, and `[[`. In the case of the latter two, there are a mess of single-letter flags to test things about a file or condition[0]. I'm not sure what makes them "fake syntax", but I also don't know that much about bash.

It's all reasonable enough if you go and look it up, but the script immediately becomes harder to reason about. Conditionals shouldn't be this hard.

[0]: https://tldp.org/LDP/Bash-Beginners-Guide/html/sect_07_01.ht...

replies(1): >>44383927 #
14. pxc ◴[] No.44383927{4}[source]
You don't need any of those to write an if statement. I frequently write if statements like this one

    if ! grep -qF something /etc/some/config/file 2>/dev/null; then
      do_something
    fi
The `test` command is there if you want to use it, but it's just another command.

In the case of Bash, `test` is a built-in command rather than an external program, and it also has two other names, `[` and `[[`. I don't like the latter two because they look, to a naive reader, like special syntax built into the shell— like something the parser sees as unique and different and bear a special relationship to if-statements— but they aren't and they don't. And in fact you can use them in other shells that don't have them as built-ins, if you implement them as external commands. (You can probably find a binary called `[` on your system right now.)

(Actually, it looks like `[[` is even worse than "fake syntax"... it's real special syntax. It changes how Bash interprets `&&` and `||`. Yikes.)

But if you don't like `test`, you don't have to use it; you can use any command you like!

For instance, you might use `expr`:

  if expr "1 > 0"; then
    echo this will always run
  else
    echo this will never run
  fi
Fish has some built-ins that fall into a similar niche that are handy for simple comparisons like this, namely `math` and `string`, but there are probably others.

If you really don't like `test`, don't even need to use it for checking the existence or type (dir, symlink, socket, etc.) of files! You can use GNU `find` for that, or even sharkdp's `fd` if you ache for something new and shiny.

Fish actually has something really nice here in the `path` built-in, which includes long options like you and I both wish `test` had. You can write:

  if path -q --type=dir a/b/c
    touch a/b/c/some-file
  end
You don't need `test` for asking about or asserting equality of variables, either;

  grep -qxF "$A" <<< "$B"
is equivalent to

  test "A" = "$B"
or with the Fish `string` built-in

  string match --entire $A $B
The key is that in a shell, all commands are truthy in terms of their exit status. `&&` and `||` let you combine those exit statuses in exactly the way you'd expect, as do the (imo much more elegant) `and` and `or` combiner commands in Fish.

Finally, there's no need to use the likes of `test` for combining conditions. I certainly never do. You can just write

  test "$A" = "$B" && test "$C" = "$D"
instead of something like

  [ "$A" = "$B" -a "$C" = "$D" ]


If-statements in shell languages are so simple that there's practically nothing to them. They just take a single command (any!) and branch based on its exit status! That's it.

As for readability: any program in any language is difficult to understand if you don't know the interfaces or behaviors of the functions it invokes. `[`/`test` is no different from any such function, although it appears that `[[` is something weirder and, imo, worse.

15. colonial ◴[] No.44384656[source]
This is a decent heuristic, although (IMO) you can usually get away with ~100 lines of shell without too much headache.

Last year I wrote (really, grew like a tumor) a 2000 line Fish script to do some Podman magic. The first few hundred lines were great, since it was "just" piping data around - shell is great at that!

It then proceeded to go completely off the rails when I went full sunk cost fallacy and started abusing /dev/shm to emulate hash tables.

E: just looked at the source code. My "build system" was another Fish script that concatenated several script files together. Jeez. Never again.

16. passwd ◴[] No.44396166{3}[source]
The difference being, as far as I know, that `[[` is the real syntax. This from what I remember helps in avoiding certain class of issues, gives better error messages and is more certain to be a bash built-in.

What I would worry about more is that it breaks `sh` compatibility.