Most active commenters
  • cogman10(3)
  • tapoxi(3)
  • acdha(3)

←back to thread

466 points CoolCold | 39 comments | | HN request time: 1.358s | source | bottom
Show context
kbar13 ◴[] No.40208219[source]
systemd has been a net positive for the linux ecosystem. remember when you had to write bash scripts to start, stop, restart services and handle any other signals you want to send it? nowadays it's a unit file (basically just an ini file) away with relatively straightforward API. and you can actually declare startup dependencies and other useful relationships past just "prepend a number signifying when it should run globally to the front of the filename". it's provided an extensible platform with which higher level orchestration frameworks like ansible / ignition can easily templatize services or other system configuration.

since the beginning of systemd people have moaned about how complex it is and how we're reinventing the wheel. yet time and time again the people actually working on the project show that the solution they've come up with is the result of the problem they're facing on a daily basis. it's quite annoying that the armchair linux experts complain about how "lol systemd is so stupid for reinventing the wheel, give me my shell scripts back", maybe think about whether or not you have a legitimate issue not being addressed by the solution proposed or if you are just getting rage baited by a headline.

replies(17): >>40208249 #>>40208286 #>>40208374 #>>40208481 #>>40209110 #>>40209185 #>>40212620 #>>40212965 #>>40214704 #>>40214800 #>>40214923 #>>40215163 #>>40215552 #>>40215793 #>>40216445 #>>40217144 #>>40217617 #
agilob ◴[] No.40214923[source]
> remember when you had to write bash scripts to start, stop, restart services

This was a really big pain, yes, but I also remember how I could `tail -f *.logs`. I remember how I didn't need to remember about `--no-pager` and `--follow`. I knew where the files were, what they were called. I remember how I didn't have to google how to find logs between 10 days ago and 4 days ago, because the logs would be in a .tar created by logrotate with a date in the filename.

The init system was probably peak of systemd, after that they started reinveting things in a more complicated way. Do we really need journald, systemd-boot, machinectl, systemd-networkd, sd-bus, systemd-resolved, systemd-nspawn? Do people actually use it all? Are there any metrics to show how many systems have it installed and in use?

replies(8): >>40215028 #>>40215089 #>>40215154 #>>40215742 #>>40216065 #>>40216893 #>>40217407 #>>40218927 #
1. lolinder ◴[] No.40215154[source]
> I could `tail -f *.logs`. I remember how I didn't need to remember about `--no-pager` and `--follow`.

journalctl supports -f, which by your own account you were already using for tail, so I'm not clear what's worse there.

> I remember how I didn't have to google how to find logs between 10 days ago and 4 days ago, because the logs would be in a .tar created by logrotate with a date in the filename.

My memory of this time was that every single application had its own unique method of handling logs and its own unique location for storing them. So sure, once you found the logs you didn't have to Google how to find the relevant dates if the application is using logrotate as you describe, but finding the logs in the first place was always a challenge. Systemd is nice in that it provides a single place where all logs go and a single interface for navigating them.

replies(7): >>40215510 #>>40215749 #>>40216628 #>>40216689 #>>40216818 #>>40218944 #>>40225179 #
2. saghm ◴[] No.40215510[source]
> journalctl supports -f, which by your own account you were already using for tail, so I'm not clear what's worse there

Yeah, this is a bit of an odd complaint. If I were this bothered by having to type those, I'd just make an alias to "journalctl --follow --nopager"` and would have forgotten about it years ago.

replies(2): >>40215734 #>>40215815 #
3. stoperaticless ◴[] No.40215734[source]
Never tried it, but if the complainer wrote: tail -f *.logs

Which involves glob’ing. I assume it allows to monitor several services at the same time.

Can systemd do that?

replies(2): >>40215882 #>>40215992 #
4. cogman10 ◴[] No.40215749[source]
I think my biggest gripe with journalctl is (and this may just be because I'm novice at using it) that discovering what sort of logs are on the system feels a lot more complex.

With regular logs, I can go to /var/log, ls the dir, and get a nice list of what's being logged. There will be, for example, an `apache.log` file that has all the logs relevant to apache.

replies(1): >>40215866 #
5. mdaniel ◴[] No.40215815[source]
> I'd just make an alias to "journalctl --follow --nopager"` and would have forgotten about it years ago.

You sure would have forgotten about it when you tried to get onto another machine without the magic set of aliases

I am also prepared for someone to chime in that "log egress tools exist" to move up the Maslow's Hierarchy of Log Needs

6. tapoxi ◴[] No.40215866[source]
It's just journalctl -u apache -f, assuming your distribution calls the unit file "apache", and the last few messages of the logfile appear with systemctl status apache

And you don't need to worry about "is this service rotating logs appropriately?"

replies(1): >>40215915 #
7. tapoxi ◴[] No.40215882{3}[source]
Yes, journalctl -u postgres -u redis -f etc
replies(1): >>40216005 #
8. cogman10 ◴[] No.40215915{3}[source]
The important thing for me at least isn't he `-u apache` but rather knowing that `apache` exists as one of the installed services on this box.

In the old ways, that was apparent because of the presence of the log file.

Granted, this isn't the problem it used to be for us, however, it would come up because we'd have our services named things like `foo-ws` and knowing that `foo-ws` existed on a given box might be tricky were it not for the log files.

replies(4): >>40215947 #>>40216154 #>>40216539 #>>40219917 #
9. tapoxi ◴[] No.40215947{4}[source]
I mean "systemctl status" shows all services and their hierarchy, there's also "systemctl list-unit-files" if you want to see things that aren't part of the current target (runlevel).

It's much easier in the systemd world to see the state of the system, the state of the service, and the logs of a service because it enforces this consistency.

replies(1): >>40216009 #
10. _flux ◴[] No.40215992{3}[source]
It's the basic function what journalctl -f does by default: follow the global log all jobs write into. -u etc are for filtering that.
11. mintplant ◴[] No.40216005{4}[source]
Or just `journalctl -f` to follow all logs.
12. cogman10 ◴[] No.40216009{5}[source]
Ah, good to know. I figured it might be the case that I just needed to RTFM.
13. tadfisher ◴[] No.40216154{4}[source]

    journalctl -f -u <TAB>
Shows you all installed units if you have working shell completion, or 'a<TAB>' gets you all units starting with 'a', etc.
replies(3): >>40217538 #>>40221208 #>>40221683 #
14. kbar13 ◴[] No.40216539{4}[source]
honestly that's such a niche problem to have that i dont think it's worth throwing the baby out with the bathwater. anyways you shouldn't be discovering installed services by logfile presence, it should be something like querying your package manager.
replies(1): >>40219622 #
15. cduzz ◴[] No.40216628[source]
I appreciate that someone's invented shoes you inflate and snap. I can see some advantages.

That said, when I'm in a hurry and going to do a thing that I've been doing every day of my life, I'm really not a big fan of having to stop what I'm doing and reexamine a tool I've used all my life to figure out how to use it. I learned this stuff decades ago and I'm not anxious to relearn how to do it for a fractional improvement, and I'm not even really 100% sure that this new "inflate and snap" shoe model is actually an improvement, except for very specific use cases.

replies(1): >>40216961 #
16. Asmod4n ◴[] No.40216689[source]
Oh the joy it is to have a corrupted log file which the program that wrote it refuses to even read.

Once you deal with that you’ll realize journald is just in the way and needs to be modified to output a human readable log by default.

17. citrin_ru ◴[] No.40216818[source]
'journalctl -u service -f' takes many seconds to start showing logs (even with 4GB journal size limit which not that much). 'tail -f /var/log/service' works without a noticeable delay (be it 1kb or 100Gb log file). For me it's a huge regression.
replies(1): >>40216992 #
18. acdha ◴[] No.40216961[source]
Don’t think of it as relearning a tool, think of it as learning one tool which works for everything. You no longer have to implement your own log rotation, compression, etc. or work out which configuration each program needs to do that in the manner you expect – for example, “is it safe to rotate a log file?” requires you to know implementation details on each program.
replies(1): >>40219219 #
19. rezonant ◴[] No.40216992[source]
While I'm very pro journalctl, this is very true. They should put some work into the initial read performance on big journals.
20. amaterasu ◴[] No.40217538{5}[source]
In which shell?
replies(2): >>40217978 #>>40218228 #
21. ac29 ◴[] No.40217978{6}[source]
Works in fish, I'll have to remember that one
22. alright2565 ◴[] No.40218228{6}[source]
bash & zsh are supported by upstream: https://github.com/systemd/systemd/tree/main/shell-completio...
23. SoftTalker ◴[] No.40218944[source]
Not my memory at all. In almost all cases the logs were in /var/log and written by syslog. Unless the whole service was in a chroot directory, or the program was written by an amateur who didn't follow logging conventions.
24. llm_trw ◴[] No.40219219{3}[source]
>think of it as learning one tool which works for everything.

For how long?

I've been around for decades at this point with people telling me how the newest and greatest thing will obsolete everything that came before it.

The best way to look through logs is still to materialize them in as text in a files hierarchy and use find with grep to look for issues.

replies(2): >>40220025 #>>40221429 #
25. rixed ◴[] No.40219622{5}[source]
I wouldn't say it's a niche problem. I also miss being able to go to /var/log and see instantly what's going on.

There should be a command like "journalctl --top" that would list all logging services with amount of logs, oldest and latest log times, and last few lines of each, to improve that experience. Maybe there is already a way to do something similar?

26. pcthrowaway ◴[] No.40219917{4}[source]
`systemctl list-unit-files` is too difficult?
27. olddustytrail ◴[] No.40220025{4}[source]
> For how long?

Debian switched to systemd about 12 years ago, so at least that long.

replies(2): >>40220229 #>>40220627 #
28. llm_trw ◴[] No.40220229{5}[source]
Find came out in 1974.
29. cduzz ◴[] No.40220627{5}[source]
A quick google indicates that it was made default in debian 8 in 2015, which is 9 years ago, not 12.

Perhaps it was available prior to then, but so were daemontools, upstart, and probably a variety of other alternatives.

replies(2): >>40225617 #>>40225940 #
30. foxhill ◴[] No.40221208{5}[source]
correct me if i’m wrong, but i believe the point that is trying to be made is;

a system user/admin has an intuition about files. saying that ‘journalctl -f -u’ (fu, indeed :) and whatever else is inherently undiscoverable, and is a.. basically orthogonal mechanism for handling what should be a simple task. i.e., viewing some logs. it’s far easier to compose and extend from files (what if i only care about the mtime of the log, for instance), than this.

look, i think systemd isn’t.. terrible. i also think it’s suffered a bit of complexity fetishisation, and it seems as though that this resulting complexity may have become invisible to you.

run0 doesn’t seem like a bad idea. but i am wincing a bit at the thought of unrestricted javascript determining access control.

replies(1): >>40238624 #
31. acdha ◴[] No.40221429{4}[source]
I’ve been doing this for decades, too, but I don’t miss having to deal with everyone inventing multiple log hierarchies, different rotation conventions, compression strategies, etc. not to mention all of the other things which systemd removed from daily toil around process management.
32. vbezhenar ◴[] No.40221683{5}[source]
With `ls -l` I can see file modification dates. How do I do it with journalctl? And why do I need to that that in the first place?

Journald would be perfect if I didn't have to know a thing about it. Work in background, move stuff from stdin/stdout to /var/log/journald/service.log, move old stuff in /var/log/journald/archive/service/2024-04/15.log.gz. I'd be happy. Why do I need this cryptic CLI I don't understand. It brings nothing but pain.

33. hmmm-i-wonder ◴[] No.40225179[source]
This and many other discussions misses the point that most people that don't like systemd have with it.

It can largely do everything it replaces, but differently, in some cases with improvements. Every year it increases its scope of 'problems' its solved.

But the issue is it hasn't _ACTUALLY_ solved a problem for me in about a decade, its only introduced problems. Its replaced things that worked perfectly for me and millions of other users for years/decades with something that... also works... but differently. Now to accomplish the _SAME_ outcome without any personal benefit I have to re-learn their new and usually opinionated way.

There was a time I could log into a system, find logs, parse dates and do everything in these discussions reletave to logs, rotation. Regardless of service because the standards were close enough and the basic tooling (tail/find/tar/grep/awk etc.) worked everywhere always, and you never had to google even if the service was one you never worked on before.

For systemd its the opposite, its almost a guarantee you have to google something along the line when interacting with new systems and services. And as soon as their newly enforced opinions and way of doing things is getting comfortable they go and replace something else that was also working... perfectly fine.

If it actively solved problems for the majority of users we wouldn't hate it so much. Instead it requires more mental overhead than anything other than XORG to deal with and constantly manage than anything else in the linux ecosystem.

replies(1): >>40234760 #
34. acdha ◴[] No.40225617{6}[source]
One thing to remember is that a lot of people use the testing distribution so they would have seen it before the Debian 8 release changed the default. The debate in the community took ages but one factor in concluding it was that people were using it for years without it being anything like the more hyperbolic predictions some opponents made.
35. olddustytrail ◴[] No.40225940{6}[source]
Yes, you're right. I saw when it was added not when it became the default. Still 9 years suggests it's not going to disappear overnight!
36. intelfx ◴[] No.40234760[source]
> If it actively solved problems for the majority of users we wouldn't hate it so much

It does and we don't. The only "we" who are "hating it so much" is a tiny vocal minority.

replies(1): >>40234829 #
37. hmmm-i-wonder ◴[] No.40234829{3}[source]
Ahh the vocal minority in support makes its appearance!

I wouldn't call the opposition tiny, although the vocal portion of any group tends to be the smallest. This would be my first online comment on it so I'm certainly part of the non-vocal group you ignore, that in my experience is significantly larger than the vocal side of group you dismiss.

38. tadfisher ◴[] No.40238624{6}[source]
I think you have me confused with someone who cares about the difference between binary and text logs. I have no pony in this race; my comment was just made to help.
replies(1): >>40242150 #
39. foxhill ◴[] No.40242150{7}[source]
fair enough :) apologies!