←back to thread

299 points miguelraz | 2 comments | | HN request time: 0.486s | source
Show context
skissane ◴[] No.45894420[source]
My personal opinion-psuedoterminals should be enhanced to provide some mechanism for sending out-of-band data, like the existing TIOCPKT but on steroids. Maybe something like… if both ends support it, they can exchange JSON-RPC messages over this out-of-band channel, with it being possible to discover if the other side supports this or not. Possibly this is just a new TIOC* ioctl

Why? Well one reason is escape sequences are really limited and messy. This would enable everyone to gradually and backward-compatibly transition to a more modern alternative. Once you have a JSON-RPC channel, the two ends can use it to negotiate what specific features they support. It would be leveraging patterns already popular with LSP, MCP, etc. And it would be mostly in userspace, only a small kernel enhancement would be required (the kernel doesn’t have to actually understand these JSON-RPC messages just offer a side channel to convey them).

I suppose you could do it without any kernel change if you just put a Unix domain socket in an environment variable: but that would be more fragile, some process will end up with your pty but missing the environment variable or vice versa

Actually I’d add this out-of-band JSON-RPC feature to pipes too, so if I run “foo | bar”, foo and bar can potentially engage in content/feature negotiation with each other

replies(2): >>45894471 #>>45895203 #
robot-wrangler ◴[] No.45895203[source]
I think the modern unix'y thing to do here is always send all messages intended for the user to stderr (regardless of whether they are actually errors), and always respond with machine-friendly JSON on stdout by default. AFAIK there's no downsides to this.. you can still have color on stderr, you can still draw sixel pictures or whatever. Meanwhile pipes and redirection still work as expected.

No need for content/feature negotiations.. machine readable just defaults to JSON unless there's a --format flag for something else. And if you add that on the generation-side of the pipe, you just need to remember to put it on the consumer-side.

replies(1): >>45895648 #
zzo38computer ◴[] No.45895648[source]
There are many downsides.

There are problems with using JSON for this; other formats would be better. JSON needs escaping, cannot effectively transfer binary data (other than encoding as hex or base64), cannot use character sets other than Unicode, etc. People think JSON is good, but it isn't.

Also, you might want to use less or other programs for the text output, which might be the primary output that you might also want to pipe to other programs, redirect to a file (or printer), etc. This text might be separate from the status messages (which would be sent to stderr; these status messages are not necessarily errors, although they might be). If you use --help deliberately then the help message is the primary message, not a status message.

(In a new operating system design it could be improved, but even then, JSON is not the format for this; a binary format would be better (possibly DER, or SDSER, which is a variant of DER that supports streaming, in a (in my opinion) better way than CER and BER does).)

(Another possibility might be to add another file descriptor for structured data, and then use an environment variable to indicate its presence. However, this just adds to the messiness of it a little bit, and requires a bit more work to use it with the standard command shells.)

replies(2): >>45895784 #>>45895880 #
robot-wrangler ◴[] No.45895880[source]
I guess I don't see those as big downsides because I don't think people usually want binary data or quoted strings back from a CLI command, nor do they want column oriented output, nor "user friendly" tables.

Answering --help with JSON is a good example, how bad is it really if the response is JSON? Well, using less works fine still and you can still grep if you want simple substring search. Wanting a section is probably more common, so maybe you'd "grep" for a subcommand with `jq .subcommand` or an option with `jq .subcommand.option`, and maybe get yourself a fancier, JSON-friendly version of less that handles escaped quotes and newlines. Tables and tab-or-space delimited output overflow char limits, force the command-generator to figure out character wrapping, and so on. Now you need a library to generate CLI help properly, but if you're going to have a library why not just spit JSON and decouple completely from display details to let the consumer handle it.

Structured output by default just makes sense for practically everything except `cat`. And while your markdown files or csv files might have quoted strings, looking at the raw files isn't something people really want from shells or editors.. they want something "rendered" in one way or another, for example with syntax highlighting.

Basically in 2025 neither humans nor machines benefit much from unstructured raw output. Almost any CLI that does this needs to be paired with a parser (like https://github.com/kellyjonbrazil/jc) and/or a renderer (like https://github.com/charmbracelet/glow). If no such pairing is available then it pushes many people to separately reinvent parsers badly. JSON's not perfect but (non-minified) it's human-readable enough to address the basic issues here without jumping all the way towards binary or (shudder) HTML

replies(1): >>45895958 #
zzo38computer ◴[] No.45895958[source]
Structured output would be helpful in many ways, but JSON is not a good format for this (neither is YAML nor XML nor HTML).

> JSON's not perfect but (non-minified) it's human-readable enough to address the basic issues here without jumping all the way towards binary or (shudder) HTML

It does not address most of the real issues. Programs that deal with pictures, sounds, non-Unicode text, structures of the kinds that JSON does not have, etc, will not do as well; and the input/output will involve converting escaping. (One format that I think is better is DER, although, it is binary format. I did write a program to convert JSON to DER, though.)

replies(1): >>45896063 #
1. robot-wrangler ◴[] No.45896063[source]
For raw data type of applications, it's definitely important to be able to preserve pipeline oriented use-cases like `cat img.png | convert-stdin-to-jpg | convert-stdin-back-to-png | imgcat`. But in the hypothetical world where all CLI I/O moves towards JSON, I'd still argue this is not only possible but now strictly easier because you can just explicitly embed mimetype info + b64 instead of assuming/detecting encoding or requiring user to specify it, or working with heuristic file-magic. (Jupyter notebooks work like this to persist images instead of just flat text.) And presumably a smarter suite of JSON oriented tools like jcat/jless would either actually display that data, or notice the type and avoid spamming the screen with raw b64.
replies(1): >>45896271 #
2. zzo38computer ◴[] No.45896271[source]
That can be helpful, although JSON is still a bad format for this, especially since now it requires base64 encoding. People think JSON is a good format but it isn't. (MIME is also not a very good format for identifying file formats, since you can only specify one format; although there is a "added on" specification like "+zip" and "+der" this isn't very good.)