In the Beginning Was the Command Line (1999)

This seems to deny the possibility of equivalence of any sequence of actions taken in a bounded spaces of entities (named widgets thus type:id) and actions to another 'representation' (e.g. text):

   { select[i]@dropdown:states > click@button:submit }

The fact that we don't have this (yet) does not mean it is not possible. In fact, given that the current darling of tech LLMs can 'compute visually' based on tokens (text) should make it clear that any 'representation' can ultimately be encoded in text.

So a 'record' feature on GUI 'a' can create action encoding that can be emailed to your peer looking at GUI 'b' and 'pasted'.

MacOS kind of had this with AppleScript, including recording. It's a little disappointing that it isn't widespread now but I realize that demand for GUI automation is extremely niche.

I really wish one of the little GUI frameworks would develop this, but alas, the evidence is that "nobody" actually wants this. As evidenced by the things like Applescript that existed, and died.

I feel like it could be done if it was really a goal from day one, and there were things like "record this set of actions as a script" built into the toolkit. Even Applescript was still an afterthought, I think, albeit a very well supported one.

In the meantime, given the comprehensive failure of UI toolkits to support this style, it is completely fair for people to act and speak as if it doesn't exist.

Applescript is so clunky and annoying. I have a few applescript launcd functions on my mac for some personal projects, and only because apple forces you to go through stuff like applescript to use stuff like the system notifications. It was a pain as the documentation is just so very poor for applescript, so clearly neglected by anyone at apple even when they built the tool; its not like good documentation only came about in recent years in programming. Then again most of what apple documents is crap. The launchd documentation isn't any better and I get undefined behavior I can't make head or tail of from it (basically functions working fine when ran as a user but failing silently when the same functions are ran through launchd), I'm guessing from some permissions related issue between launchd running as a certain privileged process and maybe the limits of their system integrity protection, I can't be sure because the documentation is so poor. And what is even worse is that there aren't many people using tools like launchd, at least not nearly compared to people who use systemd, so the various stackoverflow or blog posts you could look for reason in these more popular tools are not there for the apple tooling.

Both Applescript and Automater are still around, and still provide recording.