←back to thread

Things Zig comptime won't do

(matklad.github.io)
458 points JadedBlueEyes | 3 comments | 20 Apr 25 15:57 UTC | HN request time: 0s | source
Show context
karmakaze ◴[20 Apr 25 17:14 UTC] No.43745047[source]▶
> Zig’s comptime feature is most famous for what it can do: generics!, conditional compilation!, subtyping!, serialization!, ORM! That’s fascinating, but, to be fair, there’s a bunch of languages with quite powerful compile time evaluation capabilities that can do equivalent things.

I'm curious what are these other languages that can do these things? I read HN regularly but don't recall them. Or maybe that's including things like Java's annotation processing which is so clunky that I wouldn't classify them to be equivalent.

replies(3): >>43745080 #>>43745506 #>>43745688 #
foobazgt ◴[20 Apr 25 18:25 UTC] No.43745506[source]▶
Yeah, I'm not a big fan of annotation processing either. It's simultaneously heavyweight and unwieldy, and yet doesn't do enough. You get all the annoyance of working with a full-blown AST, and none of the power that comes with being able to manipulate an AST.

Annotations themselves are pretty great, and AFAIK, they are most widely used with reflection or bytecode rewriting instead. I get that the maintainers dislike macro-like capabilities, but the reality is that many of the nice libraries/facilities Java has (e.g. transparent spans), just aren't possible without AST-like modifications. So, the maintainers don't provide 1st class support for rewriting, and they hold their noses as popular libraries do it.

Closely related, I'm pretty excited to muck with the new class file API that just went GA in 24 (https://openjdk.org/jeps/484). I don't have experience with it yet, but I have high hopes.

replies(1): >>43746810 #
pron ◴[20 Apr 25 22:00 UTC] No.43746810[source]▶
Java's annotation processing is intentionally limited so that compiling with them cannot change the semantics of the Java language as defined by the Java Language Specification (JLS).

Note that more intrusive changes -- including not only bytecode-rewriting agents, but also the use of those AST-modifying "libraries" (really, languages) -- require command-line flags that tell you that the semantics of code may be impacted by some other code that is identified in those flags. This is part of "integrity by default": https://openjdk.org/jeps/8305968

replies(1): >>43747870 #
foobazgt ◴[21 Apr 25 01:43 UTC] No.43747870[source]▶
Just because something mucks with a program's AST doesn't mean that it's introducing a new "language". You wouldn't call using reflection, "creating a new language", either, and many of these libraries can be implemented either way. (Usually a choice between adding an additional build step, runtime overhead, and ease of implementation). It just really depends upon the details of the transform.

The integrity by default JEPs are really about trying to reduce developers depending upon JDK/JRE implementation details, for example, sun.misc.Unsafe. From the JEP:

"In short: The use of JDK-internal APIs caused serious migration issues, there was no practical mechanism that enabled robust security in the current landscape, and new requirements could not be met. Despite the value that the unsafe APIs offer to libraries, frameworks, and tools, the ongoing lack of integrity is untenable. Strong encapsulation and the restriction of the unsafe APIs — by default — are the solution."

If you're dependent on something like ClassFileTransformer, -javaagent, or setAccessible, you'll just set a command-line flag. If you're not, it's because you're already doing this through other means like a custom ClassLoader or a build step.

replies(1): >>43750896 #
pron ◴[21 Apr 25 11:55 UTC] No.43750896[source]▶
> Just because something mucks with a program's AST doesn't mean that it's introducing a new "language".

That depends on the language specification. The Java spec dictates what code a Java compiler must accept and must reject. Any "mucking with AST" that changes that is, by definition, not Java. For example, many Lombok programs are clearly not written in Java because the Java spec dictates that a Java compiler (with or without annotation processors) must reject them.

In Scheme or Clojure, user-defined AST transformations are very much part of the language.

> The integrity by default JEPs are really about trying to reduce developers depending upon JDK/JRE implementation details

I'm one of the JEP's authors, and it concerns multiple things. In general, it concerns being able to make guarantees about certain invariants.

> If you're not, it's because you're already doing this through other means like a custom ClassLoader or a build step.

Custom class loaders fall within integrity by default, as their impact is localised. Build step transforms also require an explicit run of some executable. The point of integrity by default is that any possibility of breaking invariants that the spec wishes to enforce must require some visible, auditable step. This is to specifically exclude invariant-breaking operations by code that appears to be a regular library.

replies(1): >>43764250 #
foobazgt ◴[22 Apr 25 17:07 UTC] No.43764250[source]▶
Thanks for clarifying your role in the JEP.

I feel like we're talking right past one another. The ultimate reality is that annotation processors are pretty terrible for implementing functionality that a lot of Java developers depend upon. You could say annotation processors "weren't designed for that", but then you're just agreeing with me. This is sad, because arguably something quite similar to annotation processors could make the jobs of all of these developers a lot easier, instead of having them falling back to other mechanisms.

If your concern is integrity by default, why not just add yet another flag for can-muck-with-the-ast-annotation-processors? Or we can continue with the status quo.

replies(1): >>43764760 #
pron ◴[22 Apr 25 18:06 UTC] No.43764760[source]▶
> If your concern is integrity by default, why not just add yet another flag for can-muck-with-the-ast-annotation-processors?

There is such a flag (or, rather, a set of flags), and that's exactly what the Lombok compiler uses to change javac to compile Lombok sources rather than Java sources.

However, we think there are much better solutions to the problem those languages try to solve than allowing AST manipulation.

replies(1): >>43765473 #
1. foobazgt ◴[22 Apr 25 19:33 UTC] No.43765473[source]▶
You've referenced Lombok a lot here, and some Google searches later, I can see that you're in conversations all over the internet re: Lombok (and similar projects like Manifold). Their purpose is to extend the Java language. The class of code I'm referring to is more like those you already mention in your JEP: logging, tracing, profiling, serialization, authn/authz, mocking, ffi, and so on. I would describe all of those as fitting under the umbrella of "cross-cutting" and needing a "meta-programming" facility.

> However, we think there are much better solutions

I'd like to hear more. Can I discuss this further with you in a more appropriate venue than this forever thread?

replies(1): >>43765573 #
2. pron ◴[22 Apr 25 19:45 UTC] No.43765573[source]▶
> The class of code I'm referring to is more like those you already mention in your JEP: logging, tracing, profiling, serialization, authn/authz, mocking, ffi, and so on. I would describe all of those as fitting under the umbrella of "cross-cutting" and needing a "meta-programming" facility.

Those are traditionally offered in Java in the form of bytecode transformation rather than AST transformations, as the notion of "compile time" in Java is not as clear as it is in, say, Zig; Project Leyden will make it even more vague, as it will allow caching JIT output from one run to the next.

> Can I discuss this further with you in a more appropriate venue than this forever thread?

Sure, you can email me at the email address I use on the JDK mailing lists (e.g. loom-dev).

replies(1): >>43765959 #
3. foobazgt ◴[22 Apr 25 20:29 UTC] No.43765959[source]▶
> Those are traditionally offered in Java in the form of bytecode transformation

And we've come full circle. I think they're traditionally written as bytecode transformations, because the entire pipeline for both writing and using many kinds of program transformations in bytecode is far simpler, more accessible, and more performant than implementing and executing a source-to-source compiler that feeds into another java compiler.

That said, there are also times you wish to perform transforms on programs for which you don't have access to source, in which case your hand is forced. Ideally, you would be able to write many classes of transforms agnostic to that context.

> Sure

Thanks!