Most active commenters
  • pjmlp(9)
  • apwell23(6)
  • mythz(6)
  • high_na_euv(4)
  • Pxtl(4)
  • svieira(3)
  • megadal(3)

←back to thread

201 points olvy0 | 60 comments | | HN request time: 0.612s | source | bottom
Show context
high_na_euv ◴[] No.41878416[source]
LINQ is so fucking useful and well designed feature of .NET ecosystem that it is unreal when you gotta use lang which doesnt have such a thing.

C# design team is/was unparalleled

replies(7): >>41878459 #>>41878543 #>>41878588 #>>41878686 #>>41879163 #>>41879194 #>>41879315 #
1. pjmlp ◴[] No.41878459[source]
LINQ is largely based on FP stuff, also how Smalltalk collections work.

It is relatively easy to find similar capabilities in most languages nowadays, unless one is stuck on Go, C and similar.

replies(7): >>41878547 #>>41878579 #>>41878702 #>>41878783 #>>41878792 #>>41878816 #>>41879057 #
2. sanex ◴[] No.41878547[source]
Do you know an equivalent for Linw to EF in kotlin or Java because I have not found it.
replies(2): >>41878638 #>>41878713 #
3. kumarvvr ◴[] No.41878579[source]
Any pointers to such libraries in python?
replies(1): >>41878736 #
4. stanac ◴[] No.41878638[source]
Those are linq expressions. They are indeed wonderful. You get an abstract tree from which you can create SQL or API commands to access the data source. I remember in the early days (.NET 3.5?) there were multiple examples of LINQ2X like Linq2Csv, Linq2Rss Linq2Drobox (I'm paraphrasing, I don't remember actual examples, but it was wild).

There is also relinq library which transforms linq expressions into expressions which are easier to understand/use.

5. apwell23 ◴[] No.41878702[source]
No it isn't easy to find similar capabitites in java, go, python, ruby.

Maybe you do simulate some of this using meta programming in ruby but its certainly not 'easy to find'.

replies(2): >>41878746 #>>41879346 #
6. pjmlp ◴[] No.41878713[source]
Rather quick search, as I am more of a myBatis person,

Java: https://www.jooq.org/

Kotlin: https://www.ktorm.org

7. pjmlp ◴[] No.41878736[source]
itertools would be the starting point, unfortunelly Python is rather limited due to the way it only supports one line lambdas.
8. pjmlp ◴[] No.41878746[source]
It certainly is, unless you are talking about SQL like syntax, which is basically syntax sugar for classical FP.

And I explicitly left Go out of my list.

replies(1): >>41878887 #
9. blackoil ◴[] No.41878783[source]
One difference with LINQ is its ubiquity. It works with database, in memory data structures, on disk files. You can use your skills/code across all the system.
replies(1): >>41878827 #
10. jiehong ◴[] No.41878792[source]
I've found [0] for clojure, which maps the best IMO, but it also contains links to the same LINQ examples in other languages (java, kotlin, swift, elixir, python, ...).

[0]: https://github.com/mythz/clojure-linq-examples

replies(1): >>41879147 #
11. John23832 ◴[] No.41878816[source]
It's surprising that Go didn't ship with it, but given that they just added iterators, it's coming.

Rust has combinators, which is the same thing.

Most new languages are recognizing that functional support (even if they don't consider themselves FP languages) is necessary.

replies(3): >>41879028 #>>41879919 #>>41880179 #
12. John23832 ◴[] No.41878827[source]
It's just built on top of anything that is Iterable. If a language has first class iterator support, they could do something similar.
replies(2): >>41878964 #>>41879023 #
13. apwell23 ◴[] No.41878887{3}[source]
>unless you are talking about SQL like syntax

yes thats what linq is?

https://learn.microsoft.com/en-us/dotnet/csharp/linq/

"Language-Integrated Query (LINQ) is the name for a set of technologies based on the integration of query capabilities directly into the C# language." With LINQ, a query is a first-class language construct, just like classes, methods, and events.

doing this in java is not LINQ imo

  List<Integer> lowNums = filter(toList(numbers), new 
  Predicate<Integer>() {
        @Override
        public boolean apply(Integer n) {
            return n < 5;
        }
    });
replies(5): >>41879130 #>>41879238 #>>41879328 #>>41879460 #>>41879995 #
14. naasking ◴[] No.41878964{3}[source]
Mainly first-class functions I think. If you have those, you can just use fold in the core combinators.
15. mythz ◴[] No.41879023{3}[source]
Takes a lot more than that, LINQ providers work by accepting a LINQ Expression Syntax tree instead of an opaque function, which allows providers to inspect and traverse the Expression's AST and translate it into the data source it's implementing.

This Expression AST is constructed by the compiler, not something that can be tacked on by a library later.

replies(2): >>41879354 #>>41879511 #
16. rw_panic0_0 ◴[] No.41879028[source]
nah it's not coming, functions like maps and filters won't come to go by design, iterators are not only about FP stuff
17. whizzter ◴[] No.41879057[source]
Yes and no, the LINQ syntax being coherent between IEnumerable<> and IQueryable<> hides a lot of good magic.

IEnumerable<> is regular in-memory lambdas/streams, same what you find in many places.

IQueryable<> relies on the LINQ expressions, those CAN be JIT compiled for direct execution, but the fact that they are data-objects is what allows the translation to SQL and execution on the server rather than locally and can give massive gains since processing can be done where the data lives.

replies(1): >>41879229 #
18. high_na_euv ◴[] No.41879130{4}[source]
To be fair, 99% of linq usage is method chaining syntax, not query syntax
replies(1): >>41879292 #
19. nightski ◴[] No.41879147[source]
The difference is many of those are dynamically typed languages. It's still useful, but a lot of the a beauty of LINQ comes from the fact that it is within a statically typed language.
20. whizzter ◴[] No.41879229[source]
For reference, to achieve what IQueryable does with 100% normal code in JavaScript you need something like Qustar that was posted here a month ago.

Regular transform code in JS (Like IEnumerable)

const ids = users.filter(user => user.age<18).map(user => user.id);

IQueryable like to be transformed to the server:

const ids = users.filter(user => user.age.lt(18)).map(user => user.id);

In C# it'd look identical, but in JS or Java this would be achieved via proxy-object hacks (the .lt() function in the filter instead of the < operator and the .id property getter for the user map to send a flag under the hood for the mapping function).

https://github.com/tilyupo/qustar

21. svieira ◴[] No.41879238{4}[source]
These days it would be

    var lowNums = Arrays.stream(numbers).filter(n -> n < 5).toList();
2024's Java is also quite a bit better than 2013's Java.

Which still isn't as nice as LINQ, but this way we've painted the alternative in its best light, not in the light that makes C# look the best.

replies(2): >>41879467 #>>41880381 #
22. apwell23 ◴[] No.41879292{5}[source]
LINQ = Language Integrated Query

Its even in the name. What do you mean by "method chaining is linq" ?

replies(3): >>41879342 #>>41879380 #>>41882948 #
23. ygra ◴[] No.41879328{4}[source]
I guess the very fact that LINQ is so many things makes such discussions a bit annoying at times because everyone refers to a different set of features.

Is it the SQL-like query syntax? LINQ to objects? LINQ to SQL? Expression trees in general?

Expression trees and LINQ to SQL/EF/etc. are hard to find elsewhere. The query syntax often doesn't seem to be that big of a deal, especially since not all methods are available there, so pure query syntax often doesn't work anyway.

24. ygra ◴[] No.41879342{6}[source]
Considering the methods live in the System.Linq namespace, I think the extension methods may also be called LINQ.
25. svieira ◴[] No.41879346[source]
There are easy ways to do some subset of what LINQ does in Java (using annotation processors), in Go (using generators), in Python (using double-underscore methods to capture all the operations the expression is working with at runtime, see SQLAlchemy) and in Ruby.

There isn't a seamless way to do what LINQ does in any of those languages. But if the runtime supports a LISP then you can do more than what LINQ does (Clojure for the JVM, something like zygomys for Go, Hy for Python, and ... well, Ruby for Ruby).

26. megadal ◴[] No.41879354{4}[source]
Yes, but I think the point is practically every high level language can already do this pretty trivially.

If it's scripted you can typically just get a string representation of the function.

If it's Java, JAR inspection/dynamics have been a thing for a long time. And in other languages, they usually directly support metaprogramming (like Rust) and plugging code into the compilation logic.

replies(3): >>41879513 #>>41879690 #>>41880343 #
27. high_na_euv ◴[] No.41879380{6}[source]
"var results = list.Where(x => x.Salary > 12345).Select(x => x.Name).ToList())"

Giant majority of ppl refers to this when talking about LINQ.

But yea, it is LINQ method chaining.

SQL like syntax is LINQ query syntax

replies(1): >>41879493 #
28. SideburnsOfDoom ◴[] No.41879460{4}[source]
> yes thats what linq is?

The link that you gave says "LINQ is the name for a set of technologies" which includes the "SQL like syntax".

Includes is not the same as "is".

It isn't the most often used part of LINQ.

replies(1): >>41879503 #
29. Rohansi ◴[] No.41879467{5}[source]
One of the unfortunate pieces missing from Java's streams is the ability to easily extend them with additional operators. In C# they are all extension methods for the IEnumerable interface so you can add your own methods into the chain but that's not possible in Java.
replies(1): >>41880005 #
30. apwell23 ◴[] No.41879493{7}[source]
> But yea, it is LINQ method chaining.

You mean like fluent interface? https://en.wikipedia.org/wiki/Fluent_interface

What does this have to do with LINQ or C#. I remember doing 'method chaining' in 1990s .

replies(1): >>41879518 #
31. apwell23 ◴[] No.41879503{5}[source]
sure but you cannot just remove 'sql like syntax' and claim you can do linq in any language.
replies(1): >>41886462 #
32. Pxtl ◴[] No.41879511{4}[source]
Having used it since its inception, I've come to the conclusion that the SQL translator is kind of a misfeature. It creates so many weird bugs and edge-cases and tedium.

I love LINQ, I love having a typesafe ORM as a standard feature of C#, but the convenience of being able to reuse my Pocos and some expressions for both in-memory and in-SQL don't outweigh the downsides.

If I were designing SQL/LINQ today, I'd keep the in-memory record classes and in-database record classes distinct and use some kind of codegen/automapping framework for keeping them synched up. Maybe allow predicate operators to return things other than booleans so we could make `a == b` return some kind of expression tree node.

For ad-hoc queries using anonymous classes? Support defining an interface inline in a generic so you can say

    public T MyQuery<interface {string Firstname{get;set;}; string Lastname{get;set:}} T>();
Like, to elaborate, if you were doing some kind of JSON-based codegen (alternately you could do something where you have a separate hand-written POCO Model assembly and use reflection against it to generate your DbModel classes so it's still Code First). Yes, I know MS tried and abandoned this approach, I used LinqToSQL and EF3.5 and whatnot and suffered all that pain.

like, your master datatable file would be something like

    ```json
    "tables" : [
      "persons" : {
        "dataRecordClass" : "DataRecordsNamespace.DbPerson",
        "objectClass" : "PocosNamespace.Person"
      },
      "columns : {
        "PKID" : {
          "type" = "integer",
          "isPrimaryKey" = true,
          "isAutoGenerated" = true,
        }
        "firstName" : {
          "type" : "nvarchar(255)",
          "allowNull" : true,
        }
        "lastName" : {
          "type" : "nvarchar(255)"
          "allowNull" : false
        }
      }
    ]
    ```
which would generates something like

    ```cs
    public class DataRecordsNamespace.DbPerson : DbRecord {
      public DbPerson() { throw ThisIsAFakeClassException(); }
      public DbInt PKID{
        get => throw ThisIsAFakeClassException();
        set => throw ThisIsAFakeClassException();
      }
      public DbNVarChar {
        get => throw ThisIsAFakeClassException();
        set => throw ThisIsAFakeClassException();
      }
    }

    public partial class PocosNamespace.Person {
      public AutoGenerated<int> PKID{ get; init; }
      public string FirstName { get; set; }
      public string LastName { get; set; }
    }

    public class MyDbModel : DbModel {
      public DbTable<DbPerson> Persons => DoSomeLazyStuff();
    }

    public static class MyDbContextExtensions {
      public static List<Person> Resolve(this DbQuery<DbPerson> dbPersons) 
      {
        //call code to execute the actual query.
      }
    }
    ```
Am I making sense? Then you wouldn't have the problem of "oops I used an untranslateable method or member of Person", because MyDbModel can't have any of those. You'd lose the ability to to switch from whether a query is in-memory or in-database just by removing the ToList(), but I'd argue that's a misfeature, and better-handled by having some kind of InMemory implementation. Like, having DbQuery have a simple `.ToLocalMemory()` function that is a hint that the next part should be done locally instead of in the database would be a better way to do that. Then you could still do

    ```cs
    List<Person> myPersons = connection.MyDbModel
      .Persons
      .DoSomeInDatabaseQueryStuff()
      .ToLocalMemory()
      .DoSomeLocalMemoryStuffToOffloadItFromDatabase()
      .Resolve()
      .DoSomeDotNetStuff()
      .ToList();
    ```
edits: fix some of the HN pseudomarkdown
replies(2): >>41879691 #>>41880535 #
33. mythz ◴[] No.41879513{5}[source]
If it were trivial you'd see LINQ-like providers implemented in "practically every high level language".

Source code of the function means you have to implement the parser/lexer to convert it into a usable AST which is bad for both runtime performance and library size.

Very much doubt this is available in Java, which Java ORM lets you use native Java language expression syntax to query a database?

replies(2): >>41879899 #>>41887222 #
34. high_na_euv ◴[] No.41879518{8}[source]
>fluent interface

Various names, same concept.

"fluent interface is an object-oriented API whose design relies extensively on method chaining."

>What does this have to do with LINQ or C#.

Check the name of the namespace where all those APIs like Where, GroupBy, etc. are implemented, it is "System.Linq"

So thats why majority of ppl think about them when talking about LINQ.

Query syntax has like less than 1% of the "market share" versus method chaining style

35. mythz ◴[] No.41879691{5}[source]
Guess everyone has their preferred style, I personally avoid code-gen data models like the plague and much prefer code-first libraries.

Here's how you'd do something similar in our OrmLite ORM [1]:

    public class Person
    {
        [AutoIncrement]
        public int Id { get; set; }
        public string? FirstName { get; set; }
        [Required]
        public string LastName { get; set; }
    }
Create Table:

    var db = dbFactory.Open(); // Resolve ADO.NET IDbConnection
    db.CreateTable<Person>();  // Create RDBMS Table from POCO definition
Execute Query:

    // Performs SQL Query on Server that's returned in a List<Person>
    var results = db.Select<Person>(x => x.FirstName.StartsWith("A") && x.LastName == "B");

    // Use LINQ to further transform an In Memory collection
    var to = results.Where(MemoryFilter).OrderBy(MemorySort).ToList();
Everything works off the POCO, no other external tools, manual configuration mapping, or code gen needed.

[1] https://docs.servicestack.net/ormlite/

replies(1): >>41879863 #
36. eknkc ◴[] No.41879690{5}[source]
Wait what? Am I gonna include a source code parser and AST analyser to my JavaScript library for example, to examine the provided expression source and do this? This reads like the infamous Dropbox comment from when it first got released.
replies(1): >>41887194 #
37. Pxtl ◴[] No.41879863{6}[source]
My problem with this approach is that this falls apart if you write:

    db.Select<Person>(x => Regex.IsMatch(x.FirstName, "^A.*"));
This would fail at run-time instead of compile-time.

That's why I'd rather see the DB classes auto-generated with a mapper to convert them. Having the "master" be POCOs instead of JSON/XML/YAML/whatever isn't something I'm convinced on in either direction, but imho the in-database classes being not real POCOs is the important part because it reduces the the problem of somebody writing Person.MyMethod() and then blowing up because it's not a SQL function.

replies(1): >>41879948 #
38. pjmlp ◴[] No.41879899{6}[source]
jOOQ would be one such example, https://www.jooq.org/

Not that I use this, I am a myBatis person in what concerns database access in Java, and Dapper in .NET for that matter, not a big ORM fan.

And case in point most people use LINQ for in-memory datastructures, not the database part.

replies(1): >>41880000 #
39. pjmlp ◴[] No.41879919[source]
Go culture is quite clearly against this kind of niceties.
40. mythz ◴[] No.41879948{7}[source]
Isn't this just `.StartsWith("A")`?

How would you perform this regex query with your code generated solution? What would have to be code generated and what would the developer have to write?

As there's a lot more features available in different RDBMS's than what's available in C# expression syntax, you can use SQL Fragments whenever you need to:

    var results = db.Select(db.From<Person>()
        .Where(x => x.LastName == "B")
        .And("FirstName ~ '^A.*'"));
replies(1): >>41882485 #
41. pjmlp ◴[] No.41879995{4}[source]
I discovered someone stuff in Java pre-history days.

Because I am feeling nice,

     Arrays.stream(new int[]{1, 2, 3, 4, 5, 6, 10, 23, 45}).filter(n -> n < 5).forEach(System.out::println)
42. mythz ◴[] No.41880000{7}[source]
This is a custom expression language to work within the expressive limitations of the language:

    create.select(BOOK.TITLE)
      .from(BOOK)
      .where(BOOK.PUBLISHED_IN.eq(2011))
      .orderBy(BOOK.TITLE)
If Java supported LINQ you'd be able to use a more intuitive and natural Java expression syntax instead:

    create.from<Book>()
     .where(x -> x.publishedIn == 2011)
     .orderBy(x -> x.title)
     .select(x -> x.title);
replies(1): >>41880030 #
43. pjmlp ◴[] No.41880005{6}[source]
It is coming, it is called Gatherers, another approach, same result.
replies(2): >>41880273 #>>41903726 #
44. pjmlp ◴[] No.41880030{8}[source]
Java streams are what you're looking for.

If you insist in telling LINQ === EF, well that isn't what most folks in .NET use System.Linq for.

And back to the ORM thing, jOOQ is one way, there are others, and even if it isn't 1:1 to "from x select whatever" the approach exists.

replies(2): >>41880082 #>>41880971 #
45. mythz ◴[] No.41880082{9}[source]
> If you insist in telling LINQ === EF

I don't use EF, nor have I ever mentioned it.

You're replying to a thread about what it takes to implement a LINQ provider, which was dismissed as every high level language implements it with iterables, then proceed to give non-equivalent examples.

46. devjab ◴[] No.41880179[source]
You can do functionality similar to LINQ with chaining as long as you don’t need to call a method with a generic different from the one defined. If you do need that, you’re going to have to do it without chaining. You can still do something similar but it’ll be a lot less elegant than how C# does it.

It’s part of the design philosophy of Go though. They don’t want any magic. It’s similar to why they enforce explicit error handling instead of allowing you to chose between explicit and implicit. They want you to write everything near where it happens and not rely on things you can’t see.

It’s probably the primary reason that Go is either hated or loved. I think it’s philosophy is great, a lot of people don’t. I have written a lot of C# over the years, so I’m a little atypical in that regard, I think most C# developers think Go is fairly inferior and in many regards they are correct. Just not in the ones that matter (come at me!). To elaborate a little on that, Go protects developers from themselves. C# is awesome when it’s written by people who know how it works, when it’s not you’ll get LINQ that runs in memory when it really shouldn’t and so on.

47. svieira ◴[] No.41880273{7}[source]
I'm a huge fan of Gatherers, lovely API!
48. jayd16 ◴[] No.41880343{5}[source]
Not that I agree it's trivial but even if it was, so what?

This just feels like sour grapes.

replies(1): >>41887202 #
49. apwell23 ◴[] No.41880381{5}[source]
i posted that example from a comment that linked to this repo https://github.com/mythz/clojure-linq-examples

my point was that laguange support for sql like sytax is part of what makes LINQ linq. Java niceties is not relevant.

50. magicalhippo ◴[] No.41880535{5}[source]
Saw EF now supports custom SQL queries, so been considering that once we've moved to MSSQL (old db server isn't supported by EF).

We're quite accustomed to writing our own SQL select statements and would like to continue doing that to have known performance, but the update, insert and delete statements are a chore to do manually, especially for once you're 4-5 parent child levels deep.

replies(1): >>41883541 #
51. recursive ◴[] No.41880971{9}[source]
IQueryable<> manipulation has other tools available to it than brute-force iteration, like streams do. Streams may be the closest thing java has, but it's still a fundamentally different thing.
52. Pxtl ◴[] No.41882485{8}[source]
Yes, it's a trivial example. I'm not looking to support it, I'm looking to catch it at compile-time.

if "Person.FirstName" is a string, then that encourages users to use string-operations against it, which will fail if this expression is being translated to SQL for executing in the DB.

if "Person.FirstName" is some other type with no meaningful operations supported on it (which will get converted into a string when the query is executed) then it prevents many many classes of logic errors.

53. Merad ◴[] No.41882948{6}[source]
LINQ is the name for the overall system. LINQ can be written using two different styles:

    // Method syntax
    var evenNumbers = numbers.Where(num => num % 2 == 0).OrderBy(n => n);

    // Query syntax
    var evenNumbers = from num in numbers
        where num % 2 == 0
        orderby num
        select num;
Method syntax and query syntax are both part of LINQ (query syntax is syntactic sugar). .Net developers tend to overwhelmingly prefer method syntax.
54. Pxtl ◴[] No.41883541{6}[source]
Quick word of advice when dealing with deep parent/child reloationships:

Do not use lazy loading feature. That way lies madness.

replies(1): >>41883854 #
55. magicalhippo ◴[] No.41883854{7}[source]
We're not doing that where we come from. All child tables have the main id so we can load the data for all child rows with just one query per child table, and we load everything at once.

We were planning on sticking with this, it has worked well so far, but good to know to avoid getting tempted by the alternative.

56. SideburnsOfDoom ◴[] No.41886462{6}[source]
You can in fact ignore "sql like syntax" and find "similar capabilities" in many other languages. After all, most C# coders ignore the "SQL like syntax" in C# itself.

And when languages imitate features of a different language, they tend to go for the features that people like and use. No-one is going to add "similar capabilities" to the feature that no-one wants in the first place. People who say "C#'s LINQ is awesome!" just aren't talking about "sql like syntax", and people who say "without sql like syntax it's just not on the same level as LINQ" are misguided.

57. megadal ◴[] No.41887194{6}[source]
You could also bundle your JS. Or pretend like any number of other solutions like caching parsed ASTs exist instead of being as obtuse as possible, or something idk
58. ◴[] No.41887202{6}[source]
59. megadal ◴[] No.41887222{6}[source]
I mean they kind of are. You can find a library in almost every language that transpiles source code ASTs.

They're just not core features.

In Haxe, it's extremely common :) but Haxe is just one other high level language.

60. Rohansi ◴[] No.41903726{7}[source]
Good that there's a solution coming up but it's not as nice as C# IMO. Gatherers are way more verbose both in usage and especially in implementation. In C# they can be written as generator methods which is a very intuitive way to work with streams of data.