LINQ is so fucking useful and well designed feature of .NET ecosystem that it is unreal when you gotta use lang which doesnt have such a thing.
C# design team is/was unparalleled
replies(7):
C# design team is/was unparalleled
It's pretty well put together but it was very hard to work out the patterns of what it was doing underneath e.g. I could not tell you now how to implement a custom IQueryable (I know where to look but couldn't tell you the rhythms of it) for some database and I am the type of person who usually does go and find that kind of thing out before using something for a "serious" project.
Maybe it's just a microcosm for C# as a whole - very productive language, good design team, quite hobbled by it's weird/philistine upbringings: Bjarne said within C++ there is a simple language trying to escape, in C# you basically have a simple language buried in nouns.
There is also relinq library which transforms linq expressions into expressions which are easier to understand/use.
There's a lot hidden in there, but basically they expect you to use EF. Writing an IQueryable is a similar amount of work to writing a SQL query planner. You get passed a tree of Expression objects.
https://learn.microsoft.com/en-us/archive/blogs/mattwar/linq...
* Not understanding when something is evaluated.
* Not understanding the computational complexity of multiple chained operations/aggregates.
* Not understanding the expectation that Single() requires exactly one of something.
* Not understanding how damn hard it is to test LINQ stuff.
Java: https://www.jooq.org/
Kotlin: https://www.ktorm.org
Sorry, but idk how it is footgun of LINQ. It is like complaining about 0 or 1 based indexing
>Not understanding how damn hard it is to test LINQ stuff.
Any examples? Because I struggle to see such
The F# compiler is slower than the C# compiler, but it's still more than fast enough for building large applications.
Rust has combinators, which is the same thing.
Most new languages are recognizing that functional support (even if they don't consider themselves FP languages) is necessary.
> Not understanding how damn hard it is to test LINQ stuff.
I disagree with this. Just run the LINQ query on a compatible iterable.
yes thats what linq is?
https://learn.microsoft.com/en-us/dotnet/csharp/linq/
"Language-Integrated Query (LINQ) is the name for a set of technologies based on the integration of query capabilities directly into the C# language." With LINQ, a query is a first-class language construct, just like classes, methods, and events.
doing this in java is not LINQ imo
List<Integer> lowNums = filter(toList(numbers), new
Predicate<Integer>() {
@Override
public boolean apply(Integer n) {
return n < 5;
}
});This Expression AST is constructed by the compiler, not something that can be tacked on by a library later.
IEnumerable<> is regular in-memory lambdas/streams, same what you find in many places.
IQueryable<> relies on the LINQ expressions, those CAN be JIT compiled for direct execution, but the fact that they are data-objects is what allows the translation to SQL and execution on the server rather than locally and can give massive gains since processing can be done where the data lives.
This is really the thing with the entire .NET stack that’s very hard to communicate. The standard library and framework design are so well thought out relative to anything else out there. More than that, the support within VS is beyond any other dev tool that exists for any other language - it’s not even particularly close. Edit-and-continue comes to mind, which despite how many times people confuse the two is not hot reload, and is wildly more productive and useful.
I remember back close to 20 years ago DHH was espousing Ruby/Rails and that the concept of types at all were a code smell, and thinking “you’re just very wrong, and clearly aren’t familiar with what else is out there”. Eventually a lot of that crowd moved to Node, then to typescript, and came around.
VS Enterprise (expensive as it is) had features 15 years ago that still seem magical when I show them to JS/TS folks now. IntelliTrace is one that comes to mind - there’s nothing remotely close to it’s snapshot debugging that I’ve seen anywhere else, and I’ve really looked.
The big problems with the ecosystem are that the docs are exhaustive but terribly boring, and not well explained from a learning-something-for-the-first-time perspective. They also really expect that everything you do is the Microsoft way, so if you’re trying to interface your code with something like an Avalonia UI, you’re on your own.
The language is absolutely wonderful though, even when used with Rider. The productivity relative to node/typescript is better enough that it crushes my soul having to go back to wrestling tsconfig and imports after working with .NET references for even small changes. So many of the little things I used to take for granted really just work, and work well. It’s just a wonderful piece of work executed over decades, held back by poor community outreach and badly written documentation.
With scripting languages, it's all JIT :)
The C# teams progress on this has been slow. Keep in mind the CIL bytecode has had such capabilities for at least 20 years now and only in the past like decade are we seeing more features and optimizations around LINQ and System.Reflection.Emit.
Dynamics were extremely slow in C# and if you look at the CIL generated you see why. It's possible for example to use something like a Haxe anonymous types[1] to optimize Dynamics so that CallSite caching is way more performant.
I am pretty sure in C# the only way to accept an anonymous type is as a dynamic value, so even though the type of the structure is well-defined at compile-time, it will still rely heavily on runtime reflection/DLR with no additional caching beyond what DLR does for any other dynamic type.
Anyways, this leads to niche libraries being built for handling dynamic data like JSON performantly.
Which leads to annoying things like .NET libraries/apps being incompatible (without some adapter) if they use for example, different JSON libraries under the hood. (See [2]).
Problems like these (the lack of actually good JIT/dynamic code support) in my opinion significantly slow down the .NET ecosystems development, that's why it always feels like .NET is just catching up with features other popular languages have.
To be fair though, much of C#'s lag is owed to Microsoft's contribution to .NET being mostly technical debt. Almost everything good that came out of .NET came from open source/non MS teams (like Mono).
[1] - https://haxe.org/manual/types-anonymous-structure.html
[2] - https://learn.microsoft.com/en-us/dotnet/standard/serializat...
Regular transform code in JS (Like IEnumerable)
const ids = users.filter(user => user.age<18).map(user => user.id);
IQueryable like to be transformed to the server:
const ids = users.filter(user => user.age.lt(18)).map(user => user.id);
In C# it'd look identical, but in JS or Java this would be achieved via proxy-object hacks (the .lt() function in the filter instead of the < operator and the .id property getter for the user map to send a flag under the hood for the mapping function).
var lowNums = Arrays.stream(numbers).filter(n -> n < 5).toList();
Which still isn't as nice as LINQ, but this way we've painted the alternative in its best light, not in the light that makes C# look the best.
Developer experience is far ahead any other technology out there.
Std lib and its API design is world class, I wish cpp had as good stdlib. Tooling is strong, especially debugger
So what? I see LINQ used all the time, and it is almost entirely (extension) methods IEnumerable<T>
Could I implement IEnumerable<T>? I think I did once, as an exercise. It's not that complex. Not that interesting to be able to do it either.
LINQ is useful without EF. LINQ is not EF and EF is not LINQ.
I don't agree. I don't feel any expectation to use EF. It would not be relevant anyway to our code.
LINQ is not EF and EF is not LINQ. EF uses LINQ but not vice versa. LINQ is useful without EF.
The LINQ extension methods that we use constantly are on IEnumerable<T> so EF and IQueryable is of no importance to us, but LINQ is used everywhere.
Is it the SQL-like query syntax? LINQ to objects? LINQ to SQL? Expression trees in general?
Expression trees and LINQ to SQL/EF/etc. are hard to find elsewhere. The query syntax often doesn't seem to be that big of a deal, especially since not all methods are available there, so pure query syntax often doesn't work anyway.
There isn't a seamless way to do what LINQ does in any of those languages. But if the runtime supports a LISP then you can do more than what LINQ does (Clojure for the JVM, something like zygomys for Go, Hy for Python, and ... well, Ruby for Ruby).
If it's scripted you can typically just get a string representation of the function.
If it's Java, JAR inspection/dynamics have been a thing for a long time. And in other languages, they usually directly support metaprogramming (like Rust) and plugging code into the compilation logic.
Giant majority of ppl refers to this when talking about LINQ.
But yea, it is LINQ method chaining.
SQL like syntax is LINQ query syntax
Linq is lazy. .ToList() reifies. there, that's the gist of what you need to know. Not hard.
> Not understanding the expectation that Single() requires exactly one of something.
eh? There are a bunch of these methods, Single, SingleOrDefault, First, FirstOrDefault, Last, LastOrDefault and you can look up and grasp how they differ. It's fairly simple. I don't know what the problem is, outside of learning it.
> Not understanding how damn hard it is to test LINQ stuff.
Hard disagree. LInq chains can be unit tested, unless your Db access is mixed in, which is not a LINQ issue at all, it is a database query testing issue. LINQ code, in itself, is easily unit testable.
> Enumerable.Single Method - Returns a single, specific element of a sequence.
Some overload descriptions:
- Returns the only element of a sequence that satisfies a specified condition, and throws an exception if more than one such element exists.
- Returns the only element of a sequence, and throws an exception if there is not exactly one element in the sequence.
https://learn.microsoft.com/en-us/dotnet/api/system.linq.enu...
> Enumerable.FirstOrDefault Method - Returns the first element of a sequence, or a default value if no element is found.
https://learn.microsoft.com/en-us/dotnet/api/system.linq.enu...
When everything just works, you have a lot more time to focus on your problem. Once you can focus on your problem, you may quickly find out you don't actually care about it. Focusing on tools is a great way to hide from this reality.
The link that you gave says "LINQ is the name for a set of technologies" which includes the "SQL like syntax".
Includes is not the same as "is".
It isn't the most often used part of LINQ.
You can write hard to read code with any framework. Yes it takes effort sometimes to make linq code clear, but you should not give up on it.
You mean like fluent interface? https://en.wikipedia.org/wiki/Fluent_interface
What does this have to do with LINQ or C#. I remember doing 'method chaining' in 1990s .
I love LINQ, I love having a typesafe ORM as a standard feature of C#, but the convenience of being able to reuse my Pocos and some expressions for both in-memory and in-SQL don't outweigh the downsides.
If I were designing SQL/LINQ today, I'd keep the in-memory record classes and in-database record classes distinct and use some kind of codegen/automapping framework for keeping them synched up. Maybe allow predicate operators to return things other than booleans so we could make `a == b` return some kind of expression tree node.
For ad-hoc queries using anonymous classes? Support defining an interface inline in a generic so you can say
public T MyQuery<interface {string Firstname{get;set;}; string Lastname{get;set:}} T>();
like, your master datatable file would be something like
```json
"tables" : [
"persons" : {
"dataRecordClass" : "DataRecordsNamespace.DbPerson",
"objectClass" : "PocosNamespace.Person"
},
"columns : {
"PKID" : {
"type" = "integer",
"isPrimaryKey" = true,
"isAutoGenerated" = true,
}
"firstName" : {
"type" : "nvarchar(255)",
"allowNull" : true,
}
"lastName" : {
"type" : "nvarchar(255)"
"allowNull" : false
}
}
]
```
```cs
public class DataRecordsNamespace.DbPerson : DbRecord {
public DbPerson() { throw ThisIsAFakeClassException(); }
public DbInt PKID{
get => throw ThisIsAFakeClassException();
set => throw ThisIsAFakeClassException();
}
public DbNVarChar {
get => throw ThisIsAFakeClassException();
set => throw ThisIsAFakeClassException();
}
}
public partial class PocosNamespace.Person {
public AutoGenerated<int> PKID{ get; init; }
public string FirstName { get; set; }
public string LastName { get; set; }
}
public class MyDbModel : DbModel {
public DbTable<DbPerson> Persons => DoSomeLazyStuff();
}
public static class MyDbContextExtensions {
public static List<Person> Resolve(this DbQuery<DbPerson> dbPersons)
{
//call code to execute the actual query.
}
}
```
```cs
List<Person> myPersons = connection.MyDbModel
.Persons
.DoSomeInDatabaseQueryStuff()
.ToLocalMemory()
.DoSomeLocalMemoryStuffToOffloadItFromDatabase()
.Resolve()
.DoSomeDotNetStuff()
.ToList();
```
Source code of the function means you have to implement the parser/lexer to convert it into a usable AST which is bad for both runtime performance and library size.
Very much doubt this is available in Java, which Java ORM lets you use native Java language expression syntax to query a database?
Various names, same concept.
"fluent interface is an object-oriented API whose design relies extensively on method chaining."
>What does this have to do with LINQ or C#.
Check the name of the namespace where all those APIs like Where, GroupBy, etc. are implemented, it is "System.Linq"
So thats why majority of ppl think about them when talking about LINQ.
Query syntax has like less than 1% of the "market share" versus method chaining style
No. When it was release (circa 2007), very few mainstream languages embraced "Code as Data" the way C# did. In Java, there was no way to pass an expression (as an AST) to an SQL library. Which is why LINQ is so much more ergonomic than Hibernate. In C#, you could use language features you're already familiar with (such as "order.id > 100 && order.id < 200") in your queries, whereas Hibernate made you learn the framework's specific patterns (add Criteria etc etc, I don't recall now). Java just wasn't expressive enough for this.
In fact, you couldn't do this even today in a language like say Python or JS. I mean, not without running it through something like babel to get an AST, and having arbitrary rules on what's code and what's data. C# had this in the spec; based on whether it was IQueryable.
> Almost everything good that came out of .NET came from open source/non MS teams (like Mono).
My team adopted Mono very early - like in 2005. Your statement is not true at all. C# and the Framework was a very good spec irrespective of what Open Source / Mono did, and while Mono existing might have accelerated .Net's transition into Open Source, it would have happened anyway due to the definitive swing towards Open Source in the 2000s. Linq-to-SQL, Asp.Net MVC, EF etc didn't come out of Mono.
Here's how you'd do something similar in our OrmLite ORM [1]:
public class Person
{
[AutoIncrement]
public int Id { get; set; }
public string? FirstName { get; set; }
[Required]
public string LastName { get; set; }
}
var db = dbFactory.Open(); // Resolve ADO.NET IDbConnection
db.CreateTable<Person>(); // Create RDBMS Table from POCO definition
// Performs SQL Query on Server that's returned in a List<Person>
var results = db.Select<Person>(x => x.FirstName.StartsWith("A") && x.LastName == "B");
// Use LINQ to further transform an In Memory collection
var to = results.Where(MemoryFilter).OrderBy(MemorySort).ToList();
db.Select<Person>(x => Regex.IsMatch(x.FirstName, "^A.*"));
That's why I'd rather see the DB classes auto-generated with a mapper to convert them. Having the "master" be POCOs instead of JSON/XML/YAML/whatever isn't something I'm convinced on in either direction, but imho the in-database classes being not real POCOs is the important part because it reduces the the problem of somebody writing Person.MyMethod() and then blowing up because it's not a SQL function.
Not that I use this, I am a myBatis person in what concerns database access in Java, and Dapper in .NET for that matter, not a big ORM fan.
And case in point most people use LINQ for in-memory datastructures, not the database part.
How would you perform this regex query with your code generated solution? What would have to be code generated and what would the developer have to write?
As there's a lot more features available in different RDBMS's than what's available in C# expression syntax, you can use SQL Fragments whenever you need to:
var results = db.Select(db.From<Person>()
.Where(x => x.LastName == "B")
.And("FirstName ~ '^A.*'")); create.select(BOOK.TITLE)
.from(BOOK)
.where(BOOK.PUBLISHED_IN.eq(2011))
.orderBy(BOOK.TITLE)
create.from<Book>()
.where(x -> x.publishedIn == 2011)
.orderBy(x -> x.title)
.select(x -> x.title);If you insist in telling LINQ === EF, well that isn't what most folks in .NET use System.Linq for.
And back to the ORM thing, jOOQ is one way, there are others, and even if it isn't 1:1 to "from x select whatever" the approach exists.
I don't use EF, nor have I ever mentioned it.
You're replying to a thread about what it takes to implement a LINQ provider, which was dismissed as every high level language implements it with iterables, then proceed to give non-equivalent examples.
It’s part of the design philosophy of Go though. They don’t want any magic. It’s similar to why they enforce explicit error handling instead of allowing you to chose between explicit and implicit. They want you to write everything near where it happens and not rely on things you can’t see.
It’s probably the primary reason that Go is either hated or loved. I think it’s philosophy is great, a lot of people don’t. I have written a lot of C# over the years, so I’m a little atypical in that regard, I think most C# developers think Go is fairly inferior and in many regards they are correct. Just not in the ones that matter (come at me!). To elaborate a little on that, Go protects developers from themselves. C# is awesome when it’s written by people who know how it works, when it’s not you’ll get LINQ that runs in memory when it really shouldn’t and so on.
my point was that laguange support for sql like sytax is part of what makes LINQ linq. Java niceties is not relevant.
I was discussing IQueryable with someone else to whom it was important. In reply to
> I could not tell you now how to implement a custom IQueryable (I know where to look but couldn't tell you the rhythms of it) for some database
And "for some database" the default answer is "use EF" as the intermediary between LINQ queries and the database itself, rather than delving into IQueryable.
LINQ-to-objects is very important and useful but I was talking about something else.
We're quite accustomed to writing our own SQL select statements and would like to continue doing that to have known performance, but the update, insert and delete statements are a chore to do manually, especially for once you're 4-5 parent child levels deep.
I'm certainly guilty of this! What's the difference?
That said, C# / .NET shops did have a tendency to mindlessly buy into all sorts of terrible Microsoft enterprisey stuff. That drove me crazy and ultimately is what made me head out for greener pastures.
if "Person.FirstName" is a string, then that encourages users to use string-operations against it, which will fail if this expression is being translated to SQL for executing in the DB.
if "Person.FirstName" is some other type with no meaningful operations supported on it (which will get converted into a string when the query is executed) then it prevents many many classes of logic errors.
Modern C# is probably the best general purpose language out there with the best tooling along with the dotnet framework. Too bad the guides and public information all align with the latest trends Microsoft are pushing to appear relevant. Blazor, MAUI, Aspire e.t.c. are all distractions to maintain the appearance of being modern. None of which are production ready or actually good for that matter.
Back to my original point. If you want to create a new web app then you're REALLY pushed to use Blazor, which is confusing, has many flaws, is basically alpha and is just a bad idea in general. For some reason you're shown a laughably simple guide spread over eight pages which could be a single page. You finish the "guide" and so you go to the "documentation". That documentation page is full of buzzwords that confuses new developers and belittles old developers. The end of this page links back to the pathetic guide. It's seriously like this for everything they do. There's tiny little nuggets of information scattered over thousands of useless pages.
I may sound blunt but it's a fantastic technology ruined by terrible management, poor communication and clearly the worst developer relations team any tech company has ever assembled. How can any company with this much money, this much recognition and this great of a technology fumble it so badly. Well... I actually do know why and it's obvious to anyone capable of critical thinking.
// Method syntax
var evenNumbers = numbers.Where(num => num % 2 == 0).OrderBy(n => n);
// Query syntax
var evenNumbers = from num in numbers
where num % 2 == 0
orderby num
select num;
We were planning on sticking with this, it has worked well so far, but good to know to avoid getting tempted by the alternative.
Edit-and-continue allows for changing the code and then updating the output directly in memory without re-compilation or restarting the execution. It sounds similar but in practice it allows for much more rapid iteration and is profoundly more useful. If you’re pretty deep into an application or web app for example (e.g. added to basket -> checkout -> process payment) and are 30 or 40 calls deep in a stack and realise you’ve a simple bug or change to make, you can edit the code in memory, drag the debugger back to the line, re-execute it and move to the next statement. The benefits of this compound really quickly for anything more than trivial scenarios, so much so that I’ll often code directly in a debugging session as it’s just handier to have a full rewindable call stack right there for simple cases where I’ve forgotten a property name or need to correct and XPath or something.
The surprising thing is that this isn’t even new, VS has had this for at least 20 years (and I think 25 or more as i know VB6 had it. Yes I’m old.)
Edit: 27 years ago in VC++5 (1997).
The docs are clearly not written by engineers and it really shows.
It’s a shame too - MAUI should be excellent. Best-in-class even. They’ve had the most resources and best tech to throw at the problem and are a distant second at best to React Native. (It might see less use than Flutter these days I’ve no idea).
Also having the C# dev kit for VS Code be non-free is just insane. They’re actively giving the market over to node.
You'd be right to point out that confusing README of these two does not do .NET any justice however.
And when languages imitate features of a different language, they tend to go for the features that people like and use. No-one is going to add "similar capabilities" to the feature that no-one wants in the first place. People who say "C#'s LINQ is awesome!" just aren't talking about "sql like syntax", and people who say "without sql like syntax it's just not on the same level as LINQ" are misguided.
Yes, I understand that implementing IQueryable is a beast, but let's not pretend that writing a database connectivity layer was a trivial, everyday activity before IQueryable. It's a specialised, tricky task. IQueryable may not make it easier, but it never was "easy" per se. Nor common, or usually necessary. Or something to do as an exercise "before using something for a serious project" as the grandparent post suggests.
YMMV as to how important IQueryable is to your code.
On the other hand, if you can write extension methods over "this IEnumerable<T> source" you can extend LINQ. IQueryable being complex is no barrier to that, in any way.
Anyway, EF is cool, but probably every .NET dev has an EF/LINQ performance related horror story (the generated queries are ridiculous).
A self compiling language is more impressive to me than ASP.NET MVC.
And C# is just lacking for what is actually capable in CIL bytecode. Or _was_ when I last used.
There have definitely been improvements, but in my opinion, they have just been kind of slow.
When I think of Microsoft's impact on .NET and it's culture, I think of stuff like SOAP, the SmtpClient, breaking changes in APIs every year and the technical debt left by it, the dogmatic fanboys, etc...
Quoting from the docs (emphasis mine): > .NET Hot Reload applies code changes, including changes to stylesheets, to a running app without restarting the app and *without losing app state*
That sounds more like how you described edit-and-continue to me.
> probably every .NET dev has an EF/LINQ performance related horror story (the generated queries are ridiculous)
> There have definitely been improvements, but in my opinion, they have just been kind of slow.
> much of C#'s lag is owed to Microsoft's contribution to .NET being mostly technical debt. Almost everything good that came out of .NET came from open source/non MS teams (like Mono).
Do you actively use .NET (any modern target in the last, say, 3-4 years or so)?