Net 9.0 LINQ Performance Improvements

"LINQ Performance improvements" really should read: "our own List<T> implementation performance improvements". Microsoft seem to spend their time improving what they need, rather than spending time on general improvements. LINQ (the syntax, not the method extensions) needs investment. Primarily around lambda allocation and potentially on compile-time reduction of lambdas.

It's time to have value-type local lambdas, or a strategy that doesn't make lambda allocation the overhead that it is. And also there really should be wildcard support (`_`) for LINQ variables by now. Which was been completely ignored when they were brought in for lambdas. It should also be possible to use a lifted-type (like IEnumerable<T>, Option<T>, etc.) as the final item in a LINQ expression, rather than `select ...`. The select adds overhead that isn't needed for certain use-cases and limits things like tail-recursive LINQ expressions.

Libraries like mine that go all-in on LINQ [1], but aren't using `IEnumerable` or `IQueryable`, or any of the LINQ extensions, continually get ignored because MS are focusing purely on improving the performance of their own projects.

A good example is the improved lambda inference. It was only brought forward because ASP.NET Core needed it for its minimal API. It seems like many of the features of the language/framework are driven by their own needs rather than those of the community. The absolute worst thing is the ever expanding set of 'magic methods' (like the LINQ extensions Select, SelectMany, and Where, but also GetAwaiter, and the rest). MS are adding capabilities for themselves (the compiler mostly) rather than do what is really needed and add proper higher-kinded traits to resolve the magic. So everything is weakly typed and only vaguely discoverable by the compiler :/

LINQ is one of those key differentiators between languages, yet it's been wallowing, pretty much untouched since C# 3. I think it's a crying shame that it has basically been ignored since then and, even now, they think LINQ is only useful for iterating lists. And primarily only their own list implementations.

/rant

Don't get me wrong, I appreciate all performance improvements, I'm sure it will help plenty of users. But the focus always seems to be narrowly targeted, which limits the potential.

[1] https://github.com/louthy/language-ext/

Your library looks very interesting, but I think you might be setting things up ahead of time to be dismissed. There are a large number of using statements, which is not a big deal for those who understand breaking up projects into appropriate pieces for exactly what you need, and keeping boundaries between different concerns. Most developers don't structure their projects the way that you do, and I say that as a positive towards your implementation, but often little things like this can hold back "the most common-denominator" when it comes to developers. I will admit right off the bat that you are far more well-versed in C# and .NET than I am, but I'm coming to this as someone that has worked with lots of junior developers that already struggle with standard LINQ syntax, methods, and (in particular) performance.

You do mention these things in your README, and I appreciate it, as most are set on "selling" their library instead of actually listing what it exceeds at and the purpose. Your comment on being non-idiomatic is another issue that I see (again, talking about people learning and getting used to C#/.NET). Only in my opinion, I feel like Microsoft want to keep their tools and languages following certain practices, and I get that your naming follows more functional programming practices to make them natural to FP. I feel like often, little issues like this are very large to Microsoft when it comes to looking into improvements.

I starred your repo, and am very interested in what you've put together here. On the last few major applications I've built (I normally build content management systems that are highly customized, that I tend to not call apps), I've used a Result<T> type that seems to be roughly the same as Option<T>. I wrote that sentence, and then went back to look at your library, and as much as I thought I knew about FP, I really don't. I think I'm really decent at C# and doing some complex things with it (I mainly use Umbraco CMS, and I've gotten my code to run websites under the minimal recommended requirements of the base Umbraco software). With that said, I read a lot about FP, but still have tons of issues picking it up. F# For Fun And Profit is the closest it's made sense to me. In the end, all this is to say that it's nothing that you're doing wrong. Microsoft is going to target the average or beginner developer, because that is the vast majority of developers that they have in their environment.

I hope that you library can gain some internal improvements, because it looks like you've spent an enormous amount of time on this, and you have enough stars on GitHub to at least indicate that people are actively using your library and getting benefit from it. I apologize if any of this came off as dismissive, I think what you've done here is exciting, and feel like you've set up enough documentation where I can slowly pick up concepts that I'm unfamiliar with.

re: usings. These are the only ones that are necessary:

   using LanguageExt;
   using static LanguageExt.Prelude;

I list all of the others so that people can easily add them to their global-usings. The key is `using static LanguageExt.Prelude`, because static-usings aren't flagged by tooling.

It's a conscious choice not to pander, I don't mind if the C# FP community doesn't include everybody in the C# community, I'm simply presenting an opinionated approach that is close to the 'norms' of FP languages/frameworks rather than trying to fit into the C# conventions. It's intentionally going 'all in' rather than just trying to augment the existing approach with types like Result or LINQ (the extensions). Most of the benefits of pure-FP come from a complete change of approach; I'm saying "leave the previous C# world behind and let's try a different way". Some people really won't like it and that's fine. In a company it needs an advocate that pushes the narrative, if that isn't there, then most won't get past the use of Option, or simple stuff that doesn't move the needle much.

I've been a CTO for 20 years and have mentored many juniors in my time, including in FP. A willing junior with a good mentor will find FP easier than OO once they grok it, because pure FP becomes more intuitive eventually (as it's based on maths, so you get to a point where - if your code compiles - you feel like it must be right, which is very powerful reinforcement for everyone, but especially for juniors).