|
MeziLu wrote: Plus, because you can split operations between cores\processors, it is very fast.
That's PLINQ.
Read my comment again.
Real programmers use butterflies
|
|
|
|
|
Thank you for your comment. I meant it as part of the entire package, not as a singled out feature to make the point that you need to consider Linq in its entirety, not this feature or that feature.
MeziLu
|
|
|
|
|
I feel like I did, which is why I made a whole exception for PLINQ. I even agree with you, if you'll read my comment that PLINQ is where LINQ finally pays for itself.
Where we disagree is on basic LINQ. Not PLINQ.
I covered readability. I don't think LINQ helps that, because I operate under the idea that if you can't understand exactly what your code is doing it doesn't matter if the code is concise. LINQ is concise, but not easy to read: It's harder to translate a LINQ call of any real world complexity mentally into the series of iteration operations its going to perform than it is to do the same with nested foreach and if statements. LINQ is shorter, but it doesn't lend itself to readability, it just means the code is concise, which isn't the same thing, IMO.
Real programmers use butterflies
|
|
|
|
|
Wow, I must be wired differently. Readability is one of the things I like most about Linq. I place each operation on its own line and it’s like this “outline” or flow diagram that I and my colleagues can instantly understand. Anyway, good chatting with you but I need to take care of some urgent things now
MeziLu
|
|
|
|
|
One final quick comment: There are actually two different approaches to Linq: extension methods and lambda-like. I totally agree with you about the latter. I can’t follow or understand that to save my life and lever use that approach. I avoid the lambda-like approach like the plague. Extension methods are dreams come true, but that might be because I write a lot of extension methods myself.
MeziLu
|
|
|
|
|
One other point about parallel processing: When I do that, I always actually check the speed by setting a stopwatch to make sure I am getting a faster speed. Some times it is actually slower, but for the kinds of things I do, it is most often a lot faster. Just a word of advice to not assume speed boosts. Overall, without parallel processing, I find that Linq is blisteringly fast.
MeziLu
|
|
|
|
|
Performance should never be the only consideration, and in most cases not even the most important one. One big advantage of LINQ is that you have the same set of functions, no matter what data source lies behind the data. It might be a local System.Collections.Generic.List generated on-the-fly, but it might as well be an SQL or SOAP connection providing the data collection. Either way, you should always consider the cost when calling LINQ functions. So calling Enumerable.Count() without important reason is usually not a good idea, as this will iterate over all items.
Another advantage is readability. Of course only when you know how to read LINQ.
In performance-critical scenarios though, the cost of generating an enumerator just for the sake of being able to use LINQ (e.g. for an array, which doesn't come with its own enumerator) might be relevant. But you should decide on a case-by-case basis instead of generalizing the decision for or against LINQ. "Only a Sith deals in absolutes." 
|
|
|
|
|
I feel justified in making the general observations I made about linq.
And my comment wasn't just about performance. It was also about cognitive load in terms of understanding what your code is doing as well.
And performance considerations are indeed important *if* they influence architecture, at which point potential perf problems are best identified at design time rather than after you've already architected something that will not perform to requirements.
Using LINQ to implement all of your functional-programming style operations is a "Bad Idea(TM)" when you're doing loads of heavy iteration, like building parser or scanner tables. Most pure functional languages like Haskell handle iteration a lot better than LINQ if nothing else than for the fact that it's a first class operation. Enumerators in .NET were designed not as first class operations but built on top of the existing operations in .NET, and practically, that comes with performance considerations, like all the object creation it does.
If you don't believe me, write a LALR(1) parser generator using LINQ and then one without. As long as you know what you're doing latter will be at least twice as fast.
Real programmers use butterflies
|
|
|
|
|
The other issue is that for those of us who write in VB.Net, the LINQ syntax is convoluted and ugly. At least in C# the LINQ syntax is concise.
As for clarity, simple LINQ statements can be clear, but a loop can always be clearer. The other issue with LINQ is trying to figure out what the actual <t> in the IEnumerator<t> is. This is why C# added the var statement, because LINQ doesn't lend itself to clean typing.
|
|
|
|
|
Actually, I would discourage the use of var and encourage creating a class up front. This forces you to design what you want up front, which not only makes designing easier, but more importantly, you can serialize the class (with its groups, lists, etc) to an xml file and review it to make sure you are getting what you want. Var, IMHO, is a terrible way to go, especially for long term maintenance should something unexpected pop up. Serializing a class is extremely useful, especially for groupings, joins, and select many, etc.
The tree view like structure in a serialized xml file is incredible to look at for complicated solutions.
Guess another thing I am saying is that using var promotes backwards design. Create the Linq and hope for the best vs know what you want and design Linq to get you there, using serialization to test.
Further, having the class greatly improves readability, a concern some have expressed. It’s a lot easier to know what Linq is doing if you can see what it produces.
Regarding speed: I use PLinq for Next Generation Sequencing of the human genome. Six billion data points. It is plenty fast. Same goes for my CODIS search engine. CODIS is what you see on crime lab TV shows: here is the DNA profile from a crime scene. Are there any matches? PLinq was a godsend for that complicated process and is literally a million fold faster than Sql. Example: a 47 minute Sql search was 18 seconds for PLinq.
MeziLu
modified 11-Feb-21 10:07am.
|
|
|
|
|
|
I started reading it, and it seems we're on the same page as far as general approach toward optimization.
Optimization starts in the design phase during requirements gathering. Performance is either an explicit or unwritten requirement of any application. No application can take forever to perform. How long is acceptable is a question of design.
Optimization continues through project planning - choosing the platforms and tools, and even right data structures and patterns to accomplish your tasks. You don't garbage collect driver code. You don't use a Dictionary where a LinkedList would be more appropriate. These are design decisions, the first one high level, the second one more specific, but still, design decisions.
Only after that does the phrase "optimization is evil" come into play. Because at this point, if you're optimizing, you're optimizing something you should have optimized during design, or you're trying to bit twiddle to work around something that again, should have been optimized during design.
It's way more efficient to optimize up front during the design and planning phase, rather than after the fact when you are locked in and your options for improving performance are limited to bit twiddling.
Unless you're doing embedded though, counting cycles isn't important. Optimizations should be done on the algorithmic level - look for a Big O figure, not how to shave a cycle here or there.
Real programmers use butterflies
|
|
|
|
|
"I tend to... make my iteration operations explicit and long hand."
Thanks! I thought I was the only one. So much more readable when I come back later, IMHO.
|
|
|
|
|
I think the premise of Linq was to separate database operations (T-SQL) from C# developers.
Personally I think they have replaced T-SQL for something that is not easy to learn and understand. Exactly the opposite of what T-SQL is.
If one does not do Linq each and every day, but once in a while then Linq is difficult to quickly comprehend what is going on for someone asked to maintain it.
Second, in Microsoft's universe there are ample resources(developers, fast computers, fast internet connections, and Azure is free. Most of their big clients are in a similar situation. And Microsoft can afford to hire those that are the best of the best. That is not necessarily true for their big clients, and probably not true for the bottom of their customer pyramid.
Very few within Microsoft have to maintain complex codebases for long periods of time. And we can see what happens when they have to. How many repetitive breaking bugs have you seen in VS? New stuff is the focus, not fixing what is broken.
Consider any product demo or new feature demo. It is thrown together by the back office developers to demonstrate the concepts of the new capabilities, carefully avoiding any and all complexities that would pop up in a production environment. The life time is a few weeks, because by then Microsoft devs have moved on to next thing, a new iteration, and terminology has changed.
If performance is not an use, and one would prefer to keep database operations separate from the dedicated C# team then Linq is the way to go.
You just need to ask yourself is this Microsoft flavor of the day the best solution for me?
And to the person who thought the orderby and groupby operations were easy to understand, you should make a YT video and disperse your knowledge. Once can look at 5 Linq videos on this subject and be none the wiser.
|
|
|
|
|
honey the codewitch wrote: Now, there's a case where all of the above doesn't matter, and that's PLINQ.
..that's where my argument ends. PLINQ on an in-memory database (SQLite).
Everything else is storage. And that's either blobs or relational.
Bastard Programmer from Hell 
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
|
|
|
|
|
I have always hated LINQ, because it involves learning new syntax and introducing limitations that make my coding life more difficult.
It's for people, as in new Microsoft employees, educated in OOP but apparently not able to handle SQL.
What's bizarre, is that now, with .NET Core 3.1, 5.0, etc, I can't use LINQ to update the database, because I use database views and stored procs to manage the data, and EF Core has no capability to handle that in LINQ, so I have to use their Exec methods, which, besides having to list all the parms and values, have limitations regarding return values, which have to be predefined as classes in the database context and models, even for a simple integer return.
|
|
|
|
|
Yeah, that's a whole different can of worms I didn't (and couldn't due to lack of experience) address in my OP.
Microsoft's DB access has always left a lot to be desired. Every time they introduce some new RDBMS api layer it just builds on what's there without seeming to add much (linq over the DB and I'm basing this on your assessment above) - or when it does, it's brittle as hell, like the entity framework.
I would appreciate the syntax, personally, for *functional programming* but I don't consider the .NET enumerator paradigm up to the task of first class functional operations like guided iteration. It's too inefficient, for starters.
So LINQ builds on something that's not up to the job. My secondary criticism has to do with syntax and learning curve which i briefly addressed in my op, but on reflection i suppose my main criticism is introducing LINQ into an imperative language (C#) rather than say, making F# less arcane for people that are used to languages like Haskell.
Real programmers use butterflies
|
|
|
|
|
one example is the illegal monopolistic tactics they used many years ago to destroy WordPerfect`s market share, ditto.. NetScape browser, DBase etc., etc.
After working on it for many decades, MSFT still can`t get Windows OS to be anything more than a disgusting, overly-bloated security nightmare.
|
|
|
|
|
 That's horrible!
Even if I write it using Windows, I still agree with you that what they do is evil.
Dear mr. Gates, if you read that, I can change my mind about your system if you give me a job 
modified 3-Jun-21 21:01pm.
|
|
|
|
|
Member 14971499 wrote: Dear mr. Gates,
A response (referring to Bill Gates, who hasn't had much to do with Microsoft for years), to a post referring to a lawsuit that took place over two decades ago.
How do I get out of this timewarp?
|
|
|
|
|
Be honest: so do most other companies.
Look at Corel for example: When Paintshop Pro was a JASC product, it was small, light, powerful and fast. It walked all over Corel's offering Photo Paint, so they bought JASC out. Now it's huge, bloated, powerfull, buggy and a damn sight slower.
Or Samna's Ami Pro: the best word processor available, way, way better than Word. So Lotus bought it and it sank without trace.
Or ... you get the idea.
Microsoft isn't a virus: it's a huge company where not only does the right hand not know - or care - what the left hand is doing, it actually has a couple of dozen hands, and not even the fingers want to chat to each other!
This is perfectly normal "large group" behaviour, and you can see it everywhere: Corporate, military, government - they all work the same way!
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony
"Common sense is so rare these days, it should be classified as a super power" - Random T-shirt
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
Amen. This is how people operate in groups. It's the same dynamic that gives rise to empires and nation states.
This thread reminds me of a passage from The Tao of Programming:
A novice asked the Master: "In the East, there is a great tree-structure that men call 'Corporate Headquarters'. It is bloated out of shape with vice presidents and accountants. It issues a multitude of memos, each saying 'Go Hence!' or 'Go Hither!' and nobody knows what is meant. Every year new names are put onto the branches, but all to no avail. How can such an unnatural entity exist?"
The Master replied: "You perceive this immense structure and are disturbed that it has no rational purpose. Can you not take amusement from its endless gyrations? Do you not enjoy the untroubled ease of programming beneath its sheltering branches? Why are you bothered by its uselessness?"
Real programmers use butterflies
|
|
|
|
|
|
But imagine how much happier you'd be if you could! 
Real programmers use butterflies
|
|
|
|
|
modified 3-Jun-21 21:01pm.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
