Re: How would you describe what an IEnumerable is to a bright student of C# - C# Discussion Boards

Re: How would you describe what an IEnumerable is to a bright student of C#

26-Jan-22 5:31

Is there any real reason to use an IEnumerable ?
Yes; first off IEnumerable was Microsoft implementation of functional programming algebras for a List. It is essentially a monadic data type that encapsulates List; monadic data types are typically designed to conform to a fairly standard set of functional algebras i.e. providing a common API for computations.

One major difference with Microsoft’s implementation; is that Microsoft decided to depart from the more familiar naming conventions used in functional programming circles, for example:
- Select is more common referred to as a Functor; and its method is more typically named Map.
- SelectMany is more commonly referred to as a Monad, and its method is more typically named FlatMap.
Microsoft's choice was to align their monadic implementation with SQL for obvious reasons; they had an existing group of programmers who would be already quite accustomed to SQL; so adopting that style of API would simplify adoption.

Category Theory
Function programming algebras are directly tied to Category Theory; hence there are specific tests (axioms) that can be performed to verify that these common computations conform to the axioms of a Functor; for example, it must preserve:
- Identity morphisms
- Commutativity re composition of morphisms
Similarly said there are rules governing the other algebras like SelectMany; which as I mentioned is a monad, etc…

The benefit of IEnumerable is the same as for any other functionally conforming type; it allows the use of a common set of algebras (common API) on the type that is encapsulated by IEnumerable. In Functional Programming; IEnumerable is not the only monadic data types that are used; there are also for example:
- Identity Monad; this is the most trivial of the monadic types; encapsulating a single value.
- Maybe Monad; a type that encapsulates an optional value; similar to the Option type in C#
- List Monad; a type that encapsulates a list of values; similar to IEnumerable type in C#
- Either Monad; a type that encapsulates a value with one of two possible outcomes.
- Validate Monad; a type like Either that is used to write data validators; because it has been designed to accumulate errors with e.g. an input form with multiple input validations that could be in error at the same time.
- etc.
This is by no means a comprehensive list of monadic data types… both for functional programming and on the large, but its also not comprehensive in terms of the C# types that have some level of implementation of functional programming algebras.

What's so useful about deferred evaluation ?
Deferred evaluation is more commonly referred to as lazy evaluation; why is it useful in Linq is simple. it allows a more expressive use of the api without incurring additional computation cost, for example:
- Multiple Where statements in a single Linq computation without the cost of performing each Where computation separately -- it's combined into a single computation.

Allow you to only incur a computational cost when its needed.
For example; some lazy evaluations may be built up during a computation block; and then never executed because of e.g. a branched operation like a cancellation; where this lazy evaluation becomes unnecessary; hence it would be preferable to not incur any computational cost until its actually needed, and most certainly not at each step (e.g. for each dot chained method call).

Infinite Lists
The other less obvious use of lazy evaluation is infinite lists; lists that are computationally far too expensive and/or impossible to compute on the whole… that is where method calls like Take, Skip come into play, for example:
- I can choose to Skip the 1st 3 entries in an infinite list of primes, and Take the next 4; without incurring the computation cost of trying to 1st populate an infinite list of primes.

Isn't deferred evaluation dangerous because you have no idea of what might have changed in the time between assembling the IEnumerable and iterating/evaluating it ?
No not at all; because the final computation would still work on essentially a snapshot of the encapsulated value at the time the lazy evaluation was computed.

Plus its considered bad design to build any system that incorporates data races — a data race occurs when two computations access the same value concurrently, and at least one of the accesses mutates the value. The problem is that it makes testing impossible, because you have no way of knowing the state of the encapsulated value; worse the mutation step may only be partially computed; leaving the value in a compromised state. It makes it impossible to test code with objects that are in an indeterminate state.

Why can't I test an IEnumerable to see if it's empty, or null; why do I have to trigger iteration with 'Count() to see how many items it has, and, why can I only use 'Any to see if there are no items in it without triggering iteration..
…because its a lazy evaluation — which is typically impossible (if not difficult) to predetermine in advance of actually computing the lazy evaluation. However Microsoft as of .Net 6 has made available a new Linq method called TryGetNonEnumeratedCount which attempts to determine the number of elements in a sequence without forcing an enumeration; failing which it reverts to forcing the compute.

Personally I would advise against this style of coding completely + in the functional programming context; we specifically avoid these types of method calls, because they are indeterminate; worse this method mutates an input parameter (out keyword) -- which makes testing that much more difficult. In functional programming; there is huge merit in trying to keep a majority of the codebase as pure as possible. In the C# context; it is quite possible to have the core of the codebase pure; with a very small cut out for mutation where its unavoidable, for example: UI. Leaving a codebase that is for greater part very easy to test, and hence less prone to errors.

Anyway that's a window into the rabbit hole that is functional programming and its basis in category theory; hopefully its helpful.

modified 26-Jan-22 12:01pm.

Gerry Schmitz26-Jan-22 6:13

Re: How would you describe what an IEnumerable is to a bright student of C#

26-Jan-22 6:13

It's an interface to a forward reading state machine who's final implementation is left up to the author. The whole "internal list" concept is an illusion.

I use IEnumerable to create custom tokenizers for certain text files. It's when you need it, and realize it, that its usefulness becomes clear.

"Before entering on an understanding, I have meditated for a long time, and have foreseen what might happen. It is not genius which reveals to me suddenly, secretly, what I have to say or to do in a circumstance unexpected by other people; it is reflection, it is meditation." - Napoleon I

endo funk26-Jan-22 9:40

Re: How would you describe what an IEnumerable is to a bright student of C#

26-Jan-22 9:40

Most certainly Yes... in the context of considering IEnumerable as just an interface for iteration over a collection of sorts.

What however muddles the water is the questioning of its usefulness in relation to the Linq Extension Methods for IEnumerable, as mentioned by OP:
- Select
- Where
- Aggregate

Similarly deferred evaluation aka lazy evaluation and Count()

Whilst most any data type can conform to some degree with IEnumerable, and by virtue its Linq extensions; it's predominant practicality is with in memory collection types like List, where a majority of the Linq algebras would equally be useful, performant, etc.

Gerry Schmitz27-Jan-22 16:58

Re: How would you describe what an IEnumerable is to a bright student of C#

27-Jan-22 16:58

There is no muddling; until I yield, I can pretty well do anything I like, including chaining.

Count() is not a method of IEnumerable. A state machine can learn; you could probably get a good estimate the 2nd time around; all other things being equal.

If the IEnumerable returns an object, you could even have it return a status report every so often; you're not limited to returning a particular type.

endo funk27-Jan-22 19:35

Re: How would you describe what an IEnumerable is to a bright student of C#

27-Jan-22 19:35

Count() is not a method of IEnumerable.
It is an extension on IEnumerable<TSource>, the state machine is sub typed to IEnumerable; see reference runtime code below:
runtime/Count.cs at dotnet/runtime · GitHub

A state machine can learn; you could probably get a good estimate the 2nd time around; all other things being equal.
I see no example of this in the code for Linq. In .Net 6 Microsoft has added TryGetNonEnumeratedCount to try to compute the compute the count without enumeration; failing which they fallback on enumerating the collection, reference runtime code below:
runtime/Count.cs at dotnet/runtime · GitHub

If the IEnumerable returns an object, you could even have it return a status report every so often; you're not limited to returning a particular type.
It's as mentioned before is a state machine that is sub typed to IEnumerable; so barring guesstimate extension methods like TryGetNonEnumeratedCount; it requires enumeration to return a result. Select allows mapping from 1 encapsulated type to another; however the collection object (IEnumerable) is resolute for Linq.
Modifying the outer collection object; usually involves a traverse map; Linq AFAIK doesn't have any traversal extension methods; but it's fairly easy to build your own; re the building blocks for it are already there... Aggregate, SelectMany, and Select.
Side note: Arguably ToList and ToArray are basic traversals.

modified 28-Jan-22 1:52am.

Gerry Schmitz28-Jan-22 5:50

Re: How would you describe what an IEnumerable is to a bright student of C#

28-Jan-22 5:50

My references are to IEnumerable; you keep wanting to make it about LINQ, extensions, and generic types.

"I see no example". And that means what? You only believe in "prior art"?

You keep ignoring it's an "interface"; there is no "code" (LINQ or otherwise) that you can make claims about.

endo funk28-Jan-22 6:40

Re: How would you describe what an IEnumerable is to a bright student of C#

28-Jan-22 6:40

IEnumerable in the small in defined as just an interface; for example (.Net sourcecode):

internal interface IEnumerable {
  System.Collections.IEnumerator GetEnumerator();
}

Which does nothing but define the need for a method called GetEnumerator that returns a IEnumerator.
If that's what you mean by "My references are to IEnumerable"; then following "your rules"; you should avoid saying:

Count() is not a method of IEnumerable. A state machine can learn; you could probably get a good estimate the 2nd time around; all other things being equal.

If the IEnumerable returns an object, you could even have it return a status report every so often; you're not limited to returning a particular type.

..because "your rules" are stipulating that we ignored all the extension methods that are linked to IEnumerable and by distinction all the sub types of IEnumerable. Meaning you can't imply its a "state machine" because that would require you to first acknowledge the many extension methods for IEnumerable; because (if you missed it), the extension methods are the bridging code between IEnumerable as an interface to IEnumerable as a state machine i.e. part of what's commonly known as Linq.

What value is there in ignoring reality; Linq (for IEnumerable) exists as a large set of extension methods and sub types, etc.

Whilst its possible to create your own custom interfaces and types to tie in with Linq; that fact alone doesn't discount the large part of extension methods, ... that are considered .Net System.Linq and which is directly typed to the IEnumerable interface.

modified 28-Jan-22 12:49pm.

Gerry Schmitz28-Jan-22 7:29

Re: How would you describe what an IEnumerable is to a bright student of C#

28-Jan-22 7:29

You're fixated on one type of implementation; the implementation is up to the author. The author decides how, when, and what to "yield"; which is the whole point.

There are no "rules" .

endo funk28-Jan-22 8:46

Re: How would you describe what an IEnumerable is to a bright student of C#

28-Jan-22 8:46

Nope... but I'm sure we can agree its become rather pedantic.

An interface on its own is not a state machine; whilst the outcome of this type of implementation:

public class Id : IEnumerable {
    private readonly int Value;
    public Id(int value) => (Value) = (value);
    public IEnumerator GetEnumerator() {
      yield return Value;
    }
}

..technically yields a state machine; it does not make either IEnumerable or IEnumerator themselves a state machine... they're an implementation requirement.. but then (pedantically) we're getting into the weeds.

Plus this line of discussion has no bearing on the scope of the OP's question wrt to either:

...they've had no problems mastering 'Select and 'Where, 'Aggregate, etc.
What's so useful about deferred evaluation ?
why do I have to trigger iteration with 'Count() to see how many items it has...

In that context IEnumerable should not be considered without Linq; re Select and 'Where, 'Aggregate, etc. are a reference to Linq; and in that context it is the extension methods of Linq that bridge IEnumerable to Linq's implementation of a state machine, and similarly Count().

I also said...

Whilst its possible to create your own custom interfaces and types to tie in with Linq

It is not a requirement for custom Linq types to implement IEnumerable to use both syntactic styles of Linq. Nor would it be possible for product and sum types to conform to IEnumerable; because its an implementation that is tied to IEnumerator; an API designed for iterating over a collection.

BillWoodruff26-Jan-22 23:52

BillWoodruff

26-Jan-22 23:52

Gerry Schmitz wrote:
It's an interface to a forward reading state machine who's final implementation is left up to the author. The whole "internal list" concept is an illusion.

I like the analogy of a finite state machine, even if that ... for my students ... is an advanced concept that would probably distract them. i'd rephrase that as: "forward reading finite iteration machine."

"list ... illusion:" trying to prevent the student from assuming there is an internal list is a priority when i teach.

"custom tokenizers:" interesting ... i assume you mean 'Select based IEnumerables.

«The mind is not a vessel to be filled but a fire to be kindled» Plutarch

Re: How would you describe what an IEnumerable is to a bright student of C#

BillWoodruff26-Jan-22 23:39

BillWoodruff

26-Jan-22 23:39

@endofunk I am stupefied by the idea anyone guiding a "bright young student" would smother them in such a pile of tangled, redundant, overblown, irrelevant, rhetoric.

«The mind is not a vessel to be filled but a fire to be kindled» Plutarch

Re: How would you describe what an IEnumerable is to a bright student of C#

endo funk27-Jan-22 6:44

Re: How would you describe what an IEnumerable is to a bright student of C#

27-Jan-22 6:44

Tsk tsk... that type reaction is IMO simply a reflection on an inability to grok the underlying complexity; not on the information and/or its relevance and/or its accuracy. It was after-all intended to provide an indication of the complexity underlying the architecting of Linq.

Ps. Plus my response was to you, not the student -- a good teacher would be able to gauge the level of their audience and adjust the presentation of a complex topic to match that.

BillWoodruff27-Jan-22 23:44

BillWoodruff

27-Jan-22 23:44

the sweat trying to show off produces often tastes like non-sequiturs Smile | :)

«The mind is not a vessel to be filled but a fire to be kindled» Plutarch

Re: How would you describe what an IEnumerable is to a bright student of C#

endo funk28-Jan-22 2:11

Re: How would you describe what an IEnumerable is to a bright student of C#

28-Jan-22 2:11

An accusation of "showboating"; concludes with an incongruous use of a Latin phrase... Derp Roll eyes | :rolleyes:

No knowledge is ever to be wasted or ever to be despised
-- Joseph Needham

jschell30-Jan-22 7:08

Re: How would you describe what an IEnumerable is to a bright student of C#

30-Jan-22 7:08

endo funk wrote:
Tsk tsk... that type reaction is IMO simply a reflection on an inability to grok

Tsk tsk...That reply suggests one or more of the following
1. You did not read the original post
2. You didn't care what the original post asked.
3. You don't understand the context of what student might mean in the original post.

endo funk30-Jan-22 14:22

Re: How would you describe what an IEnumerable is to a bright student of C#

30-Jan-22 14:22

Nope. I quoted his words (question), and responded to that... a fact you apparently missed.

jschell17-Feb-22 10:22

Re: How would you describe what an IEnumerable is to a bright student of C#

17-Feb-22 10:22

endo funk wrote:
Nope. I quoted his words (question), and responded to that... a fact you apparently missed.

Your first response in this thread (not sub thread)...

Did not quote anything.

Apparently failed to understand what "bright young student" means.

endo funk17-Feb-22 14:22

Re: How would you describe what an IEnumerable is to a bright student of C#

17-Feb-22 14:22

Pssst.

...to avoid coming across as either an illiterate or a moron, or a bit of both.
I suggest you compare the Big Bolded Headings before each response paragraph; for example:

Is there any real reason to use an IEnumerable ?

What's so useful about deferred evaluation ?

etc...

...these are in fact direct quotes from the OP's post. Not sure how you missed that..

jschell30-Jan-22 7:07

Re: How would you describe what an IEnumerable is to a bright student of C#

30-Jan-22 7:07

I agree. Nothing in that would have been something that one should present in the context of the original question.

jschell30-Jan-22 7:06

Re: How would you describe what an IEnumerable is to a bright student of C#

30-Jan-22 7:06

endo funk wrote:
Yes; first off IEnumerable was Microsoft implementation of functional programming algebras for a List. It is essentially a monadic data type that encapsulates List; monadic data types are typically designed to conform to a fairly standard set of functional algebras i.e. providing a common API for computations.

Err..no it wasn't.

IEnumerable existed long before linq. It wasn't added to the libraries in any way shape or form to support functional programming.

Now one could make an argument that they should have started from scratch when they did add linq but that is an argument and not a statement of fact.

One could also argue that they should have redid IEnumerable to support linq. However that argument would have been completely wrong since C# was a practical language not a university exercise. Redoing it would have broken the entire existing body of practical work that was using it. And would have been exactly the wrong thing to do for any language when attempting to enhance and not replace that language.

endo funk30-Jan-22 14:21

Re: How would you describe what an IEnumerable is to a bright student of C#

30-Jan-22 14:21

Extension methods... apparently so obvious that you missed it.

jschell17-Feb-22 10:28

Re: How would you describe what an IEnumerable is to a bright student of C#

17-Feb-22 10:28

endo funk wrote:
Extension methods... apparently so obvious that you missed it.

Apparently you do not know what you said...

"first off IEnumerable was Microsoft implementation of functional programming algebras for a List. "

And again, no it was not. It existed long before anything associated with functional programming was in the language.

Refutation of that statement has nothing to do with extension methods which were added long after IEnumerable. I can only suppose that you are claiming that they could have used extension methods when implementing linq. Perhaps. But still that has nothing to do with your original incorrect statement.

endo funk17-Feb-22 14:38