|
Ah, interesting. Very good info. So back to the Queue, it is.
There are always trade-offs.
I was sorting the Dictionary first via
waitingFiles.OrderByDescending(of => of.Value.FileCreated).First();
But that is really intensive because from what I can tell those LINQ objects are instantiated to search for the one that matches.
So, I was hoping (and it is really fast and nice) that since I load the data into the Dictionary in ascending order (oldest files first) that I could count on just calling First().
I mean it does work now. I was actually testing the order that the files are then processed.
However, I understand that it is not guaranteed.
Interesting and why I'm trying not to make assumptions about which is better ( Dictionary or Queue ) in this instance.
|
|
|
|
|
Question, do you need to check all or any of the elements in the list, or is it enough to check the last one?
If so you would do better off with a linked list.
There are no concurrent linked lists though. Apparently it was around in the beta versions of dotNet 4.0 but it was left out from the release version as there were no advantages over using a normal linked list with locking.
But a simple lock would still be faster since you don't need to enumerate the collection.
|
|
|
|
|
Thanks for chiming in on the conversation. The challenge I have here is that I need to insure the item is not currently anywhere in the collection -- an item may have been loaded at any location.
There are probably other ways around this but just attempting to do it the easiest, yet most correct way which uses the least resources.
The collection is used in two ways :
1. caller will request the oldest unique item
2. collection will periodically load newer items into itself
The interesting thing about this is that the Queue makes on of those requirements easy and the Dictionary makes the other requirement easy.
The first one is easy with the Queue since you just get the first in item.
The second one is difficult with a Queue (there is no key) so you have to determine that the object is not already contained in the Queue for every item, each time the items are retrieved from db.
The second one is easy with a Dictionary since you can do TryAdd() with your unique key and it fails fast if the item has already been added).
However, the first one is more difficult with a Dictionary because the items are not (necessarily) ordered in a significant way and we need to get the oldest item.
It's an interesting problem.
|
|
|
|
|
If you can afford the extra memory usage, the simplest option would probably be a custom class to store the data twice - once in a HashSet<T> , and once in a Queue<T> .
You'd obviously need to add your own locking code to ensure it was thread-safe.
Off the top of my head, something like this might work:
public sealed class UniqueQueue<T> : IEnumerable<T>, IDisposable
{
private readonly Queue<T> _queue = new Queue<T>();
private readonly HashSet<T> _set = new HashSet<T>();
private readonly ReaderWriterLockSlim _lock = new ReaderWriterLockSlim();
public int Count
{
get
{
bool lockTaken = false;
try
{
lockTaken = _lock.TryEnterReadLock(Timeout.Infinite);
if (!lockTaken) throw new InvalidOperationException();
return _queue.Count;
}
finally
{
if (lockTaken)
{
_lock.ExitReadLock();
}
}
}
}
public void Dispose()
{
_lock.Dispose();
}
public T Dequeue()
{
bool lockTaken = false;
try
{
lockTaken = _lock.TryEnterWriteLock(Timeout.Infinite);
if (!lockTaken) throw new InvalidOperationException();
T oldestItem = _queue.Dequeue();
_set.Remove(oldestItem);
return oldestItem;
}
finally
{
if (lockTaken)
{
_lock.ExitWriteLock();
}
}
}
public void AddRange(IEnumerable<T> itemsToAdd)
{
if (itemsToAdd == null) throw new ArgumentNullException(nameof(itemsToAdd));
bool lockTaken = false;
try
{
lockTaken = _lock.TryEnterWriteLock(Timeout.Infinite);
if (!lockTaken) throw new InvalidOperationException();
foreach (T item in itemsToAdd)
{
if (_set.Add(item))
{
_queue.Enqueue(item);
}
}
}
finally
{
if (lockTaken)
{
_lock.ExitWriteLock();
}
}
}
public UniqueQueueEnumerator GetEnumerator()
{
T[] copyOfItems = Array.Empty<T>();
bool lockTaken = false;
try
{
lockTaken = _lock.TryEnterReadLock(Timeout.Infinite);
if (!lockTaken) throw new InvalidOperationException();
copyOfItems = new T[_queue.Count];
_queue.CopyTo(copyOfItems, 0);
}
finally
{
if (lockTaken)
{
_lock.ExitReadLock();
}
}
return new UniqueQueueEnumerator(copyOfItems);
}
IEnumerator<T> IEnumerable<T>.GetEnumerator() => GetEnumerator();
IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
public struct UniqueQueueEnumerator : IEnumerator<T>
{
private readonly T[] _items;
private int _position;
public UniqueQueueEnumerator(T[] items)
{
_items = items;
_position = -1;
}
void IDisposable.Dispose()
{
}
public void Reset()
{
if (_items == null) throw new InvalidOperationException();
_position = -1;
}
public bool MoveNext()
{
if (_items == null) return false;
return ++_position < _items.Length;
}
public T Current
{
get
{
if (_items == null) throw new InvalidOperationException();
if (0 > _position || _position >= _items.Length) throw new InvalidOperationException();
return _items[_position];
}
}
object IEnumerator.Current => Current;
}
}
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
This is a very good suggestion and I saw something very similar at StackOverflow.
But, as you mentioned I was attempting to keep the memory size low too and it is interesting that this does store the objects in two collections.
Also, I am actually using the ConcurrentQueue and ConcurrentDictionary which are thread-protected automatically so that would solve that part of the problem.
Thanks for the input and discussion.
|
|
|
|
|
raddevus wrote: I am actually using the ConcurrentQueue and ConcurrentDictionary which are thread-protected automatically
Ish.
Modifications to one of those collections will automatically be thread-safe. But if you're trying to read or update the two in a single atomic operation, you'll still need your own locking mechanism.
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
I had a feeling it wouldn't be that simple.
Richards suggestion would've been my second one, minus the code sample though. (impressive that!)
Second question though, you wrote in another post that you load the objects from a database.
Are the objects in the database not unique? How do you ensure the uniqueness if you have popped an object from the list?
Or is the uniqueness coming from the identity column in the database? If so you could still use the linked list. Using a paged and sorted query.
modified 9-Feb-18 13:14pm.
|
|
|
|
|
raddevus wrote: I needed to know that when I went to determine if the queue already had the item, if it would return faster when the item was found at the top (next item out) versus found at the bottom (last out position).
I assumed, but wasn't sure.
This code gives a good definitive answer
You forgot the important qualifier through -- this code gives a good definitive answer for build XXX of the library. In the next build, the implementer may find a bug with the way they're doing things and have to change it, altering the performance. If you're writing code that depends that much on the performance of some class, you need to write the class yourself.
|
|
|
|
|
|
Without showing code there's little interesting here (especially since "nothing" was fixed ) ...
There's 'yer chuckle though.
kmoorevs wrote: fixed That's like some sort of like quantum foam, right?
if i'm not mistaken
|
|
|
|
|
for the pun.
"Go forth into the source" - Neal Morse
|
|
|
|
|
Is decimal portion not required?
TOMZ_KV
|
|
|
|
|
Tomz_KV wrote: Is decimal portion not required?
Yes, which was the problem...division casts to the datatype with highest precision for parameters in the equation. (I'm probably saying that wrong!)
The correction was to convert one of the parameters in the division to a decimal.
I only posted here 'cause we no longer have a 'hall of shame'!
"Go forth into the source" - Neal Morse
|
|
|
|
|
List<object> ccList = new List<object>();
ccList.Add(cc);
foo.Items = ccList.ToArray();
- Create a list.
- Add the item to the list.
- Convert the list to an array.
Latest Article - Code Review - What You Can Learn From a Single Line of Code
Learning to code with python is like learning to swim with those little arm floaties. It gives you undeserved confidence and will eventually drown you. - DangerBunny
Artificial intelligence is the only remedy for natural stupidity. - CDP1802
|
|
|
|
|
Or:
var ccList = Enumerable.Range(0,1);
var objects = ccList.Select(x=> (object)cc);
foo.Items = objects.ToArray();
|
|
|
|
|
That code will work, but, like my signature says, that doesn't mean it is a good way to do it.
Just because the code works, it doesn't mean that it is good code.
|
|
|
|
|
BillW33 wrote: That code will work, but, like my signature says, that doesn't mean it is a good way to do it. And I suppose this is exactly why he is posting it in this forum
M.D.V.
If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about?
Help me to understand what I'm saying, and I'll explain it better to you
Rating helpful answers is nice, but saying thanks can be even nicer.
|
|
|
|
|
Nelek wrote: And I suppose this is exactly why he is posting it in this forum
Darn, I'm so predictable.
Latest Article - Code Review - What You Can Learn From a Single Line of Code
Learning to code with python is like learning to swim with those little arm floaties. It gives you undeserved confidence and will eventually drown you. - DangerBunny
Artificial intelligence is the only remedy for natural stupidity. - CDP1802
|
|
|
|
|
great minds think alike
M.D.V.
If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about?
Help me to understand what I'm saying, and I'll explain it better to you
Rating helpful answers is nice, but saying thanks can be even nicer.
|
|
|
|
|
Thanks for sharing this great snippet of codez, bro! It will help me speed up my development work.
Oh sanctissimi Wilhelmus, Theodorus, et Fredericus!
|
|
|
|
|
Someone who learned to "program in C# in 21 days"?
Bastard Programmer from Hell
If you can't read my code, try converting it here[^]
|
|
|
|
|
How come nobody came up with the one line solution ?
foo.items = new List<object>() { cc }.ToArray();
“That which can be asserted without evidence, can be dismissed without evidence.”
― Christopher Hitchens
modified 24-Jan-18 10:07am.
|
|
|
|
|
Did you do APL when you were a boy?
|
|
|
|
|
Never heard of it until you asked - so I had to look at wikipedia.
“That which can be asserted without evidence, can be dismissed without evidence.”
― Christopher Hitchens
|
|
|
|
|
As the Wikipedia article says: "In nearly all versions of APL, it is theoretically possible to express any computable function in one expression, that is, in one line of code" - the one-liner Game of Life is an excellent example.
I learned APL as early as 1975-76, on one of the worlds earliest "PCs", the IBM 5100 - APL was almost exclusively an IBM thing then - and it was a well known joke that IBM was working on writing the entire OS360 (the first operating system for the 360/370 mainframe series) in a single line of APL
There are still elements of APL that I miss, in particular the "workspace" concept: You do your stuff in a sandpit where you throw in functions, variables and whatever stuff, and throw them out when no longer needed - while the "program" (workspace) running. On the 5100, you could save the entire workspace on disk in its current state, or you could declare selected variables as persistent. There were file system operations, but you rarely needed it. There have been languages with similar concepts (I believe Smalltalk comes close), but mainline programming 40 years later still are based on concepts that could be compared to "Of course you have to tell how much space to reserve for a file before starting to use it" (that's essentially how IBMs mainframe file systems were at the time). APL was so much more flexible and dynamic...
I'm getting carried away. Maybe I tonight should curl up in my recliner in front of my fireplace with the old APL book, memorizing what the world was like when I was a youngster...
|
|
|
|