|
Yes, I have read it quite a bit, should mention that the Lee Cambell also has some valuable answers on the MSDN page as well:
Msdn forums - Reactive Extensions (Rx)[^]
However they all seem to have most of the activity from 2012 or before.
|
|
|
|
|
Hmm, the Scheduler.Default and the NewThreadScheduler.Default seem to be the exact same command? They both live in the System.Reactive.Concurrency namespace, and I get the same error, that its not the UI Thread.
|
|
|
|
|
If you want the UI thread scheduler for the ObserveOn I think what you want instead is to use the .ObserveOnDispatcher() extension method, in the System.Reactive.Linq namespace.
This, apparently (from looking at the version 2 source!), is the recommended way to use the DispatcherScheduler.Current scheduler.
================
[Edit: I see that you already found this!]
"Fairy tales do not tell children the dragons exist. Children already know that dragons exist. Fairy tales tell children the dragons can be killed."
- G.K. Chesterton
|
|
|
|
|
Either Im missing something here, or Im doing something wrong?
var buttonClick = Observable.FromEventPattern<RoutedEventHandler, RoutedEventArgs>(
h => btnMainWindow.Click += h,
h => btnMainWindow.Click -= h);
var EnumerableTest = Enumerable
.Range(1, 10)
.ToObservable()
.ObserveOnDispatcher()
.SubscribeOn(NewThreadScheduler.Default);
var DisposableEnumerable = EnumerableTest
.Subscribe(evt =>
txtMainWindow.Text += evt + Environment.NewLine
);
buttonClick.Subscribe(evt =>
{
DisposableEnumerable.Dispose();
});
There is no change if I replace:
.SubscribeOn(NewThreadScheduler.Default)
with
.SubscribeOnDispatcher()
If I understood things, the SubscribeOn should specify where the disposable lives, so if I set it on a different thread, what happens then?
|
|
|
|
|
I have a class with 11 string properties, together with an IEnumerable (Initialised to a generic List) property, each item of which has an int property, a long property and 3x Nullable int (int?) properties. The typical number of items in this sub property list is 4-6.
I have an IEnumerable containing ~700K of these, and I'm breaking them out into batches of 1000 using the following code;
var enumerator = items.GetEnumerator();
enumerator.MoveNext();
List<object> entityBatch = new List<object>();
DateTime start = DateTime.Now.Ticks;
for (int iterStep = 0; iterStep < batchSize; iterStep++)
{
entityBatch.Add(enumerator.Current);
enumerator.MoveNext();
}
DateTime end = DateTime.Now.Ticks;
However, to process this loop is taking around 12 seconds (as measured using the start / end tick count)....that in itself is bad, but the real question is can anybody shed any light on the fact that if I remove the list property, the same data is processed in 0.25 seconds - so several orders of magnitude more quickly?
I realise there will be some casting going on from objects to my actual types, but the difference makes me think I'm missing something pretty significant.
C# has already designed away most of the tedium of C++.
modified 24-Feb-16 9:33am.
|
|
|
|
|
Don't measure the performance of your code using DateTime; use the Stopwatch class[^] instead.
Did you make sure to "warm up" the code before you timed it, and run the test over many iterations?
Since you know the maximum number of items you'll be adding to the list, you should set its capacity when you create it:
List<object> entityBatch = new List<object>(batchSize);
If you don't set the capacity, it starts at 4 and doubles each time it runs out of space.
You need to check the value returned from enumerator.MoveNext; if it returns false, you've reached the end of the sequence. Your code currently continues adding the final items from the sequence to the list until you reach the batch size.
You should consider using proper generic types, rather than a List<object>; that way, you avoid having to cast the items back to the correct type when you read the batches.
It's probably best to wrap this sort of thing up as a extension method. Here's the one I use:
public static class EnumerableExtensions
{
private static IEnumerable<IEnumerable<TSource>> BatchIterator<TSource>(IEnumerable<TSource> source, int size)
{
int count = 0;
var bucket = new TSource[size];
foreach (var item in source)
{
bucket[count] = item;
checked
{
count++;
}
if (count == size)
{
yield return bucket.Select(x => x);
bucket = new TSource[size];
count = 0;
}
}
if (count != 0)
{
yield return bucket.Take(count);
}
bucket = null;
}
public static IEnumerable<IEnumerable<TSource>> Batch<TSource>(this IEnumerable<TSource> source, int size)
{
if (source == null) throw new ArgumentNullException(nameof(source));
if (size < 1) throw new ArgumentOutOfRangeException(nameof(size));
return BatchIterator(source, size);
}
}
If you can, try to reproduce the problem in a small example, and either post it here, or create a .NET Fiddle[^] and post the link.
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
Hi Richard - many thanks for the quick reply...
Richard Deeming wrote: Don't measure the performance of your code using DateTime; use the Stopwatch class[^] instead.
I'm not measuring the performance per-se, just getting an idea, and in any case that part of the code won't be in the prod version...
Richard Deeming wrote: Did you make sure to "warm up" the code before you timed it, and run the test over many iterations?
Absolutely - same perf over many MANY runs 
Richard Deeming wrote: Since you know the maximum number of items you'll be adding to the list, you should set its capacity when you create it:
List<object> entityBatch = new List<object>(batchSize);
If you don't set the capacity, it starts at 4 and doubles each time it runs out of space.
Valid point - made the change and no real effect - still around 12s
Richard Deeming wrote: You need to check the value returned from enumerator.MoveNext; if it returns false, you've reached the end of the sequence. Your code currently continues adding the final items from the sequence to the list until you reach the batch size.
Again, valid point, but in this instance it's not the problem - I can be certain that it's not at the end.
Richard Deeming wrote: You should consider using proper generic types, rather than a List<object>; that way, you avoid having to cast the items back to the correct type when you read the batches.
Also a valid point, but not possible in this instance - it's in an abstract base class, which operates over several types of similar but not related types and exists in a (necessarily) singleton class....my thinking is leading me to the point that it may well be an area to look at, but given that IEnumerator.Current returns an object type anyway I'm not sure.
Richard Deeming wrote: It's probably best to wrap this sort of thing up as a extension method
Ultimately that's a possibility, but until I can bottom out the cause of the several orders of magnitude difference it's sort of neither here or there - appreciate the comment however.
Many thanks again - I'll have to keep digging into it and see what's what!
C# has already designed away most of the tedium of C++.
|
|
|
|
|
RichardGrimmer wrote: not possible in this instance - it's in an abstract base class, which operates over several types of similar but not related types and exists in a (necessarily) singleton class....
So you can't do something like this?
public abstract class MyBaseClass
{
public void ProcessList<T>(IEnumerable<T> listToProcess) where T : MyBaseClass
{
...
}
}
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
Unfortunately not (admittedly not thought it all the way through, but the types in question don't share a common base class)....I do appreciate that I could re-structure so that they do however, but again, given the fact that the problem manifests itself a s difference of several orders of magnitude, I'm in a "bigger fish to fry" type of situation - thanks again for the suggestion however - I may well end up doing something similar "just for grins" and see what happens....just for background, the code is used to index items from several different data sources into an Elastic Search instance, hence the object type being used....
C# has already designed away most of the tedium of C++.
|
|
|
|
|
public class ChildItem
{
public ChildItem()
{
Count = int.MaxValue;
LongCount = long.MaxValue;
Nullable2 = int.MaxValue - 3;
Nullable3 = Nullable2.Value - 2;
}
public int Count { get; set; }
public long LongCount { get; set; }
public int? Nullable1 { get; set; }
public int? Nullable2 { get; set; }
public int? Nullable3 { get; set; }
}
public class MasterList
{
private List<ChildItem> children = new List<ChildItem>();
public MasterList()
{
for (int i = 0; i < 6; i++)
{
children.Add(new ChildItem());
}
Master1 = "Hi";
Master2 = "Hi";
Master3 = "Hi";
Master4 = "Hi";
Master5 = "Hi";
Master6 = "Hi";
Master7 = "Hi";
Master8 = "Hi";
Master9 = "Hi";
Master10 = "Hi";
Master11 = "Hi";
Master12 = "Hi";
}
public IEnumerable<ChildItem> Children => children;
public string Master1 { get; set; }
public string Master2 { get; set; }
public string Master3 { get; set; }
public string Master4 { get; set; }
public string Master5 { get; set; }
public string Master6 { get; set; }
public string Master7 { get; set; }
public string Master8 { get; set; }
public string Master9 { get; set; }
public string Master10 { get; set; }
public string Master11 { get; set; }
public string Master12 { get; set; }
}
public class TestIt
{
private List<MasterList> items = new List<MasterList>(700000);
public TestIt()
{
for (int i = 0; i < 700000; i++)
{
items.Add(new MasterList());
}
Console.WriteLine("Ready");
}
public void StartTest(string text = "Dummy run")
{
var enumerator = items.GetEnumerator();
enumerator.MoveNext();
List<object> entityBatch = new List<object>();
Stopwatch sw = new Stopwatch();
sw.Start();
for (int iterStep = 0; iterStep < 700000; iterStep++)
{
entityBatch.Add(enumerator.Current);
enumerator.MoveNext();
}
sw.Stop();
Console.WriteLine(text);
Console.WriteLine("Total time {0}", sw.ElapsedMilliseconds);
}
}Running this through a few times consistently returns a time of less than 50 milliseconds.
This space for rent
|
|
|
|
|
Hi Pete,
So the classes in question represent a name and a collection of dates of birth associated with the name...classes below;
public partial class DateOfBirth
{
public int DateOfBirthId { get; set; }
public Nullable<int> Day { get; set; }
public Nullable<int> Month { get; set; }
public Nullable<int> Year { get; set; }
public long PepDeskRecordId { get; set; }
}
public class ExpandedFoo
{
public string FooId { get; set; }
public string BarId { get; set; }
public string Title { get; set; }
public string Gender { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public string FullName { get; set; }
public string OtherNames { get; set; }
public string Country { get; set; }
public string BlahId{ get; set; }
public string CombinedName { get; set; }
public IEnumerable<DateOfBirth> DateOfBirths { get; set; }
public ExpandedFoo()
{
this.DateOfBirths = new List<DateOfBirth>();
}
}
(Apologies for the naming, edited for a variety of corporate reasons!)
All in all pretty basic classes, and similar to those you've put together...
C# has already designed away most of the tedium of C++.
|
|
|
|
|
OK - I've cracked the issue, but I'm not even going to pretend I know why...it appears it was down to how I was populating my lists....
The pop code originally looked like;
var expandedEntities = Foontities.Select(entity => new ExpandedFoo
{
FooId = entity.fooId.ToString(CultureInfo.InvariantCulture),
BarId = entity.BarId.ToString(CultureInfo.InvariantCulture),
Title = entity.Title,
Gender = entity.Gender,
FirstName = entity.FirstName,
LastName = entity.LastName,
FullName = entity.FullName,
OtherNames = entity.OtherNames,
CombinedName = entity.CombinedName,
Country = entity.Country,
BlahId= entity.BlahId.ToString(),
DateOfBirths = (from DateOfBirth dob in dateOfBirths
where dob.FooId== entity.FooId
select new FooDateOfBirth()
{
DateOfBirthId = dob.DateOfBirthId,
Year = dob.Year,
Month = dob.Month,
Day = dob.Day
}).ToList()
});
Which was causing the 12s time to extract 1000 records, but (as mentioned, this is to index into Elastic Search), converting to Json and building a bulk command using PlainElasic.net was <.2s....
Removing the ToList(), to make the DOB population code;
DateOfBirths = (from DateOfBirth dob in dateOfBirths
where dob.PooId == entity.FooId
select new FooDateOfBirth()
{
DateOfBirthId = dob.DateOfBirthId,
Year = dob.Year,
Month = dob.Month,
Day = dob.Day
})
Fixed the extraction problem and took the time to milliseconds - BUT caused the Json conversion and bulk command building to take (coincidentally) 12s...
HOWEVER, doing;
DateOfBirths = (from DateOfBirth dob in dateOfBirths
where dob.FooId == entity.FooId
select new FooDateOfBirth()
{
DateOfBirthId = dob.DateOfBirthId,
Year = dob.Year,
Month = dob.Month,
Day = dob.Day
}) as List<DateOfBirth>
Made it run in the region of .001s for both the extract and the convert and bulk command creation....
I think I need to look at the IL and see what's going on!
Many thanks to both you and Richard for all your help - if I can figure out the issue / differences, I'll post back and let you both know!
Thanks again gents!
C# has already designed away most of the tedium of C++.
|
|
|
|
|
Just as a test, go back to your ExoandedFoo and change the IEnumerable<DataOrBirth> into a List<DataOfBirth> and populate it using the ToList() method.
This space for rent
|
|
|
|
|
Hi Pete,
Actually I was all sorts of wrong - on deeper investigation, the as statement was failing silently (as it of course should) - a cast rather than the as did indeed throw exceptions left right and centre.
Not a problem, I've taken a different tack now...I may at a later date circle back around and look at it in more detail, but for now it'll do as it is.
Many MANY thanks for your time on this one
C# has already designed away most of the tedium of C++.
|
|
|
|
|
Hi Richard,
I've just posted an update above - very odd, but thought you might like to take a look - it was (apparently) down to how I was populating my data...
Many thanks for all your help and suggestions!
C# has already designed away most of the tedium of C++.
|
|
|
|
|
Dear All,
I am making a C# window program with a function playing a local video using window medial Player. I need a function like capture a image frame from the playing video using a button click that will transfer capture directly into a picturebox, and then save into local desk with a button(save button).
Thank you for your help in advance,
|
|
|
|
|
And what code do you have so far? What, exactly, are you expecting us to provide? I can make an awful lot of assumptions here but assumptions are a bad thing.
This space for rent
|
|
|
|
|
Pete O'Hanlon wrote: I can make an awful lot of assumptions here
And the first one is probably correct. 
|
|
|
|
|
|
So I'm still playing around with Rx, and usually, I found it very neat:
private void Window_Loaded(object sender, RoutedEventArgs e)
{
var Moves = Observable.FromEventPattern<MouseEventArgs>(this, "MouseMove").Select(evt=>evt.EventArgs.GetPosition(this));
Moves.Subscribe(evt => {
this.Title = evt.ToString();
});
}
I do however find it more useful in most cases to send the value using the Select statement. But when I pass this around, I can't unsubscribe to it like so:
private void Window_Loaded(object sender, RoutedEventArgs e)
{
var Moves = Observable.FromEventPattern<MouseEventArgs>(this, "MouseMove").Select(evt=>evt.EventArgs.GetPosition(this));
Moves.Subscribe(evt => {
this.Title = evt.ToString();
});
Moves.Dispose();
}
How can I dispose of the subscribtion?
|
|
|
|
|
Nevermind I'm stupid 
private void Window_Loaded(object sender, RoutedEventArgs e)
{
IObservable<Point> Moves = Observable.FromEventPattern<MouseEventArgs>(this, "MouseMove").Select(evt=>evt.EventArgs.GetPosition(this));
var DisposableSubscribtion = Moves.Subscribe(evt => {
this.Title = evt.ToString();
});
DisposableSubscribtion.Dispose();
}
|
|
|
|
|
public string ConfidenceConnected(string filename)
{
string ConfidenceConnected = Path.GetDirectoryName(AppDomain.CurrentDomain.BaseDirectory) + "\\image\\ConfidenceConnected.exe";
int seedX = 100;
int seedY = 78;
string file = Path.GetDirectoryName(AppDomain.CurrentDomain.BaseDirectory) + "\\image\\" + filename + " output.dcm" + " " + seedX + " " + seedY;
//string Out = " output"+ filename+".dcm";
string cmd = "/c" + ConfidenceConnected + file;
System.Diagnostics.Process process = new System.Diagnostics.Process();
process.StartInfo.UseShellExecute = false;
process.StartInfo.FileName = "cmd.exe";
process.StartInfo.Arguments = cmd;
process.StartInfo.RedirectStandardOutput = true;
string return_string = " ";
try
{
process.Start();
return_string= process.StandardOutput.ReadToEnd();
}
catch (Exception e)
{
return_string = e.Message;
}
return return_string;
}
|
|
|
|
|
And what problem do you have? That's just a code dump.
This space for rent
|
|
|
|
|
Spaces are a problem here.
First off, you need a space between ConfidenceConnected and file, and before ConfidenceConnected:
string cmd = "/c" + ConfidenceConnected + file;
Otherwise your command string becomes:
/cC:\path\image\ConfidenceConnected.exeC:\path\image\filename output.dcm 100 78
Second, if you want paths with spaces, they need to be delimited with double quotes:
/c "C:\path\image\ConfidenceConnected.exe" "C:\path\image\filename output.dcm" 100 78
But please try to use Path.Combine[^] instead of building your paths manually!
It's a good idea to build your parts, then finalize with a String.Format:
string cmd = string.Format("/c \"{0}\" \"{1}\" {2} {3}", ConfidenceConnected, file, seedX, seedY);
It makes it a lot easier to see the overview of what you are trying to achieve.
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
|
|
|
|
|
The other day I posted about a problem[^] with FileSystemWatcher raising events before the file is completely copied into the watched folder.
Thanks for Gerry Schmitz[^] I think I've come up with a workable solution:
I would appreciate any comments on this idea.
[1] The FileSystemWatcher detects the file. It's a large file that take a while to get fully copied into the folder:
private void OnCreated(object sender, FileSystemEventArgs e)
{
if (HasAnotherFileEventOccuredRecently(e.FullPath))
{
return;
}
QueueFile(e);
}
The HasAnotherFileEventOccuredRecently method is called to see if this is the first time a FileSystemWatcher event has been raised for the file. Multiple events are raised for the file, so if any events for the file have already been called, then ignore them. I didn't include this code for brevity.
[2] Next QueueFile is called:
private void QueueFile(FileSystemEventArgs fileSystemEventArgs)
{
var worker = new BackgroundWorker();
worker.DoWork += Worker_DoWork;
worker.RunWorkerCompleted += Worker_RunWorkerCompleted;
_queue.Add(worker);
worker.RunWorkerAsync(fileSystemEventArgs);
}
The BG Worker snippets...
private void Worker_DoWork(object sender, DoWorkEventArgs e)
{
var args = (FileSystemEventArgs)e.Argument;
if (args != null)
{
while (!IsFileAvailable(args.FullPath))
{
}
}
e.Result = args;
}
and
private void Worker_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
var args = (FileSystemEventArgs)e.Result;
var worker = _queue.FirstOrDefault(x => x == sender);
if (worker != null)
{
worker.Dispose();
_queue.Remove(worker);
}
this.OnCreated(args);
}
[3] and finally the IsFileAvailable method:
public bool IsFileAvailable(string filename)
{
var results = false;
try
{
using (var inputStream = File.Open(filename, FileMode.Open, FileAccess.Read, FileShare.None))
{
results = inputStream.Length > 0;
}
}
catch (Exception)
{
results = false;
}
return results;
}
If it's not broken, fix it until it is
modified 22-Feb-16 16:59pm.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
