|
What you might want to do is outlyer detection, and just encode those results with some kind of symbol on the chart like a red line or something like that. Or, as others have said, just drop them and leave them as gaps.
Also, you can compute the median and standard deviation for your series, and then just plot everything within a few standard deviations of the median, which should be pretty close to the same thing.
|
|
|
|
|
How do I calculate the median. That would be a straight line - I want a 'median' between lows and highs that moves up and down with the lows and highs. I suppose I could do it point for point and calculate a median curve.
|
|
|
|
|
Ok, that gets into how you display the data. For a single static graph, I'd do it for the entire graph. That would give the bounds for the entire image.
If I was calculating a rolling average, I'd plot that as a curve overlay on top of the existing data. Check out various stock charts for that kind of visualization.
To calculate a rolling average, you just assign a window to each point, for example the last 30 data points and calculate the statistics on that window. The trick is really to use a window that makes sense for the underlying data.
|
|
|
|
|
By a window, you mean calculate the average low for 30 points, the same for high, and the mean of those is one point in your running central mean curve?
|
|
|
|
|
for the 30 point window it would be just calculate the stats for n-30 to n, or n-15 to n+15, or whatever window makes sense and graph those values for n. It's just a sliding window to do your calculations.
|
|
|
|
|
Also, if the y scale varies across the width of the graph according to the data, that is going to be very difficult to interpret, as you will be "straightening" out the data and none of the variation will be absolute.
Sometimes it's just easier to us a log scale for wildly varying data.
|
|
|
|
|
It's not at all widely varying. It's mains voltage over the day, and most samples vary by7 a few volts on 240V. It's juts a very few funny points that I think I will just ignore. You can't really depend ona graph as an accurate source of data, and I have a report of all the exact values.
|
|
|
|
|
Personally I would start in Excel - just to be able to play easily with various stats functions and see the result easily.
To determine your Y bounds:
Since we are looking at mains voltage, the maximum is of interest, so your Y max should be based on the maximum of your data.
Take the average M.
Calculate the standard deviation s.
In Excel take M - Ns for the lower bound - play with N to find a reasonable value. I suspect that 3 will probably work.
Now extract all those points below the lower bound and put them aside for later.
Now take the minimum of the remaining points round it to the nearest 5, 10 etc. based on the final range that you use.
Round the maximum in the same way.
(Note: depending on the measuring equipment you might get spurious high values too, you can do the same with the maximum bound if you want, but might need a different value for N)
This should give you the plot that you want.
Now, those pesky low values....
These should figure as points along the bottom of the graph, downward pointing arrows would be nice, since we need to be scientifically correct and show that the data has been "massaged".
Set their value to Ymin plus 7% of the Y range for example.
Hope this helps
|
|
|
|
|
Since the zero readings are anomalies, then I would exclude them from the chart data (thereby cleaning up the chart display), and keep a separate record of the zero readings viewable in a table with time of reading.
That way you get a clean chart & a separate table of anomalous values if you need to look at them. Or you could chart the zero readings in another way - number of zero readings per hour against time perhaps?
|
|
|
|
|
If you want to show all the data you could try splitting the chart. Split the data in two and plot the graphs stacked. An ASCII representation of what you would aim for is:
250 V |-------------------------
240 V |-------------------------
230 V |-------------------------
~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~
1 V |-------------------------
0 V |-------------------------
|
|
|
|
|
Chart everything and then use the chart's axis controls to limit the low and high values that are shown on the chart.
|
|
|
|
|
Removing zeroes is not good practice unless you have a legitimate reason to discard them (e.g. the measuring device failed). Can't you use a log scale or simply calculate axis span to ignore the lowest 10% of values or those values more than 2 standard deviations below the mean or some other systematic criterion? As for the second question, can't you use some sort of best-fit curve along with an indication of range, such as standard deviation to determine lower and upper bounds, or better still the standard error of the mean?
|
|
|
|
|
I updated my graphics package from PaintShop Pro 9 (Lovely - but 10 years old, and the last version before Corel bought it) to *gulp* a nearly-the-latest version: Corel PaintShop Pro X6.
And I bought a book to help me learn the new version. And I've found time to read it. Well, I've found time to read the first chapter. OK, about a third of the first chapter.
And today, I found I had to knock up a quick "Good Luck in You New Job!" card, so I dithered. Do I use the old version, and know it will work and be an easy job? Or do I bite the bullet and throw myself into the new one. Bearing in mind it's a Corel product, and they had a (well deserved) reputation for mucking up working features and adding bugs? Lots of Bugs? A Corel version is a bit like a Lotus car: Loads Of Trouble, Usually Serious...which is why I hung on to PSP9 for so long...
Bite the bullet.
And...all my old, familiar shortcuts worked. Layering still worked. Print layouts still worked. Things actually worked better than the old version.
And: It Didn't Crash 
OK, I didn't do anything too complicated...but in the days of "Corel Photo Paint" and "Corel Draw!" just saving your work at the wrong moment was dangerous. 
I'm impressed!
I should have done it sooner...but now I have even less impetus to read the damn book! 
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
|
|
|
|
|
I can't believe it. They probably now have sneakier ways than they used to. Now they wait until you have gained some confidence before...
OriginalGriff wrote: And: It Didn't Crash 
I should also have ended my post below this way. 
The language is JavaScript. that of Mordor, which I will not utter here
Panza llena Corazon Contento
|
|
|
|
|
CDP1802 wrote: They probably now have sneakier ways than they used to
Damn! I should have guessed...just lulling me into a false sense of security... 
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
|
|
|
|
|
I strayed from Corel years ago but I tried touching up some photos using PSP about 4 months ago and it had a lot of nice features but was slower than snot on a pump handle in January.
New version: WinHeist Version 2.1.0 Beta
Have you ever just looked at someone and knew the wheel was turning but the hamster was dead?
Trying to understand the behavior of some people is like trying to smell the color 9.
I'm not crazy, my reality is just different than yours!
|
|
|
|
|
I have PaintShop Pro 7 and PaintShop Pro X5. I still prefer 7.
|
|
|
|
|
I love PSP9 - but it's a PITA if you are doing Win7 screenshots: since it doesn't support Aero, it's a case of
1) Take screenshot
2) Start PSP9 (which switches to "classic")
3) Check screenshot and save
4) Close PSP9 (to return to Aero display)
5) Repeat from 1.
Wastes so much time! 
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
|
|
|
|
|
OriginalGriff wrote: "Good Luck in You New Job!"
I hope you fix the typo in your final product.
Within you lies the power for good - Use it!
|
|
|
|
|
|
OriginalGriff wrote: And I bought a book to help me learn the new version. And I've found time to read it.
You read the manual?!?
Jeeze, there's something dreadfully amiss when a developer reads a manual.
I wanna be a eunuchs developer! Pass me a bread knife!
|
|
|
|
|
Well...I tried...and it's not the Corel manual, if that helps: PaintShop Pro X6 for Photographers[^]
And in my defence, I'm upgrading by ten years, so there will be a heck of a lot I don't know exists if I don't read up on it! 
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
|
|
|
|
|
OriginalGriff wrote: a heck of a lot I don't know exists if I don't read up on it!
Feh.
Real men just go to the "All Tools" toolbar-customisation option.
Reading manuals, indeed.
Shameful. really shameful.
mutter mutter
I wanna be a eunuchs developer! Pass me a bread knife!
|
|
|
|
|
You might want to let them know--they'll probably want to have this documented.
|
|
|
|
|
Not me, as long as I can have Airwolf.[^]
The picture shows my T-Rex 450 in its new Airwolf costume after a first test flight. Just asome careful hovering behind the house. Better weather and a few adjustments are needed before I can risk a real maiden flight.
- A flybarless rotor head did not fit, so I also had to keep the old rotor head, gyro and receiver. The new electronics (with integrated microbeast 3 axis flybarless system) will have to wait for my next heli.
- The tail rotor is not driven by a belt anymore and now the old gyro actually does its job quite well. It may just need a little more signal gain. The tail rotor still makes buzzing sounds when the heli is turned.
- The rotor head and swashplate are well adjusted and let the heli hover well and without much drift.
- The tail is too heavy and moves the center of gravity too far aft. I had to trim foeward quite a bit to compensate, but this can't stay that way. Some lead in the nose will help, but that's more dead weight. Instead, I could get a heavier battery with higher capacity and a stronger motor for more power and a longer flight time.
- The retractable landing gear is blocked somewhere. I hope this did not kill the servo.
Edit: You can't see it in the picture, but the tail was full of tiny bits of chopped grass. Could it be that it still believes to be a lawn wolf, not Airwolf?
The language is JavaScript. that of Mordor, which I will not utter here
Panza llena Corazon Contento
modified 20-Sep-14 12:06pm.
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
