|
At work, we have time series data collected from multiple remote sites.
On occasion, a communication break with the remote site occurs and collected data is buffered; when the communication break is resolved the buffered data is passed to the receiving system. All of the data is time stamped from when it originated, not from when it is received at corporate.
There are two types of calculations:
event based - execute the calculation when an input event is received and time stamp the resultant value with the originating event time stamp
periodic - execute the calculation on a timed basis and use whatever values are available; the resultant value is time stamped with the execution time
So.. now the issue.. if the data is buffered, the periodic calculations are useless, but, periodic calculations are a business requirement.
I need to devise a system to:
detect a communication break
detect when the communication break is resolved and the buffered data has been received
delete the periodic values for the affected site for the affected time frame
re-execute the periodic calculations for the affected site for the affected time frame keeping in mind calculation dependencies; an output from periodic calculation 'A' may be used in periodic calculation 'B'
make the system that gathers the periodic data aware of the recalculated data so they can reload it
I have a conceptual plan written... starting to work on the pieces.
|
|
|
|
|
You also need a system to devise what is the appropriate forum to ask these questions on 
|
|
|
|
|
Not asking a question; just stating what I'm working on.
|
|
|
|
|
Yep - read the entire thing, no questions.
I figured you were just procrastinating.
|
|
|
|
|
No procrastinating, just finding time to start on it.
|
|
|
|
|
Never begin a new project on a Friday.
|
|
|
|
|
I've been mulling it over in my mind since about August; just now have time to actually start to dump the thoughts.
|
|
|
|
|
Damn you!
I am sat here at work with nothing to do but wait for any problems to rise and I've had one support call today and new development projects start next year 
Every day, thousands of innocent plants are killed by vegetarians.
Help end the violence EAT BACON
|
|
|
|
|
delete from ALL_PERIODIC_DATA
sp_reconstructAllPeriodicData();
That's the way we'd do it here. 
|
|
|
|
|
SQL Server is not an option... the calculation engine is part of a vendor supplied package.
But... it is a great challenge!
|
|
|
|
|
video recordings? ... you need some protocol to detect when something when offline and log it?
Caveat Emptor.
"Progress doesn't come from early risers – progress is made by lazy men looking for easier ways to do things." Lazarus Long
|
|
|
|
|
We have essentially two different 'types' of data feeds; for one, we can detect that we are not receiving data, and upon resumption, how much data is buffered.
However, the other type is more problematic: it is largely a 'copy from here to here' system that doesn't report how much data is waiting to be sent... so my challenge will be to find out how to detect the break and resumption in the other system.
Again, just stating what my challenge is.
Many of these sites have low band width and video would overwhelm the system.
|
|
|
|
|
mmm a NVR feature... well this is a project for the new year !!!
Caveat Emptor.
"Progress doesn't come from early risers – progress is made by lazy men looking for easier ways to do things." Lazarus Long
|
|
|
|
|
We are visiting my wife's mother in upstate New York (we live in South Carolina) and my wife's 8 year old nephew spent the night. All good...
My 12 year old daughter came out and put on snow pants, snow jacket and gloves. The conversation follow:
Nephew: What are you doing?
Daughter: Getting my breakfast.
Nephew: But those are snow pants, what are you doing?
Daughter: Getting my breakfast.
Nephew: What is the bowl for?
Daughter: Getting my breakfast. Snow and then maple syrup.
It is her daily ritual when here during snow season.
|
|
|
|
|
Tim Carmichael wrote: Snow and then maple syrup.
Ah, the nutritional requirements of children!  Hopefully it's real maple syrup!
|
|
|
|
|
Yes, we are at the south end of Lake Champlain; Vermont is spitting distance away.
|
|
|
|
|
I know the area fairly well. I have spent a lot of time in Burlington, VT.
|
|
|
|
|
|
|
Be careful. Whenever I'm on the west coast of New England I spit at New York.
|
|
|
|
|
Tim Carmichael wrote: Yes, we are at the south end of Lake Champlain;
Brrr. It's cold enough in Albany where I work!
|
|
|
|
|
Woke up to -17 the other day; at home, it was in the 40s.
|
|
|
|
|
Yes, but it's a dry cold. I used to live in Oregon and +40 degrees there felt colder than -5 in VT did.
|
|
|
|
|
Yet another tedious, drink sodden round of pointless and instantly forgettable social events is past.
Just one more to avoid entirely then it is back to business. Can't wait!
|
|
|
|
|
Are you saying you're being forced to drink?
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
