|
I want to perform a standard type of file backup where the files are compressed and encrypted into a single backup file.
I think I'll simply keep a log file of which files I couldn't open for exclusive access, and leave it to the user to close whatever application is using the file.
The difficult we do right away...
...the impossible takes slightly longer.
|
|
|
|
|
This may sound very simple, but these parts of my brain are a bit rusty from disuse. I need to report on insurance policies, stating whether a policy is current, in advance/credit, or behind/in arrears. Am I correct in presuming it is as simple as below, providing the monthly payments remain constant?
Received = Tally of all monthly payments on the policy.
Expected = Policy age in months x monthly premium.
If Received < Expected, policy is in arrears.
If Received > Expected, policy is paid in advance.
If Received == Expected, policy is current.
|
|
|
|
|
Yes.
Will Rogers never met me.
|
|
|
|
|
Unless I am missing something, don't you have monthly payment object where you track due date, payment day, payment amount, payment made etc. If so you should be able to query month by month.
If you are trying to do a quick check, then you logic is correct. I am assuming you meant
Received = Tally of all monthly payments on the policy [for the policy period]
|
|
|
|
|
Thing is, even if this month, I received the full monthly payment, before the due date, the policy will be current for this month, but if I missed last month's payment, it is overall still in arrears, unless I include a brought-forward amount. I think it's more accurate and practical to sum all payments made on the policy, instead of implementing an extra brought-forward field, and having to keep this up to date at the end of each month. This would in fact required a monthly run, where this system is only intended for MIS and on demand, not monthly, reports.
I don't understand your qualification [for the policy period]. There surely should not be any payments for policy outside the policy period?
|
|
|
|
|
How does the business define being in arrears?
My guess is that you should use the expected sum to date compared with the actual sum to date.
So I think you are right - it is that simple.
Regards
David R
---------------------------------------------------------------
"Every program eventually becomes rococo, and then rubble." - Alan Perlis
The only valid measurement of code quality: WTFs/minute.
|
|
|
|
|
I don't have any definition from the business, and I haven't been able to elicit one yet, so for the first, prototype release I'm goinf with arrears == expected > actual.
|
|
|
|
|
Brady Kelly wrote: I don't understand your qualification [for the policy period]. There surely should not be any payments for policy outside the policy period?
Here is my understanding here in US. This is personal experience and I have not worked in the insurance industry.
I've a car insurance, which is my policy. I happen to buy it on the month of December. So my policy period runs from Dec'10 to Dec'11 for the current year. So, when they are calculating my payment to figure out the status they only need to look into the policy period. Last years policy period is irrelevant. Of course by the end of the policy period last year, I must have been current otherwise the insurance company would not have renewed it for this year.
Brady Kelly wrote: Thing is, even if this month, I received the full monthly payment, before the due date, the policy will be current for this month, but if I missed last month's payment, it is overall still in arrears, unless I include a brought-forward amount.
I agree, in fact if you go back and see my original post, what I said was this method will give you a quick way of checking the status of the policy. But you won't be able to know why it is so.
|
|
|
|
|
Looks like what I'm working on is life, not auto, so it doesn't have a yearly renewal, but a payout date, so it's actually assurance, but the 'spec' is in Afrikaans, which I read and speak well, but doesn't translate that well into a business spec for me, and I'm not in direct contact with the user. I'm working through an intermediary who has not been very available this week.
|
|
|
|
|
Ah, the plot thickens!
My first answer assumed the whole scenario was as simple as explained. But in most businesses, and in accounting generally, an aging report breaks amounts into groups - Current, 30 days, 60 days, 90 days, and >90 days in arrears. This is very likely what your client wants to see. Of course, it's a lot more work, but it gives a much better report.
Will Rogers never met me.
|
|
|
|
|
Roger Wright wrote: Current, 30 days, 60 days, 90 days, and >90 days
No, it's not the actual finance software, it's just a sort of basic 'dashboard' for the broker. They probably use an accounting package for the real aging, the brokers probably use a pen. I'll leave room for more detail though - take this baby Enterprise. Mwaahaaaahaaa.
|
|
|
|
|
Brady Kelly wrote: just a sort of basic 'dashboard'
Oh, then I'll stick with my original answer - comparing the sums is a good way to do it.
Will Rogers never met me.
|
|
|
|
|
I'm a little tired of staring at these few lines and not seeing the problem. Take this code:
volatile int WorkingBuff = 0;
volatile int NextBuff = 1;
volatile int RenderBuff = 1;
void stuff()
{
NextBuff = WorkingBuff;
int usedId = 0;
do
{
usedId = RenderBuff;
for(int i=0; i<3; i++)
{
if(i != NextBuff && i != usedId)
WorkingBuff = i;
}
} while(RenderBuff != usedId);
}
void render()
{
RenderBuff = NextBuff;
}
Could anybody point out the situation where WorkingBuff and RenderBuff have the same value? I'm just not seeing it.
|
|
|
|
|
1. NextBuff = WorkingBuff; in Thread A.
2. Thread A is blocked and Thread B starts.
3. RenderBuff = NextBuff; in Thread B.
|
|
|
|
|
Sorry, I should have elaborated a bit. I mean after the do-while loop had run.
I'm having a problem it seems, where I'm trying to render with stuff I'm in the middle of updating.
|
|
|
|
|
You could use lock () around the places you absolutely don't want to execute concurrently. This would remove all the uncertainty and make your code more reliable.
|
|
|
|
|
True, but I was aiming for a lock-free solution as they are surprisingly expensive.
|
|
|
|
|
It's surprisingly complex for such a short fragment because of the possible interactions between threads, and the shuffling of values among four variables. Maybe it could be rewritten in a simpler way.
A useful step in analyzing code like this is identifying invariant conditions, that are guaranteed to always be true at some point in the code. But it looks like the variable shuffling makes this impossible.
|
|
|
|
|
Yeah, I've temporarily gone with a lock which solves the problem for now. Then I'm going to try Luc's suggestion of using a couple of queues. I already have a lock-free queue that's working elsewhere which I should be able to use, since there would only be one producer for each queue (a limitation for lock-free queues it would seem).
|
|
|
|
|
SK Genius wrote: there would only be one producer ... a limitation for lock-free queues it would seem
a limitation for lock-free whatevers I would say. 
Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.
|
|
|
|
|
I am sure you can find a solution for adding more producers, if you are good at lock-free algorithms. This page suggest that it is not impossible.
http://www.1024cores.net/home/lock-free-algorithms/queues
|
|
|
|
|
when you have a producer, a consumer, and a number (at least 2) of equivalent buffers, I find it most easy to have two queues, one holding "empty" buffers (emptied by the consumer, to be filled by the producer), one holding "full" buffers (filled by the producer, to be processed by the consumer). That is an approach that is simple to understand; it works well, it doesn't create new objects once initialized, and it doesn't really need locking, except the Queue.Dequeue operation should wait for a buffer being returned.
In pseudo-code:
emptyQ=CreateQueue();
fullQ=CreateQueue();
for(N times) emptyQ.Queue(new Buffer());
forever {
buf=emptyQ.Dequeue();
...fill buf
fullQ.Queue(buf);
}
forever {
buf=fullQ.Dequeue();
... process buf
emptyQ.Queue(buf);
}
Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.
|
|
|
|
|
Good call, I might be able to fit that in. With the change that the consumer won't release a buffer until it has new one to render so it would be holding 2 buffers at times, so the minimum number of required buffers becomes 3.
|
|
|
|
|
What is the best (by that I mean the fastest) way to merge some smaller files of sorted data into a larger file, and then later merge the larger files into a single sorted file.
Some particulars:
Input: text files each 6 GB, short records of 500,000,000 10 digit numbers in three formats, sequential ascending, sequential descending, and random selection order, long records with null strings to 65533 byte strings in random order, with many duplicate strings, and then as unique strings where the last 10 digits of each line are replaced with one of the random short strings. A total of 5 different files for 5 different tests.
Memory is allocated with VirtualAlloc (3 GB enabled in boot.ini and in the Link line and 4 GB if memory in the system), memory blocks of 2 GB, 1 GB, and 47 MB, plus 2 more small blocks are allocated.
The first step does the initial sort. The 2GB buffer is used as an input buffer, the 1 GB buffer is used for building mod 16 aligned and sized strings and for the nodes for a RedBlackTree, the 47 MB buffer is used to hold pointers to the strings for the sort tree and for the records moved after the sort. For the short records, this results in about 211 sorted blocks written to a file. This step is complete for now.
The next step combines the 47 MB sorted blocks into a big sort block (2 GB) using the 1 GB buffer for tree nodes and 28 sort blocks are combined into each big block (limited by the node space available - 16 BYTES for each record in the tree), and there are 8 big sort blocks written to a file. This step is complete for now, but could be changed if there is a better merge solution.
The final step needs to merge these big blocks into a single output file, but has to be done in a fragmented mode because there is not enough memory to hold the files. The 2 GB buffer is split into pieces (8 in this case for the 8 big sort blocks). The header from each sort block and the initial pointers from each sort block and the initial associated records from each sort block are read into the 8 pieces of the 2 GB buffer (with an initial header pointing to each big sort block).
Essentially you need to sort the big blocks in ascending sequence by the first record in each big block (how to do this efficiently?), then select and output all the records in the first big block that are lower than the first record in the second big block (how to do this efficiently?), then unlink the first buffer and re-link it with the other set of buffers in ascending sequence (how to do this efficiently?), refilling each block buffer as it exhausts, deleting the block as all of its records are selected and written.
Unfortunately, 6 GB is not necessarily the maximum size file so the 8 big blocks could become thousands so linked lists are not efficient. Arrays of thousands of pointers have insertion timing property problems. Trees are better at insertions than arrays, but have timing problems as well (RedBlackTree recursive rotations, especially with deletions).
Which way is best (overall) for merging, and why?
Dave.
|
|
|
|
|
Sorry Dave, that does not make much sense to me. You will be wasting lots of time reading, decoding, sorting, writing data, all of which isn't very useful.
I would:
- avoid such an amount of data if at all possible;
- avoid large files; I prefer several smaller files;
- avoid text files; binary files are smaller and get processed much faster;
- probably use a database.
Luc Pattyn [Forum Guidelines] [My Articles] Nil Volentibus Arduum
Please use <PRE> tags for code snippets, they preserve indentation, improve readability, and make me actually look at the code.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
