|
External sorting is just a term to mean a sort algorithm that can sort a large load of data without requiring it to all be loaded into memory at the same time.
The simplest example is the external merge sort. You just divide the data into equal sized blocks that you can handle. Sort each block and write it back to disk. Then load the first few values of each block into input buffers, and merge them into an output buffer. When the output buffer fills, write it to disk and clear it. If an input buffer empties, load some extra values from it's associated block.
This wikipedia article has a step by step process, and a few useful links. http://en.wikipedia.org/wiki/External_sort[^]
Before you consider this though, make sure you really need it. If you are sorting hundreds and hundreds of megabytes of data, then yes you need to do something like this. How big is the data file you are sorting? If it can sensibly be held in ram in one go, then you probably don't need to do external sorting.
Simon
|
|
|
|
|
I appreciate the reply, and I am sorting text files that could possibly contain up to 4 million lines, I have written a method to sort lines of text by a column, or columns selected on the first line. this requires creating several copies of this 4 million line doc in memory. and leads to an out of memory exception, my method works perfect with 500 thousand lines or less, my test doc is 1 million lines, and it runs out of memory.
I get the idea, and i have read that wiki page already. I get the idea. I just don't understand how you can sort in chunks, I understand how to separated the document into parts, and sort each of those, then loading segments of each part and sorting those together and appending them to a file. but that doesn't actually sort all the data does it?
it doesn't seem right.
or at least I can't seem to get it to work right.
|
|
|
|
|
Easiest thing is probably an example.
Take the numbers 0 to 9.
1 7 3 5 2 9 8 4 6 0
Lets assume you can only load 5 at a time into memory.
Load the first block of 5.
1 7 3 5 2 - Lets call this block A.
Now sort block A, using what ever method you want.
1 2 3 5 7.
Now save block A to disk.
Now load the next block.
9 8 4 6 0 - Lets call this block B.
Sort block B.
0 4 6 8 9.
Now save block B to disk.
A - 1 2 3 5 7
B - 0 4 6 8 9
Now you create an input buffer for each block, and 1 output buffer
In this case, we've already said we can manage 5 in memory at a time, so lets make our input buffers 1 item big, and our output buffer 3 items big.
Load the first items from blocks A & B into the input buffers.
InA: 1
InB: 0
Out: Nothing yet.
Now you take the lowest of all the input buffers. In this case buffer B. Move it to the output buffer.
InA: 1
InB: Empty
Out: 0
As buffer B is now empty, you load the next item from B into the buffer. (In this case 4)
InA: 1
InB: 4
Out: 0
Now you just repeat the process. So you move the lowest value to the output buffer.
InA: Empty
InB: 4
Out: 0 1
Then load the next item from A into the empty input buffer.
InA: 2
InB: 4
Out: 0 1
Then move the lowest to the output buffer
InA: Empty
InB: 4
Out: 0 1 2
Now the output buffer is full (remember we said the output buffer was 3 big). So you write the output buffer to disk.
InA: Empty
InB: 4
Out: Empty
Final sorted on disk: 0 1 2
Now you need to load the next value from block A (because InA has become empty)
InA: 3
InB: 4
Out: Empty
Then move the lowest to the output buffer
InA: Empty
InB: 4
Out: 3
Then load the next for any empty input buffers
InA: 5
InB: 4
Out: 3
Move lowest to the output buffer
InA: 5
InB: Empty
Out: 3 4
Load for any empty input buffers
InA: 5
InB: 6
Out: 3 4
Is it starting to be clear now?
1st, Divide into manageable blocks.
Sort each block and write back to disk.
Load the first bit of each block into an input buffer.
Process the input buffers, and move the lowest value into the output buffer.
Every time an input buffer runs empty, you load the next few data items from the associated block.
Every time the output buffer fills up, you write it to disk, and append it to the data already there.
Obviously, this is a simple example. You wouldn't really do this with a list of 10 numbers. And your block sizes and buffers would be much bigger.
Essentially, it's just a merge sort, but your not keeping each block in memory.
Good luck
Simon
|
|
|
|
|
 Thanks, It makes perfect sense now. I appreciate it endlessly.
DiamondDrake
|
|
|
|
|
|
how can i use that treeview in crystal report viewer? it´s always blank.
At least in the sample codes i download!!! 
nelsonpaixao@yahoo.com.br
trying to help & get help
|
|
|
|
|
I have a database that has many types of variables; varchar, int, date, image.
I connect it to my c# application, even builted a fine dataset to fit my report.
OK, all data is present in my report, but the images aren´t there!!!
The IBLOBfieldObject suitable for pictures is always blank.
In dataset the image is a byte[], right? YES
I even trigger the same procedure in a datagridview and the pictures appear there!!!
I dont have a clue
nelsonpaixao@yahoo.com.br
trying to help & get help
|
|
|
|
|
You do not have the correct format recorded images in a database
You can use this code
private void Button3_Click(object sender, System.EventArgs e)
{
Stream imgStream = UploadFile.PostedFile.InputStream;
int imgLen = UploadFile.PostedFile.ContentLength;
string imgContentType = UploadFile.PostedFile.ContentType;
byte[] imgBinaryData = new byte[imgLen];
int n = imgStream.Read(imgBinaryData,0,imgLen);
String idis = Request.QueryString["id"];
int busid = System.Convert.ToInt32(idis);
int RowsAffected = SaveToDB( imgBinaryData,imgContentType, busid);
}
private int SaveToDB(byte[] imgbin, string imgcontenttype, int busid)
{
SqlConnection sqlConnection1 = new SqlConnection(ConfigurationSettings.AppSettings["povezava"]);
if (sqlConnection1.State==ConnectionState.Closed)
{
sqlConnection1.Open();
}
SqlCommand command = new SqlCommand( "update posta1 set signature = @imgData " +
" where id = @ID", sqlConnection1 );
SqlParameter param1 = new SqlParameter( "@imgData", SqlDbType.Image );
param1.Value = imgbin;
command.Parameters.Add( param1 );
SqlParameter param4 = new SqlParameter( "@ID", SqlDbType.VarChar,10);
param4.Value = dfPosta.Text;
command.Parameters.Add( param4 );
int numRowsAffected = command.ExecuteNonQuery();
sqlConnection1.Close();
return numRowsAffected;
}
|
|
|
|
|
Hi Ted,
thanks a lot for the post, iam going to check it out.
I really dont get many post/anwsers on crystal reports
again thanks 
nelsonpaixao@yahoo.com.br
trying to help & get help
|
|
|
|
|
Hi, ted
you are missing assembles, and by the way i already have a routine to read image (byte []), not the question here.
I was asking about loading the pictures in crystal report, it seems everthing is ok but i didn´t know that CR doesn´t load all image types.
I know you are only trying to help
nelsonpaixao@yahoo.com.br
trying to help & get help
|
|
|
|
|
Hy
I think CR only .gif format can't load
|
|
|
|
|
Hi all,
ı wanna learn that how can i write as html on a word document. When i send the string with .typetext(); it writes it as a string. How can i do that ??
thx...
thanks for everything i have...
|
|
|
|
|
TALHAKOSEN wrote: ı wanna learn How can i do that ??
Less typing, more reading. Let us know if that helps.
led mike
|
|
|
|
|
In order to write *anything* on a word document, you have to make sure you lay it on a flat surface, and smooth it out completely (wrinkles in the document tend to distort newly added text). Then - and only then - can you write on the document. After you're done writing on the document, make sure you spray some sort of fixative over the text so it doesn't run when you spill fluids on the document.
"Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"...the staggering layers of obscenity in your statement make it a work of art on so many levels." - Jason Jystad, 10/26/2001
|
|
|
|
|
Alternatively, he can always use some form of whiteout on the screen whenever he makes a mistake.
|
|
|
|
|
Pete O'Hanlon wrote: use some form of whiteout on the screen whenever he makes a mistake
Ah, yes, that's always the classic solution 
"The clue train passed his station without stopping." - John Simmons / outlaw programmer
"Real programmers just throw a bunch of 1s and 0s at the computer to see what sticks" - Pete O'Hanlon
"Not only do you continue to babble nonsense, you can't even correctly remember the nonsense you babbled just minutes ago." - Rob Graham
|
|
|
|
|
|
yes i have already seen that article but i didnt type my text as html on word document. I found this
object pasteType = Word.WdPasteDataType.wdPasteHTML;
range.PasteSpecial(ref oMissing, ref oMissing, ref oMissing, ref oMissing, ref pasteType ,ref oMissing, ref oMissing); but it gives me an error says {"The specified data type is unavailable."}
thanks for everything i have...
|
|
|
|
|
I found the solution, maybe somebody need it...
to write as html on c# first take clipboard as html then use stringbuilder
Object m = System.Reflection.Missing.Value;
DataObject clipDO = new DataObject();
clipDO.SetData(DataFormats.Html, HtmlClipboardData(strHTML));
Clipboard.SetDataObject(clipDO, true); // Save to clipbaord
object typeHtml = (object)Word.WdPasteDataType.wdPasteHTML;
s.PasteSpecial(ref m, ref m, ref m, ref m, ref typeHtml, ref m, ref m);
http://www.codeproject.com/script/Membership/Profiles.aspx?mid=252028
knowledge is valuable when it is shared !!!)
and nobody has a right to make fun with anyone!!
thx all
thanks for everything i have...
|
|
|
|
|
Hi TALHAKOSEN,
I know it's been 2 years on this stream, But is there any chance that you can tell me what's HtmlClipboardData(strHTML)?
Thanks for sharing your knowledge.
N
|
|
|
|
|
hi,
How can I send and receive data from clipboard to a COM port?
Best wishes
|
|
|
|
|
For sending and receiving data from COM port there is a SerialPort class. For clipboard manipulation you can use Clipboard class.
|
|
|
|
|
Hello there,
I wanna to create an application that transfer data between two computer over the internet. Can any body help me?
Please give a web link of a sample, if there is any things.
|
|
|
|
|
Majid_grok wrote: I wanna to create an application that transfer data between two computer over the internet
Good for you. Please let us know how that turns out for you.
Why is common sense not common?
Never argue with an idiot. They will drag you down to their level where they are an expert.
Sometimes it takes a lot of work to be lazy
Individuality is fine, as long as we do it together - F. Burns
|
|
|
|
|
heh heh ... 
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
