A new free open-source external sorter written in C

I am now a big fan of Git source control and I am dumping SVN after 5 years.

Admittedly the Git command line tools are terrible, but you should use TortoiseGit which is the same as the SVN version http://code.google.com/p/tortoisegit/[^].

If you need a Git server for team development (which you don't really as Git works in disconnected mode also) check out this http://gitblit.com/[^] which is marvelous (it is java based but still great to work with).

Its the man, not the machine - Chuck Yeager
If at first you don't succeed... get a better publicist
If the final destination is death, then we should enjoy every second of the journey.

http://pedroarilla.com/en/valentina

"Valentina is a classic didone that follows some of the canons proposed by Bodoni in the eighteenth century but incorporates many of the characteristics of the antique Spanish punches of the time. It is a complete font of 457 glyphs, in which there are 125 alternative lower cases or the 46 ligatures."

Some very large size examples of letter-forms here:

http://www.behance.net/gallery/Valentina-typeface-free-font/3527197

best, Bill

"Humans are amphibians ... half spirit and half animal ... as spirits they belong to the eternal world, but as animals they inhabit time. This means that while their spirit can be directed to an eternal object, their bodies, passions, and imaginations are in continual change, for to be in time, means to change. Their nearest approach to constancy, therefore, is undulation: the repeated return to a level from which they repeatedly fall back, a series of troughs and peaks.” C.S. Lewis

modified 3-May-12 23:23pm.

"Brits, take control of the web. Spiffing allows you to write your CSS and stylesheets in conformance to proper British English (also known as correct English) grammar and spelling regulations."[^]. On GitHub:[^].

Appears to be specific to PhP 5.2, which I don't use, so I have not tried it, and being a colonial, gone expat-native in a non-western culture, have no need to.

best, Bill

"Humans are amphibians ... half spirit and half animal ... as spirits they belong to the eternal world, but as animals they inhabit time. This means that while their spirit can be directed to an eternal object, their bodies, passions, and imaginations are in continual change, for to be in time, means to change. Their nearest approach to constancy, therefore, is undulation: the repeated return to a level from which they repeatedly fall back, a series of troughs and peaks.” C.S. Lewis

Very funny: stopped spelling like a Brit many years ago: just easier to spell American.

"If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." Red Adair.
nils illegitimus carborundum

me, me, me

40+ great open source apps and games

Not a free tool per se, but a great list of them. I particularly like Launchy and Virtual Desktop.

WMI Code creator[^] from Microsoft.

Bastard Programmer from Hell Suspicious | :suss:

You need USB View.

Long story[^]

in case that gets nuked:
Direct link to FTDI Utilities page[^] referred to in the link above.

Henry Minute

Girl: (staring) "Why do you need an icy cucumber?"
“I want to report a fraud. The government is lying to us all.”
I wouldn't let CG touch my Abacus!
When you're wrestling a gorilla, you don't stop when you're tired, you stop when the gorilla is.
Cogito ergo thumb - Sucking my thumb helps me to think.

Last night I made a sort-showdown package Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT.zip 4.75 MB (4,981,422 bytes), these are notes which I wanted to share with all C programmers:

Back in 1992 RPSORT was a phenom, just wanted to see how my experimental console sorter Sandokan (written in C) behaves among Windows sort, ported GNU sort and the 16bit Bob Pirko's excellent tool (written in assembler).

Note 1: The package can be downloaded freely at: www.sanmayce.com/Downloads/Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT.zip
Note 2: This console-sort-test needs Windows 32bit because many 64bit Windowses do not support 16bit code.
Note 3: You may use/start the shortcut 'KAZE prompt.lnk' - it offers easy-to-the-eyes prompt i.e. font/color/size.
Note 4: Sources of all (except Windows' one) participants are given.
Note 5: Sandokan executables 32bit/64bit are included, Intel 12.1 and Microsoft VS2010 compilers were used, yes 4 EXEs in total.

Just run the batch file: Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT.bat

To sort (on my laptop T7500 2200MHz 4MB L2 4GB DDR2, Windows XP 32bit) first 3 million (129 bytes long strings) Knight-Tours took 33s/39s/94s/21s respectively for Sandokan/Windows-sort/GNU-sort/RPSORT.

Robert Pirko still rules... Hats down I guess.
Can anyone share with us a similar High-Performance [not] free, [not] open-source (for Windows to be more specific) console sorter?

Enjoy!
Kaze, 2012-Apr-11

Resultant log on my laptop T7500 2200MHz 4MB L2 4GB DDR2, Windows XP 32bit:

E:\Sandokan_r3-++\Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT>dir/og/on
 Volume in drive E is SSD_Sanmayce
 Volume Serial Number is 4C99-82D8

 Directory of E:\Sandokan_r3-++\Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT

04/11/2012  04:28 AM    <DIR>          .
04/11/2012  04:28 AM    <DIR>          ..
03/27/2012  08:08 AM             1,632 KAZE prompt.lnk
04/09/2012  12:03 AM            24,490 Knight-tour_r8dump.c
04/11/2012  04:20 AM            73,728 Knight-tour_r8dump_Microsoft_V16_32bit_Ox.exe
03/16/2004  08:37 PM           898,048 libiconv2.dll
10/09/2004  05:25 PM           101,888 libintl3.dll
04/11/2012  05:05 AM             8,313 README.TXT
12/15/1992  04:41 PM            18,597 rpsort.com
02/04/2009  03:42 AM            89,116 RPSRT102.ZIP
04/11/2012  01:44 AM           552,448 Sandokan_Logo.doc
04/11/2012  01:47 AM           300,367 Sandokan_Logo.pdf
04/11/2012  04:01 AM         2,316,781 Sandokan_Logo.png
04/11/2012  03:26 AM           145,079 Sandokan_QuickSortExternal_4+GB.c
04/11/2012  04:23 AM            91,648 Sandokan_QuickSortExternal_4+GB_Microsoft_V16_32bit_Ox.exe
04/11/2012  04:27 AM         1,076,165 Sandokan_r3-++.zip
04/11/2012  04:29 AM               769 Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT.bat
10/22/2007  02:47 PM             9,458 sha1sum.c
04/11/2012  04:20 AM            62,464 sha1sum_Microsoft_V16_32bit_Ox.exe
04/08/2005  09:15 PM            70,172 sort_GNU_530.c
04/20/2005  07:41 PM            96,256 sort_GNU_530.exe
07/29/2011  10:12 AM           140,095 sort_GNU_813.c
03/27/2012  08:08 AM             4,096 Timer.exe
              21 File(s)      6,081,610 bytes
               2 Dir(s)  24,177,815,552 bytes free

E:\Sandokan_r3-++\Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT>type Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT.bat
@echo This test takes some 9 minutes to complete...
@ver
Knight-tour_r8dump_Microsoft_V16_32bit_Ox.exe a8 3000000>KT3million.txt
dir kt*
timer Sandokan_QuickSortExternal_4+GB_Microsoft_V16_32bit_Ox.exe KT3million.txt /fast /descend
timer sort /R /T c: KT3million.txt /O KT3million.txt.XP
timer sort /M 1048576 /R /T c: KT3million.txt /O KT3million.txt.XP
timer sort_GNU_530 -oKT3million.txt.GNU -Tc: -r KT3million.txt
timer sort_GNU_530 -oKT3million.txt.GNU -Tc: -r -S1024M KT3million.txt
timer rpsort.com KT3mil~1.txt KT3mil~1.rp /A /R
sha1sum_Microsoft_V16_32bit_Ox.exe "QuickSortExternal_4+GB.txt"
sha1sum_Microsoft_V16_32bit_Ox.exe KT3million.txt.XP
sha1sum_Microsoft_V16_32bit_Ox.exe KT3million.txt.GNU
sha1sum_Microsoft_V16_32bit_Ox.exe "KT3MIL~1.RP"
E:\Sandokan_r3-++\Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT>Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT.bat
This test takes some 9 minutes to complete...

Microsoft Windows XP [Version 5.1.2600]

E:\Sandokan_r3-++\Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT>Knight-tour_r8dump_Microsoft_V16_32bit_Ox.exe a8 3000000 1>KT3million.txt

E:\Sandokan_r3-++\Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT>dir kt*
 Volume in drive E is SSD_Sanmayce
 Volume Serial Number is 4C99-82D8

 Directory of E:\Sandokan_r3-++\Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT

04/11/2012  04:32 AM       390,000,000 KT3million.txt
               1 File(s)    390,000,000 bytes
               0 Dir(s)  24,000,847,872 bytes free

E:\Sandokan_r3-++\Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT>timer Sandokan_QuickSortExternal_4+GB_Microsoft_V16_32bit_Ox.exe KT3million.txt /fast /descend
Timer 9.01 : Igor Pavlov : Public domain : 2009-05-31
Sandokan_QuickSortExternal_4+GB r.3-++, written by Kaze, using Bill Durango's Quicksort source.
Size of input file: 390,000,000
Counting lines ...
Lines encountered: 3,000,000
Longest line (including CR if present): 129
Allocated memory for pointers-to-lines in MB: 22
Assigning pointers ...
sizeof(int), sizeof(void*): 4, 4
Trying to allocate memory for the file itself in MB: 371 ... OK! Get on with fast internal accesses.
Uploading ...
Sorting 3,000,000 Pointers ...
Pass #1: Quicksort commenced ...
- RightEnd: 000,001,500,225; NumberOfSplittings: 0,000,456,671; Done: 100% ...
Pass #2: Insertionsort commenced ...
\ i: 000,003,000,000 ...
NumberOfComparisons: 71,492,936
The time to sort 3,000,000 items via Quicksort+Insertionsort was 23,953 clocks.
Dumping the sorted data ...
Dumped 3,000,000 lines.
OK! Incoming and resultant file's sizes match.
Dump time: 5,047 clocks.
Total time: 33,953 clocks.
Performance: 11,201 KB/s.
Done successfully.

Kernel Time  =     1.437 =    4%
User Time    =    27.296 =   80%
Process Time =    28.734 =   84%
Global Time  =    33.993 =  100%

E:\Sandokan_r3-++\Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT>timer sort /R /T c: KT3million.txt /O KT3million.txt.XP
Timer 9.01 : Igor Pavlov : Public domain : 2009-05-31

Kernel Time  =     0.515 =    1%
User Time    =    36.656 =   91%
Process Time =    37.171 =   92%
Global Time  =    39.985 =  100%

E:\Sandokan_r3-++\Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT>timer sort /M 1048576 /R /T c: KT3million.txt /O KT3million.txt.XP
Timer 9.01 : Igor Pavlov : Public domain : 2009-05-31

Kernel Time  =     0.453 =    1%
User Time    =    36.796 =   93%
Process Time =    37.250 =   94%
Global Time  =    39.251 =  100%

E:\Sandokan_r3-++\Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT>timer sort_GNU_530 -oKT3million.txt.GNU -Tc: -r KT3million.txt
Timer 9.01 : Igor Pavlov : Public domain : 2009-05-31

Kernel Time  =     1.984 =    2%
User Time    =    92.218 =   94%
Process Time =    94.203 =   97%
Global Time  =    97.089 =  100%

E:\Sandokan_r3-++\Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT>timer sort_GNU_530 -oKT3million.txt.GNU -Tc: -r -S1024M KT3million.txt
Timer 9.01 : Igor Pavlov : Public domain : 2009-05-31

Kernel Time  =     1.562 =    1%
User Time    =    92.515 =   97%
Process Time =    94.078 =   99%
Global Time  =    94.533 =  100%

E:\Sandokan_r3-++\Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT>timer rpsort.com KT3mil~1.txt KT3mil~1.rp /A /R
Timer 9.01 : Igor Pavlov : Public domain : 2009-05-31

RPSORT v1.02  Dec. 15, 1992, Copyright 1991 by Bob Pirko, All rights reserved

Sort successfully completed.
Processing Took 00:00:20.32

Kernel Time  =     0.000 =    0%
User Time    =     0.000 =    0%
Process Time =     0.000 =    0%
Global Time  =    21.069 =  100%

E:\Sandokan_r3-++\Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT>sha1sum_Microsoft_V16_32bit_Ox.exe "QuickSortExternal_4+GB.txt"
a053faa74ffb6cad41e61e273a1a0e0049cb25e7  QuickSortExternal_4+GB.txt

E:\Sandokan_r3-++\Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT>sha1sum_Microsoft_V16_32bit_Ox.exe KT3million.txt.XP
a053faa74ffb6cad41e61e273a1a0e0049cb25e7  KT3million.txt.XP

E:\Sandokan_r3-++\Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT>sha1sum_Microsoft_V16_32bit_Ox.exe KT3million.txt.GNU
a053faa74ffb6cad41e61e273a1a0e0049cb25e7  KT3million.txt.GNU

E:\Sandokan_r3-++\Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT>sha1sum_Microsoft_V16_32bit_Ox.exe "KT3MIL~1.RP"
a053faa74ffb6cad41e61e273a1a0e0049cb25e7  KT3MIL~1.RP

E:\Sandokan_r3-++\Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT>

Resultant log on laptop Pentium Dual-Core E6500 2.93GHz, 2048 kB L2-Cache, 4GB DDR2, Fedora 16 x86_64:
The script Runtest.sh below:
uname -a
echo
gcc --version
echo
sort --version
echo
./Knight-tour_r8dump a8 3000000 > KT3million.txt
time ./Sandokan_QuickSortExternal_4+GB KT3million.txt /fast /descend
time sort -r KT3million.txt -o KT3million.txt.GNU
./sha1sum QuickSortExternal_4+GB.txt
./sha1sum KT3million.txt.GNU

[Kaze@Sandokan Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT]$ sh Runtest.sh
Linux swan.sineva.net 3.3.0-4.fc16.x86_64 #1 SMP Tue Mar 20 18:05:40 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

gcc (GCC) 4.6.3 20120306 (Red Hat 4.6.3-2)
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

sort (GNU coreutils) 8.12
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Mike Haertel and Paul Eggert.

Sandokan_QuickSortExternal_4+GB r.3-++, written by Kaze, using Bill Durango's Quicksort source.
Size of input file: 387,000,000
Counting lines ...
Lines encountered: 3,000,000
Longest line (including CR if present): 128
Allocated memory for pointers-to-lines in MB: 22
Assigning pointers ...
sizeof(int), sizeof(void*): 4, 8
Trying to allocate memory for the file itself in MB: 369 ... OK! Get on with fast internal accesses.
Uploading ...
Sorting 3,000,000 Pointers ...
Pass #1: Quicksort commenced ...
- RightEnd: 000,001,500,225; NumberOfSplittings: 0,000,456,671; Done: 100% ...
Pass #2: Insertionsort commenced ...
\ i: 000,003,000,000 ...
NumberOfComparisons: 71,492,936
The time to sort 3,000,000 items via Quicksort+Insertionsort was 22,060,000 clocks.
Dumping the sorted data ...
Dumped 3,000,000 lines.
OK! Incoming and resultant file's sizes match.
Dump time: 1,050,000 clocks.
Total time: 25,340,000 clocks.
Performance: 14,535 KB/s.
Done successfully.

real	0m25.975s
user	0m24.002s
sys	0m1.353s

real	1m0.220s
user	1m34.929s
sys	0m2.532s
b647ce39fcb7459f9692a479ce736ae36dab87af  QuickSortExternal_4+GB.txt
b647ce39fcb7459f9692a479ce736ae36dab87af  KT3million.txt.GNU
[Kaze@Sandokan Sandokan_vs_Windows-sort_vs_GNU-sort_vs_DOS-RPSORT]$

Despite the fact Sandokan r.3-++ being an initial release it behaves well on short files, for some reason Fedora 16 sort is too slow, since I am not into Linux perhaps there is a faster way to run it.

Get down get down get down get it on show love and give it up
What are you waiting on?

Hi SanMayce,

This looks very interesting, but may I suggest that this belongs on CP as an article, where you can include links to the source code, and your tests, or at the very least a "tip/trick."

If you take the time to publish an article, you may get some very interesting feedback on optimizing your own algorithm further.

This seems really out of place here in the "free tools" forum, but I think this can be a good solid contribution to CP if included as an article.

best, Bill

"Humans are amphibians ... half spirit and half animal ... as spirits they belong to the eternal world, but as animals they inhabit time. This means that while their spirit can be directed to an eternal object, their bodies, passions, and imaginations are in continual change, for to be in time, means to change. Their nearest approach to constancy, therefore, is undulation: the repeated return to a level from which they repeatedly fall back, a series of troughs and peaks.” C.S. Lewis

Hi BillWoodruff,
glad for your feedback, you are right but (at least for now) I have got nothing new to share on sort front.

>If you take the time to publish an article, you may get some very interesting feedback on optimizing your own algorithm further.
Agree, but it turns out that I am not good at writing articles, I have been criticized many times that my first article is an example of how not to write an article. I think I am into tests not into explaining. Moreover my level of understanding is still of a beginner and don't feel I am the one who is to write, my current desire is to outline the subject not to run into it.

I just wished to give some benchmarks and to emphasize that good things (well-written code here) don't age by reminding Bob Pirko's exemplary work.

Also THOMAS NIEMANN wrote a very good pure C tool for external sort which in my view is an excellent example how to deal with external sorting at:
Work C++ Algorithm of External Natural Merge Sort with Non-decreasing and Decreasing Ordered Sub Sequences[^]

As you can see my way of approaching things is a practical one with emphasis on testing.
The man who has got something to say is Mr. Emerald, I prefer to be guest rather than host.

Please give me (if possible) a link to your dragon avatar in better quality, I wonder what are its eyes made of.

Best regards,
Kaze

Get down get down get down get it on show love and give it up
What are you waiting on?

Hi Sanmayce,

I think there is another alternative you may consider, if writing an article is just "not your thing:" how about posting this content on the C / C++ / MFC forum ?

best, Bill

"Humans are amphibians ... half spirit and half animal ... as spirits they belong to the eternal world, but as animals they inhabit time. This means that while their spirit can be directed to an eternal object, their bodies, passions, and imaginations are in continual change, for to be in time, means to change. Their nearest approach to constancy, therefore, is undulation: the repeated return to a level from which they repeatedly fall back, a series of troughs and peaks.” C.S. Lewis

Thanks for your support,
Regardsm

tăng rank alexa , tăng thứ hạng website , tăng traffic , tăng lượng truy cập website, tăng thứ hạng alexa, tăng page view, Thực phẩm chức năng

The author asks, for some of the various groupings of graphics, in return, a "Tweet," which seems quite fair to me[^].

best, Bill

"While I complain of being able to glimpse no more than the shadow of the past, I may be insensitive to reality as it is taking shape at this very moment, since I have not reached the stage of development at which I would be capable of perceiving it. A few hundred years hence,in this same place, another traveller, as despairing as myself, will mourn the disappearance of what I might have seen, but failed to see." Claude Levi-Strauss (Tristes Tropiques, 1955)

Where is the source please?

Follow the link - it's the "^" character at the end of the original post.

Ideological Purity is no substitute for being able to stick your thumb down a pipe to stop the water

Pretty obscure but thanks.

This application can be used by DotNet Developers to download assembly(s) from Global Assembly Cache (GAC).
It contains features like Copy Assembly(s) to Clipboard or Copy to some Folder in Local Machine. Best part it supports DotNet 4.0 GAC structure.

Try this out Smile | :)

https://sourceforge.net/projects/gacexplorer/[^]

Nice tool!

Read more about it here[^]
Originally for Windows, but now spreading across the platform landscape. (Ask Google!)
Does what it says on the packet, and you can't ask for more than that.

Been using it for years with no drama.

Peter

Software rusts. Simon Stephenson, ca 1994.

I like the product but do not forget your master password! Did that and am out of luck. Smile | :)

We use it a work very effectively.

Started using it after hearing about it here at CP... and I'm very happy with it. Works on all my devices (Android, Windows XP/7, and Ubuntu w/ Wine).

tangible FREE UML modelling tools[^] Thumbs Up | :thumbsup:

Another excellent tool by the same contributor.
tangible T4 Editor plus modeling tools for VS2010 [^] Thumbs Up | :thumbsup:

Wonde Tadesse
MSCS, MCTS

very good, any support for vs 2012 or any other free tool that supports?

Tim Toady Bicarbonate

Is there any tool to check if UI controls on a desktop app or a web app are properly aligned, the distance between them is consistent and the controls are following UI standards?

Because if there isn't any already, I was planning to write one. Smile | :)

It's better to know some of the questions than all of the answers.

Pravin.

Wrong forum thread. Perhaps you can ask your question here.http://www.codeproject.com/Questions/ask.aspx[^]

Wonde Tadesse
MSCS, MCTS

Free Tools