Whatever you try to squeeze from
StringBuilder, it won't help you if you really approaching the memory limitation. As Nishant and Griff rightfully pointed out, the problem is your design.
It looks like the mere size of your file is already a stress for you memory if you try to keep it in memory at once. The techniques using such files are quite different. For one thing, you can use memory-mapped file instead, see
http://en.wikipedia.org/wiki/Memory-mapped_file[
^]. This is a relevant Microsoft help page with a sample to demonstrate the technique:
http://msdn.microsoft.com/en-us/library/dd997372.aspx[
^].
From the other hand, the file remains unstructured to you, but you probably wanted to break the file into structured fragments, put each fragment to
StringBuilder to have a random access to each fragment. For example, the fragment is the text-file line. As lines have different lengths, it is not possible to know the file position of each before you read the whole file.
The simplest popular technique is remembering index map of the file: you pre-read the whole file, but instead of putting all the content in memory, you just remember file position (which is 64-bit integer); for random access you would put those positions in a list. When this is done, you keep your file open. When you need a fragment itself, you find a file position by index, go to this position again and read appropriate amount of byte again. A modification of this technique is putting some extra information in the list: fragment length, classification, etc. You can use this technique with a regular file stream (with random access) or combine with memory-managed file.
I don't know what you're trying to achieve, but the above can merely give you the idea how to approach a big file.
Good luck!
--SA
Submit an article or tip