C# Discussion Boards - CodeProject

Re: I want to modify/replace some data from a binary file without loadig the file in memory.

30-Dec-09 6:07

My quest is simple and I have some ideas, but I wonder if you have a better idea Smile | :)

I have a binary file with some data in it, and I want to modify/replace data from position "x".
I've seen that if I want to modify/replace some data (bytes) from a binary file, I need to replace with the same bytes size.

Now I need a solution to do this.
I taught at a simple solution:
>read the binary file with a handler, and write to a new binary file with another handler
>when reading arrive at my "x" position, I write my own binary data.

My code is shown below:

public void MyWriteMethod()
        {
            int b;
            int x = 153; // the position I want to insert binary data

            Stream str = File.Open("dT.man", FileMode.Open);
            Stream strIndex = File.Open("dT.index", FileMode.CreateNew);
            
            BinaryReader br = new BinaryReader(str);            
            BinaryWriter bw = new BinaryWriter(strIndex);
            
            while ((b = br.BaseStream.ReadByte()) != -1)
            {
                if (br.BaseStream.Position == x)
                {
                    // below is the data that I want to write
                    bw.Write(1);
                    bw.Write("StringData");
                    bw.Write("StringData");
                    bw.Flush();
                    continue;
                }
                bw.Write(b);
            }

            bw.Flush();
            bw.Close();
            br.Close();

            str.Close();
            strIndex.Close();           
        }

Hope there is a better solution, something like seek into the binary file and replace the data. Big Grin | :-D

Cheer's,
Alex Manolescu.

ProtoBytes30-Dec-09 6:39

ProtoBytes

30-Dec-09 6:39

The Stream class has an overridable method seek

from MSDN:

For an example of creating a file and writing text to a file, see How
 to: Write Text to a File. For an example of reading text from a file, see
 How to: Read Text from a File. For an example of reading from and writing
 to a binary file, see How to: Read and Write to a Newly Created Data File.

Use the CanSeek property to determine whether the current instance supports seeking.

If offset is negative, the new position is required to precede the 
position specified by origin by the number of bytes specified by 
offset. If offset is zero (0), the new position is required to be the 
position specified by origin. If offset is positive, the new position 
is required to follow the position specified by origin by the number of 
bytes specified by offset.

Stream.Seek() method[^]

Re: I want to modify/replace some data from a binary file without loadig the file in memory.

Alex Manolescu30-Dec-09 6:56

Re: I want to modify/replace some data from a binary file without loadig the file in memory.

30-Dec-09 6:56

Thanks you verry much for answer!
I know about the method Seek, but there is a problem!
I want to modify/replace some binary data starting from the position "x", but when I try to modify data there is a problem with data integrity!
The new data may be bigger (in size) that the replaced data!!
I've seen that if I want to modify/replace some data (bytes) from a binary file, I need to replace with the same bytes size.

To be more concret, supose the binary file is organised like this: 1 integer, and 2 string. (this is called, by me, a record).
When I want to replace data from position "x", I have to replace with the same amount of bytes like the old data to not damage the other records!

Now, what if I don't want to replace with the same amount of bytes? What if the amount of bytes is bigger/smaller than the old data bytes size? Can I do this without damage other data (records) ? What is the best solution to do this?

Cheer's,
Alex Manolescu.

ProtoBytes30-Dec-09 7:07

ProtoBytes

30-Dec-09 7:07

Hmm, Okay well an easy approach would be to add some padding to your file so it don't not matter, just use a greedy shuffling algorithm to move things around in the original.

If you create a data structure in your app, fill it with data, and then serialize it to a file with padding you can add what ever the data structure will allow. You would have a set size file, once you get bigger then the file size you would have to recreate the file.

Think of it as a loading problem, similar to a Hashtable. You create the Hashtable at approximately the correct working size. If the number falls within 70% of the total allocated space for the hashtable it 're-hashes' and increases to a larger size, usually 2x bigger.

Re: I want to modify/replace some data from a binary file without loadig the file in memory.

Alex Manolescu30-Dec-09 8:27

Re: I want to modify/replace some data from a binary file without loadig the file in memory.

30-Dec-09 8:27

Thanks you for answer!
I'll give it a try and come back with news.

Happy New Year!

#realJSOP30-Dec-09 7:08

#realJSOP

30-Dec-09 7:08

Alex Manolescu wrote:
The new data may be bigger (in size) that the replaced data!!

Then you'll have to read the entire file into memory, change the part that needs changing, and write the entire thing back out to the file again.

.45 ACP - because shooting twice is just silly
-----
"Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"The staggering layers of obscenity in your statement make it a work of art on so many levels." - J. Jystad, 2001

Re: I want to modify/replace some data from a binary file without loadig the file in memory.

Alex Manolescu30-Dec-09 8:29

Re: I want to modify/replace some data from a binary file without loadig the file in memory.

30-Dec-09 8:29

Thanks you for answer!
I think my method, to read and write bytes from file to file without loading the memory, is working great for big files.

Happy New Year! Smile | :)

Luc Pattyn30-Dec-09 6:49

Luc Pattyn

30-Dec-09 6:49

Hi,

some coding comments:

1. that continue is ugly; what you want is an if (...) ... else ...
2. please drop the flushes, they don't make any sense just before a close.
3. you can replace the close() by using a using construct.

FYI: BinaryWriter.Write(string) "Writes a length-prefixed string to this stream in the current encoding", which may or may not be what you want. And of course, inserting something in the middle of the file may break its format. Don't try it like that with an EXE file!

Smile | :)

Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles]

Merry Christmas and a Happy New Year to all.

Re: I want to modify/replace some data from a binary file without loadig the file in memory.

Alex Manolescu30-Dec-09 7:00

Re: I want to modify/replace some data from a binary file without loadig the file in memory.

30-Dec-09 7:00

Thanks you verry much for answer!
You're right about the flushes, but I am missing something at that

continue

?
Is more efficient to use a if/else statement or why is ugly?
I am just curious and I want to know Smile | :)

I've wrote a replay above yours.
Hope I've been more specific to what I want!

Cheer's,
Alex Manolescu.

Luc Pattyn30-Dec-09 7:30

Luc Pattyn

30-Dec-09 7:30

Hi,

1.

if (br.BaseStream.Position == x) {
    // below is the data that I want to write
    bw.Write(1);
    bw.Write("StringData");
    bw.Write("StringData");
} else {
    bw.Write(b);
}

is much cleaner; performance is the same.

2.
damage to file format depends on file type. You could not do that for a Word document, an EXE file, etc.
You could do it for your own data files if your apps accept such changes.

Smile | :)

Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles]

Merry Christmas and a Happy New Year to all.

Re: I want to modify/replace some data from a binary file without loadig the file in memory.

Alex Manolescu30-Dec-09 8:25

Re: I want to modify/replace some data from a binary file without loadig the file in memory.

30-Dec-09 8:25

Thanks you for answer!
Happy New Year! Smile | :)

petercrab30-Dec-09 17:59

petercrab

30-Dec-09 17:59

what you have written already looks ok to me. Like others have said, so long as binary file is not too large you can get a speed increase by loading entire file to memory in one read, modifying, then writing the output file.

many thanks,

petercrab

Abstracts: can I do better than this?

MollyTheCoder30-Dec-09 4:45

30-Dec-09 4:45

I know I'm missing something simple. I have a base class, Device, and a bunch of derived classes Dev1, Dev2... Depending on the type of device active at the moment, I have an abstract method for each derived type. Deciding which to use leads me to:

if (SelectedDevice.GetType() == typeof(Dev1))
{
SetDevice((Dev1)SelectedDevice);
}
else if (SelectedDevice.GetType() == typeof(Dev2))
{
SetDevice((Dev2)SelectedDevice);
}    
else if ...

It works, but it just feels ugly.

Molly

Luc Pattyn30-Dec-09 4:56

Luc Pattyn

30-Dec-09 4:56

rather than comparing types explicitly, you could use the is keyword.
and what is wrong with a simple SetDevice(SelectedDevice);?
or perhaps SelectedDevice.Set()?

Smile | :)

Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles]

Merry Christmas and a Happy New Year to all.

MollyTheCoder30-Dec-09 5:22

30-Dec-09 5:22

My apologies for using the term 'abstract' a little too loosely. The SetDevice methods control display within a form, and are not part of the Device class.

I started with SetDevice(SelectedDevice); but the compiler kicks that back because it can't be resolved at compile time.

Molly

#realJSOP30-Dec-09 5:14

#realJSOP

30-Dec-09 5:14

You could establish an enum and then have an appropriate property in each class. Setting/getting a property probably has less impact on performance than casting.

DaveyM6930-Dec-09 5:20

DaveyM69

30-Dec-09 5:20

I've had this problem in the past. What I decided on was to have an enum and pass the type of the device to the base classes constructor. Something like...

public enum DeviceType
{
    TypeA,
    TypeB
}

public abstract class DeviceBase
{
    private DeviceType deviceType;

    internal DeviceBase(DeviceType deviceType)
    {
        this.deviceType = deviceType;
    }

    public DeviceType DeviceType
    {
        get { return deviceType; }
    }
}

public class DeviceTypeA : DeviceBase
{
    public DeviceTypeA()
        : base(DeviceType.TypeA)
    { }
}

public class DeviceTypeB : DeviceBase
{
    public DeviceTypeB()
        : base(DeviceType.TypeB)
    { }
}

Dave

BTW, in software, hope and pray is not a viable strategy. (Luc Pattyn)
Why are you using VB6? Do you hate yourself? (Christian Graus)

Paulo Zemek30-Dec-09 5:23

Paulo Zemek

30-Dec-09 5:23

Why can't you do something like:
SelectedDevice.Set(this)

Which Set being virtual (abstract), so the right one is called without the IF and also, if newer classes appear, you don't need to change the "Deciding" code.

Re: Abstracts: can I do better than this? [modified]

Eddy Vluggen30-Dec-09 6:56

Eddy Vluggen

30-Dec-09 6:56

MollyTheCoder wrote:
I have a base class, Device, and a bunch of derived classes Dev1, Dev2

SetDevice((Device)SelectedDevice);

This one?

I are Troll Suspicious | :suss:

PIEBALDconsult30-Dec-09 7:20

PIEBALDconsult

30-Dec-09 7:20

Is the casting required at all? Even if SetDevice is overloaded for each type there shouldn't be a problem should there? And if not, then you still don't need to cast to each type.

Edit: I just tried it and I see the problem.

More information about SetDevice would be helpful.

And, as Luc said, use is rather than comparing types.

Edit: Or perhaps SetDevice should handle determining which type of device it was passed:

private static void
SetDev
(
    DevBase Dev
)
{
    System.Console.WriteLine ( "DevBase" ) ;

    if ( Dev is Dev1 ) SetDev ( (Dev1) Dev ) ;
    else if ( Dev is Dev2 ) SetDev ( (Dev2) Dev ) ;

    return ;
}

modified on Wednesday, December 30, 2009 2:01 PM

MollyTheCoder30-Dec-09 7:43

30-Dec-09 7:43

I have a form with device information for for each type of device. Most of the information is similar, so I'm sharing a single form, using SetDevice() to setup fields and controls specific to each derived device type. (right now, all SetDevice methods are inside the form, not the Device classes).

SelectedDevice is instantiated as a Device, then cast to whichever derived class is appropriate. The cast is necessary because at compile-time, SelectedDevice is just a Device, but at run-time will have a specific derived type.

PIEBALDconsult30-Dec-09 7:52

PIEBALDconsult

30-Dec-09 7:52

I wonder whether or not dynamics in C# 4 will do it.

MollyTheCoder30-Dec-09 7:58