Socket transmission corruption

Question

0.00/5 (No votes)

See more:

Hi! I am a professional programmer but I am very new to network programming.
As an hobby, I am studying asio and socket programming.

I have create rudimentary client/server programs (one connects the other accepts).
Nothing fancy really.

I run both client and server on the same computer and I transfer an image of roughly 8mb (really its just an 8mb buffer but I interpret it as an image) from the client to the server then from the server and the client. A simple roundtrip test using asio_read/asio_write.

Now this is where things are getting weird at least to me.

If my client connects on the server using either "localhost" or the router given address of the server (192.168.1.16) everything is fine - all the time - meaning I can do the roundtrip test hundreds of time without a single byte difference.

Now, I have setup port forwarding in my router and tried the same test with my public IP address given by my ISP. Once the client is connected using this IP I can do the roundtrip test again but now I see *some* garbage data in the image. The transfer is not longer or shorter... just some "random" data overwrites in the middle of the buffer.

I have used a hexdiff program to see the actual bytes of garbage and while I can't understand all of it, I see snippets of strings of what looks like some broadcast messages.

stuff like :
- "googlerpc-3._googlerpc.tcp.local"
- "Netgearxr500"
appears in the "garbage" data.

With all that in mind, I got a few questions:

- Is it really possible for the router or some external actor to inject crap in a TCP connection without actually connecting to the socket? (I do not see more than one client connection on the server)

- So should I be coding fail-safes for corrupted data in TCP transfers?
(not for security reasons but just so my stuff works?)

- I understand that 8mb is big for a single asio_read command but at the same time I thought that it was safe to use like that. Could my problem be related to asio somehow? I know asio_read / asio_write are compound operation made up of "small" transfer operations? Maybe I need to configure something on asio initialization so it does not "sniff" the network?
(that was a colleague suggestion - "maybe the socket is sniffing the network?")

- Another colleague suggested that I split large transfers into smaller ones with some hash/checksum to verify integrity. All good but I thought that that was the point of TCP?

What I have tried:

- Verified all memory allocation and made sure the data is present until the async operations are completed.
- Updated the router + factory reset just in case.
- Disconnected all google home devices that I suspected could broadcast data.
- Made both client & server programs completely single threaded.

Posted 10-Feb-21 2:41am

Gabriel Lassonde

Updated 10-Feb-21 5:18am

Add a Solution

Comments

Hritik Debnath 19-Jan-23 1:59am

Hey, I'm Hritik, currently learning socket programming, trying to implement the same but facing the hurdles of file corruption even in the local host, and can't even do the round trip of the data in local network, i thing your help would be appriciated.

2 solutions

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Rick York · Answer 1 · 2021-02-10T05:18:00

If you are using TCP/IP for transferring data then it is not corrupting the data. The protocol has checksums and integrity checks built-in and packets are sent in order. If your data is corrupt then either you are sending it that way or it is getting mangled on the receiving end. Using Wireshark is a good suggestion because it can help you verify what is being sent over the wire. I recommend starting small. Something like a few KB so you can verify your logic of sending and receiving and then work up to bigger transfers. It doesn't hurt to include your own checksum but it is somewhat redundant. When I did this kind of thing, I included a small header that described how much data was going to be sent and it had a simple checksum. I sent a few MB at a time and never had an issue with any errors.

RickZeeland · Answer 2 · 2021-02-10T03:13:00

Solution 1

Could be a buffer problem as you exceed 65K, see: boost::asio::async_write, writing data larger than 65536 bytes - Stack Overflow[^]

Also see CodeProject article: Socket Programming in C++ using boost.asio: TCP Server and Client[^]

As suggested in the reply k5054 you can use a tool like WireShark to analyze the network traffic, but this is not for the faint of heart, it requires some study:
network-protocol-analysers[^]

Posted 10-Feb-21 3:13am

RickZeeland

Updated 10-Feb-21 5:16am

v3

Comments

Gabriel Lassonde 10-Feb-21 10:19am

Thank you for the links.
Regarding boost::asio::async_write, writing data larger than 65536 bytes - Stack Overflow[^]
I release the async_read buffer memory in the completion hander passed to async_read. I expected this is sufficient to ensure that the read operation is completed before releasing its memory.

k5054 10-Feb-21 11:04am

Have you looked at the packet contents? Wireshark https://www.wireshark.org/ is my tool of choice. It should be available as an APT or DNF/YUM package if you are using linux, and is available for download for windows from their site. At least then you would know if your ISP is injecting something (naughty, naughty!), or if its a buffer overflow/reuse issue.

Gabriel Lassonde 11-Feb-21 23:20pm

Thank you for the suggestion!
I was able to use Wireshark to analyze the packets in both directions!
What a nice application.

I first looked at the packets sent (local ip to public ip) and I can confirm that the packets looked all good!

So I looked at the received packets (public ip to local ip) and the garbage data was exactly where I was expecting it.

a few things I noticed:
- Data was sent in "large" chunks... packets with 60k+ bytes.
- Data was received in fairly small chunks ~1400 bytes.
- at the middle of the transmission wireshark reported a bunch of "out of order" packets - is that problem?
- the garbage data data was just 16 bytes (out of 8mb) at the start of the data section one of the received packet.

so.... what conclusion should I take from this?

RickZeeland 11-Feb-21 23:36pm

In computer networking, out-of-order delivery is the delivery of data packets in a different order from which they were sent. One of the functions of TCP is to prevent the out-of-order delivery of data, either by reassembling packets in order or requesting retransmission of out-of-order packets

RickZeeland 12-Feb-21 1:13am

Can not say more about it, as I'm not a network expert. But I can say that in our main software written in C# we never send large chunks of data via TCP, we always break it up in smaller chunks, so I guess that's because of experiences in the past with networks that couldn't cope.