|
Not only "might have a point", but "That's it!"
Network people ALWAYS count bits, not bytes. They always did.
First: Networking standards were established before a byte was fixed to be 8 bits. At least up to 1980, a byte could be anywhere from 5 bits and upwards - 7 was a common size, as was 9. A byte was the space requrired to store a single character.
Second: Communication overhead comes is bits (or even half bits, in modem communication). What's interesting to the cable guys is how much they have to carry, whether usable data, check digits, start/stop bits, preamble bits, link layer bit stuffing or whatever. It doesn't matter whether that 36 bit Univac word carries six 5-bit Fieldata characters, five 7-bit ASCII characters or four 9-bit characters.
Another thing to remember: Communication guys have ten fingers. Like in 56kbps channels - they are 56,000 bits/sec, not 57,344 (that is 56 * 1024) bits/sec. Or in more modern unit: 1 gpbs is 1,000,000,000 bits/sec, not 1,073,741,824. (So you get 7% less than you expected.)
About B/b: Some computer guys (those not working with communication) has tried to establish a convention of B = Byte, b = bit - but without success. Certainly not in communication; those guys do not have any 'byte' concept. They carry bits, period. Besides, the computer guys are not at all consistent themselves: You frequently see the size of a data structure given as, say, 1.5 kb, references to 4kb disk pages etc. The context tells you that these are byte sizes, not bit sizes, whether you use upper or lower case b. A communication context is similar: They are bit sizes, not byte sizes, whether you use upper or lower case b.
|
|
|
|
|
Well put. 
|
|
|
|
|
As a former subscriber to Comcast (I moved out of their area, and now I'm stuck with Charter's bullshit instead), I can at least confirm that the 300GB data cap they list is actually 300 gigabytes, as it reads. I frequently crossed it on my 50up/10down gbps connection, but they were not enforcing the cap at the time, and my router's logs (I have and had an ASUS RT-AC66U, behind my own Motorola Surfboard modem) agreed with Comcast's.
I do wonder how accurate OP's router logs are. I normally wouldn't have any doubts that Charter was overcounting (and 800GB seems excessive even considering the following concern), but my household crossed 300GB regularly with only two users, and habits similar to those described by OP. If his logs are showing only 150GB with more users, then I have a hard time trusting those logs.
|
|
|
|
|
Just FYI: On a wire, if you want to send 8 bits, you may need, 8, 9, 10, or 11 bits. It depends on the electrical wiggle that sends a bit. Some wire standards use simple DC levels to signal bits. If you don't have a "start bit" that is always a non-default value, there's no way to tell the difference between a stream of 0's from a disconnected cable, and no way to tell when the next nonzero byte begins. These days there are a million different wire protocols (level 1 for you ISO network model fans). But for reasons that depend on the exact nature of the wire wiggle, you usually send some extra bits with each byte (um, octet) of data. Comparing byte rates among different wire protocols with different goals might cause fistfights at IEEE meetings. Bit rates can't meaningfully be compared either, but they mean something useful to the RF engineers and DSP programmers who build cable boxes, so we're stuck with 'em. You can't just divide the bit rate by 8. Say that at a standards meeting and see how long it takes to restore order.
|
|
|
|
|
SeattleC++ wrote: On a wire, if you want to send 8 bits, you may need, 8, 9, 10, or 11 bits
I would like to see some citations that indicate that protocols on the optic fiber that net work providers are laying in the modern era are using anything but 8 bit protocols.
|
|
|
|
|
Are you telling us that you still, in 2015, connect to the internet through a sub-9600bps modem line?
Most of us use technologies that requires somewhere between 500 and 1000 bits on the line if you want to transfer only a single octet.
First, there is some physical framing (e.g. the preamble, if you are on a bus CSMA/CD net, sync etc. - or, on a modem line, the start/stop-bits that you refer to. Some link protocols (such as HDLC) are 'self-clocking' and do not require any physical framing. ATM has a 40 bit cell header, which may seem a lot for a payload of 384 bits, but considering that it both handles end-to-end routing, multiplexing, out-of-band data, flow control and header checksumming, you get a lot for those 40 bits! If you need it, that is.
Then there is a link layer frame: An Ethernet frame, HDLC frame, FR frame or whatever. It could have address fields getting you to the destination, like a MAC address or FR end-to-end address, but that doesn't really matter for IP; it won't trust it any way but provides its own. The number of bits of link layer overhead can vary a lot; for good old HDLC it was a minimum of 40 bits. When Ethernet frames were broadcast on a bus, the disctinction between physical and link framing was rather diffuse; together they filled 304 bits (including the 'intepacket gap' before the next frame).
IP has its own header of minimum 160 bits, including the address field of the destination machine (and the source) - but that is the machine only, not the entity within that machine! So we need another header at the Transport layer, usually the TCP header of 128 bits minimum.
Now we are at the receiving entity. But to know what that byte is for - whether it is a keystroke, an alarm signal or the eight most significant bits of a 32 bit integer, it is almost without exception wrapped in some sort of application protocol, tagging it one way or the other. And the TCP payload is always padded to a multiple of 32 bits.
That's in IPv4. Comes IPv6, and the minimum IP header has grown to 320 bits. Fortunately, the TCP header is roughly the same (checksum calculation is modified), and the link layer and physical layer is independent of the IP version, so that overhead is unchanged. But the header sizes of IP and TCP are minimum sizes; there may be optional fields at both layers.
The good thing about this terrible overhead is that if you transmit not one byte, but a thousand bytes, the overhead will probably stay the same, in absolute numbers - and 1/1000 as much in percent of the payload size. (There could be cases of packet fragmentation, increasing overhead slightly, but that doesn't happen often with 1000 byte packets.)
All large-volume data (such as streaming video or music) is transmitted in large packets with only a few percent overhead. But for interactive services, e.g. web sites reacting to every keystroke you make, the protocol overhead may easily multiply the actual data volume transmitted by a factor of 10 to 100, from that which is visible in the application.
|
|
|
|
|
charlieg wrote: Any ideas from you other techies about tracking data usage like this?
Turn off WiFi, connect a single computer that is running a proxy server and have all other computers/devices get to the internet via the proxy server. This will give you your true usage figures that you can compare with the cable company.
Failing that, a quick trip to said cable company and a quick beating may be in order.
Michael Martin
Australia
"I controlled my laughter and simple said "No,I am very busy,so I can't write any code for you". The moment they heard this all the smiling face turned into a sad looking face and one of them farted. So I had to leave the place as soon as possible."
- Mr.Prakash One Fine Saturday. 24/04/2004
|
|
|
|
|
Michael,
I understand what you are suggesting - a single point - which is what I thought I had. This is my setup:
Wall -> cable coax -> Cable modem (mine btw) -> router -> every other device in the house.
I'm measuring at the router point. The frustrating thing is that they don't even bother to listen to the data. I'll keep the beating as a last resort. 
Charlie Gilley
<italic>Stuck in a dysfunctional matrix from which I must escape...
"Where liberty dwells, there is my country." B. Franklin, 1783
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
|
|
|
|
|
charlieg wrote: Michael,
I understand what you are suggesting - a single point - which is what I thought I had. This is my setup:
Wall -> cable coax -> Cable modem (mine btw) -> router -> every other device in the house.
I'm measuring at the router point. The frustrating thing is that they don't even bother to listen to the data. I'll keep the beating as a last resort.
You do have a single point in the router, and since it is yours it is probably more reliable than if it was cable company supplied. But it is still a black box and will be limited in what information it will provide.
The proxy server is obviously more work and if you use Squid (or whatever the new thing is now) it is open source and able to get what information you need.
In the end though, you are stuffed as you say the cable company are not listening to you anyway. Maybe you should build a home made digital clock and take it down to them.
Michael Martin
Australia
"I controlled my laughter and simple said "No,I am very busy,so I can't write any code for you". The moment they heard this all the smiling face turned into a sad looking face and one of them farted. So I had to leave the place as soon as possible."
- Mr.Prakash One Fine Saturday. 24/04/2004
|
|
|
|
|
I have a Linux box sitting in my closet somewhere. Leftover from my days of attempting to build my own web filter. It was easier to just take laptops and tablets away. Even so, I'm sure it still runs, has Ubuntu, so we should be close.
The router goes okay, but if I want pure customization, I have to flash it with dd-wrt or tomato.
Stuffed is what's pissing me off. If I were using the data, I would pay for the data. In a prescient moment, I left my old UVerse connection on for a month or two. I just disconnected the router from the cable modem and to the UVerse box. We'll see what happens to the usage then muahahaa
Charlie Gilley
<italic>Stuck in a dysfunctional matrix from which I must escape...
"Where liberty dwells, there is my country." B. Franklin, 1783
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
|
|
|
|
|
charlieg wrote: I'll keep the beating as a last resort.
It's Comcast. The beating needs to be the 1st resort, and the 3rd, and the 4th...
CPallini wrote: You cannot argue with agile people so just take the extreme approach and shoot him.
:Smile:
|
|
|
|
|
charlieg wrote: I'll keep the beating as a last resort.
Maxim 6[^]: If violence wasn’t your last resort, you failed to resort to enough of it.
Did you ever see history portrayed as an old man with a wise brow and pulseless heart, waging all things in the balance of reason?
Is not rather the genius of history like an eternal, imploring maiden, full of fire, with a burning heart and flaming soul, humanly warm and humanly beautiful?
--Zachris Topelius
Training a telescope on one’s own belly button will only reveal lint. You like that? You go right on staring at it. I prefer looking at galaxies.
-- Sarah Hoyt
|
|
|
|
|
charlieg wrote: Wall -> cable coax -> Cable modem (mine btw) -> router -> every other device in the house.
I fought a similar battle with Cox a few years ago and I implemented a SmoothWall as my single access path from my LAN to the Cox WAN network
Wall -> Cable Model -> SmoothWall PC -> Linksys Router -> Every other device in the house
In my case, I discovered that the Cox usage figures were accurate. Before I stood up the Smoothwall router/firewall PC between my LAN and WAN I had my Linksys Router connected directly to the cable modem and quickly determined the Linksys bandwidth monitoring was way off, by an enormous amount (like 50 GB of usage or so during my first month of analysis).
Like others mentioned, a Squid proxy server or a firewall PC (SmoothWall, PfSense, etc) sitting between your LAN and WAN is the only accurate way to measure bandwidth utilization.
|
|
|
|
|
Comcast is the devil!
There are two types of people in this world: those that pronounce GIF with a soft G, and those who do not deserve to speak words, ever.
|
|
|
|
|
You mentioned three times about changing passwords. I assume that was to keep unauthorized users off. Is that outsiders or family members?
charlieg wrote: ...but nothing to absurd levels - calculating the daily rate, we're averaging 150 GB / month.
You need to do this on a daily basis. In other words, instead of gathering a few days's usage and extrapolating that out to a full month (e.g., a 100MB difference per day would result in a 3GB difference for the month), you need to monitor each connection for a day and compare that to your ISP's numbers for the same time period. Narrowing the scope down like that will help you to zero in on why your usage numbers differ from theirs.
The only other thing I could suggest would be to tighten the rein on family members. That shouldn't be too hard to do.
"One man's wage rise is another man's price increase." - Harold Wilson
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
"You can easily judge the character of a man by how he treats those who can do nothing for him." - James D. Miles
|
|
|
|
|
Outside users. The router is setup as a secure WIFI. Password has been changed twice in the last two weeks. This weekend, I will restrict access to only MAC addresses I recognize (that's about as tight as you can get). However, I can see the list of MAC addresses accessing my router. There are only 3 that I have no identified. I think one is a roku, another is a phone, and another is a tablet somewhere. But they are not using any data.
Agreed on the sampling. Has been started.
Charlie Gilley
<italic>Stuck in a dysfunctional matrix from which I must escape...
"Where liberty dwells, there is my country." B. Franklin, 1783
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
|
|
|
|
|
It has been my experience that the default settings for a Comcast modem is to have an unsecured WiFi presence called xfinitywifi. If that hole isn't plugged, then you may have traffic that bypasses your router.
Fletcher Glenn
|
|
|
|
|
|
jschell wrote: That is in fact a specifically a selling point of xfinity.
Yes, but the real question is whether or not the xfinitywifi traffic adds to his total usage.
Fletcher Glenn
|
|
|
|
|
"family members" - I have a fairly technically literate 18 yo. She's the main bw hog (my opinion). But her router #s are not unreasonable. I have a theory that Comcast gives extra credit to any streaming service that isn't from them. Just a theory.
Charlie Gilley
<italic>Stuck in a dysfunctional matrix from which I must escape...
"Where liberty dwells, there is my country." B. Franklin, 1783
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
|
|
|
|
|
Before investing much energy into wrong direction, take them here[^]
Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning
|
|
|
|
|
good link! I'll save it.
Charlie Gilley
<italic>Stuck in a dysfunctional matrix from which I must escape...
"Where liberty dwells, there is my country." B. Franklin, 1783
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
|
|
|
|
|
Do you have X1? Most of the system is very web dependent, especially the DVR.
I just got X1 about 4 weeks ago. No problems but I also haven't tracked the data usage yet on the new system. Now you've got me curious. I used to use about 350GB a month before the switch so I have a good baseline to look at.
|
|
|
|
|
I do, and this is what Comcast "says":
Will live TV and/or XFINITY On Demand streaming on the XFINITY TV app or XFINITY TV website (xtv.comcast.net) count against my XFINITY Internet data usage threshold?
No. Live TV and select XFINITY On Demand streaming does not count against your Internet data usage threshold. Streaming XFINITY TV Go On Demand content, however, will count against your Internet data usage.
Charlie Gilley
<italic>Stuck in a dysfunctional matrix from which I must escape...
"Where liberty dwells, there is my country." B. Franklin, 1783
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
|
|
|
|
|
Yeah, that's live TV and select OnDemand streaming. You don't see a reference to DVR in there, do you? 
Also, the last time I looked at my bill, there was a message that said the data caps we not being enforced.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
http://arstechnica.com/tech-policy/2014/12/comcast-sued-by-customers-for-turning-routers-into-public-hotspots/