The Year-Long Comcast Saga

Many of you have been around for the Chronicles of Comcast. This is a synopsis for those of you who don’t want to bother with reading through each post in order.

The problem started sometime in December of 2006, around the time that Comcast bought Adelphia. I noticed my connection speed immediately increased, but the connection dropped once in awhile. The linked post was created on a previous version of the Blog, so the date is way off.

There were over a dozen trouble calls before I finally started blogging about it. In July, I began what Comcast refers to as “Regular contact” to see about correcting the problem. Sometimes they’d blame it on the weather without even bothering to check my multi-page history of calls. Other times they’d mention they’d heard of me and transfer me directly to Supervisor Kieth.

Sometimes they’d run signal checks and look puzzled at my 1×10^-9 Bit Error Rate (One line error per Gigabit transferred). This is the number they strive for, and every test had shown this same error rate. They expected a much higher frequency (i.e. to Comcast, “down” refers to a bit error rate of 1×10^-6 or higher), but never arrived in time to see it occur.

Side Note: For the uninitiated, when a TCP packet is sent in error (i.e. pretty much any standard persistent Application Layer protocol; HTTP, Telnet, FTP, SMTP, POP3, IMAP, etc), the entire packet would be discarded due to an invalid CRC checksum, thusly increasing the Packet Loss. Unfortunately, Comcast doesn’t seem to measure that at the customer level. With UDP protocols such as streaming video, the packet would simply be dropped without the need for retransmission.

After signal checks usually came Modem replacements. We’ve gone through four. Most Comcast modems can be accessed via HTTP at 192.168.100.1 (i.e. http://192.168.100.1). One of my dedicated ICMP monitoring windows is configured to ping my modem continuously (i.e. ping -t 192.168.100.1).

Then they’d start replacing Coaxial cable. Sometimes it would be as simple as the span from the modem to the wall. Sometimes they would crawl under the house and replace some of those spans; maybe even the main splitter. The really motivated ones would go as far as to replace the fitting in the wall outlet that connects the intra-wall coax to the extra-wall coax.

One Comcast Technician went so far as to replace the Coax from the outside of the house to the Tap in the backyard, and to put us on a different port on that tap, in hopes the problem would migrate to a neighbor. No such luck. It took three weeks for another contractor to show up to bury the new coax. A previous tech had replaced every coaxial plug and cable from the modem to the fitting that the new coax connected to; so now we had new cabling from the modem to the tap.

For awhile I was the subject of Comcast E-mails throughout Colorado. Every Supervisor in the state knew my name and situation by August. One guy answered the IVR call simply by transferring me to Kieth (The Sup who had been taking a personal interest in my ongoing issue).

During an escalation call, I learned some things about Comcast’s WAN Infrastructure. They operate on a Tiered-Star Network Topology, meaning that each city has a centralized network called the “Head End”, which connects to a system of Tier 1 Nodes, each of which connects to smaller Tier 2 Nodes around the city. I’m on a Tier 1 node with 57 other connections on it, but it doesn’t have any incoming Tier 2 fibers.

One Line Technician (”Network Technician”)went as far as to show me some of their cool toys, such as a Google Maps API that overlays all Comcast customers in the map area with an icon representing their services and current connectivity status. There’s one guy on my node with Comcast Digital Voice; apparently their VOIP solution is less than popular.

During this visit, I learned a good deal about Rattlesnake. Rattlesnake used to be our Head-End during the Adelphia days; But, since Comcast took over they’ve been replacing Fiberoptic spans throughout the city and migrated to a single-tier Start topology — Though the people who used to go through Rattlesnake still do. The Network guy explained it as a “Giant Fiber Splice”. Sounds to me like it’s either another Tier Node, or a Tier 2 Head End; either way, my node doesn’t exactly connect to the Head End directly. I also learned that the sattelite connections they used to use from city to city across the US have been replaced by slightly-thicker Fiber spans.

So I guess it’s good that they’re laying and replacing so much old infrastructure, but I wish they’d do it in such a way that I didn’t lose my connection five minutes at a time, six times a day.

I also learned that when the Adelphia-Comcast conversion occurred, they converted from a QPSK compression to a modulation of 256 QAM (Quadrature Amplitude Modulation). I was, however, impressed to hit the Hardware Cap of my modem at 3.8MBps one day; I love this service when it works. I wonder if this new modulation technique may be related to my outages; but the Network techs all say no.

I have learned that I should have a downstream receive power level of 0 +/- 15 dBmV with a minimum SNR of 30dB. Evidently it’s hard to push this past 40, but possible. The default credentials for a Motorola SBV5220 modem are (UN: admin - PW: motorola).

I am further preturbed by an article that explains that 256-QAM signals are susceptible to… Degraded Bit Error Rate. Go figure; the timing’s right. Also from that article:
“Headend combiner isolation problems have been known to result in degraded BER performance. This is a little harder to troubleshoot and may require temporarily removing one or more signals from service”
This sounds very familiar; but I’ll get to that later.

I briefly posted some Tips for monitoring your Unreliable Comcast Internet Connection.

I had noticed a Comcast crew working on my Node several times a week for a few months. Confused and infuriated, I went over to inquire as to the network outage repair status, only to find that the Comcast Network Techs don’t believe in Comcast. They say the delays are due to red tape and they must replace everything along the way before they can work on the Head End, where they believe the problem to be. This is when I gave up on the Comcast issues.

Finally I worked out a credit deal with Supervisor Kieth. He credited my account $120 and set my account up for three months at the increased 8MBps service without an increase in service cost. it hardly fits their “Credit for the full day” policy (i.e. $1.50/day (Comcast fee pro-rated by 30 days) times 270 days (9 months of issues) = $(1.5 * 270) = $405. Really, I’m getting $((3*40)+(3*10)) = $(120 + 30) = $150; a loss of $255; but at least I’m getting something.

So, I continue to call in with problems. I suppose I’ll get monthly credits; they keep changing their IVR around so I don’t want to call every day anymore. Sometimes the IVR tells me about Pay-Per-View events I don’t care about since I don’t watch TV. Sometimes the IVR resets my modem without asking. Sometimes I have to wait 30 minutes to get to a human. If I call after 11:00PM, I get someone in Freaking Canada. FYI: The Canadian Comcast people don’t seem to follow the same policies as the Americans on alot of things - Including credits and what constitutes an outage.

My Internet connection dropped three times during the compilation of this Entry. I’d use this as fodder to ward you away from Comcast, but it’s better than DSL or Satellite. What can we do?