How to Monitor Home Network Traffic

I’m the “victim” of Comcast’s insanely-stupid 250 GB data cap — which is inclusive of all data transfers, uploads, and downloads (ack and syn). It’s caused me to become uber-paranoid about any network usage here at home. I was a Business Class subscriber up until recently, when I realized I could save a ton of money and get twice as much download speed. My pocketbook thanks me, but Comcast is forcing me into the corner of buyer’s remorse.

This morning, when I checked the barebones chart that Comcast offers me in their control panel (seriously, it’s a bar that doesn’t tell you squat other than where you are in terms of data usage for the month), I discovered that yesterday’s data transfer amount didn’t align with the changes that I thought I had made on my network. I had stopped all online backup services, shared file utilities like Dropbox, etc. Still, I had transferred an insane amount of data — above and beyond my expectations.

I needed to find a way to monitor network traffic without getting a certification in how to use 99% of the traffic monitoring applications out there. I run my network through a series of AirPort Extremes — which have been the best routers / bridges / access points I’ve ever used in any environment. I had something called “SNMP” (Simple Network Management Protocol) enabled, but no tool to better view the logs being generated.

Of course, I searched for “network monitoring tool” and found a glut of crap. In my journey, I discovered likely the most simple free packet sniffer for the Mac (Packet Peeper) — but that’s not what I needed for this task.

I decided to dive into the App Store, where a quick search for “network monitor” came up with only a handful of results. One of these tools was the SNMP Test Utility — a freebie which would tell you if you were eligible for the more comprehensive NetUse Traffic Monitor tool. After quite a bit of undue fuss (due to a poor user experience), I was finally able to get the SNMP tool to work — and then purchased the traffic monitor tool from there. This version, apparently, wasn’t much help.

I emailed the developers and they responded quickly with an updated version of the testing tool. In it, I was able to figure out more about which network interface I needed to watch: mgi1 (the WAN / Internet connection). Maybe, at some point, mgi0, bwl0, bwl1, lo0, wlan0, wlan1, and bridge0 will be interesting to monitor — but not for this exercise of trying to figure out which device on my network was causing the most amount of bandwidth transfers.

At this point, you have to manually enter your SNMP device’s IP address (my AirPort Extreme is 10.0.1.1) and if you’ve not set a password, stick with the default (“public”). They don’t tell you that in the instructions! Either way, once it’s recognized, this is what I saw:

SNMP Monitor

Once I saw the mgi1 upload rate jumping at ~1MB a second, I knew I had a hole to fill — but the tool hasn’t been developed to the point where I could figure out which device was causing the data transfer. I started to unplug devices, one by one, until I believe I found the culprit: a simple security camera that was constantly streaming data out to a remote server even when it wasn’t being actively used. Ugh.

Thanks, Comcast, for forcing me to compromise my home security for the sake of playing by your anti-competitive rules. I’d flip back to Business Class Internet, but you’ve locked me into a two-year contract and I’d further lose thousands of dollars in savings with my Residential TV service over that time.

Either way, I’m grateful to have found the SNMP Test Utility to give me a better idea as to what was going on with data transfers on my home network. It’s not perfect, and they definitely have a TON of improvements to make, but at least I was able to better know exactly how much I was transferring across the network (and when).

The next step: finding a tool to better help me monitor specific device data usage on my home network. Maybe the NetUse Traffic Monitor tool will get there soon?

Article Written by

Chris has consistently expressed his convictions and visions outright, supplying practical information to targeted audiences: media agencies, business owners, technology consumers, software and hardware professionals, et al. He remains a passionate personality in the tech community-at-large. He's a geek.

  • http://www.facebook.com/profile.php?id=1155064355 John T Mcf Mood

    Any ideas on network monitors for those of us ‘stubborn’ PC-aholics? I know Mac, I can do Mac, I even own one, but I stubbornly refuse to use it. My father’s nickname was even Mac. But alas, I am a PC.

    • http://chris.pirillo.com/ Chris Pirillo

      Difficult to say. There’s a lot of junk Windows software out there claiming to be something they’re not. Every tool I found seemed to be so esoteric and beyond difficult for an average human being to use.

  • http://www.facebook.com/profile.php?id=1155064355 John T Mcf Mood

    Any ideas on network monitors for those of us ‘stubborn’ PC-aholics? I know Mac, I can do Mac, I even own one, but I stubbornly refuse to use it. My father’s nickname was even Mac. But alas, I am a PC.

    • http://chris.pirillo.com/ Chris Pirillo

      Difficult to say. There’s a lot of junk Windows software out there claiming to be something they’re not. Every tool I found seemed to be so esoteric and beyond difficult for an average human being to use.

  • Anonymous

    “The next step: finding a tool to better help me monitor specific device data usage on my home network. Maybe the NetUse Traffic Monitor tool will get there soon?”

    The problem is that while SNMP can (theoretically) provide that sort of information, I’m not sure that Apple’s MIB, and the device itself will support that sort of information.

    Here’s the problem:
    SNMP is providing information about each interface on the Airport (physical or virtual I believe).
    This is limited to the bytes going in/out on the interface (and some other info on the interface itself).

    What it DOESN’T do is decode the packets and see who the talker is.
    In a business environment, that is what network Probes are for (disclaimer, I work for a company that makes network monitoring appliances :) ).

    If you have only one device plugged into each interface, all well and good, easy to figure out. If you have multiple devices plugged into a Hub/Switch, and then the hub plugged into the interface, then the most the SNMP of the Airport will be able to do is narrow the problem down to that Hub/Switch. If the Switch supports SNMP (used to be less likely on consumer switches, but hey, look at Apple :) ), then you can try looking at its interface information, etc.

    With a small-ish network, what you did is probably the easiest approach (keep unplugging till the network traffic drops). With a larger network (or if you’re so inclined), the alternative is to look at the network traffic itself directly. There is a free tool out there called WireShark ( http://www.wireshark.org , formerly Ethereal if that means anything). This is a full (and free) network capture/decode suite that supports multiple platforms (including OSX and Windows natively).

    Once you have the network set up to allow you to capture the traffic (an exercise I won’t go into here because it depends on how your network is set up, what resources you have available, etc.), you can use WireShark to capture a sample of your network traffic, and then with a few easy clicks see a traffic breakdown from the capture, either by Conversation, or by End Point (it has quite a few other features that recommend it over Packet Peeper, but Packet Peeper looks very simple if you just want to capture some packets and look inside, so it sounds like it lives up to its name).

    • http://chris.pirillo.com/ Chris Pirillo

      Yeah, Wireshark didn’t come near to what I was looking for. ;) There’s a reason I didn’t recommend it. It’s beyond difficult to interpret any data that comes through it. Powerful, yes. Friendly, hell-to-the-no.

      • Anonymous

        I’ll concede getting the trace is more challenging, and I will certainly concede that I wouldn’t recommend Wireshark for the Faint of Heart, but if you can get a trace and load it into Wireshark, clicking on “Statistics -> Conversations” or “Statistics -> Endpoints” gives you exactly what you wanted (a list of devices and how much they sending onto the network), so I’m not sure I agree with it being “beyond difficult to interpret any data that comes through”. :)

        The trickier part personally is how to get a capture of your network’s in/out bound traffic. :)

        • http://chris.pirillo.com/ Chris Pirillo

          That does help a tiny bit, but doesn’t come close to being interpretable.

          • Anonymous

            Click on the “IPv4″ tab in either Conversations or Statistics and it will give you a list by their IPv4 address.

            I just checked, and by default it gives you the Ethernet Address.

            Looking at the Endpoints screen of the version I have:
            – There is a row of tabs across the top to select What address type to look at.
            The columns from left to right are:
            Address, Packets, Bytes, Tx Packets, Tx Bytes, Rx Packets, Rx Bytes
            (where Tx= Transmitted, and Rx= Received)

            I don’t get the interpretation problem (really, not trying to pick a fight, just trying to help).

            I’ll admit (and have repeatedly :) ) that Wireshark has a lot going on, can be confusing, and for simpler tasks an easier to use packet capture software is great, but Wireshark is Professional level software (and is in use in IT departments at thousands of companies around the globe).

            I get your frustration with it though, I really do.

            I feel equally frustrated and annoyed when I open up Photoshop or GarageBand and try to do what I think should be easy things. :)

    • http://chris.pirillo.com/ Chris Pirillo

      From the comment thread on https://plus.google.com/107234826207633309420/posts/Z5FeoZwCkK3 – I just learned about a MUCH better alternative to the unwieldy Wireshark: http://www.baurhome.net/software/downloads/ (Eavesdrop). It’s a must for Mac users who want to see traffic patterns on their local machine. Still no word for full-on home network monitoring, though (and don’t mention Wireshark again *wink*).

      • Anonymous

        “(and don’t mention Wireshark again *wink*).”
        Wouldn’t even think of it, an easy to use tool, that people feel comfortable with, and that does what they want is definitely a Good Thing. :)

        • http://chris.pirillo.com/ Chris Pirillo

          But it’s not an easy tool – it’s unintelligible. It crams too much information down your throat – which is awesome if THAT is what you want. I don’t want that. I want something that simply shows me which devices on my network (according to their human-readable DHCP name) are sending data out to the Internet at any given time. Wireshark, again, does NOT do that. ;)

          • Anonymous

            Umm … the “Easy Tool” I was referring to Eavesdrop. :)

          • http://chris.pirillo.com/ Chris Pirillo

            My bad. ;) I appreciate the assistance, though!

  • Anonymous

    “The next step: finding a tool to better help me monitor specific device data usage on my home network. Maybe the NetUse Traffic Monitor tool will get there soon?”

    The problem is that while SNMP can (theoretically) provide that sort of information, I’m not sure that Apple’s MIB, and the device itself will support that sort of information.

    Here’s the problem:
    SNMP is providing information about each interface on the Airport (physical or virtual I believe).
    This is limited to the bytes going in/out on the interface (and some other info on the interface itself).

    What it DOESN’T do is decode the packets and see who the talker is.
    In a business environment, that is what network Probes are for (disclaimer, I work for a company that makes network monitoring appliances :) ).

    If you have only one device plugged into each interface, all well and good, easy to figure out. If you have multiple devices plugged into a Hub/Switch, and then the hub plugged into the interface, then the most the SNMP of the Airport will be able to do is narrow the problem down to that Hub/Switch. If the Switch supports SNMP (used to be less likely on consumer switches, but hey, look at Apple :) ), then you can try looking at its interface information, etc.

    With a small-ish network, what you did is probably the easiest approach (keep unplugging till the network traffic drops). With a larger network (or if you’re so inclined), the alternative is to look at the network traffic itself directly. There is a free tool out there called WireShark ( http://www.wireshark.org , formerly Ethereal if that means anything). This is a full (and free) network capture/decode suite that supports multiple platforms (including OSX and Windows natively).

    Once you have the network set up to allow you to capture the traffic (an exercise I won’t go into here because it depends on how your network is set up, what resources you have available, etc.), you can use WireShark to capture a sample of your network traffic, and then with a few easy clicks see a traffic breakdown from the capture, either by Conversation, or by End Point (it has quite a few other features that recommend it over Packet Peeper, but Packet Peeper looks very simple if you just want to capture some packets and look inside, so it sounds like it lives up to its name).

    • http://chris.pirillo.com/ Chris Pirillo

      Yeah, Wireshark didn’t come near to what I was looking for. ;) There’s a reason I didn’t recommend it. It’s beyond difficult to interpret any data that comes through it. Powerful, yes. Friendly, hell-to-the-no.

      • Anonymous

        I’ll concede getting the trace is more challenging, and I will certainly concede that I wouldn’t recommend Wireshark for the Faint of Heart, but if you can get a trace and load it into Wireshark, clicking on “Statistics -> Conversations” or “Statistics -> Endpoints” gives you exactly what you wanted (a list of devices and how much they sending onto the network), so I’m not sure I agree with it being “beyond difficult to interpret any data that comes through”. :)

        The trickier part personally is how to get a capture of your network’s in/out bound traffic. :)

        • http://chris.pirillo.com/ Chris Pirillo

          That does help a tiny bit, but doesn’t come close to being interpretable.

          • Anonymous

            Click on the “IPv4″ tab in either Conversations or Statistics and it will give you a list by their IPv4 address.

            I just checked, and by default it gives you the Ethernet Address.

            Looking at the Endpoints screen of the version I have:
            – There is a row of tabs across the top to select What address type to look at.
            The columns from left to right are:
            Address, Packets, Bytes, Tx Packets, Tx Bytes, Rx Packets, Rx Bytes
            (where Tx= Transmitted, and Rx= Received)

            I don’t get the interpretation problem (really, not trying to pick a fight, just trying to help).

            I’ll admit (and have repeatedly :) ) that Wireshark has a lot going on, can be confusing, and for simpler tasks an easier to use packet capture software is great, but Wireshark is Professional level software (and is in use in IT departments at thousands of companies around the globe).

            I get your frustration with it though, I really do.

            I feel equally frustrated and annoyed when I open up Photoshop or GarageBand and try to do what I think should be easy things.

          • Troy Shimkus

            Thanks for your info, When I first looked at Wireshark, I was a little overwhelmed. I’m an IT guy, but not too heavy on the networking side. I’m trying to figure out where on my home network things are going on. Every now and then my network seems to just grind to a halt, and I don’t know what causes it, so I’m hoping this will help. Your explanation here makes Wireshark much more usable.

    • http://chris.pirillo.com/ Chris Pirillo

      From the comment thread on https://plus.google.com/107234826207633309420/posts/Z5FeoZwCkK3 – I just learned about a MUCH better alternative to the unwieldy Wireshark: http://www.baurhome.net/software/downloads/ (Eavesdrop). It’s a must for Mac users who want to see traffic patterns on their local machine. Still no word for full-on home network monitoring, though (and don’t mention Wireshark again *wink*).

      • Anonymous

        “(and don’t mention Wireshark again *wink*).”
        Wouldn’t even think of it, an easy to use tool, that people feel comfortable with, and that does what they want is definitely a Good Thing. :)

        • http://chris.pirillo.com/ Chris Pirillo

          But it’s not an easy tool – it’s unintelligible. It crams too much information down your throat – which is awesome if THAT is what you want. I don’t want that. I want something that simply shows me which devices on my network (according to their human-readable DHCP name) are sending data out to the Internet at any given time. Wireshark, again, does NOT do that. ;)

          • Anonymous

            Umm … the “Easy Tool” I was referring to Eavesdrop. :)

          • http://chris.pirillo.com/ Chris Pirillo

            My bad. ;) I appreciate the assistance, though!

  • http://frasercain.com Fraser Cain

    That sounds like a huge business opportunity. Here in Canada there are some serious restrictions on bandwidth. You could imagine that parents would want to be able to monitor their network usage to see if the kids are running huge bandwidth. Reach out to your network of programmers and get something built Chris.

  • http://frasercain.com Fraser Cain

    That sounds like a huge business opportunity. Here in Canada there are some serious restrictions on bandwidth. You could imagine that parents would want to be able to monitor their network usage to see if the kids are running huge bandwidth. Reach out to your network of programmers and get something built Chris.

  • Jim Simon

    Reading through all your postings about your ordeal seriously has me worried about the immediate future of cloud computing and the growing trend of ISP’s capping their bandwidth. I understand your issue at present is with Comcast but just like Verizon capping their mobile data plans when they saw AT&T wasn’t losing subscribers over capped plans, all the other ISP’s will be capping theirs too. Leaving the average consumer stuck in the fast fading present.

    What do you see the next 3 years of the internet bandwidth issues looking like?

    • http://chris.pirillo.com/ Chris Pirillo

      What do I see them looking like? One word: frightening.

  • Jim Simon

    Reading through all your postings about your ordeal seriously has me worried about the immediate future of cloud computing and the growing trend of ISP’s capping their bandwidth. I understand your issue at present is with Comcast but just like Verizon capping their mobile data plans when they saw AT&T wasn’t losing subscribers over capped plans, all the other ISP’s will be capping theirs too. Leaving the average consumer stuck in the fast fading present.

    What do you see the next 3 years of the internet bandwidth issues looking like?

    • http://chris.pirillo.com/ Chris Pirillo

      What do I see them looking like? One word: frightening.

  • http://www.bsitko.com Bill Szczytko

    I thought SNMP tools was for the PC crowd. Didn’t realize MAC users used that stuff. HA!!!

    • http://chris.pirillo.com/ Chris Pirillo

      Mac users are geekier than you think – we just expect a design ethos to come along with our tools. ;)

      • http://www.bsitko.com Bill Szczytko

        Hard to disagree with you from a design standpoint. PC was never meant to be beautiful. Honestly I have to ask you … why did you switch from Business to Home with Comcast? For the $? I’m disappointed in you sir…

        • http://chris.pirillo.com/ Chris Pirillo

          Comcast 105/10 – Business is $369.95 vs Residential at $100 on a two-year special. You do the math – and it doesn’t add up.

          I’m not asking for unlimited – I’m just asking for the caps to rise accordingly. I’m paying more for more speed, I should also see a higher cap that’s in line with the download bandwidth (in this case, twice as much as the 50/10 product, so I should have a 500GB cap and would likely fall within that without hassle).

          If, as Comcast contends, I’m a 1% edge case – then there shouldn’t be an issue. Right? Right.

  • http://www.bsitko.com Bill Szczytko

    I thought SNMP tools was for the PC crowd. Didn’t realize MAC users used that stuff. HA!!!

    • http://chris.pirillo.com/ Chris Pirillo

      Mac users are geekier than you think – we just expect a design ethos to come along with our tools. ;)

      • http://www.bsitko.com Bill Szczytko

        Hard to disagree with you from a design standpoint. PC was never meant to be beautiful. Honestly I have to ask you … why did you switch from Business to Home with Comcast? For the $? I’m disappointed in you sir…

        • http://chris.pirillo.com/ Chris Pirillo

          Comcast 105/10 – Business is $369.95 vs Residential at $100 on a two-year special. You do the math – and it doesn’t add up.

          I’m not asking for unlimited – I’m just asking for the caps to rise accordingly. I’m paying more for more speed, I should also see a higher cap that’s in line with the download bandwidth (in this case, twice as much as the 50/10 product, so I should have a 500GB cap and would likely fall within that without hassle).

          If, as Comcast contends, I’m a 1% edge case – then there shouldn’t be an issue. Right? Right.

  • http://twitter.com/no_substitute Kim Nilsson

    I was just about to suggest nTop. I’ve used it before and it too delivers lots of information, but some of the graphs and lists actually ARE humanly readable :-) http://www.ntop.org/overview.html

  • http://twitter.com/no_substitute Kim Nilsson

    I was just about to suggest nTop. I’ve used it before and it too delivers lots of information, but some of the graphs and lists actually ARE humanly readable :-) http://www.ntop.org/overview.html

  • http://www.facebook.com/profile.php?id=515784343 Arne Quanbeck

    Sad to say it, but I’m glad to have Clearwire. I got throttled last week after watching a few movies and updating a Windows Vista machine twice, but at least my bill won’t go up. Even in throttled mode it was fine for surfing, just streaming was broken.

  • http://www.facebook.com/profile.php?id=515784343 Arne Quanbeck

    Sad to say it, but I’m glad to have Clearwire. I got throttled last week after watching a few movies and updating a Windows Vista machine twice, but at least my bill won’t go up. Even in throttled mode it was fine for surfing, just streaming was broken.

  • Anonymous

    I know how it feels to have a data cap on my internet services .The only internet I can access except dial-up where I live is through satellite service. I’m using a commercial
    account and my data cap is 525 MB/day. This service costs over $125.00/month.

  • Denis Paley

    I know how it feels to have a data cap on my internet services .The only internet I can access except dial-up where I live is through satellite service. I’m using a commercial
    account and my data cap is 525 MB/day. This service costs over $125.00/month.