AVG LinkScanner Causes More Problems

Grisoft, the makers of AVG, are now being accused of destroying web analytics with it’s LinkScanner technology. Grisoft had purchased the LinkScanner technology in December of 2007, and had incorporated the technology in its latest AVG Version 8 software. The purpose of LinkScanner is to alert the user to the validity of websites through search engines and warns us of bad sites. But using LinkScanner is now being suspected as causing rogue traffic to web sites as well. In this article it states:

In fact, LinkScanner analyses results from search engines (not just Google) and is browser independent. This may sound like a good idea from a security point of view, however, from a webmaster/website owner point of view, this is not good at all.

If your site appears well in the search engines, as everyone strives to do, your website is or is going to be hugely affected by this. Essentially this means, that everytime your site appears in a users results, regardless of whether they click on it, your website logfiles and thefore your statistics will show that person as a real visitor coming to your site. Now, because the IP address is the users IP address, we can’t filter on that, at first look it would appear we can filter on this useragent, unfortunately I spotted another one.

This one however, is even worst. This time it’s a legitimate user agent which means you can’t filter it out or rewrite it to another page on your site without the risk of blocking or harming real visitors. The first user agent is different, due to lack of a space (or plus) between the last semi-colon and the 1813, it doesn’t follow the standard pattern used by Microsoft.

So, we get to crux of the problem, AVG has destroyed web analytics for people who use a logfile analysis tool. Not only have they done this, they are also wasting our bandwidth and our disk space on servers!

Because of this I recommend that users of the new AVG 8 security software may wish to turn off LinkScanner for now. To do this one needs to do the following:

Click on Tools – Advanced Settings – LinkScanner and uncheck Search Shield.

If you wish to add protection back to your web searches give McAfee SiteAdvisor or WOT a try as a replacement.

Thanks to DougCuk for this information.

Comments welcome.

Source

Article Written by

I have been writing for LockerGnome since relocating to Missouri seven years ago, where I continue to be a technology enthusiast who enjoys playing with the newest and latest gadgets.

  • Agarath

    I turned it off almost immediately after installing it, as I found (rightly so) it was the most likely culprit to the immense slow down of my browsing.

    Good article though, thanks

  • Paul

    Reccomending McAffee over AVG… In any form is a crime.

    But yes I agree with your comments.

  • n00b

    Why would I care about screwing up a site’s web analytics? Might as well tell someone who’s being stalked not to run any yellow lights so that the car following them won’t lose them in traffic.

    I would personally turn this off because I feel comfortable with my security practices and would rather not use up CPU cycles on link scanning or any other kind of real-time virus protection. I simply do a virus scan on files individually after I download them (with AVG 7.1, about to install version 8).

    But the next time I’m over at my mother’s house, I’ll definitely be upgrading her to version 8 and leaving on both the linkscanning and the real-time protection. I don’t see any downside in doing so.

  • http://wp3.lockergnome.com/nexus/blade/ Ron Schenone

    Thanks for the comments and for sharing your thoughts.

  • DougCuk

    The two most telling quotes from my original post may be helpfull in understanding the impact of Link-scanner.

    From an AVG v8 user:
    “I had Link-scanner turned on and have Google set to display 100 hits per page. In the process of scanning links, Linkscanner downloaded over 900 MegaBytes of data in one day!”

    No problem if you have no bandwidth restrictions – but if you have a monthly cap of 5, 10 or even 30 Gig a month this is a big problem. Then multiple that by millions of AVG users all doing the same. All that EXTRA data is coming from websites that have to pay bandwidth costs. I run a small website and planned my hosting package around the estimated monthly bandwidth I would require – looks like I may have to upgrade.

    One webmaster is quoted as saying “traffic has spiked as much as 80 per cent on some sites. And this is more than just an inconvenience. After all, sites live and die by their traffic numbers. And net resources aren’t free.”

  • leftystrat

    I have never had a website so I lack knowledge of administration. Like noob, I’m having trouble, strictly from a safety and privacy standpoint, developing concern for stats. This is not to say that I don’t feel for sites – just that I’m into privacy.

    I have upgraded to AVG8 on Win boxes but turned off a few of the services because I don’t like all-inclusive/invasive software. I’m pretty sure I turned off link scanning because it didn’t look relevant.

    All this indicates to me that I need to find a light antivirus for my Win boxes. Maybe Clam… or I have to banish Win entirely (not for lack of trying)…

    AVG just went the Symantec way and bloated itself out of a job.

  • Pingback: antivirus - a (sort-of) overview from a corporate pov… ~ ThermionicEmissions

  • Ryan

    I have to be amused with this AVG crapfest, aside from my system being impervious to Windows viruses even if I did click on a bad link, and using OpenDNS + Firefox 3, which will both alert me in the event I click through to most any phishing scam, without extra software being installed on my computer.

    Mozillazine even lists AVG’s Linkscanner in it’s extensions to avoid:

    “Problems include crashes, blocking use of ENTER from location bar, and high CPU usage. Part of the “Link Scanner” component installed with AVG 8 Free Anti-Virus. ”

    Their solution is to either disable it or uninstall it.

    As far as Windows goes, users are used to paying “protection money” to “Fat Tony” to keep them safe from “Nicky the Nose”, but all silliness aside, I think that the protection afforded by hosing down your computer with extraneous and liberal amounts of do nothing, feel good crap, is a pretty bad way to secure it.

  • DougCuk

    It appears Google have implemented their own server based checking for malicious code in websites that they index. If this technique proves effective it should provide similar protection to Link-scanner within Google searches – which sounds much more effecient than every user checking every website themselves.

    The Google Help article can be read here:
    http://www.google.co.uk/support/bin/answer.py?answer=45449

    The quote below is from this article:
    http://www.darkreading.com/document.asp?doc_id=154607

    The search giant (Google) this month quietly added a new, free service called the Safe Browsing Diagnostic Page that tells whether a site flagged by Google as potentially dangerous is hosting malware, or helps distribute malware, for instance.

    Google’s new diagnostics service provides information about any bad behavior by the site within the past 90 days. The idea is to give owners of the compromised Websites more information to assist in their remediation and cleanup of the site, and to provide users more information on why the site has been flagged.

  • HarryH

    I turned off Linkscanner in AVG Free8. I don’t much care for the systems tray icon with it’s exclamation point telling me something’s wrong. Grisoft might be trying to help, but I don’t think the extra service is worthwhile.

  • http://wp3.lockergnome.com/nexus/blade/ Ron Schenone
  • Equitan

    I hadn’t realised that Linkscanner had been causing my recent but ever-more-frequent freezes in Firefox – but I certainly turned it off quickly when it crashed my PC within seconds of loading my freshly-upgraded Firefox 3. AVG 8 seems also to have some issues with post-virus scan routines and I quite wish I hadn’t bothered with upgrade to the latest version.

  • http://wp3.lockergnome.com/nexus/blade/ Ron Schenone

    Some people are having issues with LS.

  • http://nepacrosroads.com Richard

    As webmaster I can tell you there is more than just analytics that it will mess with, the bandwidth this piece of crap consumes is outrageous. It’s useless anyway because the user agent it uses is unique and it can be identified by any website. If I wanted too I could identify every member on my forum who is using it or any other visitor…How is that for “protection” of your privacy.

  • AVG_IS_EVIL

    Someone should file a class-action lawsuit against Grisoft/AVG for this linkscanner destructive conduct that is causing users to use up their monthly bandwidth and for webmasters to incur excessive bandwidth access costs.

  • Ryan

    http://www.theregister.co.uk/2008/06/26/avg_disguises_fake_traffic_as_ie6/

    Ron, it looks as if instead of fixing Linkscanner, AVG is trying to disguise the bogus traffic as legitimate human clicks by reporting itself as IE 6.

    That way when your web hosting bills go up because of AVG’s fraud, it doesn’t say AVG Linkscanner, it says IE 6.

    There was an interesting script a webmaster had put up that he used on his site to deflect the bogus traffic back to AVG.com to sap up their bandwidth instead of stealing yours.

    http://www.pixelbeat.org/docs/web/avg_linkscanner.html

    Anyway, I’ve been following this crapfest, it abuses webservers, it craps up the customers computer, and it makes AVG’s bulletpoint features *look* nicer.

    I think AVG is distributing malware.

  • Buffy

    So Noob and Leftystrat are indifferent to the effect of this ill-thought-out software on website owners.

    How about looking at it purely from your own selfish points of view as AVG customers: every dodgy website that LinkScanner visits will have your IP address in their usage logs. You will be identified as a legitimate visitor. Traffic from that site to your PC will be traceable via your ISP. If that site is currently under police investigation you can, potentially, expect a knock at your door.

    Your defence that it was LinkScanner that visited the site and not you is indefensable in court as LinkScanner traffic is deliberately indistinguishable from a human visitor.

    Now how comfortable do you feel having this software installed ?

  • http://www.avg.com.au Lloyd Borrett

    LinkScanner delivers the only real-time protection against web exploits currently available. Because of the transient nature of web exploits and the huge size of the web, a blacklist based solution can never be up-to-date.

    Yes, Google and others are using a blacklist based approach. But just ask the web masters whose web sites get on a blacklist about the time and trouble they have to go through to get off a blacklist. Ask them also about the impact being blacklisted has on their business and search engine rankings in the meantime. So whatever solution is used, there are consequences.

    Web exploits are a huge and rapidly growing problem. They are likely to be with us until web developers, managers and hosting companies can properly secure their web sites. And there’s no likelihood that will happen anytime soon.

    In the meantime, LinkScanner is providing millions of web users with the protection against web exploits they need in order to stay safe online today. Those who don’t wish to have this level of protection in place can simply choose not to install LinkScanner, or to turn LinkScanner off.

    With LinkScanner, AVG has responded to the problem of web exploits with a true real-time solution. With official reports showing that 25% of USA computers are running malware and the huge growth in the size and power of botnets, it’s obvious that new solutions like LinkScanner are required. Indeed they are now being sought out by those who want to be properly protected.

  • http://www.avg.com.au Lloyd Borrett

    Obviously the issue of how LinkScanner impacts on web analytics clearly raises some concerns not anticipated, and AVG has acknowledged this and is looking into the issue.

    The primary purpose with LinkScanner is to protect users against web-based threats that they cannot see. These threats are also usually invisible to web site operators, who presumably also don’t wish to be unwittingly passing infections on to their visitors. Such web exploits are a problem that can and does affect all types of web sites, big or small, and is extremely transient – which is why LinkScanner doesn’t use the static database approach cited by some as a viable alternative.

    AVG is exploring ways in which it can continue to deliver informed protection as unobtrusively as possible without adversely impacting site analytics. Any webmaster reading this post who is interested in working with AVG constructively to reach this goal is welcome to contact Pat Briion – pat.bitton(at)avg.com.

  • Buffy

    Its a shame that during the course of spouting his AVG propoganda, Lloyd Borrett didn’t have the courtesy to address the post immediately prior to his which asks, how does AVG justify a customer’s IP address showing up as a legitimate visitor on some very unsavoury sites that they would never deliberately visit ?

    Clearly that’s another concern “not anticipated” prior to RTM.

  • JLRodgers

    I wrote an article about this back in March — and the few people I could get to talk about it said I was nuts. Now it’s broken as news and they don’t think I’m as crazy lol.

    I did an experiment way back then. I did a search for my company’s domain (with 100 search results as my normal in Google), and had over 100 hits. What I never mentioned before now is, either there was a glitch in the software, or it’s be default, but it also downloaded a 250MB file to scan as well (it’s a zip file of data that’s freely available to visitors on my site). The file came up on 20 of the links…. Do the math. It’s a popular link that shows up for normal searches for the site.

    I contacted AVG within 30 minutes of installing version 8 (it had that many problems). As I told them, and they acknowledged, if you click on a link – avg scans the page. So if a user searches for your site, you’ll be scanned then. Then if they click on your link — it’ll be scanned again. The linkscanner is redundant, if you click a link – the page and memory/etc is scanned to prevent a virus. It just wastes bandwidth, processor cycles and memory.

  • http://www.avg.com.au Lloyd Borrett

    AVG has recently been made aware of the increased web traffic that the new Search-Shield component of our AVG Anti-Virus Free Edition 8.0 product is causing in an attempt to notify users about infected websites. We have actively listened to the web masters who have brought this to our attention, and as a company we have reacted quickly to solve them.

    In working with the web master community, AVG has responded immediately and on Tuesday, July 9th, AVG will issue a product modification to address the spikes that a few individuals have seen with their web traffic.

    We have modified the Search-Shield component of the product to only notify users of malicious sites. Search-Shield no longer scans each search result online for new exploits, which was causing the spikes that web masters addressed with us. However, it is important to note that AVG still offers full protection against potential exploits through the Active Surf-Shield component of our product, which checks every page for malicious content as it is visited, but before it is opened.

    We’d like to thank our web community for bringing these challenges to our attention, as building community trust and protecting all of our users is critical to us.

    AVG’s primary concern is protecting our customers. In order to do this, we have actively provided our customers with cutting edge technology. There are 20,000 to 30,000 unique pieces of malware being submitted to anti-virus labs around the world each day, and the vast majority of these will be delivered via web based exploit and social engineering tricks from hacked and rogue websites.

    Nearly all of these pieces of malware are designed to steal financial and personal information from victims. In order to protect our customers, and the world in general, we released technology in our free product that is designed to discover and block these sites. As a result of this, we included real-time, dynamic scanning in our free product that we recently released to customers.

    Because of the unique nature of our technology – we scan web links before our customers open them to ensure they are safe – we anticipated that we would see a spike in the number of sites that were analysed, however, we underestimated the popularity of our product and the resulting number of verdicts that came back to us. As a result, we did not anticipate seeing the volumes we have seen in two months for another 24-36 months.

    Today we are rendering over 1 billion verdicts per week that result in the identification of 1 infected URL per 43 searches which equates to rendering a red verdict to 1 in every nine 9 users. While this has affected web traffic analysis and marketing analytics on a handful of sites, we are dedicated to protecting our users with the best technology on the market today while at the same time not being disruptive.

  • wolf

    *Sigh*

    Nothing like hearing publicity statements from corporate bots…

    Well, I hope, Mr Lloyd Borrett is correct and that they will redefine ( or better yet – eliminate ) this aspect of their program. While they’re at it, it would be nice not to have error notifications that my viral definitions are out of date… even as soon as a few minutes after I updated them.

    ( AVG 8.0 was a big step backwards in many respects. )

  • Pingback: AVG update set to fix things? « OSBlues

  • http://www.avg.com.au Lloyd Borrett

    AVG has already responded to resolve this issue. The full response can be seen at http://www.avg.com.au/index.cfm?section=news&feature=104

    An updated version of AVG Anti-Virus Free Edition 8.0 is already available. The Search-Shield component of LinkScanner has been modified to only notify users of malicious sites. The equivalent modification to the the AVG 8.0 commercial products will be rolled out on 9th July 2008.

    Once the updated version has been rolled out to all AVG 8.0 users the issue will be resolved.

    As of this date, Search-Shield will no longer scan each search result online for new exploits, which was causing the spikes that web masters addressed with us.

    However, it is important to note that AVG still offers full protection against potential exploits through the LinkScanner Active Surf-Shield component of our product, which checks every page for malicious content as it is visited but before it is opened.

    We’d like to thank the web community for bringing these challenges to our attention, as building community trust and protecting all of our users is critical to us.

    Best Regards, Lloyd Borrett
    Marketing Manager, AVG (AU/NZ)

  • Thomas

    I noticed from my logs that one of the User-Agents (MSIE 6.0; Windows NT 5.1;1813) has already virtually disappeared, the other is hopefully to follow soon. So I reckon I’ll wait another week or two before I apply some blocking (it definitely has caused a significant increase of useless traffic for some of my sites as well (about 30%-40% over the last few months (and I am literally paying for every MB used)).

    I am just wondering anyway what the point of integrating such a blanket link scanner technology into an antivirus program is. The latter should detect any malware anyway when it is about to be opened (I am using Avast for instance, which includes a Web Shield as well, but doesn’t use any link scanner).
    Or was this all just a publicity stunt by AVG?

    Thomas

  • NB

    Not sure about link scanner but going to help someone with AVG soon and then will check it out.

    In the mean time someone said they wanted an Antivirus that was not intrusive.

    Well I have been using CA Etrus Antivirus the verison without all of the bells and whistles and have been since it was free many years ago.

    It is a good quailty product that has never let me down.

    Here is the best part. You can almost find this free for the first year after rebate if you send in a receipt proving you were using something else.

    You can get it all the time at the New CompUSA if you have one or you can find always fine it here for free or no more than 5.00.

    http://www.freeafterrebate.info/

    Please note also that this program comes with three licenses and uninstalls easy without leaving a mess. Even cleans up it’s registry. Just be sure to get the basic version without all the add on products.

  • BamaBoy

    Well, I have been unable to get any kind of support from this vendor. I had uninstall version 8.0 and re-install 7.5
    Still, when I run a scan, it never finds anything even the tracking cookies that it usually found 20 or more. I no longer have any confidence in this software and I am totally dis-satisfied with their so called tech support.
    I am looking at the PC Tools offerings as I must feel that I am getting some sort of use of the virus software that I put on my PC.

  • http://www.youtube.com/watch?v=dMTHtPywQL4 LinkScanner Video

    A video “Start safe surfing with AVG LinkScanner” has been uploaded! It’s available here