jump to navigation

An Introduction to Click Fraud and its Detection February 22, 2006

We are all familiar with the sponsored ads in the right hand column of Google, Yahoo, or any other search engine. Everytime somebody clicks on one of those ads, the advertiser pays the search engine for that click, with the hope that the lead will convert. Advertisers pay from a few cents up to several dollars for each click, so if somebody were to click several times for no reason on their ad, the money could potentially begin to add up very quickly. This is the essence of click fraud on pay per click search engines, Google AdSense, Google AdWords, or any other advertising program on the internet which is paying on a CPA basis. These programs are the money making machine which have led to the extreme profitability of companies like Google.

Using Google as an example, let’s say there are two companies who are selling some product, and they are in the number 1 and 2 spots in the keyword search for that product. Click fraud can be said to occur when company A clicks over and over again on company B’s ad, causing company B’s ad budget to be wasted on leads (company A’s clicks) which are never going to convert. Company A is committing click fraud, and company B is potentially incurring a huge loss.

Now, there are two fundamental ways in how click fraud can be perpetrated. Company A can have somebody actually sit there in front of a computer clicking away at company B’s ads, or company A can have its programmers make a script which will automate the fraud. Detection of the click fraud depends on which method is being used. Generally speaking, if the company is sitting there with a real human clicking on ads, then with few exceptions, it is not possible to know that the clicks were a priori fraudulent. Companies use statistical methods to decide AFTER THE CLICKS ARE MADE whether they were fraudulent or not. They look at ip’s, clicking times and patterns and try to ascertain whether the clicks were made with fraudulent intent, but it is by no means an exact science. Besides, the people that you as an advertiser have to pay, such as Google if you are participating in AdWords, don’t really care if you are being cheated. In the end, the more clicks that are made on your ads, the more money they make. They have next to no incentive to make sure that the clicks on your ads are good leads.

The second way in which click fraud is perpetrated, and which would appear to be even more nefarious and efficient, is by using computers to do the dirty work. The scripts and software used to do this can be made to click on just the right links, at the right times, at the right frequency. This software is called a “bot”. For the busy company trying to screw over its competitors, this is way more efficient than having to pay somebody to actually sit there and click on ads. They save money, and waste their competitors money.

People are beginning to understand the consequences of these bots. See for example: Expert: Botnets No. 1 emerging Internet threat. Also, Jeff Martin’s blog has an excellent discussion. It would in fact, be foolish to believe that companies do not use in-house produced bots for click fraud purposes. However, it is not completely hopeless. The reason for this article is that TJ and I actually have a method for detecting computer generated click fraud in real time.

A few years back in 2003, a couple of people asked us if we knew how to detect bots. Some of our friends were suspecting bot activity on their sites and desperately wanted to know if they were being assaulted without their knowledge. We sat down and came up with a method for detecting computer generated clicks at the time the click (or fake click) is made.

Being the good capitalists that we are, we thought of ways of marketing and selling this method to companies like Google, Yahoo, and other PPC search engines. However, the problem is that our method is so simple and in retrospect, obvious, that we figured that the major companies would simply reverse engineer our solution and pay us nothing. We made a software package and managed to sell a few copies to webmasters who were interested in knowing how much bot activity was on their site, but we stayed away from the major players in fear of being looted of our intellectual property.

In the past few days, we made a decision to release this software to the public under the auspices of an open source license of our choosing. We will be releasing the full source code of our software so that everybody will be able to detect bot generated clicks on their own sites with 100% certainty. Hopefully it will make the industry aware of this massive problem, and maybe it will incite some change in the policies of Google and other companies. After all, when this software is out, there won’t be much excuse for anybody not to implement this method in all their advertising and affiliate programs.

We will be putting this software up soon and going through some of the intricacies of the code to explain how it is done. In addition to the code for the fraud detection, it wouldn’t be much help if we didn’t show what we used to test it out. So we will also publish examples of the types of scripts/bots that it will detect.

Comments

Sorry comments are closed for this entry