[Darth-Apple]

I wrote this post on hardwiredgoogle.com, and figured I'd copy it over here.

Forum spam has become an increasingly prevalent issue for webmasters over the last decade. Not only do spambots waste large amounts of space in your database and make life difficult for the real users of your forum, but they also make waste of server resources and use valuable bandwidth. When spambots come at 1-2 bots/month, it is manageable, but at 50+ bots/day, deleting them one by one becomes a laborious task that takes precious time away from important aspects of running your website. So, when spambots strike, how do you strike back?

Question 1: The \"Why is my website being attacked\" question.
Spambots usually  register for sites for the sole purpose of posting backlinks to a specific website, with the intent of boosting search engine rankings for that site. Although forum spamming programs usually run for several hundred dollars, those who use them can generally get tens to hundreds of thousands of backlinks very quickly, considerably increasing the rankings in search engines such as Google, and they can generally make that money back very quickly. Such programs usually have two primary methods of finding websites to spam:

(1) Spammers sell lists of forums, just like they sell lists of email addresses. Websites in these lists tend to get more spam.
(2) Spammers also use Google search results to find forums to spam. For example, it takes less than a half second to find 106,000,000 results for \"powered by phpBB\" with google, so clearly there are plenty of forums out there. If your website can be found with a specific target keyword (such as \"Microsoft\") it is more likely to be picked up on by forum spamming utilities.

Generally, because forum spamming utilities use these primary methods for selecting sites to spam, and thousands of people use such utilities, spambots are generally coming in from thousands of different IPs from hundreds or thousands of different spammers. This is why banning the IPs rarely makes much of a difference.

Many sites have been successful fighting forum spam, even when they have been targeted from multiple sources. Even if your site is in a list sold to forum spammers, they have no reason to target you if you strike back, since there are far too many sites to spam for them to waste their time on one website.


Question 2: \"Ok, I delete the spambots and have a captcha, and it's not helping. What am I doing wrong? \"

No webmaster wants spambots taking up space in their site database, and it also gives the site a bad reputation. Spambots do not even have to post in order to get backlinks (since the links can be in profiles as well. ) Deleting the bots can be a tedious task, however. Many people go further by banning their IP addresses, or even banning IPs on a reputable blacklist as well. This will not usually help, since generally spambots come from thousands of different IPs, and banning enough to make a difference will be a tedious task. While it is undoubtedly important to delete the spambots, this generally does not do anything to prevent them from coming.

Many people resort to captchas as well. Captchas, however, will not always protect your forum either. Recaptcha, a very popular captcha, has recently been cracked by popular forum spamming programs, and thus is not very effective anymore. Although having a captcha is important, it is often not enough to protect your forum against automated submissions.

Furthermore, Q&A plugins are generally ineffective because they use questions that are very standard. Q&A turns out to be highly effective as a technique, but only if the questions are not standard. Many of the default questions that come with such plugins are too standard, and are thus easy to crack.

Email verification is also not always 100% effective as advanced forum spammers are still able to activate accounts by email automatically. If you are having spambot problems, without a doubt email verification is critical, but that alone will not block all spambot submissions.

Question 3: \"OK, so what do I need to do about it?\"

When these common techniques aren't effective, many are left with this question. The good news is, if you are willing to get a little creative, there are several techniques you can employ that are proven to be effective. One of the most effective techniques is to use creative questions with Q&A (which can often be done with or without a special plugin. For example, the questions on the registration form here are done only with custom profile fields, and we did not need to install any special plugin to set it up. )

It is generally NOT recommended simply to use a plugin for Q&A, as many of the included questions are easy for forum spamming programs to crack. If you use a plugin, make sure to change the default questions. The use of more creative questions, however, can go a long way. Some examples of questions NOT to use are:

* 16 + 1 is... This is ineffective because it is very standard and easy to crack.
* \"What causes a ball to fall when dropped?\" This is also very ineffective because the answer can simply be googled. Questions that can be put into a search engine to get an answer generally aren't effective.
* \"What year was the war of 1812 in?\" because this is a very common question, it is very easy for forum spammers to crack.

The questions used must be creative, complex, and non-standard. Some examples of more effective questions, however, include:

* \"Suppose a man had 13 pennies. A man takes four dimes away from him. How many pennies does the man have now? (hint, 2 digit number)\" This question is much more complex, and is also non-standard, decreasing the chances that forum spammer programs will crack it. Because it uses cognitive reasoning, the answer cannot simply be googled. This is an example of a more effective question.

* \"Enter any four digit number between 6000 and 7000 here.\" This is also a more complex, non-standard question. It is generally very easy to set this up with custom profile fields, simply by limiting the number of digits to 4, then requiring the number to be between 6000 and 7000.

Another effective measure to use is the \"Are you a spambot\" question. It sounds very simple, and it won't stop all spambots from coming, but it has been proven for many people to be an effective line of defense. Put several options in the menu, including options like \"human,\" etc... as a measure to throw forum spammers off. Have the menu default to \"Yes, I am a spambot,\" then set it to deny registration to all users who do not have the correct answer. This is fairly easy to set up in phpBB, and although it isn't effective 100% of the time, many forum spammer programs are not complicated enough to change the drop down menu. This is only effective if the field defaults to an unacceptable answer. (If you would like to see how this looks in a real site setup, check the registration form here. We use a similar question. )

It's generally recommended to use two or three questions, just in case spamming utilities crack them. We had to use several here at HWG. It is generally wise not to use more than what's needed, however. You do not want to make registration more difficult than it needs to be on users.

Yet another effective technique involves hidden fields. Xrumer (the most dreaded of all forum spamming utilities) is set to fill out all password fields with an identical password, whether the field is hidden or not. Obviously, a human user cannot fill out a hidden field; however, Xrumer will. This so turns out to allow clever users to block submissions from Xrumer with a little technical know-how. By setting a third \"password\" field, and setting the hidden attribute, Xrumer will fill it out upon registration. You can then set it to deny registration to all users who fill that field out. This requires more technical knowledge to set up, but is a clever way to block spambots nevertheless.

Here is a tutorial on how to hide form fields in code: http://pixelentity.com/freebies/hide-for...nd-jquery/


Other tips you can use against spambots:

Restrict access to user profiles for members only. If only members can see user profiles, Google will not be able to crawl the profiles for backlinks. If search engines cannot crawl profiles for backlinks, it reduces the incentive for spambots to register. It is a good idea to restrict profile access for members anyway, whether or not your forum is being attacked by spambots.

Furthermore, ban all .ru email addresses. Xrumer is able to automatically create .ru email addresses, and thus, any user that registers under a .ru email address is likely a spambot. It cannot be guaranteed that this is the case 100% of the time, but since it is not hard to use a different email provider, simply banning .ru is not a bad idea.

Also, if there are certain areas of your forum that tend to be more subject to spam, you can set nofollow attributes to keep search engines such as Google from crawling them. It is important to understand that Google is by no means your enemy. By preventing search engines from crawling sites, however, you can decrease the incentive for forum spammers to spam your forum. Needless to say, this can have unexpected consequences on your own search results as well, and should be used carefully. Here is a guide for using the nofollow attribute, should you decide to use this method as an additional line of defense: http://support.google.com/webmasters/bin...swer=96569

A new generation of game captchas are also beginning to show themselves as being an effective solution, however, such captchas should be used carefully as not to make registration harder for real users. (Not sure if I'm the only one, but I've been banned from facebook for not \"answering\" game captchas correctly. I had to email facebook before my account was restored. Needless to say, you don't want to put your users through the same mess )

If these techniques still don't solve your problem, here is another guide with additional tips and techniques, outlined in more detail:
http://www.mediawiki.org/wiki/Manual:Combating_spam

Generally, using a combination of these different techniques can prove to be highly effective against spambots. It is important to use non-standard techniques, as forum spammers generally tend to crack common anti-spam tools. The use of these tools together, however, can significantly reduce the weight of forum spam on your website.