One of the biggest problems that the Internet has faced for years, and is facing still, is spam. It is estimated that more than 90% of the total Internet traffic is email spam. The figure has come down considerably, after Microsoft hunted for and took down a number of botnets. Nonetheless, the volume of spam is still high enough, that it is a matter of concern.

Spam is so popular and widespread, there are businesses based on spam, which thrive on the naiveness of the casual Internet surfer. Most of these spammers gather personal data or credentials, using phishing attacks.

This time, all tech giants, namely Google, Microsoft, Yahoo!, PayPal, Facebook and LinkedIn are going after spammers. They have decided to verify if the email sender is indeed the actual sender. Ars Technica writes on this, claiming that SMTP is too old to, and the concept of emails has become complicated since its birth.

Although methods like SPF and DKIM  have been used in email phishing protection already, they have their limitations. Instead, these tech giants are working on a new protection scheme called DMARC.

As with SPF and DKIM, DMARC depends on storing extra information about the sender in DNS. This information tells receiving mail servers how to handle messages that fail the SPF or DKIM tests, and how critical the two tests are. The sender can tell recipient servers to reject messages that fail SPF and DKIM outright, to quarantine them somehow (for example, putting them into a spam folder), or to accept the mail normally and send a report of the failure back to the sender.

As DMARC stores extra information about the sender, it has to record all the senders, to function effectively. This makes a global rollout compulsory for DMARC to be effective in fighting spam. In the next few months, we will see how the Internet community receives DMARC and whether it is effective against spam.

According to a recent report from Internet security company Kaspersky Labs, India has become the top source of spam emails for the third quarter of 2011.

During this period, about 79.8% of total emails sent were spam and out of this, 14.8% originated in India. The second and third positions are also held by developing nations Indonesia with 10.6% and Brazil with 9.7%. All of the top ten sources are Asian, South American or Eastern European countries.

With limited or no laws at all to tackle the issue of spam, these countries have become the safe haven for criminals looking to exploit the internet community by spamming.

India’s huge internet user base (which is currently the third largest behind China and US) and lack of awareness among the general public about general security practices could have been the reason for India’s rise as the world’s spam capital.

Some of the other important details from the Kaspersky Spam Report are -

· In Q3 of 2011, the share of spam in mail traffic was down 2.7 percentage points compared to the previous quarter, averaging 79.8%.

· The percentage of fraudulent emails in spam traffic increased twenty times, reaching 2%.

· Asia and Latin America remain the most prominent sources of spam.

· The share of partner program spam went up 5.7 times, accounting for 29% of all spam.

· The percentage of emails with malicious attachments grew by 1.17 percentage points and averaged 5.03%.

· The share of phishing emails averaged 0.03%. Three social networks were among the Top 5 organizations targeted by phishers.

You can read the entire report here.

If you are one of those users on Facebook who have downloaded the ChatSend  application, you may notice that the application has been acting shady of late.  The application has been linked to spammy Facebook message and is enticing Facebook users into installing the application that inaptly  advertises itself.

What is ChatSend Application?

According ChatSend, the application -

…creates advanced, innovative technologies to efficiently deliver large files across the Internet. Our innovative clientless peer to peer file sending platform, enable us to virtually send files without having the cost of heavy infrastructure, thus we provide the service for FREE .

Now, according to a report by GFI Labs, messages that link to ChatSend’s Facebook page started to appear on Facebook users’ inbox, which were posted without the user’s knowledge. The application not only posted a message, but also sent the same message via Facebook chat (if enabled) to all users in the list.

Although the Facebook Security team has blocked the ChatSend” app, it is reported that the app is still promoting itself among friends of users who had installed it.

The app download link has been spreading in a way similar to how scam messages spread – by sharing or “liking” the post. It was noted that the app had received more than 134,000 likes. The message contains a link to download the ChatSend application and upon installing the app on the user’s computer, it may lead to installing a browser toolbar for Chrome, Firefox and Internet Explorer that will set the default search engine to “Web Search.”

While installing the app, you will notice under the Terms and Agreements, three boxes which are already checked by default, each of them validating the installation of toolbars, search engines and predefined URLs as homepage. Once the installation is done, the app starts sending automated messages to your friends on Facebook, each of which contains a unique link pointing to the download site.

If in case you have installed this application by mistake, you can remove it from the Add/Remove Programssection of the Control Panel. Also, it is recommended that you run a full system anti-virus scan, just to make sure that there no Trojans or any the malicious programs installed. If you  receive  any message  related  to ChatSend, then DO NOT click on any links provided in the message and be sure to delete it immediately.

It is always advised that you avoid clicking links on Facebook, which may cause a potential threat your Facebook account or your system. If in case you really want to click on a link or try out some new programs, then make sure that you have read enough reviews about it before installing it on your computer.

Read:  How to Avoid Facebook LikeJacking Scams

Twitter users are being hit by a new phishing attack where affected users are sending DMs to their friends, which contain links to fake website that looks exactly like the Twitter login page.

If you receive a direct message on Twitter that says, “haha the look on your face in this pix is priceless! [LINK], DO NOT click on it. Clicking on the link will take you to a fake Twitter login page, where you will be asked to re-enter your username and password.

After clicking the link, notice the URL in the address bar of your browser. If a user provides the log-in credentials, the credential will be sent to the attackers, after which they will full control over the user’s account and can retweet the phishing message from that account.

There are a dozen of scams and phishing attacks that occur on Facebook and Twitter every day. We constantly report attacks like this, so that it helps users stay cautioned about it. Recently, another fake message that was spreading on Twitter stated Pics of Osama Bin Laden Are Finally Released

I suggest you to avoid clicking on the link and alert your followers about the attack. If you have accidently clicked on the link  and entered your login details, then change the password of your Twitter and all other accounts immediately. Also, contact your followers to avoid the message and ask them to change their account password as well.

To learn how you can avoid falling victim to a phishing scam and keeping your Twitter account secure, please read Twitter’s official guide to Keeping Your Account Secure. Here are some tips that will help you protect your Twitter account -

• Use a strong password.
• Watch out for suspicious links, and always make sure you’re on Twitter.com before you enter your login information.
• Use HTTPS for improved security.
• Don’t give your username and password out to untrusted third-parties, especially those promising to get you followers or make you money.
• Make sure your computer and operating system is up-to-date with the most recent patches, upgrades, and anti-virus software.

Sophos’ Graham Cluley explains a simple way of creating a complex hard-to-guess password – and how you should never use the same password on different sensitive websites.

It has been nearly one month since Google rolled out the famous Panda update, which is geared towards returning high quality sites on search results instead of content farms, scraper sites and sites who don’t produce original content.

Since the day Google’s Panda update went global, a lot of websites have seen an overwhelming change in their organic traffic. A lot of high traffic websites practically disappeared from Google search while Google’s own properties saw an increase in search traffic. Search metrics has the complete details about the early losers and winners of Google’s Panda update.

Recently, Google gave some additional guidelines to webmasters affected by the Panda update and how they can focus on delivering the best possible user experience. Google clearly says that the Panda algorithmic update incorporates user feedback as a strong signal, so website owners should focus on making their content or service as user friendly as possible.

Some Questions To Ask and Ponder Upon

The webmaster tools blog post list all the questions site owners should ponder upon, I will highlight some specific questions from that article here:

• Would you trust the information presented in this article? As a user, trust comes by looking at the cover and the symbol associated with it. If I have never heard about a website before, it has ugly design,   bombards me with popups and subscription boxes every second it is most likely that I won’t trust the content or the service provided on the website. And there is a good chance that I will also tell my friends not to visit that website again.Inference: Blog design matters. So does usability.
• Is this the sort of page you’d want to bookmark, share with a friend, or recommend? Google wants to know the feel goodfactor about a particular page by judging whether users would want to share the content with their friends or family members. I think this is doubtful because not all users can judge the usefulness of every page, at first glance.Example: Consider a set of users who are searching   for information about Mothers day. All they want is to read quotations and facts about Mom’s day and if you take them to this Wikipedia page , chances are that many won’t read it top to bottom. They might instead prefer this page which has a list of quotations about Mom’s day. This is a fact all users can’t judge the usefulness of a webpage, which is why we need search engines to return them the best possible page.Inference: No doubt Google is taking social sharing as one of their ranking signals, which can be gamed thanks to retweet clubs, spamming each others Facebook fan pages, paying someone to get 100 stumbles the list is endless.
• Was the article edited well, or does it appear sloppy or hastily produced? Google wants to know the grammatical level of a webpage and how well written it is. This is obvious, because search engines don’t want to return a page which is sloppy.But there is no guarantee that a page which is written badly, does not contain useful information. I know a couple of friends who don’t have good writing skills, but they are really good at their subject. They don’t know how to write like a seasoned blogger, who on the other hand knows how to collect information from different sources, mask it and produce a blog post.
• Does this article have an excessive amount of ads that distract from or interfere with the main content?

There are more questions on the blog post and the probable answers depend from one webmaster level to another. However, the biggest consequence of Google’s Panda update has remain unanswered scraper sites outranking the sources for the original content they have written.

Google’s Algorithm Doesn’t Make Exceptions

An algorithm works for everyone whether you are a spammer or a genuine source, an algorithm doesn’t give a damn care. It will continue to work the way it is designed and this is exactly where the problem begins.

There is no way a machine or a computer program can accurately determine whether John or harry wrote a piece of content. Before the Panda update was rolled out, Google did a good job keeping the scrapers away and showed the original article on top.

Here are a few examples which show that Google’s so called algorithm is not able to differentiate between the real source and the copycats.

Example # 1: Matt Cutt’s Personal Blog

Performing a search for this string (not an exact match) from Matt’s post on overdoing URL removals shows the following result:

So what do we have here.

1. Matt’s blog post is nowhere to be found on the first page.

2. The same scraper site is ranking for the first three results on SERP’s which violates Google’s own theory that they tend to show content from different domains on search results. There is more, Google thinks that this piece of content in exclusively genuine and unique on the spam site, so they are showing the little suggestion Read more content from this source. Ridiculous !

3. Google also offers a translated version of another spammer site who has completely copied the original article from top to bottom.

Now consider the following facts

• Matt’s blog is highly informative and considered an authority site on the subject.
• Matt’s blog is a trusted source among users.
• There are ZERO advertisements on the blog.
• Has good quality backlinks, high domain age, good social influence and a decent design.
• Google pagerank: 7.

So why is that Matt’s page is not shown at all in the SERP’s?

Example #2: Search Engine Land

Performing a search for this string (not an exact match) from   Greg’s post at SearchEngineLand shows the following results:

The same thing holds true for SearchEngineLand’s article. The original article is nowhere to be found on the first page, while Google thinks it is good to return even the auto generated RSS feed on the scraper site? And why is that Googlebot fails to read the title Latest News?

For the record, SearchEngineLand has Google PR 7 and it’s a high quality site with genuine content and reports on search   analysis. In fact, it is one of the oldest sites to break developments and news about search engines, in this case the latter is not giving the due credit to the former.

Example #3: Techcrunch.com

Performing a search for this string (not an exact match) from Alexia’s post on Techcrunch, shows the following result:

Techcrunch’s original article is nowhere shown on the first page, while the first result takes you to an ad laden page with no content.

Note: On performing the above example searches, you might see different results on search result pages. Search rankings of a particular phrase can change any second and it also depends upon your geographical location amongst other factors. If you are seeing different results than those shown in the screenshots above, you might want to check the following video where I have performed the above example searches one by one:

Now looking at the above examples, we go back to the same echo chamber Physician, first heal thyself.

The sole purpose of Google’s new algorithm was to remove content farms, scraper sites and people who blindly copy whole or part of the article from the original source. But the results speak a different story there are numerous occasions when original content is nowhere to be found on the search results. And it’s not that only we are saying this, the folks at Seomoz and Ubergizmo have produced their reports here and here

Judging quality of content comes later, first you should find out who the real source is and Google is failing terribly here.

If you are a webmaster and find that scrapers are outranking for the content you have written, I am afraid there is nothing much you can do here. Because this is an algorithm which detects and differentiates between the source and the scraper and if it is failing to do it’s job, nothing is in your hands.

You can file DMCA complaints and take down the scraper sites one by one, but this is impossible for sites having thousands of pages.

Come on Google ! We have circled back to the point we were before – meaningful results. That’s what Google search is known for and everybody used to appreciate that. You have changed the rules of the game, but please don’t take away the real players and let robots dominate the search results.

Wake up !

My Facebook stream just went for a toss with people posting new wall updates which is sort of a competition about who joined Facebook first. First of all, Facebook does not usually publish information on when a user joined in their APIs, so it is highly unlikely that this app is accurate at all.

Secondly, this app basically tags friends and also asks for permissions to post to your wall and gains access to your email account too along with tons of other information as you can see from the image above.

Thought this app is technically not Facebook scam, it is definitely a spam and inaccurate app. I have also seen other similar apps like "who will get married first" and other stupid quiz apps which are deceptive at best.

There have been several similar scams in the past where apps were created to tell users who visited your Facebook profile and my Total Facebook views scam among other things.

Apps like these are dime a dozen and will continue to exist in the Facebook ecosystem. However, it is not really hard to spot the one’s that have spam written all over it. If you find it difficult to differentiate genuine vs bad links, you can always read our earlier article about Avoiding Facebook Lifejacking and Clickjacking scams.

Also, if you have posted an update like this make sure to delete it and then remove the app from your account. Learn more about How To Remove Applications from Facebook.

A new worm has spread on Twitter and is spreading like wildfire. Users are spreading messages saying "Have you heard of HCG diet? AMAZING AND IT WORKS, I lost 2lbs in the last 3 days [link redacted]" to their friends.

This is a spam message so please do not click on the message. A quick Twitter search shows that an number of people have been affected by this and the number of updates that are coming through are astonishing.

First of all don’t click on this link or you might just spread the work yourself. Second, tell your friends who have posted this update to delete the message and reset their passwords if possible.

Update: The website in question is a landing page for some sort of dietary medicine. However, the page has scam written all over it so don’t click on anything. I am trying to investigate on how this worm is spreading and will update this post shortly with more information.

Update 2: If resetting your password does not help, it might be because of an app that has been added to your account. Learn more about revoking permissions for Twitter apps.

More to come…

(h/t @gaberivera)

Email spam might be dropping, but that doesn’t mean that spam is about to go away. It’s just that the spammers have found new, and possibly more fruitful, vehicles for spreading junk. This includes search engines, twitter, and of course, SMS. In spite of the establishment of the NDNC (National Do Not Call) registry, SMS spam is rapidly increasing in India. I personally consider SMS spam to be much more of a nuisance than email spam, simply because, although there are sophisticated spam filters available to tackle email spam, when it comes to SMS spam we are mostly helpless. Manual blacklisting of repeat offenders is the best most of us can do.

For separating spam from ham, most email filters utilize two techniques:

Heuristic Approach: The software learns to distinguish spam from experience by learning from the content of already processed messages.
Bayesian Approach: It’s a statistical approach that employs a probabilistic model to determine if a message is spam, based on pre-defined classifiers.

Unfortunately, these methods alone are not very effective when it comes to tackling SMS spam. The short length of messages, coupled with the use of abbreviations and vernacular languages make it very tough for machine learning algorithms to work with acceptable accuracy.

Now, a team of students at the Indraprastha Institute of Information Technology (Delhi) are trying to tackle this problem by employing the intelligence of the crowd. The team lead by Dr. Ponnurangam Kumaraguru, includes Vinayak Naik, Kuldeep Yadav, Atul Goyal, Ashish Gupta, Dipesh Kumar Singh, and Rushil Khurana.

For developing the initial proof-of-concept, the team ran an incentivized crowd-sourcing scheme in the IIIT-D campus (organized through Facebook) to collect sample spam messages. Pictured below is the tag cloud of the initial database of 4,318 messages, out of which nearly half were spam.

Tag Cloud for SPAM (left) and HAM (right)

Some of the interesting observations made by the team from the initial training set are:

• Almost all messages including an URL are spam.
• Certain special characters like /’ are frequently present in spam messages.
• Typically word count of spam messages is higher. Also the average word length in legitimate messages is shorter due to the presence of abbreviations.

SMSAssassin Architecture

For spam filtering, two techniques were explored by the research group Bayesian and SVM (Support vector machine). SVM is a supervised machine learning technique commonly used for classification. With SpamAssassin, Bayesian approach yielded lower than desired accuracy with spam classification; however, SVM was too computationally heavy for low and mid-range mobile devices, and it had a lower success rate with classification of hams. Dr. Ponnurangam’s team is currently working on an online module that will run a pre-trained SVM based classifier on the server and pass on the results to the app.

SMSAssassin Accuracy

A Symbian app, which will offer full featured spam protection on mobile phones with or without data connection, is currently in the final stages of development. The choice of Symbian as the launch platform might surprise some; however, the decision was likely inspired by the ground situation in India. Nokia still has a significant presence in India, and it dominates the mid-range segment. In terms of volume, I suspect Android is still quite far behind Symbian. That being said, an Android app is planned, and will possibly be released later in the summer. In the meanwhile you can check out the research paper for getting a better understanding of the underlying technology.

With the average computer user getting more tech savvy each day and the average mobile user just catching up, spammers are seeing the mobile platform as a viable option for expanding their notorious business.

Awareness against spam has gone up over the last one year and as a direct result, email spam has gone down visibly. As ZDNet says in this post, only 80% of the current email traffic is spam. Although the figure is nothing to be proud of, it is a vast improvement over earlier spam levels. So much so, that this is supposedly the lowest spam we have seen after 2007. However, in the last quarter of 2010, mobile spams increased up to 46 percent when compared with the fourth quarter of 2009.

The two primary platforms that are victims of mobile spam are undoubtedly Symbian, because it is so popular and Android, because it is so vulnerable. A quote from McAfee as it appears on ZDNet says,

In the past three months, we saw the lowest spam volumes since 2007, but at the same time, we identified attacks on new devices such as smartphones using the Android operating system. Mobile malware and threats have been around for years, but we must now accept them as part of the mobile landscape, both in awareness and deployment.

Spam can be fought with simple awareness and now that businesses are becoming more prominent on the mobile platform too, mobile users need to get aware against spams immediately. However, a comment on the same post at ZDNet has led to the question if these reports are bloated. The comment is quoted below.

This is like pharmaceutical companies not releasing cures for diseases but just symptomatic medications because you can make more money. If your entire business model relies on there being bad stuff out there you want people to think there is lots of bad stuff out there. Biased doesn’t begin to describe.

A few days ago Google gave some hints on how Google web spam team is changing the way web pages are ranked and trying to implement a redesigned “document level classifier” in Google search algorithm.  This whole saga of Google search spam got ignited by Jess Atwood’s post at CodingHorror; so if you have missed the details, read that article and our response article on the issue.

Earlier today, Google Engineer Matt Cutts announced that a new algorithmic change has been launched which will rank the content scraping sites lower in search results. Hence, users are more likely to see those sites higher in search results, who wrote the original content.

Matt said that this change was very much  targeted  and geared towards improving the overall experience of users.

Note that Matt says “lower in search results” and not that these scraped content will never appear in a search result page.

Let’s quickly take an example.

We announced an Apple Ipad Giveaway on 23rd december 2010, which received 254 responses and was pretty much successfull.

Some websites thought it’s wise to scrap that content and put their own “mix” to the scraped material. They changed the Title, modified the URL (thinking that adding the author’s name might just work) and added the same sentences, keywords over and over in the meta description. The result is something like this

The first search result points to this site (thanks Google for your algo).

My point is: Some users will still click the second and third links and arrive to that scraped website who have no original content. Because they are not organizing the Giveaway and can never give an Apple iPad on behalf of Techie Buzz. ( Those who need proof, can read this research article on “Eye tracking in  search results” (PDF), learn the facts and then comment on this post).

So what happens is that some users arrive to the scraped website, can’t find what they are looking for and quit. The will simply go elsewhere.

The Result: We lose those prospective readers who are searching our website,  just because Google showed the scraped sites on search result pages. Agree the number of such readers is way less, but it’s never ZERO.

Our Suggestion To Google Web Spam Team

Don’t show the scraped sites at all. Never. I mean “Why ?”

Google Engineers can easily judge whether the content is an exact photocopy of the source website or not, so there is no point in showing these sites on any of the search result pages. Not even on the 99th page.

Learn from Bing

Surprisingly, the same search at Bing.com shows only one spam link (third one).

This is just an example and I am not saying that Bing is better than Google. But as you can see – Bing shows less number of scraped sites, when you consider a long tail of search.

@Google: We are Praying for better search results.