<< Back to the main Twitter Blog page

Thursday, August 07, 2008

Making Progress on Spam

We've previously discussed our efforts to defend Twitter against spam. We are making progress—there are bugs while we find our way—but we're making progress too. Recently, we've seen significant impact by introducing limits around how many accounts can be followed on Twitter under certain conditions. These limits are designed to not affect the vast majority of users. However, some people (who are not spammers) have (and will) run into them. We want to shed some light on what we've done and why.

A Work In Progress

First of all, let me note that these limits are still a work in progress. We've had some bugs in them that are still being worked out. And we need to tweak them to adapt to what we learn about our ever-changing system. However, it's clear from our vantage point they've already had a very positive effect in reducing (though not eliminating) the most common type of Twitter abuse: "Follow spam."

What is "Follow Spam?"

Follow spam is the act of following mass numbers of people, not because you're actually interested in their tweets, but simply to gain attention, get views of your profile (and possibly clicks on URLs therein), or (ideally) to get followed back. Many people who are seeking to get attention in this way have even created programs to do the following on their behalf, which enable them to follow thousands of people at the blink of any eye.

As you can imagine, this is a problem. In extreme cases, these automated accounts have followed so many people they've threatened the performance of the entire system. In less-extreme cases, they simply annoy thousands of legitimate users who get an email about this new follower only to find out their interest may not be entirely...sincere. On rare occasions we may see a person who is mass following and actually cares about every tweet—there is an opportunity for us to learn more about this use case and work to provide a better experience.

There Is No Magic Number

So, our challenge is to curb this type of behavior without interfering with non-spammy users—some of whom may just be very enthusiastic followers. What is a reasonable number of people to follow, anyway? Most users may have a hard time finding 500 accounts they are interested in—while others would think a limit of 10,000 is too low.

Also, people approach Twitter in different ways. Some think you should follow everyone who follows you. Personally, I don't because that would render Twitter unusable for me. I "only" follow about 700 accounts—less than 5% of the 16,000 who follow me. (Mr. Obama may have time to keep up with 50,000 people, but I'm a busy guy!)

The point is, there is no right or wrong. And there is no perfect formula. We do our best by taking a multi-dimensional approach. We look at a number of factors—including how many people are following you back—before applying limits. We don't reveal exact limits, because it's somewhat complicated and, more importantly, if you were to tell spammers exactly what the filtering rules are on your email or, say, Google's PageRank, they'd just engineer their way around them much more easily.

Making Progress

Like I said, this is still a work in progress. The good news for most people is that we're taking measures to reduce junk in the system—and it's working. The bad news for some is that it's possible you'll run into a limit and get frustrated. If that happens, please let us know. We want to learn how people want to use Twitter. (Note: We intend to allow you to follow at least as many people as follow you, though there are cases where that might not yet be the case. We will fix that.)

By the way, this is only a small part of our approach to spam in general. We'll be talking more soon about other measures we're taking. Thanks for hanging with us as we figure everything out.

24 Comments:

Blogger Dan Hollings said...

Twitter Invented in 1935? Who would have thunk! I wonder if they had spam?

8/7/08 3:18 PM  
Anonymous Brad said...

It's been 8 days since I sent opened a support ticket with Twitter support about this very issue and have not received 1 response yet.

You ask people to contact support, but then we hear nothing?

8/7/08 3:53 PM  
Anonymous Greg said...

Forget follow spam, what apbout spammers who are forcing follows so they appear in your updates if you like it or not?

8/7/08 4:30 PM  
Anonymous Ian said...

I would suggest a limit by time and number. For example, limit to following 200 accounts in any 72 hour period. Any more than that should require pre-approval. This would mean spammers would only get about 10 clicks on their tweeted advert per week, and maybe only 2 sales, so it wouldn't be worth it for them.

Any chance of a mobile version of this blog?!

8/7/08 5:42 PM  
Anonymous giania said...

I agree with this overall methodology, and I wish you the best of luck enforcing it. There are actual people, not just robots, who think that following MASSIVE numbers of people is a good idea.

With your recent support of (and subsequent acquisition of) Summize, you would think that people would get that finding out what's hopping on twitter doesn't mean following every user in the system.

8/7/08 6:03 PM  
OpenID Vicki Brown said...

> we intend to allow you to follow at least as many people as follow you,

that's not good enough. Some people follow many to invent their own public timeline (in their preferred language). Some may want to give many people an ability to DM.

Some people (let's face it) like to read but they themselves are boring.

If you "want to learn how people want to use Twitter" then that's what you need to do. Don't try Design y Omiscience. That trick never works.

And make a new category for help tickets - "I've hit a limit and I'm deeply frustrated" so that people like brad don't wait 8 days with no response!

8/7/08 10:26 PM  
Blogger Joey said...

For most users like myself, this is not a problem.

But thanks for the warning.

8/8/08 10:24 AM  
Anonymous Michael said...

I am the owner of one of the most effective and largest online advertising groups in the world. Just wanted to say that a lot of my traffic comes from you guys. Thanks and keep up the good work!
M.C.

8/9/08 1:25 AM  
Blogger Gavin Knight said...

intercepting follower spam at inception is exactly why I have asked multiple times for the option to have to approve new followers even though my updates are public

8/10/08 4:24 AM  
Anonymous Ashok said...

If the email notifying me of a new follower just listed how many people the person follows (and ideally how many were added very recently) then I'd find it easier to distinguish the mass-followers from interesting people.

You'd need to tinker to find a good cut off for 'very recently', but I'd have thought a few hours would work pretty well.

(If it also held back on sending the notification for a small while, then the recent-follows count should show up batch following even for people who are towards the start of the batch; that might be a bit too sophisticated though.)

8/10/08 9:15 AM  
Blogger AxedbyDax said...

I think it is really hard to find out those spammers and those who are not.There are also people who wanted to have or meet new friends and we might consider them spammers as well and that will be unfair to them.Cheers.

8/10/08 9:12 PM  
OpenID feedle said...

Here's a suggestion.

Automatic account suspension / deletion if you have a significant number of "blocks" against your account. Not a percentage, but a numerical account.

You have to wonder if somebody generates 100 blocks they're doing something to annoy and harass people, and probably shouldn't be on the service. Period.

8/10/08 10:50 PM  
Anonymous michael said...

To fight spammers on Twitter, sounds like a job that can be very system intensive. Developers and staff at Twitter have the data and real-time tools to study, track, and monitor abuse and strange behavior.

In addition to monitoring nefarious activities by spammers, the staff must still keep the system running with as little interruption as possible. It could be a difficult, but is a necessary task. It is quite unfortunate that with Web 2.0's advances that individuals seek to exploit its usefulness and features cherished by countless users.

8/11/08 9:41 AM  
Blogger Stephanie Booth said...

You guys are probably doing it, but I guess factoring in time and tweets into the limit makes sense. Somebody who has 3 tweets, a one-week-old account, and is trying to follow 2000 people is probably not legit.

OTOH, Roberts and Loïcs who have been around a while, are clearly active users, get significant numbers of @ and DMs... well, I guess those are hints that they are probably legit, even though they may follow a spam-like number of people.

8/12/08 6:29 AM  
Anonymous PaulFeldsAtGmailDotCom said...

Plain upper limit, period.

Twitter have a public timeline and open API. There is no point in following 50,000+ people. Period. Just cache the public timeline.

Why would someone follow 50,000+ people rather than follow the public timeline?

A) The public timeline is too spam filled.

Easy spam protection there. One sees a Tweet is a spam it is easily reported on the web interface and through Twhirl etc. Spam keyword/link catching and reporting in the public timeline (even take it to somewhere else public but way obscured from people and robots maybe). [Or rather, this is the way things should be.]

B) Those that follow in person rather than using the API.

I guess (especially in Twitter's present formative stages) they're cacheing in on the fact that most Twitters that get the 'follow' update for are real people and would 'listen in' back at you. Soa knee jerk is to listen back.

In this sense, S**ble and the like are massive psychological spammers (I'm deeming myself important, I have many followers, so you should deem me important too). He does not know me. Already following 50,000 people is he really interested in my Tweets? Is there a staff reading them or just a selective few following a few preferred people but replying through the massive follower account to keep up the appearances of really following all? Why is he following me?

I find this odd and from that he mildly disturbes me. But my timeline is public so if he wants is thrills that way so be it.

Sounded harsh but that's the only reasonng I can think of, so:

B.20) Absolute resource limit. If really want to actively follow X00,000 people rather than do it through the public API then I must have some people working for me (and paying them either dollars or cult points) or ignoring a whole bunch I wouldn't do if I focussed more. And I create a real drain on the service. So charge me, simple.

C) Absolute limits work to prevent super-spammers but fail simply because I could create a single account following 100,000 people or 100 accounts all synced to follow 1000 people.

As suggested above, charge those that use an excessive amount of single human resources.

Keep a limit on each account for follower growth per day and keep a follower:followed limit but cross-check the 'circles' formed by who follows whom. Do-able.

My 2c. I am available for consultation ;)

Just follow me: followafollowerwhore@twitter.twit.comish

;)

8/12/08 8:44 AM  
Blogger Observador said...

Please,

Put a captcha engine into FOLLOW button, to minimize spammers actions.

8/12/08 9:29 AM  
Blogger Seocracy said...

I think this is an excellent move on your guys' part. Lets hope it works!

8/12/08 10:28 AM  
Blogger Duane said...

I'm actually just starting work on integration of Twitter with my existing application, and wondering now if I should skip it. Here's the thing - I only ever want to send direct messages to users. I want to do it via Twitter, rather than SMS or Email or something else, because it seems like the optimal combination of timeliness and delivery mechanism (i.e. the user gets what tweets they want, how they want them). But in order to message a user, I have to follow that user. Thus I'll eventually be following lots of people, even though I'm not listening to anything they're sending me. It's strictly a one-way thing. Maybe I'd implement a simple bot where people could send commands to tweak app statuses, but that'd be the only time the # of followers would come into it.

8/12/08 10:39 AM  
Blogger ltdraper said...

> they simply annoy thousands of legitimate users who get an email about this new follower only to find out their interest may not be entirely...sincere

People really misuse the word "spam".

Isn't it Twitter that's sending the unwanted email? And why don't users just change their preferences to not receive email from twitter?

Following someone is like walking up to someone at a party and saying "Hi, how are you?" If they don't want to talk, then you can move on. But you both went to the party, so classifying that as spam is a bit ridiculous. Or is Twitter's idea of going to a party only talking to people you already know?

And why in the world would someone not be allowed to follow additional people when they have more people following them than they follow?

8/12/08 11:09 AM  
Blogger Otto said...

Here's an idea: make it easier for me to block/flag somebody as a spammer. Because when I get follow spam now, blocking those people is kind of a pain in the ass.

8/13/08 7:11 AM  
Blogger dave said...

Suggestion: In new follower emails, could you put a new follower's numbers for following/followers, or even just a ratio? That way I don't think I have a new follower, click on their profile, only to see someone following 2000 people and only having one follower...

8/14/08 8:50 AM  
OpenID chmac said...

I think HashCash could solve these problems. It could make follow spam (and many other types of spam) impractical, but would be unnoticed by the average user.

8/14/08 9:17 PM  
Anonymous Dr Saxe said...

50,000, that is a huge number that Mr. Obama is being known for. This will unfortunatley always be a challenge, but in the mean time, good luck with it all.
Respectfully,
Dr Saxe

8/16/08 8:33 PM  
Anonymous Anonymous said...

On block, have it pop a dialogue that says 'are you blocking for personal reasons, or is this a spammer?'

If flagged as the latter by several disconnected people (other than via the account in question), the account is suspended immediately (cannot follow new people) and then reviewed (which could lead to it being canned).

If the account is blocked for personal reasons, this could also lead to a review - but the threshold would be different (both thresholds would be unpublished).

An account in review would still be visible to other connected accounts, but might be moved into a new category (to visually seperate it from following and followers)

If an account adds X follows per day then it's flagged for review. (I reckon 100 would do it, with the threshold for being marked as a spammer being a fraction of the above).

(With user mod, you'd have to have a review or democrats would flag McCain, if he's on the system, and republicans would flag Obama, who is on the system)

8/20/08 1:50 AM  

Post a Comment

Links to this post:

Create a Link

<< Home