3 min read

Twitter releases open source Anomaly Detection tool

Graham CLULEY

January 11, 2015

Ad One product to protect all your devices, without slowing them down.
Free 90-day trial
Twitter releases open source Anomaly Detection tool

Most of the time we’re pretty normal.

We do the same thing every day. We get up, go to work, come home, go to sleep. At weekends we pop out to the cinema, and maybe take our partner to a restaurant.

Nothing very unusual.

But every now and then, fed up with the bad weather or just tired of the same-old same-old, we book an overseas holiday.

And for two weeks our lives are full of fun and sunshine. We go out to dinner every day, rent a boat to take us around the island, and spend a fortune on gifts for the friends we left back at home.

From a credit card’s point of view, that behaviour looks unusual. Indeed, you might have had your bank ring you up, asking to verify that you really are visiting country, perhaps worried that your card might have been stolen.

What the bank’s computer systems have detected is an anomaly. And as part of the fight against fraud, they are checking in with you that everything’s okay before they cancel your card. This, by the way, is a good reason why it can be helpful to tell your bank if you are travelling to the different part of the world for a short time.

Anomalies, of course, don’t just happen with banks.

They can happen on websites too.

Twitter this week released an open source tool that can be used by software developers to detect various types of anomalies.

Twitter has been using the tool, called AnomalyDetection, to detect anomalies on its platform.

Anomalies might occur on Twitter when, say, there is a breaking news story, or the final episode of a cult TV show airs.

In a blog post about its anomaly detection technology, Twitter describes the spike in the number of photos it saw uploaded to Twitter on Christmas Eve, Christmas Day and New Year’s Eve.

xmas-eve-2014

That’s clearly a legitimate anomaly, because of the days on which it happened, but the tool could also be used to detect malicious behaviour on Twitter, such as the spike in traffic that could be caused by spammers or bot activity designed to augment follower counts.

If you didn’t think you were being watched on Twitter, think again.

As we engage more and more with online systems and cloud services, chances are that internet companies will be trying to learn more about our activities and find a way to make use of big data.

Sometimes it might be to determine if newly rolled-out systems are working properly, if a user interface change is making a difference in engagement, or to predict if malicious activity is taking place. Other times, it might be as a simple as determining group behaviours in order to make sense of the chaos, and make recommendations and change advertisements on the fly.

Twitter says it is open sourcing AnomalyDetection in order to encourage others to contribute their expertise to the project:

“Early detection of anomalies plays a key role in ensuring high-fidelity data is available to our own product teams and those of our data partners. This package helps us monitor spikes in user engagement on the platform surrounding holidays, major sporting events or during breaking news. The package can be used to find such bots or spam, as well as detect anomalies in system metrics after a new software release. We`re open-sourcing AnomalyDetection because we`d like the public community to evolve the package and learn from it as we have.”

You can read more about AnomalyDetection on Twitter’s engineering blog.

tags


Author



Right now

Top posts

What is medical identity theft and how to protect against it

What is medical identity theft and how to protect against it

July 27, 2022

2 min read
Curious about Omegle? Here’s how the roulette-style chat platform can threaten your online privacy and security

Curious about Omegle? Here’s how the roulette-style chat platform can threaten your online privacy and security

July 07, 2022

5 min read
Identifying and Dealing with Online Bullying Is Not Impossible - School Presentation Inside

Identifying and Dealing with Online Bullying Is Not Impossible - School Presentation Inside

June 28, 2022

2 min read
Let’s Celebrate World Social Media Day by Improving Your Privacy and Security Online

Let’s Celebrate World Social Media Day by Improving Your Privacy and Security Online

June 28, 2022

3 min read
Bitdefender Reveals the Top Cyber Threats Faced by Consumers in 2021

Bitdefender Reveals the Top Cyber Threats Faced by Consumers in 2021

June 22, 2022

1 min read
Scam alert: Cybercrooks use shady investment domain to scam keen investors out of money and data

Scam alert: Cybercrooks use shady investment domain to scam keen investors out of money and data

May 24, 2022

3 min read

FOLLOW US ON

SOCIAL MEDIA


You might also like

Chinese criminals scam kids desperate to play games for more than three hours a week Chinese criminals scam kids desperate to play games for more than three hours a week
Graham CLULEY

August 12, 2022

2 min read
Sophisticated Smishing Attack on Twilio Leads to Employee Credential Leak and Data Breach Sophisticated Smishing Attack on Twilio Leads to Employee Credential Leak and Data Breach
Silviu STAHIE

August 09, 2022

1 min read
Attackers Hit German Chambers of Industry and Commerce; All Digital Services Down Attackers Hit German Chambers of Industry and Commerce; All Digital Services Down
Silviu STAHIE

August 05, 2022

1 min read