top band

Sunday 1:50 p.m.–2:20 p.m.

Finding Spammers & Scammers through Rate Tracking with Python & Redis

Mica Swyers, Jay Chan

Audience level:
Novice
Category:
Best Practices & Patterns

Description

This talk provides an introduction to rate tracking as well as an explanation of a particularly cool way to implement it. You will learn what rate tracking is, why you would want to do it, and then how you can use build a Redis-backed “velocity engine” in Python to do just that.

Abstract

Sometimes, you want to modify a user experience based on rate, but first, you need a way to track how fast s/he is doing something. How to go about doing this? “Classic” rate tracking involves incrementing a count when a user performs a specific action of interest in a given period of time. We’ll discuss why this method is not as effective as we would like, especially given the goal of rate limiting, or restricting access based on rate, namely: - no way to archive this data - the difficulty of implementing a sliding time window - the lack of granularity that we want for writing rate limiting rules Next, we will explain the “velocity engine,” the more evolved rate tracker that we built in Python at Eventbrite. We’ll cover: - our use of the redis-py library to implement a Redis data store and how Redis versions 2.6 and 2.7 affect our implementation - how we generate keyspaces and facets in Redis, as well as the partitioning of each bucket - translating from the Redis internal structure to a more readable list of rates - our expiration strategy for keyspaces and the “housekeeper” module that keep Redis clean despite tons of rate data - how this smarter implementation allows us to do more nuanced rate limiting by writing rules with a greater granularity You will leave this talk with a better understanding of rate tracking and how smart rate tracking is a great foundation to set for better rate limiting.
bottom band background