Keeping Redditors safe in real-time with Flink Stateful Functions скачать в хорошем качестве

Keeping Redditors safe in real-time with Flink Stateful Functions

Reddit is home to more than 100,000 communities and 52 million daily active users. But it's not all cat videos and memes: Reddit's Anti-Evil teams fight spam, harassment and other threats every day. Automating the detection and actioning of harmful user behaviour and content in real-time is essential to keeping Redditors and communities on Reddit safe. To enable this work, we have built Snooron, a dynamically configurable, multi-tenant real-time stream processing platform based on Flink Stateful Functions. Snooron enables teams to focus on the business logic and data flow of their applications. Under the hood, Snooron manages resources and handles deployment and scaling by integrating with Flink, Kubernetes and Apache Kafka. Snooron is now being used for a variety of real-time use cases within Safety and beyond. In this talk, we will share our journey and lessons learned, our motivation, design considerations, implementation, roll-out and outline some of the use cases in Safety.