Lessons learnt from Migrating to a Stateful Streaming Framework

摘要：

In modern applications of streaming frameworks, stateful streaming is arguably one of the most important use cases. Flink, as a well-supported streaming framework for stateful streaming, readily helps developers spend fewer efforts on system deployment and focus more on the business logic. Nevertheless, upgrading from an existing production system to a new one with stateful streaming can still be a challenging task for any development team. In this talk, we will share our experience in migrating an existing system at Appier (an AI-based startup specialized with B2B solutions) to stateful streaming with Flink. We will first discuss how stateful streaming matches our business logic and its potential benefits. Then, we review the obstacles that we have encountered during migration and present our solutions to conquer them. We hope that our experience and tips shared in this talk hints future users to prepare themselves towards applying Flink in their production systems more painlessly.

講者簡介：

Wei-Che(Tony) Wei is a software engineer on Data Platform Team at Appier. He works on providing general facilities for internal users to access data by leveraging different open sources, such as Flink, Spark, and Kafka. Recently, he focuses on building a streaming platform to let users benefit from the advantage of the stateful streaming framework. And he has been contributing to Flink as well.
https://github.com/tony810430

• ← ONE FOR ALL! Using Apache Calcite to make SQL smart
• Develop Kafka Streams application on your laptop →