Spark Streaming use cases in eCommerce at Verizon Media Taiwan

[jazzwang]

• 演講主題 Talk Topic: Spark Streaming Use Cases in eCommerce at Verizon Media Taiwan
• 講者姓名 Name: Evans Ye
• 單位名稱 Organization & Job Title: Verizon Media, Engineering Manager
• 講者簡介 Self Introduction:

Yu-Hsin Yeh(Evans Ye) is a former VP and PMC member of Apache Bigtop. He worked for Verizon Media(Yahoo) as an Engineering Manager and oversees the system architecture for the entire data team. He loves to code, automate things, and tackling big data challenges. Aside from engineering stuff, he is also an enthusiast in giving talks to share software innovations and cutting-edge technologies. His latest talk were to introduce Apache Calcite on DataCon.TW 2018. Evans had talked about Apache Bigtop on international conferences such as Dataworks Summit 2017 San Jose and Apache: Big Data 2017/2016/2015. The SDACK architecture was proposed by him on DockerCon 2016. In addition to that, 2 times Big Data Innovation Summit speaker, 2 times HadoopCon speaker, 2 times Taiwan Hadoop user group speaker, and dozens of company internal talks make him an experienced presenter.

• 講題摘要 Abstract:

In eCommerce, real-time stream processing is the key solution to capture user's intent and serve the right recommendation to users in timely manner. In this talk, we'll reveal our solutions to enhance users' eCommerce experience through:

• Online ML predicting which emphasizes the real-time user intent over long-term user interest.
• reinforcement learning which captures users' clicks and converts into rewards and bump up the products that are liked by the majority of users.
• real-time push notification that delivers time-sensitive messages to users such as price drop notifications in Yahoo Auctions.

All the mentioned solutions are built based on Spark Streaming fronted with Kafka as reliable/replayable stream data source.