I recently finished a course called "Programming in Scala for Big Data Systems." We spent a lot of time learning the fundamentals of functional programming, and exploring the popular tools used for big data analytics. For the final project we were tasked with building an application using a streaming data library. I chose to use Akka Streams to process train arrival prediction times received from the MBTA API.
Using Akka’s built-in backpressure functionality, a stream of prediction events is throttled to one per second to simulate the speed of the MBTA streaming API. Predictions are stored in an in-memory map, and printed to the console. When an update is received for a prediction that was previously reported, an updated prediction is printed to the console. The result is a human readable stream of predicted arrival times, and updates when trains are running early or late.
The source code for the project can be found on GitHub.
