Originally posted here.
Managing and analyzing a network of more than 100,000 sensors requires a server infrastructure that can handle the amount of data it creates. For Blesh that means more than 735 million data entries, which is the amount it collected each month in 2017.
What’s more, the company recently accepted the job of deploying wireless Bluetooth beacons across Istanbul Airport—home to 41 million passengers annually. This was part of the airport’s project to help inform customers in real time about their flight status by measuring real-time passenger traffic. Blesh would need to launch its sensors across a variety of cafés, restaurants, shops, ATMs, and parking areas, which was expected to create an additional 90 million more data points a month. “We love challenging ourselves with complex projects and this project provided that,” says Uǧur Gökdere, cofounder and chief technology officer at Blesh. “We created a unique data-collection strategy involving the airport in order to work around the challenge of multiple retail units in a space that was smaller than usual. A very exciting project.”
The platform needed to integrate new cloud services that would easily measure and analyze big data at scale to help manage its client’s expectations.
Why Amazon Web Services
In early 2016, Blesh was already using Amazon Web Services (AWS) to run its location analytics service infrastructure, and because it sought better big data services, the company began utilizing Amazon Redshift. “The reason we put Amazon’s competitors aside is because of Amazon Redshift. As a data warehouse, it is really fast and incredibly cost-effective,” said Gökdere.
The fully managed data warehouse analyzes large-scale, structured data sets using standard SQL. By the end of the year, the company decided to move its entire analytics platform to AWS to prepare for the airport tracking project’s incoming data streams.
The process of migrating the infrastructure was made simple by the AWS Server Migration Service; in fact, it only took two days. Blesh uses Amazon Elastic Compute Cloud (Amazon EC2), with Elastic Load Balancing, which automatically sends incoming traffic across several instances to vary any traffic load. The Turkish beacon platform also used Amazon Route 53 to connect user requests or data points to infrastructure running on Amazon EC2 instances.
The infrastructure now runs seamlessly. Blesh stores and manages the verification and mining of big data from airport beacons through AWS. The data logs are transferred to Amazon Simple Storage Service (Amazon S3), which triggers Amazon EC2 instances to analyze the data contained in the log files. The processed data is then moved to the Amazon Redshift data warehouse for longer-term analysis.
And just in case a problem does occur, Blesh relies on AWS Business Support to help it run its operations smoothly. “The service we get from AWS Business Support is really great,” says Gökdere. “Their team is extremely helpful and they understand the importance of responding quickly as we’re running production environments that can’t afford to have any downtime.”
The Benefits
Since Blesh began using Amazon Redshift, latency has decreased from 1,000 milliseconds to 100–500 milliseconds, depending on network performance. With other products, which would have had a latency of more than 5 minutes, this feature wouldn’t have been as helpful for the airport. This low latency means that airlines or the airport itself, can send real-time information to travelers in the final 15 minutes of their journey. And as a result, Blesh, using AWS, ensures its customers can utilize its sensor network as effectively as possible.
After moving its entire analytics platform to AWS, Blesh’s capacity to handle big data has significantly increased. The availability of the service is now 99.981 percent. Using the AWS Cloud has also benefited the Blesh IT team—12 professionals as of 2017—which is now spending about 85 percent less time dealing with system maintenance than before. “If we were running a physical infrastructure, we would need at least another three administrators to run the hardware, but instead we can avoid that expenditure altogether,” says Gökdere. These savings, and the company’s 2 million daily ad impressions, have encouraged the company to launch new services and products related to proximity-based advertising and data solutions. This larger portfolio of metrics-based services means Blesh can continue to grow its relationship with customers well into the future.
Learn more about using AWS for big data analytics solutions.