ClearScale was approached by an agricultural technology company that manufactured tags for livestock health monitoring to assist them in deploying a solution using IoT technologies to capture and process information from millions of individually tagged animals. The original solution the company had implemented involved numerous individual servers scattered in key locations to capture transmitted tag data. Aggregating this data was a challenge, as was actively monitoring, diagnosing, or anticipating animal health based on the information the tags were transmitting. As the demand for this unique product grew, the company knew it needed a robust and scalable IoT implementation to streamline the entire product operation.
The ClearScale IoT Solution
ClearScale implemented a solution based on AWS best practices. The infrastructure was deployed in a VPC which spans two Availability Zones for service redundancy. AWS services such as IoT, DynamoDB, Kinesis, Kinesis Firehose, Lambda, S3, WAF, CloudFront, and Route53 are leveraged to provide world-class availability and security without additional IT management overhead. While all web application instances and instance-based database services are deployed in a private subnet, only instances/services that require external access (such as Elastic Load Balancers) are exposed to the Internet. This is done to reduce the external attack surface and move those risks onto secure AWS services (IoT, NAT Gateway, Elastic Load Balancers). AWS WAF and CloudFront provide an additional layer of protection against vulnerabilities and DDoS attacks. A duplicate Staging environment was also deployed to validate releases/changes before deployment. Data backups are delivered to a separate Region for DR purposes.
The IoT implementation was focused on data collection from millions of sensor tags attached to livestock. All tags send messages to AWS IoT with the help of a gateway device installed at the customer site. These messages are sent to Kinesis and enriched using Lambda functions. Additional Lambdas process the data and store results in DynamoDB, RedShift, and MySQL RDS for retrieval, deep analysis, and visualization/notification purposes.
The entire backend application is built on AWS services. The customer only needs to manage their application code executed by the Lambda functions. On the front-end, the customer supports the small ASG of servers that host the Web UI that customers access. Their developers can now focus on the core components of their application rather than supporting infrastructure.
In addition to the core application, ClearScale designed and deployed a centralized management capability for the gateway devices. A device image was created that had VPN clients configured to auto-connect to the management VPC and an Ansible agent for configuration management. The customer was able to deploy software updates and security patches to all gateway devices. Static IPs are not available at many customer sites and Support Technicians were able to leverage the VPN network to securely access devices for troubleshooting.
The customer is able to focus on developing their web application and processing algorithms that are the core of their business. The IT management overhead has been almost completely absorbed by AWS services. The customer is also able to stream data into multiple datastores effortlessly to satisfy different query/analytics use cases. CloudFormation templates and Elastic Beanstalk allow the customer to have a reliable DR plan as well as the ability to deploy ad hoc development environments as needed.