Rebuilding Infrastructure following AWS and DevOps Best Practices

One nonprofit organization, a client of ClearScale’s, an AWS Premier Consulting Partner, has attempted to manually deploy Craft CMS, a modern, forward-facing platform in AWS. But they ran into some troubleshooting errors and needed to review infrastructure, redesign it following AWS and DevOps best practices, and then leverage the scalability and automation that AWS can provide, in order to minimize operations team overhead from their provisioning and managing environment.

The ClearScale Solution – DevOps Best Practices

Since its inception, Amazon Web Services has created numerous cloud-based services to give its clients the ability to create phenomenal architectures with minimal investment and high scalability and redundancy. Once ClearScale had identified its new client’s needs and performed a robust audit of the client’s environment, it identified several areas of improvement and designed and built the necessary scalable and highly available infrastructure.

ClearScale isolated each application tier with security groups and then deployed them in a secure Virtual Private Cloud (VPC) across multiple subnets and Availability Zones (AZ). With exception of the OpenVPN host, NAT Gateways, and the elastic load balancers (ELB) for routing external traffic to the VPC, all EC2 servers and RDS instances were placed within private subnets for added security.

The client’s implementation in the AWS Cloud did not use auto-scaling, as this was inefficient. In order to achieve success, ClearScale configured the client’s frontend web instances to leverage Auto Scaling Groups (ASG). Once completed, ClearScale created scale-up and scale-down policies based on CPU usage in the ASG, and the entire setup was tied to the CloudWatch metrics that AWS offers as part of its services.

ClearScale also implemented Amazon’s Elastic Load Balancer (ELB) for use by the ASG. Using the ELB, ClearScale was able to set up monitoring of the health of any instance and then dynamically route traffic to healthy instances as needed, regardless of the Availability Zone they were in. By taking this approach, ClearScale provided a level of fault tolerance that our client had not had previously and allowed for true load-balancing behavior by allowing the system to monitor and route traffic as needed without manual intervention on the part of the client.

Elastic File System (EFS) was used to serve as a centralized repository for any files that needed to be shared across multiple AWS EC2 instances. As AWS doesn’t provide a built-in backup tool for EFS, ClearScale looked to Rsync, a tool that synchronizes files and directories between two different systems. ClearScale decided on using AWS Systems Manager document (SSM document) to define the Rsync command to perform. The team also recognized, that launching a backup process every day using CloudWatch was the best solution.

The client had a single database instance in their existing environment that was not configured for high availability. To overcome this, ClearScale set up, configured, and migrated the client’s database to AWS Relational Database Service (RDS) Aurora, which provides a ready-to-use scalable database solution with integrated backups and high availability features.

The final improvement implemented was the use of Terraform, a service that acts as a template for infrastructure rollouts and retains information about AWS components and their interactions. The configuration of these templates was the element in the client’s deployment process that was missing. With Terraform, our client could easily deploy environments or change infrastructure deployment templates in the future to meet the changing needs of their product or customer base.

Network Diagram

The Benefits

Since 2011, ClearScale has found unique approaches to any number of sizable challenges. Sometimes, it requires brand new architectures and home-grown services or third-party tools to interact in unique ways. Other times, it means getting back to the basics and using proven technology to solve challenging scenarios.

The approach that ClearScale adopted had immediate success. The client’s team can now be self-sufficient in launching new or customizing current infrastructure stacks, allowing them to support rapid growth. For the ops team, AWS enabled on-demand elastic scalability and high availability that does not require long procurement cycles or manual labor, at the same time keeping performance relatively high. This allows the team to save many weeks from each capacity cycle and focus on more business-critical projects.

The next phase of this project will probably enable multi-region support, so instances of the application can be launched in multiple regions with AWS presence to provide the best response times based on client locations and to provide geographical diversity and better availability.

ClearScale has shown time and again that the competencies that AWS has bestowed demonstrate our ability to not only understand the complexities of the AWS service landscape but to augment these services with creative approaches designed to meet any client-critical needs. You can learn more about ClearScale’s DevOps practices here.