Performing a Complex AWS Migration Using EXASOL for HA and Ansible for Deployment Automation

Over the years, ClearScale, an AWS Premier Consulting Partner, has encountered varied and unique requests from clients wishing to utilize AWS solutions and services in order to achieve their specific set of criteria and business goals. Despite the complexity or challenges often inherent in these requests, ClearScale has repeatedly proven that the key to success lies not only in clearly understanding each customer’s particular needs but also in optimizing our AWS-native solutions in order to reduce programmatic and process bottlenecks.

We recently put this to practice for a client who required the construction of a robust development pipeline for migration to AWS. Importantly, the deployment needed to provide several pre-production instances and several production instances for redundancy and failover. As for all projects, ClearScale spent a considerable amount of time gaining an understanding of the customer’s primary requirements and end goals. Once they evaluated and approved our proposed solution, we immediately began our implementation, all the while maintaining consistent communication with the client including regular check-ins and status reports.

Building Out the Application Development Solution using EXASOL for HA and Ansible

In order to successfully carry out our proposed solution, we had to utilize many AWS Services to their fullest. We started by configuring S3 buckets, then instantiating the AWS Virtual Private Cloud (VPC), including the various security groups and gateways that our client would need to leverage in order to access and interact with their environments. ClearScale built out load balancers using Nginx and HAProxy, along with the AWS ECS Clusters and ElastiSearch Cluster.

Since the development pipeline solution needed to work across multiple regions, we instantiated Aurora clusters coupled with EXASOL database clusters for active backup and restore. EXASOL is a high-performance, in-memory MPP database designed specifically for BI, analytics, and reporting that allows you to analyze large volumes of data in real time. The analytic database achieves performance with linear scalability by combining in-memory technology, columnar compression and storage, and massively parallel processing. It also boasts standard SQL interfaces which conferred to our client easy compatibility with pre-existing applications and data structures.

Unlike many other database systems, EXASOL comes with a dedicated cluster storage subsystem that efficiently handles data redundancy and high availability and is optimized for low latency and maximum throughput. Therefore, we implemented this tool for the client to ensure redundancy and high availability. ClearScale also set up HA VPN to allow data replication across regions. As always, the entire implementation was encrypted and AWS Identity Access Management (IAM) Service was implemented to provide authentication oversight for registered users.

ClearScale enabled AWS CloudFormation to deploy the VPC, ECS Clusters, Proxy Services, EXASOL, and Kafka Clusters. We also set up sizeable monitoring services using CloudWatch, CloudTrail, and Sensu to provide active session and audit history for all services and users.

None of the designs would have been nearly as successful had we not leveraged Ansible to help automate the deployment pipeline. Its ability to carefully monitor and curate a deployment made it the ideal choice to reduce manual processes to only a few keystrokes. Without both Ansible and Jenkins for the Continuous Integration (CI) component of the development pipeline (allowing for constant deployment, building, and testing of applications in lower and production environments), our solution would have only solved part of our client’s needs.

These tools provided a seamless and automated approach to development environment management as well as application-level deployment and testing methodologies. This all measured out to providing our client with a more hands-off approach to managing their sizable development needs and allowing them to focus more on the development of the application they are trying to deliver to their customer base.

Deploying to the Cloud Environments

With the more complex development pipeline built out, ClearScale began deploying our solution to a number of environments, including a dedicated Development environment, a Staging environment that mimicked a Production environment which was used for a robust set of testing methodologies, a Demo environment that our client could use when talking with potential and current customers to demonstrate their application in a near-production state, and Production environments that were customer-facing.

Unlike the other three environments, the Production environments required hosting in a redundant fashion to ensure uptime and recovery should anything happen. To prepare the Production environment for any type of situation, ClearScale deployed a Production instance in the AWS East region simultaneous to a similar Production environment in the AWS West region. Using the East as the primary Production instance, the West was configured to serve as the Disaster Recovery instance should anything happen to the primary instance.

Similarly, ClearScale set up a primary Production instance in one AWS region while simultaneously deploying a Disaster Recovery instance in another AWS region. This methodology allowed two primary instances to serve all customer needs and share and sync data between two locations. Meanwhile, the Disaster Recovery locations would serve as effective failovers should the primary locations go down.

ClearScale’s methodical approach of fully engaging our client early in the process makes project development straightforward, gives us the ability to discover and recommend approaches that meet client objectives, and reduces programmatic and process overhead.

At ClearScale, our goal is not only to find and implement seamless AWS solutions that deliver on customers’ desires but also to do everything in our power to streamline the implementation in a manner that reduces resource constraints and provides an optimized experience from start to finish.

Get in touch today to speak with a cloud migration expert and discuss how we can help:

Call us at 1-800-591-0442
Send us an email at sales@clearscale.com
Fill out a Contact Form
Read our Customer Case Studies