Migrating from Cassandra NoSQL to DynamoDB with AutoScaling

Data is everything when operating a modern enterprise, and it’s typically stored in massive databases. The challenge, however, is understanding which type of database will suit the organization’s needs. If a large amount of unstructured data is being persisted, relying on a relational database tends to have latency restrictions as well as the overhead of determining how to map the data into a relational schema when it has no inherent relational information, to begin with. Further, relational databases sometimes have issues with scaling up to meet operational needs based on data volume or the rate of data persistence required.

The evolution of databases has progressed to NoSQL designs due to the inherent limitations that relational databases have. This approach allows large amounts of unrelated data to be stored and queried at a rate that traditional relational databases just are not capable of doing, and it allows for a higher rate of transactional queries against it for data mining and analysis. Depending on the type and implementation of NoSQL databases, this can be a benefit or a detriment; some NoSQL databases tend to require a fair amount of upkeep and maintenance by IT staff to maintain the operational environment.

This last point was a cause for ClearScale, an AWS Premier Consulting Partner, to find a replacement for the client’s NoSQL Cassandra database that they had deployed into the AWS Cloud. The client’s operations relied heavily on AWS services, but they were struggling with finding efficient ways to maintain their Cassandra deployment. They were devoting a large number of IT personnel to the upkeep and maintenance of Cassandra and were looking for ways that ClearScale could help them identify AWS Services to help optimize this interaction to save time and cost. They also needed a way to improve functionality as well as save on some of the costs associated with their Cassandra implementation on AWS.

The ClearScale Solution – DynamoDB With Autoscaling

During the solution evaluation, ClearScale determined that the Cassandra implementation, although properly deployed, was not ideally suited for the client’s needs. The real issue was not that the client needed to optimize their approach to maintaining the Cassandra deployment, but rather the fact that the only reason they needed to perform this activity in the first place was due in no small part to the reality that Cassandra was not scaling up easily to accommodate greater volumes of data and transactional queries.

The client was in the space of providing their customers not only with a place to gather and store large volumes of information but with business analytic toolsets through a Platform-as-a-Service (PaaS) model to help provide insight for making business decisions; as such, they had a large volume of highly used functions.

After extensive research, ClearScale recommended using the DynamoDB database that could auto-scale with client needs going forward. ClearScale was able to get early access to Amazon’s new version of DynamoDB with autoscaling beta which provided that functionality. This version of the popular NoSQL database was built with the premise that a customer should not be required to actively monitor and manage their database to scale up based on operational requirements.

Without this auto-scaling functionality that was built directly into the beta version of Amazon DynamoDB, ClearScale would have needed to spend a fair amount of time leveraging AWS Lambda to perform similar functionality. With the auto-scaling service enabled, the only thing remaining before migrating the client’s data from Cassandra to DynamoDB was to develop a custom code to address DynamoDB strict requirements on how the data is added or accessed.

DynamoDB does not require customers to setup or manage the database and utilize distributed database clusters. Its power lies in leveraging event-driven triggers, accessing controls using the AWS Identity Access Management (IAM), and is flexible by providing both document and key-value data structures, which in turn provides a very robust and scalable solution for any customer’s complex data structure.

DynamoDB with autoscaling also has the added benefit of already having direct integrations with other services within AWS and heavily leverages REST APIs to make querying easy across multiple platforms without proprietary query language and logic, which is inherent to Cassandra. The added benefit of using DynamoDB further allows customers to fully utilize redundancy across multiple geographical zones as well as recovery of data.

The Benefits

Unlike other engagements where ClearScale has been brought in to design and implement a highly customized solution on AWS, this particular engagement was relatively straightforward in the sense that the effort required was spent mostly on determining what solution will serve this client’s best interests. Then, ClearScale mapped Cassandra’s data schema into the selected DynamoDB database, created a custom script to address DynamoDB-specific data requirements, then re-created the high-volume transactional functions the client required and exposed them back to the client’s data analytics toolsets.

The result of using DynamoDB with autoscaling was a continuous, non-interrupted experience with no loss of data or impact on the client’s customer base. After significant load testing and documentation processes of the new DynamoDB implementation were completed, ClearScale was able to deliver a solution that reduced and eliminated a significant amount of IT upkeep for the client, reduced overall costs, and streamlined transactional functions.

ClearScale believes in finding solutions that will benefit customers and enhance their operations while also delivering value by reducing costs and dependencies on non-optimized solutions. We evaluate each client’s unique and complex business needs to deliver industry-leading technology solutions leveraging the AWS platform.

Learn more about ClearScale cloud database services here.