There is a rapid growth in adoption of Cloud database in Asia now. There were some major challenges a couple of years ago as highlighted by Sachin Dabir (CEO, Ashnik) in his blog in 2014. Sachin had concluded his blog with elements which would bring about the change we are witnessing today, and he was apt in many ways. Things have evolved rapidly since then and they continue to evolve. Internal policies have been revised and regulatory requirements have either been flexed or have become obsolete with new data centres, availability zones and choice of cloud vendors. Today there are many different ISV and solution providers who have ported their solutions across various cloud platforms. In fact the rudimentary excuses given by operations and compliance teams is not relevant any more.
And now many enterprises take cloud first approach for new applications and their database deployment, they face different set of challenges. I have been meeting users and customers across South East Asia and have heard the challenges they have faced. Let me share a few stories to put things in perspective –
1. One of the most successful start-ups in South East Asia started with MySQL as a Service on a popular cloud platform. The reason to choose MySQL was very simple – That was the only database option available with its preferred cloud vendor at that point of time. All the skill building and hiring happened around MySQL and hence it continued with MySQL. I asked the start-up team about some of the challenges they have faced. I realized that those could have been solved quite easily with PostgreSQL or other databases. Now, don’t get me wrong here. I am not in favour of introducing unnecessary complexity and diversity but at the same time I don’t believe that databases should be treated as Swiss Knife.
2.In another case, I came across a situation antipodal to the previous one, where a start-up company diversified its portfolio of databases simply because different databases were available at the click of a button. The judgement was totally made by developers based on their “convenience”, “comfort” and “experience of working with databases”. When the start-up eventually hired a database engineer, he had a tough time managing many different databases. It is very important to use right tool for the right job but you should also spend some time trying to understand how your existing database technologies work. This certainly helps you to understand if new workloads can be accommodated within existing array of technology with a little bit of tuning.
3. We all come across bottlenecks and design flaws leading to performance issues. Same happened with one of the large enterprises we work with, but in its database setup hosted as a service in cloud. Its solution architect revealed that its operations team simply kept on scaling up and scaling out because it was as simple as clicking a button. This masked the major performance issues which they could have solved easily in the early days of application. Eventually it had to fix those issues but only when it became inevitable and after incurring a lot of cost on past scale-up/scale-out resolutions. And as the application aged, the fixes were more complex and tedious. This might not have happened with on-premise database setup, as scaling out is rarely the first option and is not so easy – given the constraints for resources and licenses/subscription.
4. When solutions are available out of box, why would you want to build them or customize them? That is exactly what happened with another successful start-up. Its reporting/warehouse schema hosted in MySQL was getting larger and complex and it was looking for a more efficient choice of database for hosting data warehouse. It settled for Amazon Redshift which is really great for very large size databases. But eventually it ran into some of major drawbacks and limitations of Redshift for that specific workload. Then it decided to build its own DW setup with S3 storage and compute nodes using S3 APIs. It optimized the cost of storage and had flexibility to scale out compute on demand. Unfortunately it had to learn it hard way but now this setup works very well for catering to different teams ranging from reporting to data scientists.
All these stories validated my thought that while cloud adoption for database is going to make operations easy, it needs to be handled differently. I would like to summarize some key take aways from these experiences –
1. Don’t choose a cloud vendor first – Choosing the cloud vendor should be your 2^nd step not the first. It is very important to choose the right technology for your workload and then see which vendor supports it. Established cloud providers like AWS offers better price benefits , vast choice of technologies and type of service. AWS marketplace sas many database vendors who provide Software-as-a-service.
2. Do not undermine the importance of a database professional – Cloud database makes the task of a database engineer or a DBA easier but certainly does not eliminate it. In my previous blog I talked about how DBA’s time can be better utilized in making design decisions instead of engaging him/her in operational tasks. You should think about engaging your DBA/DB engineer early-on in your project and seek his/her inputs before you make a choice of database platform. This will help in containing the diversity of platform and ensuring that databases are chosen and tuned as per the workload.
3. Root cause analysis – When you face performance bottleneck try to evaluate all symptoms. Look at the logs and system utilization and identify the issue instead of taking an easy solution of scaling up. This is not only important for fixing the root cause but also helps in containing your investments. Again, the role of DBA/DB engineer is pivotal in this. He/she can get all the help needed from automated monitoring or monitoring-as-a-service provided by cloud vendors. e.g. AWS Cloudwatch provides important monitoring metrics for various AWS resources.
4. Engage with right partners – A right partner can help you to identify the right choice of tools before you make a decision. This is very important as most of the ISV and some Cloud vendors might push their solutions your way. Some of them either have an interest or incentive in selling a specific solution. You should identify a partner who is agnostic to technology and works with you for solving your business problems instead of simply selling solutions/service to you. While this the most important considerations and learning from these experiences, it is often not easy to find a partner who would work with you as your team.
At Ashnik we help our customers adopt new technologies both on premise and in cloud. We have a team of seasoned solution architects and implementation engineers who are aware of various nuances of database workloads. We have engaged with our customers to provide consultation on application design, choosing right database, designing database architecture and migrating to cloud. If the stories of cloud adoption sounds familiar or interesting to you, please write to us on success@ashnik.com.