Data Migration and Automation simplified for a multi-national mobile advertising company
The Customer:
The customer is a multinational mobile advertising technology company based in India. Known to be one of the top mobile ad-serving companies, the customer’s mobile-first platform enables brands, developers and publishers to engage with their end consumers through contextual advertising.
The Challenge:
The mobile-ads technology company approached Ashnik for the mammoth task of upgrading their legacy and unsupported Postgres servers, with its complex replication & operating system, and migrate to the newer Postgres version with latest features like Logical Replication.
The Solution:
Ashnik proposed automating the Postgres deployment, streaming replication, scale-up and scale-down, maintenance jobs and table level logical replication using the Automation Solution developed by Ashnik.
The Benefits:
Ashnik helped the customer save on costs by using the Postgres enterprise version. We helped the customer improve their efficiency by upgrading their architecture and reduced the total billings for the customer in the long run.
Customer Background
The customer is a multinational mobile advertising technology company based in India. Known to be one of the top mobile ad-serving companies, the customer’s mobile-first platform enables brands, developers and publishers to engage with their end consumers through contextual advertising.
Customer Setup
During the onset of their business, the customer started off with certain technologies and platforms, and with time kept adding more applications on various platforms. Some core platform versions, infra, and technologies weren’t upgraded due to the assumed complexities and dependencies. One such core application and infra of their setup had become outdated and unsupported.
This further disabled the customer to continue using the latest features of Postgres (such as table level logical replication) and remain on the complex Slony replication – this was key as it was being used for universal data distribution across geo locations.
Besides, the customer had moved their on-prem Postgres to a public cloud using the Lift and Shift method. At the time, they also had automation for scale-up and scale-down of Postgres instances across Geo locations, supporting the increase and decrease of workload. They had 50+ production servers with a range of 10 GB to 1 TB that ran on streaming replication (for read-only and standby) and Slony replication (for reporting and search) – making it highly complex and old-fashioned.
The Challenges
The mobile-ads technology company approached Ashnik for the mammoth task of upgrading their legacy and unsupported Postgres servers, with its complex replication & operating system, and migrate to the newer Postgres version with latest features like Logical Replication.
Additionally, they intended to automate:
- Scale-up and scale-down of Postgres instances
- Maintenance tasks for Cloud Native Database using IaaS platform on Public cloud
The Consulting Solution
Ashnik worked closely with the customer’s business, application, database, and infra teams to understand their requirements and the expected outcomes keeping in mind their existing infra, Database and application landscape.
Based on the discovery sessions, we created different option plans of upgrade with tallying pros and cons. Options included:
- A native backup and restore with automation to make the migration faster
- An out of the box replication from source to target (native replication was not an option as the legacy version didn’t support the new version for replication)
- Upgradation with tools like Pentaho and Kafka for real-time data migration
The Chosen Approach
Ashnik suggested native backup and restore with automation – this would be the best-fit approach for the customer, also considering the downtime.
We also proposed automating the Postgres deployment, streaming replication, scale-up and scale-down, maintenance jobs and table level logical replication using the Automation Solution developed by Ashnik.
Since the go-live deadlines were tight, we ensured testing on Test and UAT boxes, knowing well that automation will principally help here.
The Execution
Ashnik’s DBA and Automation team started working on customizing the automation script on Public Cloud IaaS. The scripts were tested on Test and UAT boxes along with customer production clone setup.
What we did:
- Replaced the complex Slony replication with newer Logical Replication (table level) with source coming from 8 different servers and target tables spread across geos as subscriptions
- Postgres Logical Replication, based on Pub-Sub architecture, was used to extract the sub-set of data and distribute it with local write
- Setup the automation for streaming replication with cascading in different geo location
Go-Live Action Plan
Ashnik team had planned which applications and workload will move first and which later and how we tackle issues if any arise on the go-live day.
Main Challenge: More than 60 servers with over 30 applications generating millions of dollars’ worth business could be impacted across geos – needed us to painstakingly devise every aspect part of the plan.
Going Live:
Day 1 – Went live with a few large applications, observed for 24 hours – everything went smooth, no issues were observed.
Day 2 – Initial activities went well, but we faced a hitch during the data sync for table level logical replication for a few subscribers. With Publisher and Subscriber clean up solutions and some mindful techniques, we solved the challenge.
We efficiently went live and the customer declared production live for all their applications across geos. The Ashnik team was on standby to observe the data and events carefully for the next few days. Although, the migration was smooth and witnessed no downtime.
Top Benefits to Customer
- The customer was able to get an upgraded and modernized version of Postgres in lieu of their unsupported and legacy Postgres – along with their operating system and other lib upgrades
- The company saved costs on Postgres Enterprise version and its licensing fees, as the open source version was used here
- Upgraded from archaic Slony replication to new Logical replication for better efficacy
- The customer can now use automation on Public Cloud IaaS platform, thanks to the usage of the latest Postgres version – means they save further costs as opposed to using Public Cloud DBaaS for a longer duration
Key Highlights
- Over 60 servers migrated on production with scale-up and scale-down automation. Provision to add more servers whenever there is a demand. There are around 600 cores on production that got migrated to production on Azure Cloud
- With our Automation Solution, the Customer will not only use for Postgres deployment on-demand, but also for Postgres replication configuration for HA / DR, maintenance and monitoring, scale-up and scale-down, and upgrade tasks
- Ashnik provided on-demand scripts to ensure that the customer can use it as a Cloud-Native Database on the IaaS platform – saving them long term associated costs and ease of maintenance