Summary

Telegra is a next-generation telecoms provider with over 20 years of experience as a telephone network operator and telecoms application developer. Headquartered in Germany, telegra’s business is rapidly expanding across Europe and handles 30 million calls and 90 million minutes of call time a month. One of telegra’s successful products is a Web-based automatic call distribution (ACD) solution for call centers. The ACD offering enables organizations to centralize their entire telecommunication in telegra’s cloud and manage it through an integrated web-based interface.

The Challenge

The arrival of services over the internet and the consumerization of IT have significantly changed the dynamics of the telecoms market and the expectations of customers for always-on connectivity. As a carrier, telegra’s customers expect services to be available 24/7 with no downtime. The pandemic further emphasized the importance of reliability, availability, and scalability, as telegra’s call center customers had to send their agents home to work remotely during various lockdowns. As a result of widespread home working, telegra experienced a significant increase in customer adoption, with the new customers looking for out-of-the-box solutions that support remote working.

While telegra’s VoIP-infrastructure always had a primary-primary datacenter model, the web applications, however, used a binary primary-failover datacenter model, which did not allow them to respond quickly enough to changing customer demand and ensure 24/7 availability at the same time. As a result, In 2018, telegra began the process of redesigning its IT infrastructure for more agile service development. This redesign involved moving away from traditional databases, such as Oracle, and adopting an open source approach in their two datacenters in Germany.

The Solution

telegra adopted EDB Postgres Advanced Server (EPAS) with Postgres-BDR across both its new Dusseldorf and Cologne datacenters, which has enabled the company to switch from a primary-failover to a primary-primary setup. Postgres-BDR synchronizes data configuration, routing plans, and call detail records, which has helped telegra keep its customer data highly available and synchronized at all times.

The Results

• Since adopting EPAS and Postgres-BDR, the telegra team has:
• Experienced no major downtime and both datacenters are now always synchronized
• Increased the time available to build improvements into its software applications
• Reduced database complexity, increased security, and cut perpetual licensing costs
• Leveraged Postgres-BDR as a contingency plan for any potential failure or “network split-brain” scenario
• Reinvested the freed-up support time in customer-centric application innovation

Transformation of the datacenter to meet evolving customer demands

Historically, telegra ran two datacenters for call routing, one in Cologne and one in Frankfurt. The databases were run as a primary-failover setup. However, if the primary datacenter crashed, the legacy architecture made it impossible to do automatic failover or load balancing between the datacenters for the applications running on top. Moreover, the necessary manual rebuild of the database replication after a primary switch made it too costly to shutdown the primary datacenter for maintenance.

Starting in 2018, telegra kicked off a program called “Gruene Wiese” or “Greenfield.” It built new datacenters in Dusseldorf and Cologne and adopted a new approach to the software architecture. The company moved to a primary-primary setup for both applications and databases, with load balancing applied to all applications in both datacenters to ensure greater reliability and scalability.

Choosing the right database strategy

In the past, telegra used many different kinds of databases, including MySQL, Oracle, and PostgreSQL. Oracle databases in particular generated significant licensing costs and created a complex admin environment. The legacy primary-failover datacenter setup required shutting down the primary database and rebuilding it afterwards in order to switch to the failover. This involved significant manual work and left telegra with only one primary datacenter at any given time, which was not ideal.

To resolve this challenge, the company considered moving to the public cloud. However, telegra has many clients for whom data security is critical and who insist on their data being hosted in Germany by a local company. Additionally, telegra experienced high volumes of voice traffic across its network, which made delivery via the cloud prohibitively expensive compared to running services in its own data centers.

As a telecoms operator, telegra is also required to have a Point of Present (PoP) to link up with the traditional carriers like Deutsche Telekom and BT, which do not connect to services through the cloud. telegra decided the most effective solution for its architecture modernization strategy was to host the data itself and chose EPAS as its main database.

“We decided to embrace an open source software strategy, because we see it as far more transparent, reliable, and secure,” said Christian Blaesing, Head of IT, telegra. “We know we can review the software at any time, which gives us greater confidence in the functionality and its configurability. I would go as far as saying Postgres gives us greater security than with any closed source database, as well as far greater flexibility to switch vendors and avoid expensive traditional perpetual licensing.”

telegra also decided to implement Postgres-BDR as a critical element of its primary-primary datacenter setup to provide true high availability for PostgreSQL databases used in demanding industry verticals, such as telecommunications, where downtime is unacceptable. Postgres-BDR provides primary-primary replication, which enables customers to build multi-primary clusters with mesh topology and write to any server with the changes sent row-by-row to all the other servers that are part of the same BDR group.

In addition to improving availability, the implementation is cleaner, as there are no complications with a primary-primary in terms of evaluating different setup scenarios. It makes it easier to integrate legacy apps, as changes are only needed for a small number of requirements such as sequences. The availability of good documentation also enables developers to be working with the database in two to three hours.

As a result of implementing Postgres-BDR, telegra now has two datacenters running with load balancing, which helps to deal with the eventuality of a split-brain situation (a failure condition based on servers not communicating and synchronizing their data to each other). For example, if the lines between the datacenters were to go down, they could continue to operate even in a split-brain scenario, because both datacenters are able to handle all call types. Historically, telegra would have done this by closing down one datacenter, but this would have caused a service interruption for customers. Using Postgres-BDR, telegra is now able to scale their solution just by adding another BDR node to support the rapidly growing number of customers.

“Although Postgres is easy to use, it is always reassuring to have someone in the background with expertise to fix any potential challenges, especially as it is now our main database,” said Blaesing. “EDB has always been very responsive if there were any issues. This has given telegra the confidence and peace of mind that it can support the business, as it can no longer cover the range of support requirements in-house.”

Previously, telegra managed PostgreSQL in-house, but working with EDB it had confidence that the setup would be accurate and clean. An EDB BDR specialist worked with telegra using its TPAexec tool to set up the database in two to three days. This tool, which uses Ansible to build PostgreSQL clusters that follow Trusted Postgres Architecture (TPA), is also useful for on-going support issues, as it enables EDB to copy what telegra is doing with the database to reproduce issues and see what might have gone wrong.

Having adopted EPAS and Postgres-BDR two years ago, telegra has not experienced significant downtime, while the replication capability has worked smoothly, which is essential for the high availability of telegra’s services.

Providing stability for a dynamic environment

A key element of telegra’s competitive advantage is the next-generation software capabilities it offers its customers, which include call centers that can average between 300 to 600 call agents. The applications include AI-based, voice-enabled intelligent routing, which route calls more efficiently depending on the caller’s voice instructions. It has applications sitting on top of its network, such as automatic call distribution, which is a web real-time communication client. This means call agents only require a web browser to log on and have calls directed to them. telegra also provides supervisor and management tools, as well as real-time statistics and analysis capabilities.

However, such innovations are only valuable to customers if there is stability in the network. The pandemic further emphasized the importance of reliability and availability, as telegra’s call center customers had to send their agents home to work remotely during various lockdowns. The modernization of telegra’s IT infrastructure, including its primary-primary datacenter strategy, ensured that it was well-placed to provide the right support to these customers.
By adopting EPAS and Postgres-BDR, telegra had the confidence that its core applications would be reliable,robust, and scalable. In such a fast-moving world, this reassurance is critical. It also meant that telegra’s team was no longer consumed with fixing the database and understanding its complexities. Today, they are able to focus on building new functionality and better products, which has enabled them to become even more responsive to the evolving and dynamic demands of their customers.

Looking ahead, the company’s goal is to grow the business across Europe, and given the team’s past experience with PostgreSQL, telegra believes PostgreSQL will play an integral role in driving innovation across the business. As such, the company has decided to move all workloads to PostgreSQL to support its growth plans, and it estimates that in the future its PostgreSQL database will grow to around 1TB to 2TB of data.