Top Tools to Manage Postgres in an Enterprise: Administration, Performance, High Availability, and Migration
Table of Contents
This blog post lists tools and recommendations for managing Postgres in the enterprise. While this is a big topic, there are common denominators that we have seen in many Postgres projects and that we recommend for Postgres practitioners.
Our list is not exhaustive, but it provides a starting point.
Tools
We list tools for Development and Administration, Performance Tuning and Monitoring, High Availability and Disaster Recovery, Connection Pooling and Query Routing, and Migration.
Not all the tools listed below are part of the EDB support scope. Many of them are open source tools with varying degrees of support and maintenance. (**) identifies tools that are part of EDB’s support scope.
Development and Administration
| Tool | Description | License |
|---|---|---|
| psql (**) | Postgres core client https://www.postgresql.org/docs/current/app-psql.html |
PostgreSQL |
| pgAdmin (**) | Desktop or web-based; browse and modify a database schema, run queries, debug stored procedures and much more. https://www.pgadmin.org |
PostgreSQL |
| Toad Edge for Postgres | A new version of the classic DBA tool for Oracle https://www.quest.com/products/toad-edge/toad-edge-postgres.aspx |
Quest proprietary license |
| sqlSmith | A fuzz testing tool for developers. sqlSmith generates random queries https://github.com/anse1/sqlsmith |
GPL V3 |
Performance Tuning and Monitoring
| Tool | Description | License |
|---|---|---|
| Postgres Enterprise Manager (**) | All the functionality of pgAdmin, plus 24×7 monitoring and alerting and various enterprise management tools. Includes Tuning Wizard, Wait State Analyser for EDBAS, a SQL Profiler, and monitoring dashboards. https://www.enterprisedb.com/edb-docs/p/edb-postgres-enterprise-manager |
EnterpriseDB proprietary license |
| Pg_stat_statements (**) | A Postgres extension that tracks execution statistics of all SQL statements executed by a server. https://www.postgresql.org/docs/current/pgstatstatements.html |
PostgreSQL |
| Auto_explain (**) | A Postgres extension that logs execution plans of slow queries automatically. | PostgreSQL |
| pgBadger | Very popular Postgres log file analyzer with a graphical output. Help find long-running queries, demanding workloads, etc. https://github.com/darold/pgbadger |
PostgreSQL |
Disaster Recovery and High Availability
| Tool | Description | License |
|---|---|---|
| EDB Postgres Failover Manager (**) | Cluster-aware database availability monitoring, automatic failover and support of manual switchover in support of maintenance activities. https://www.enterprisedb.com/edb-docs/p/edb-postgres-failover-manager |
EnterpriseDB proprietary license |
| EDB Backup And Recovery Tool (**) | Block-level incremental backup for Postgres. https://www.enterprisedb.com/edb-docs/p/edb-backup-and-recovery-tool |
EnterpriseDB proprietary license |
| RepMgr | Replication Manager for Postgres streaming replication and failover https://repmgr.org |
GPL V3 |
| pgBackRest | Advanced backup and recovery tool for Postgres https://pgbackrest.org |
MIT License |
| Barman | Backup and recovery tool for Postgres https://www.pgbarman.org |
GPL V3 |
| Patroni | High availability tool for Postgres, mostly used in containerized deployments https://github.com/zalando/patroni |
MIT License |
Connection Management/Connection Pooling
| Tool | Description | License |
|---|---|---|
| pgPool-II (**) | Connection pooler and query router for Postgres https://www.pgpool.net |
BSD License |
| pgBouncer (**) | Lightweight connection pooler for Postgres https://www.pgbouncer.org |
BSD License |
Migration from Commercial Databases
| Tool | Description | License |
|---|---|---|
| EDB Migration Portal (**) | Web-based Oracle to Postgres migration tool. Maps DDL, DML, packages, stored procedures, and other proprietary extensions to the SQL standard from Oracle to Postgres. https://Migration.EnterpriseDB.com |
EnterpriseDB proprietary license |
| EDB Migration Toolkit (**) | Command-line tool for migration of DDL, DML, and data from Oracle, SQL Server, Sybase and MySQL to Postgres https://www.enterprisedb.com/edb-docs/p/edb-postgres-migration-toolkit |
EnterpriseDB proprietary license |
| ora2PG | Maps Oracle schemas and data types to Postgres; provides some DML transformationhttps://ora2pg.darold.net | GPL V3 |
| Cybertech migrator | Maps Oracle schemas and data types to Postgres, provide DML transformation, provide some PL/SQL transformation and an interface to debug it https://www.cybertec-postgresql.com/en/products/cybertec-migrator/ |
Cybertech proprietary license |
Recommendations
One of the most important lessons learned is: “Don’t go it alone!” Postgres is mature, and in many regards, it behaves like Oracle and SQL Server, but there are significant differences in tooling, management practices, and performance behavior.
The ROI of Postgres projects is very high. Project delays, partial implementations, and substandard performance are the enemies of a timely move to production.
Supported Postgres
We highly recommend that users work with a well-established Postgres company that has a staffed 24X7 support team with commercial-grade support SLAs. That support team must be backed by software engineers who are involved in the development of the Postgres database (check the list of committers and contributors at PostgreSQL.org — don’t work with ‘Postgres Companies’ that don’t have several full-time team members who are on that list. They will not be able to help you in a timely manner on the rare occasion where Postgres has issues).
EnterpriseDB’s Postgres Advanced Server is a managed fork of Postgres. This allows EnterpriseDB to provide features for customers (Oracle compatibility, Resource Management, Query Hints, PCI-compliant password management, etc.) that may otherwise not be possible in the community process.
Technical Account Management
Technical Account Managers (TAMs) are a staple of the commercial software world, especially for mission-critical infrastructure software. TAMs provide an intimate and knowledgeable link between the customer’s needs and the software provider’s engineering team. TAMs greatly improve communication and help make projects successful. They also act like the customer’s advocate and influence the product roadmap.
DBA Services
Specialized Postgres DBAs monitor and manage Postgres databases in the customer’s data center, in cloud IaaS deployments, or in cloud-based DBaaS projects. Postgres is close to Oracle and SQL Server, but some key management processes, such as bloat and vacuum, or backup/HA, are different. Query tuning and performance management practices also differ significantly – especially when considering the requirements of Tier 1 99.99%+ SLA solutions.
Many customers use DBA services as ‘training wheels’ for their first project; other customers focus their in-house DBAs on innovation and data management and use EDB’s DBAs to keep the lights on.
Consulting and Architecture Advice
Highly performing and reliable Postgres architectures are foundational building blocks if an enterprise wants to transition a significant part of their database estate from commercial databases to open source based ones. Understanding how to use Postgres and how to take advantage of its unparalleled innovations, its flexible data model, extensions, GIS and document capabilities, is important in order to achieve the ROI and get value from Open Source.
The Need for an Integrated Platform
The open-source community has provided a plethora of tools and capabilities around Postgres – almost everything from HA to DR and log analyzers, are available as ‘free’ building blocks. However, none of the blocks fit together seamlessly, none of them are on the same release schedule, and they are not covered by the same support SLA. None of that matters if an enterprise can invest enough into a large number of in-house Postgres knowledgeable resources, has plenty of time to mature its Postgres projects, and does not plan to use Postgres for Tier 1 mission-critical applications.
However, that is not acceptable for enterprises that want to take advantage of open source-based technology to reduce cost, drive innovation, and support digital transformation. Those users require:
- An integrated platform that includes management, monitoring, tuning, HA, and DR tools that fit together seamlessly and create a robust management platform
- One integrated release schedule to make sure that the tools work together
- An integrated SLA and support structure for the tools and the database. A whole is only as strong as its weakest part.
Conclusions
The enterprise demands imposed on an open-source based platform, such as Postgres, are in no way different from the demands imposed on closed source software. Increasing use of Postgres for mission critical apps means that the tooling and the best practices align increasingly with traditional soft practices. Integrated platforms, single vendor support solutions, and the need for an agile roadmap are obvious requirements when enterprises start betting on Postgres.