Scaleway dedicated server PostgreSQL performance

During a recent cloud migration project that had Scaleway as target cloud I stumbled upon their Dedibox Start-9-M dedicated server offering, delivering an AMD Ryzen™ 5 PRO 3600 (6C/12T, 3.6 GHz) machine with 32 GB of memory and two 1 TB NVMe disks for a price point of 39,99€/month. The two NVMe disks paired with the attractive price make this a nice offer for IO-intensive database workloads.

This post evaluates the performance characteristics of running a PostgreSQL 18 database on this machine. For comparison, I will try to match the performance using a managed AWS RDS PostgreSQL database.

Test setup

To keep the test setups comparable I tried to match Scaleway’s dedicated server specs against the offered instance types for AWS RDS, and tuned the underlying IO to match the performance numbers.

• I used PostgreSQL’s included pgbench tool to create measurable load
  • using an artificial test load generator like pgbench will naturally never reproduce the exact workload properties of your application, but it provides a good first indication of the overall performance and helps to make different systems comparable
  • all tested databases were initialized by pgbench with a scale factor of 25,000, resulting in a roughly 350 GB database
  • before each test run, a two-minute warmup phase was executed to warm up caches
  • all test runs were executed on a dedicated cloud instance with at least 8 vCPUs and 16 GB of memory in the same network as the database to avoid interference with the database itself
  • the number of simulated clients was increased until max transactions per second was hit (32, 64, 128, 256, 512, 1024)
• no instance resizing for AWS RDS instances was performed to make sure the configuration parameters generated by AWS match the instance size
  • the RDS instances were only tuned by adjusting the provisioned IOPS for the io2 storage backend
• the two 1 TB NVMe disks for the dedicated machine are used as a soft RAID 1
  • the test runs were conducted with the RAID in full sync
• The database parameters on the dedicated machine were tuned using pgtune using the dedicated machine specs (DB Version: 18, OS Type: linux, DB Type: web, Total Memory (RAM): 32 GB, CPUs num: 12, Connections num: 1024, Storage: ssd)
  • full config see here
• The testing infrastructure and code is available here

Baseline run

To get a performance baseline the first test was executed with the default of 1000 IOPS for the two AWS RDS machines. Not surprisingly the two raw NVMe disks outperform RDS instances here by a large margin.

More IOPS

To gauge the impact of more IOPS, the second run was conducted with 5000 IOPS. The needle moved a bit, but the overall performance is still lacking compared to raw metal.

Even more IOPS

For the next run the IOPS for the RDS instances was dialed up to 20,000. At 32 and 64 clients AWS RDS finally caught up to par with the dedicated server, but from 128 clients onwards the raw metal still has an advantage.

No more IOPS and cost

At this point my AWS billing alert was already screaming at the top of its lungs, and since it is my private account ( and money) I stopped the test series here and stepped back to look at what those performance numbers mean money wise:

^{3 4}

Looking at the raw numbers of performance you get per € spent, the fact that the founder of AWS is able to send rockets to space suddenly makes sense. Of course the whole picture is a little bit more differentiated and the cheap self-hosted solution has some serious drawbacks, that can be worthwhile to accept in some environments.

Conclusion

To put those numbers into perspective, operating bare-metal definitely imposes some operational overhead on the team. How much depends on the context and requirements and is almost never a black/white decision. From my experience there are some nuances that may allow you to save a lot of money without compromising on data safety or diverting too much resources from your original mission towards database operations, especially for IO-intensive workloads.

All-in vs. partial migration

Using a bare metal database need not be an all-or-nothing decision, depending on your workload it can be feasible to just move the database to dedicated machines, and have the rest still running in a Hyperscaler context. Most important aspects here are the database traffic patterns, egress can get expensive when the services send a lot of data to the DB, and latency can also be a concern.

Operational complexity

The managed AWS RDS solutions really shine when it comes to the more complex topics, like geographical distribution, cluster setups and automatic failover. To replicate this requires significant effort and knowledge. On the other hand, due to the sheer performance of modern NVMe disks, single node databases already go a long way performance wise.

Same goes for other tasks like automatic upgrades, which is nearly transparent in AWS RDS. On the self-hosted side, keeping the database and the underlying OS up-to-date needs a restart sometimes. Given the fact that e.g. PostgreSQL and Linux are very mature products, this is nothing that happens every day and from real world deployments I have seen this culminating in a few minutes needed for restarts per month which can be acceptable for some deployments.

Data safety

This is the most critical topic, because no matter what happens you obviously also do not want to risk losing any of your valuable data. Here the same assumptions as for the previous topic apply; There are a handful of mature, and battle hardened backup solutions available for PostgreSQL, that, once set up and properly monitored, provide a high level of confidence that your data is always safe and sound and that you have the option to fully recover from a total loss of your dedicated hardware.

Monitoring

Of course for robust day-to-day operations a good monitoring and alarming solution is essential. AWS RDS is obviously tightly integrated with AWS CloudWatch, for a custom solution on dedicated metal this also needs to be implemented.