top of page
Graham

Lab Hardware Evolution: Our Journey to a Proxmox Cluster


At SeraphimGate Systems, we're always looking for ways to optimize our testing environment while maintaining reliability and performance. Today, we're excited to share some significant changes to our lab infrastructure that have improved our ability to test and validate solutions for our clients.


Our Hardware Heroes


The Powerhouse: Minisforum NAB9

The Minisforum NAB9: Our primary compute workhorse

The NAB9 is a testament to how far small form factor PCs have come. Despite its compact size, this unit packs an Intel Core i9-12900HK processor, capable of handling our most demanding workloads. We have two of these units serving as our primary compute nodes.


The Efficient Guardian: GMKTEC NUC G3

The GMKTEC NUC G3: Our efficient quorum keeper

Don't let its small size fool you - the Intel N100-powered G3 plays a crucial role in our cluster's reliability.


Understanding Our Cluster Architecture


The Power of Three: Why Quorum Matters


In any distributed system, one of the biggest challenges is maintaining consistency and availability when things go wrong. This is where quorum comes into play. But what exactly is quorum, and why did we choose a three-node setup?


How Quorum Works

A quorum is the minimum number of nodes that must be available for the cluster to make decisions. In our Proxmox cluster:

- Total nodes: 3 (two NAB9s and one NUC G3)

- Quorum requirement: majority (2 out of 3 nodes)

- Each node gets one vote in cluster decisions


This setup protects against "split-brain" scenarios - a dangerous situation where the cluster becomes divided and both parts try to operate independently. Here's how:


1. Normal Operation: All three nodes communicate and agree on cluster state

2. Single Node Failure: Two remaining nodes form a majority and continue operations

3. Network Split: Only the side with the majority (2 nodes) continues operating


Why Three Nodes?

While we could run Proxmox on just two nodes, a three-node cluster offers several advantages:

Reliable Fail-over: Can lose any single node without losing cluster functionality

Maintenance Flexibility: Can perform updates on one node while maintaining high availability

Clear Decision Making: Always has a majority in case of network splits


Resource Distribution

We've carefully planned how workloads are distributed across our nodes:


Primary Nodes (NAB9s):

- High-performance VMs requiring significant CPU/RAM

- Development and testing environments

- AI/ML workloads

- Database servers


Quorum Node (NUC G3):

- Lightweight services

- Monitoring tools

- DNS/DHCP services

- Cluster management tasks


Cost-Effective Scaling

The combination of Proxmox and our chosen hardware provides an incredibly cost-effective solution:

  • Lower initial hardware costs compared to traditional server hardware

  • No expensive virtualization licenses

  • Reduced power consumption and cooling needs

  • Ability to scale incrementally as needed


Real-World Benefits

This new setup has already shown its value in several ways:

  • Reduced power consumption by more than 50% compared to our previous VMWare infrastructure based on HP Z series workstations

  • Improved density - more computing power in less space

  • Simplified backup and recovery processes. Proxmox Backup server allows us to backup duplicated and even verify without third party software like Veeam or Synology Active Backup.

  • Enhanced ability to test complex scenarios and configurations

  • Substantial cost savings from both hardware and software perspectives


Embracing Open Solutions

Our move to Proxmox represents more than just a change in virtualization platforms - it's part of our broader commitment to embracing open solutions where they make sense. This approach allows us to:

  • Provide more cost-effective solutions to our clients

  • Maintain greater control over our infrastructure

  • Contribute back to the open-source community

  • Stay at the forefront of technology trends

  • Avoid vendor lock-in

### Backup Infrastructure

For backup operations, we've repurposed an HP Mini PC as our Proxmox Backup Server. This dedicated backup node ensures our test environments are always protected without impacting the performance of our primary cluster.


Backup Strategy

Our backup configuration includes:

  • Hourly incremental backups of all VMs

  • Weekly fully verified backups

  • Two separate backup storage pools, one on the Proxmox Backup Server and another on our NAS

  • Automated verification, deduplication and garbage collection of backups

  • Retention policies based on VM importance or other requirements


Real-World Benefits


  • Zero downtime during planned maintenance

  • Seamless failover during testing scenarios

  • Reduced power consumption by more than 50% compared to previous VMware infrastructure based on HP Z series workstations - entire testing environment now runs on less than 250 watts including NAS and networking

  • Improved density - more computing power in less space

  • Better workload isolation and resource management

  • Simplified backup and recovery processes using Proxmox Backup Server

  • Enhanced ability to test complex scenarios and configurations

  • Substantial cost savings from both hardware and software perspectives


Looking Forward


As we continue to explore the capabilities of our new infrastructure, we're excited about the possibilities it opens up for testing and validating solutions for our clients. The combination of Proxmox's flexibility with our carefully chosen hardware has created a robust platform for innovation.


Whether we're testing the latest AI implementations, evaluating network configurations, or developing new solutions for our clients, our updated lab environment gives us the tools we need to deliver reliable, tested solutions.


Stay tuned for more insights from our lab as we continue to push the boundaries of what's possible with modern infrastructure!

24 views0 comments

Comments


bottom of page