IT 3300 : Proxmox

Proxmox Clustering

Proxmox Clustering - Why Cluster?

• centralized web-based management
• each node can do any/all management tasks
• pmxcfs: database-driven file system for storing configuration files, replicated in real-time on all nodes using corosync
• easy migration of vms and containers between hosts
• fast deployment
• firewall and HA available for cluster

Proxmox Clustering - Misc

• Need to have at least 3 proxmox nodes to do HA
• Should all be running same versions
• pvecm status
• pvecm nodes

Proxmox Clustering - HowTo

• On the node you want to add to the cluster:
  • pvecm add IP-ADDRESS-CLUSTER
• There is a gui way as well, but this is easiest.

Proxmox Clustering - Note

A new node cannot hold any VMs, because you would get conflicts about identical VM IDs. Also, all existing configuration in /etc/pve is overwritten when you join a new node to the cluster. To workaround, use vzdump to backup and restore to a different VMID after adding the node to the cluster.

Proxmox Clustering - Quorum

• Proxmox VE use a quorum-based technique to provide a consistent state among all cluster nodes.
• A quorum is the minimum number of votes that a distributed transaction has to obtain in order to be allowed to perform an operation in a distributed system.
• Proxmox VE assigns a single vote to each node by default.
• When you turn on nodes, or when power comes back after power failure, it is likely that some nodes will boot faster than others. Please keep in mind that guest startup is delayed until you reach quorum.

Proxmox Clustering - Guest Migration

• SHould choose to send this over separate network than the 'control' channel.
• Online (running, no local resources, in same cluster, same version)
• Offline (not running, copy from local to remote resource)

Proxmox Clustering - Online Migration

This starts a Qemu Process on the target host with the incoming flag, which means that the process starts and waits for the memory data and device states from the source Virtual Machine (since all other resources, e.g. disks, are shared, the memory content and device state are the only things left to transmit).

Once this connection is established, the source begins to send the memory content asynchronously to the target. If the memory on the source changes, those sections are marked dirty and there will be another pass of sending data. This happens until the amount of data to send is so small that it can pause the VM on the source, send the remaining data to the target and start the VM on the target in under a second.

Proxmox Clustering - Online Migration

• Online migration of virtual machines is only supported when nodes have CPUs from the same vendor. It might work otherwise, but this is never guaranteed.
• If you are interested in High Availability, you need to have at least three nodes for reliable quorum. All nodes should have the same version.
• Date and time must be synchronized.

Datacenter options

• When creating new guests the web interface will ask the backend for a free VMID automatically. The default range for searching is 100 to 1000000 (lower than the maximal allowed VMID enforced by the schema). Can edit this under options