Further work required on main web Ganeti cluster

Print Friendly, PDF & Email

Following on from work on Saturday and Sunday’s to upgrade the Ganeti cluster software to the most recent version I am planning the required work to upgrade the underlying OS of each node to Debian Lenny. There are 4 nodes in gnt-cluster1:

  • Abbott
  • Costello
  • Titania
  • Oberon

Abbott was removed from the cluster yesterday and reinstalled with Debian Lenny. This was to test that the block replication service DRBD could still sync’ with a newer kernel and that VMs could be migrated to a new Xen version. As suspected DRBD can sync’ fine but VMs need to be stopped on the source node (old OS) and failed over to the secondary (with the new OS) and started up again.

Therefore, in order to perform a rolling upgrade on the remaining machines VMs will need to experience a further ~1min downtime per VM. I will do the failover at night to lessen the exposure and propose to do this in a fairly small time frame.

Tonight between 11pm and midnight I will failover these VMs

?This will evacuate Costello so it can be rebuilt and rejoined to the cluster. Once this is done I will juggle the VMs around to get a list in order to evacuate Oberon. I will notify which VMs will be affected by this when I am ready to move them, rather than setting a time and which VMs now.

Lists of projects these servers manage are at the servers own index page. Please, check them to see if services you manage are affected by these outages. The outages are very short but if you are concerned about the downtime please let me know.