HowTo: applying kernel updates to ganeti node

Print Friendly, PDF & Email

To apply a standard kernel update to a same version kernel (e.g. 2.6.18-6 security update where the kernel version doesn’t change).

  1. Check running instances on master node:
  2. $ ssh gnt-cluster0
    $ sudo gnt-instance list
    Instance                 OS          Primary_node            Status     Memory centos-5    running      1024     debian-etch running        64     debian-etch ADMIN_down      -

  3. Failover running instances from node which is going to reboot (himalia in this example).
  4. $ sudo gnt-instance failover
    Failover will happen to image This requires a
    shutdown of the instance. Continue?
    y/[n]/?: y
    * checking disk consistency between source and target
    * shutting down instance on source node
    * deactivating the instance's disks on source node
    * activating the instance's disks on target node
    * starting the instance on the target node

  5. Check instances started correctly on target node.
  6. Failover the master node if necessary. First determine secondary node in this cluster.
  7. $ sudo gnt-cluster getmaster
    $ sudo gnt-node list
    Node                    DTotal  DFree MTotal MNode MFree Pinst Sinst 276444 152028   4095   257  3793     1     2    276444 207324   4095   257  2705     2     1

    Free node in this cluster is leda.

    $ ssh leda
    $ sudo gnt-cluster masterfailover

    Test the new master from another host.

    $ ssh gnt-cluster0

  8. If all instances migrated successfully it is safe to reboot the node.

IMPORTANT: if the kernel update involves a new release upgrade, e.g. 2.6.18-6 -> 2.6.18-7 then you must first update the drbd kernel modules and deploy them via bcfg2. This is a dangerous process and must be performed carefully!

  1. Check the contents of for files in bcfg2 that need to be updated with the current kernel version. (This might change so the files are automatically generated with the correct kernel version in the future.)
  2. Then update the kernel packages on the xen hypervisor host that the deployer sits on.
  3. $ ssh callisto
    $ sudo bcfg2 -qvn
    $ sudo bcfg2 -kqv
    $ sudo aptitude update
    $ sudo aptitude upgrade

  4. Then update the deployer
  5. $ ssh deployer
    $ sudo bcfg2 -qvn
    $ sudo bcfg2 -kqv
    $ sudo aptitude update
    $ sudo aptitude upgrade

  6. See what services a reboot will affect back on callisto
  7. $ sudo xm list

  8. Then reboot callisto
  9. $ sudo reboot

  10. Update the local debian repo packages for the DRBD modules.
  11. $ sudo /usr/local/projects/pkg-repo/bin/

  12. Then update the modules on the ganeti cluster node via bcfg2.
  13. $ sudo bcfg2 -qvn
    $ sudo bcfg2 -qv
    $ sudo aptitude update
    $ sudo aptitude upgrade

  14. Perform the steps outlined above before rebooting the node.
  15. Then you must update all ganeti cluster nodes repeating steps 6-7 on each node.
  16. When all is done remove the old kernel packages.