Instance Missing on Hypervisor

Problem

  • Instance goes into ERROR state after failing to start.

  • Running openstack server show [uuid] command shows an instance fault such as the following.

Error: Instance [UUID] could not be found on hypervisor.

Environment

  • Platform9 Managed OpenStack - All Versions

  • Nova

Causes

  1. Instance failed to migrate.

  2. Disk is full on hypervisor.

Resolution

Instance Failed to Migrate

  1. Determine on which host the instance resides on.

# openstack server show [UUID]
  1. Check if the instance is running on the host obtained from the previous output.

# virsh list --all | grep [UUID]
  1. Confirm that the instance XML file still exists on the hypervisor.

# ls -l /etc/libvirt/qemu/[UUID].xml# ls -l /opt/pf9/data/instances/[UUID]/cfg-[UUID].xml
  1. If the XML file resides on the host, redefine the instance using virsh.

# virsh define /etc/libvirt/qemu/[UUID].xml
  1. Reset the state of the instance.

# openstack server set --state active [UUID]

Disk Full

  1. Identify which partition is full.

# df -h
  1. Inspect which folder(s) or file(s) are consuming the most amount of disk space.

# du -xh [partition-path] | sort -h | tail
  1. At your discretion, optionally backup and/or remove or truncate the offending large files which are consuming the most space.

Note: If you are uncertain if a particular file may be removed or truncated belonging to PF9, please contact Support.