There are possibilities when records disappear underneath ovs-xapi-sync.
In this particular case, when VLAN network was deleted, the corresponding
record in bridge's external_ids:xs_network_ids column was not deleted by
xenserver. In situations like that handle the exceptions cleanly.