xen: clean up "extra" memory handling some more