xhci: Avoid dead ports when CONFIG_USB_XHCI_HCD=n