when the device is yanked out ath_pci_remove starts doing the cleanups,
unregistering the hardware etc. so we should bail out immediately when
we get drv_flush callback from mac80211 when the card is being unplugged.
the panic occurs after we had associated to an AP.
Cc: Rajkumar Manoharan <rmanohar@qca.qualcomm.com> Signed-off-by: Mohammed Shafi Shajakhan <mohammed@qca.qualcomm.com> Signed-off-by: John W. Linville <linville@tuxdriver.com>