Workqueue needs to be flushed early when removing card, otherwise
soft lockup issue may happen because main_process is triggered by
interrupt while card is being removed.
Signed-off-by: Marc Yang <yangyang@marvell.com> Signed-off-by: Cathy Luo <cluo@marvell.com> Signed-off-by: Amitkumar Karwar <akarwar@marvell.com> Signed-off-by: Kalle Valo <kvalo@codeaurora.org>