Since commit 511f3c53 (usb: gadget: udc-core: fix a regression during
gadget driver unbinding) usb_gadget_remove_driver will pass NULL for
the driver argument.
Signed-off-by: Maarten ter Huurne <maarten@treewalker.org> Signed-off-by: Felipe Balbi <balbi@ti.com>