net/irda: Remove unnecessary casts of private_data