spi/imx: get rid of more ifs depending on the used cpu