spi/bfin_spi: only request GPIO on first load