Store the source/destination slave address separately into the channel
structure. This moves us towards being able to avoid a configuration
call each time we use the channel.
Acked-by: Linus Walleij <linus.walleij@linaro.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk> Signed-off-by: Vinod Koul <vinod.koul@intel.com>