This reverts commit
affdb62b815b38261f09f9d4ec210a35c7ffb1f3.
The commit introduced a regression with AD codecs where the stream is
always clean up. Since the patch is just a minor optimization and
reverting the commit fixes the issue, let's just revert it.
Reported-and-tested-by: Michael Burian <michael.burian@sbg.at>
Cc: <stable@vger.kernel.org> [v3.9+]
Signed-off-by: Takashi Iwai <tiwai@suse.de>