xfs: optimize the negative xattr caching