perf: Fix a race condition in perf_remove_from_context()