mm: allow remote_vm access to non-io pfnmap memory