ANDROID: KVM: arm64: Ensure vCPU is initialised before publication

When initialising a new vCPU, a pointer to the vCPU is inserted into the
relevant index of 'hyp_vm->vcpus[]'. However, this is done without
sufficient memory barriers, allowing a concurrent attempt to load the
new vCPU to succeed with a partially initialised object.

Ensure that the 'hyp_vm->vcpus[]' entry is set with release semantics
and loaded with READ_ONCE() (relying on address dependency ordering)
when 'hyp_vm->vcpus_lock' is not held on the vCPU loading path.

Bug: 396117524
Bug: 357781595
Fixes: 7d9f60ddf7 ("ANDROID: KVM: arm64: Introduce new spinlock for hypervisor VM vCPUs[] array")
Reported-by: Ben Simner <ben.simner@cl.cam.ac.uk>
Change-Id: I8c33a81fb5f0e5868781d8f28f266e94d08e8265
Signed-off-by: Will Deacon <willdeacon@google.com>

arch/arm64/kvm/hyp/nvhe/pkvm.c

@@ -389,7 +389,11 @@ struct pkvm_hyp_vcpu *pkvm_load_hyp_vcpu(pkvm_handle_t handle,
 	if (!hyp_vm || hyp_vm->is_dying || hyp_vm->kvm.created_vcpus <= vcpu_idx)
 		goto unlock;
 
-	hyp_vcpu = hyp_vm->vcpus[vcpu_idx];
+	/*
+	 * Synchronise with concurrent vCPU initialisation by relying on
+	 * dependency ordering from the vCPU pointer.
+	 */
+	hyp_vcpu = READ_ONCE(hyp_vm->vcpus[vcpu_idx]);
 	if (!hyp_vcpu)
 		goto unlock;
 
@@ -881,7 +885,11 @@ int __pkvm_init_vcpu(pkvm_handle_t handle, struct kvm_vcpu *host_vcpu)
 		goto unlock_vcpus;
 	}
 
-	hyp_vm->vcpus[idx] = hyp_vcpu;
+	/*
+	 * Ensure the hyp_vcpu is initialised before publishing it to
+	 * the vCPU-load path via 'hyp_vm->vcpus[]'.
+	 */
+	smp_store_release(&hyp_vm->vcpus[idx], hyp_vcpu);
 
 unlock_vcpus:
 	hyp_spin_unlock(&hyp_vm->vcpus_lock);