NVIDIA: SAUCE: locking/rtmutex: use cmpxchg in mark_rt_mutex_waiters

BugLink: https://bugs.launchpad.net/bugs/2072591

Function `mark_rt_mutex_waiters` is used by lock waiters to
intentionally mark that the lock has waiters before trying to acquire
the lock. By doing so it serializes all the waiters on `wait_lock`.

`wait_lock` is implemented using raw_spinlock, which does not provide
any memory barriers. If the system is stressed enough it is possible
that `RT_MUTEX_HAS_WAITERS` is updated later in the cache. This makes it
possible for the new waiter to acquire the mutex in the fast path, even
though there are waitiers in queue.

CPU0				CPU1
lock(wait_lock)
__try_to_take_rt_mutex {
	mark_rt_mutex_waiters
	lock->owner != NULL {
		return		rt_mutex_slowunlock {
	}
}
unlock(wait_lock)
					lock(wait_lock)
					!rt_mutex_has_waiters(lock) {
						unlock(mutex_lock)
// RT_MUTEX_HAS_WAITERS flag			owner->NULL
// Updated in cache after this		}
// check			}

To fix this use `cmpxchg` instead of `cmpxchg_relaxed` in
`mark_rt_mutex_waiters`. `cmpxchg` includes memory barriers to make
sure `RT_MUTEX_HAS_WAITERS` is updated before the mutex is release in
fast unlock path.

http://nvbugs/4015141

Signed-off-by: Kartik <kkartik@nvidia.com>
Tested-by: Tonny Liang <tonnyl@nvidia.com>
Tested-by: Abhilash G <abhilashg@nvidia.com>
Reviewed-by: Abhilash G <abhilashg@nvidia.com>
Signed-off-by: Laxman Dewangan <ldewangan@nvidia.com>
Acked-by: Jacob Martin <jacob.martin@canonical.com>
Acked-by: Noah Wager <noah.wager@canonical.com>
Signed-off-by: Noah Wager <noah.wager@canonical.com>

kernel/locking/rtmutex.c

@@ -241,7 +241,7 @@ static __always_inline void mark_rt_mutex_waiters(struct rt_mutex_base *lock)
 
 	do {
 		owner = *p;
-	} while (cmpxchg_relaxed(p, owner,
+	} while (cmpxchg(p, owner,
 				 owner | RT_MUTEX_HAS_WAITERS) != owner);
 
 	/*