tmakin/ack-tegra
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge 38ee6e1fb6 ("Merge tag 'kgdb-7.1-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/danielt/linux") into android-mainline

Browse Source 
Steps on the way to v7.1-rc1

Change-Id: I14c60a74debb664b6237363b26d526a73af9e9cd
Signed-off-by: Carlos Llamas <cmllamas@google.com>
This commit is contained in:
Carlos Llamas committed 2026-05-26 21:44:48 +00:00
2 parents 78335921f3 38ee6e1fb6
commit c9c68973d3
187 files changed
+4159 -1622
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

Documentation/ABI/testing/sysfs-fs-f2fs
+6
View File
@@ -407,6 +407,12 @@ Contact: "Hridya Valsaraju" <hridya@google.com>
Description: Average number of valid blocks.
Available when CONFIG_F2FS_STAT_FS=y.
What: /sys/fs/f2fs/<disk>/defrag_blocks
Date: February 2026
Contact: "Jinbao Liu" <liujinbao1@xiaomi.com>
Description: Number of blocks moved by defragment.
Available when CONFIG_F2FS_STAT_FS=y.
What: /sys/fs/f2fs/<disk>/mounted_time_sec
Date: February 2020
Contact: "Jaegeuk Kim" <jaegeuk@kernel.org>
Documentation/arch/s390/pci.rst
+104 -47
View File
@@ -6,6 +6,7 @@ S/390 PCI
Authors:
- Pierre Morel
- Niklas Schnelle
Copyright, IBM Corp. 2020
@@ -27,14 +28,16 @@ Command line parameters
debugfs entries
---------------
The S/390 debug feature (s390dbf) generates views to hold various debug results in sysfs directories of the form:
The S/390 debug feature (s390dbf) generates views to hold various debug results
in sysfs directories of the form:
* /sys/kernel/debug/s390dbf/pci_*/
For example:
- /sys/kernel/debug/s390dbf/pci_msg/sprintf
Holds messages from the processing of PCI events, like machine check handling
holds messages from the processing of PCI events, like machine check handling
and setting of global functionality, like UID checking.
Change the level of logging to be more or less verbose by piping
@@ -47,87 +50,141 @@ Sysfs entries
Entries specific to zPCI functions and entries that hold zPCI information.
* /sys/bus/pci/slots/XXXXXXXX
* /sys/bus/pci/slots/XXXXXXXX:
The slot entries are set up using the function identifier (FID) of the
PCI function. The format depicted as XXXXXXXX above is 8 hexadecimal digits
with 0 padding and lower case hexadecimal digits.
The slot entries are set up using the function identifier (FID) of the PCI
function as slot name. The format depicted as XXXXXXXX above is 8 hexadecimal
digits with 0 padding and lower case hexadecimal digits.
- /sys/bus/pci/slots/XXXXXXXX/power
In addition to using the FID as the name of the slot, the slot directory
also contains the following s390-specific slot attributes.
- uid:
The User-defined identifier (UID) of the function which may be configured
by this slot. See also the corresponding attribute of the device.
A physical function that currently supports a virtual function cannot be
powered off until all virtual functions are removed with:
echo 0 > /sys/bus/pci/devices/XXXX:XX:XX.X/sriov_numvf
echo 0 > /sys/bus/pci/devices/DDDD:BB:dd.f/sriov_numvf
* /sys/bus/pci/devices/XXXX:XX:XX.X/
* /sys/bus/pci/devices/DDDD:BB:dd.f/:
- function_id
A zPCI function identifier that uniquely identifies the function in the Z server.
- function_id:
The zPCI function identifier (FID) is a 32-bit hexadecimal value that
uniquely identifies the PCI function. Unless the hypervisor provides
a virtual FID e.g. on KVM this identifier is unique across the machine even
between different partitions.
- function_handle
Low-level identifier used for a configured PCI function.
It might be useful for debugging.
- function_handle:
This 32-bit hexadecimal value is a low-level identifier used for a PCI
function. Note that the function handle may be changed and become invalid
on PCI events and when enabling/disabling the PCI function.
- pchid
Model-dependent location of the I/O adapter.
- pchid:
This 16-bit hexadecimal value encodes a model-dependent location for
the PCI function.
- pfgid
PCI function group ID, functions that share identical functionality
- pfgid:
PCI function group ID; functions that share identical functionality
use a common identifier.
A PCI group defines interrupts, IOMMU, IOTLB, and DMA specifics.
- vfn
- vfn:
The virtual function number, from 1 to N for virtual functions,
0 for physical functions.
- pft
The PCI function type
- pft:
The PCI function type is an s390-specific type attribute. It indicates
a more general, usage oriented, type than PCI Specification
class/vendor/device identifiers. That is PCI functions with the same pft
value may be backed by different hardware implementations. At the same time
apart from unclassified functions (pft is 0x00) the same pft value
generally implies a similar usage model. At the same time the same
PCI hardware device may appear with different pft values when in a
different usage model. For example NETD and NETH VFs may be implemented
by the same PCI hardware device but in NETD the parent Physical Function
is user managed while with NETH it is platform managed.
- port
The port corresponds to the physical port the function is attached to.
It also gives an indication of the physical function a virtual function
is attached to.
Currently the following PFT values are defined:
- uid
The user identifier (UID) may be defined as part of the machine
configuration or the z/VM or KVM guest configuration. If the accompanying
uid_is_unique attribute is 1 the platform guarantees that the UID is unique
within that instance and no devices with the same UID can be attached
during the lifetime of the system.
- 0x00 (UNC): Unclassified
- 0x02 (ROCE): RoCE Express
- 0x05 (ISM): Internal Shared Memory
- 0x0a (ROC2): RoCE Express 2
- 0x0b (NVMe): NVMe
- 0x0c (NETH): Network Express hybrid
- 0x0d (CNW): Cloud Network Adapter
- 0x0f (NETD): Network Express direct
- uid_is_unique
Indicates whether the user identifier (UID) is guaranteed to be and remain
unique within this Linux instance.
- port:
The port is a decimal value corresponding to the physical port the function
is attached to. Virtual Functions (VFs) share the port with their parent
Physical Function (PF). A value of 0 indicates that the port attribute is
not applicable for that PCI function type.
- pfip/segmentX
- uid:
The user-defined identifier (UID) for a PCI function is a 32-bit
hexadecimal value. It is defined on a per instance basis as part of the
partition, KVM guest, or z/VM guest configuration. If UID Checking is
enabled the platform ensures that the UID is unique within that instance
and no two PCI functions with the same UID will be visible to the instance.
Independent of this guarantee and unlike the function ID (FID) the UID may
be the same in different partitions within the same machine. This allows to
create PCI configurations in multiple partitions to be identical in the
UID-namespace.
- uid_is_unique:
A 0 or 1 flag indicating whether the user-defined identifier (UID) is
guaranteed to be and remain unique within this Linux instance. This
platform feature is called UID Checking.
- pfip/segmentX:
The segments determine the isolation of a function.
They correspond to the physical path to the function.
The more the segments are different, the more the functions are isolated.
- fidparm:
Contains an 8-bit-per-PCI function parameter field in hexadecimal provided
by the platform. The meaning of this field is PCI function type specific.
For NETH VFs a value of 0x01 indicates that the function supports
promiscuous mode.
* /sys/firmware/clp/uid_checking:
In addition to the per-device uid_is_unique attribute this presents a
global indication of whether UID Checking is enabled. This allows users
to check for UID Checking even when no PCI functions are configured.
Enumeration and hotplug
=======================
The PCI address consists of four parts: domain, bus, device and function,
and is of this form: DDDD:BB:dd.f
and is of this form: DDDD:BB:dd.f.
* When not using multi-functions (norid is set, or the firmware does not
support multi-functions):
* For a PCI function for which the platform does not expose the RID, the
pci=norid kernel parameter is used, or a so-called isolated Virtual Function
which does have RID information but is used without its parent Physical
Function being part of the same PCI configuration:
- There is only one function per domain.
- The domain is set from the zPCI function's UID as defined during the
LPAR creation.
- The domain is set from the zPCI function's UID if UID Checking is on;
otherwise the domain ID is generated dynamically and is not stable
across reboots or hot plug.
* When using multi-functions (norid parameter is not set),
zPCI functions are addressed differently:
* For a PCI function for which the platform exposes the RID and which
is not an Isolated Virtual Function:
- There is still only one bus per domain.
- There can be up to 256 functions per bus.
- There can be up to 256 PCI functions per bus.
- The domain part of the address of all functions for
a multi-Function device is set from the zPCI function's UID as defined
in the LPAR creation for the function zero.
- The domain part of the address of all functions within the same topology is
that of the configured PCI function with the lowest devfn within that
topology.
- New functions will only be ready for use after the function zero
(the function with devfn 0) has been enumerated.
- Virtual Functions generated by an SR-IOV capable Physical Function only
become visible once SR-IOV is enabled.
Documentation/crypto/index.rst
+1 -1
View File
@@ -13,6 +13,7 @@ for cryptographic use cases, as well as programming examples.
:caption: Table of contents
:maxdepth: 2
libcrypto
intro
api-intro
architecture
@@ -27,4 +28,3 @@ for cryptographic use cases, as well as programming examples.
descore-readme
device_drivers/index
krb5
sha3
Documentation/crypto/libcrypto-blockcipher.rst
+19
View File
@@ -0,0 +1,19 @@
.. SPDX-License-Identifier: GPL-2.0-or-later
Block ciphers
=============
AES
---
Support for the AES block cipher.
.. kernel-doc:: include/crypto/aes.h
DES
---
Support for the DES block cipher. This algorithm is obsolete and is supported
only for backwards compatibility.
.. kernel-doc:: include/crypto/des.h
Documentation/crypto/libcrypto-hash.rst
+86
View File
@@ -0,0 +1,86 @@
.. SPDX-License-Identifier: GPL-2.0-or-later
Hash functions, MACs, and XOFs
==============================
AES-CMAC and AES-XCBC-MAC
-------------------------
Support for the AES-CMAC and AES-XCBC-MAC message authentication codes.
.. kernel-doc:: include/crypto/aes-cbc-macs.h
BLAKE2b
-------
Support for the BLAKE2b cryptographic hash function.
.. kernel-doc:: include/crypto/blake2b.h
BLAKE2s
-------
Support for the BLAKE2s cryptographic hash function.
.. kernel-doc:: include/crypto/blake2s.h
GHASH and POLYVAL
-----------------
Support for the GHASH and POLYVAL universal hash functions. These algorithms
are used only as internal components of other algorithms.
.. kernel-doc:: include/crypto/gf128hash.h
MD5
---
Support for the MD5 cryptographic hash function and HMAC-MD5. This algorithm is
obsolete and is supported only for backwards compatibility.
.. kernel-doc:: include/crypto/md5.h
NH
--
Support for the NH universal hash function. This algorithm is used only as an
internal component of other algorithms.
.. kernel-doc:: include/crypto/nh.h
Poly1305
--------
Support for the Poly1305 universal hash function. This algorithm is used only
as an internal component of other algorithms.
.. kernel-doc:: include/crypto/poly1305.h
SHA-1
-----
Support for the SHA-1 cryptographic hash function and HMAC-SHA1. This algorithm
is obsolete and is supported only for backwards compatibility.
.. kernel-doc:: include/crypto/sha1.h
SHA-2
-----
Support for the SHA-2 family of cryptographic hash functions, including SHA-224,
SHA-256, SHA-384, and SHA-512. This also includes their corresponding HMACs:
HMAC-SHA224, HMAC-SHA256, HMAC-SHA384, and HMAC-SHA512.
.. kernel-doc:: include/crypto/sha2.h
SHA-3
-----
The SHA-3 functions are documented in :ref:`sha3`.
SM3
---
Support for the SM3 cryptographic hash function.
.. kernel-doc:: include/crypto/sm3.h
Documentation/crypto/libcrypto-signature.rst
+11
View File
@@ -0,0 +1,11 @@
.. SPDX-License-Identifier: GPL-2.0-or-later
Digital signature algorithms
============================
ML-DSA
------
Support for the ML-DSA digital signature algorithm.
.. kernel-doc:: include/crypto/mldsa.h
Documentation/crypto/libcrypto-utils.rst
+6
View File
@@ -0,0 +1,6 @@
.. SPDX-License-Identifier: GPL-2.0-or-later
Utility functions
=================
.. kernel-doc:: include/crypto/utils.h
Documentation/crypto/libcrypto.rst
+165
View File
@@ -0,0 +1,165 @@
.. SPDX-License-Identifier: GPL-2.0-or-later
==============
Crypto library
==============
``lib/crypto/`` provides faster and easier access to cryptographic algorithms
than the traditional crypto API.
Each cryptographic algorithm is supported via a set of dedicated functions.
"Crypto agility", where needed, is left to calling code.
The crypto library functions are intended to be boring and straightforward, and
to follow familiar conventions. Their primary documentation is their (fairly
extensive) kernel-doc. This page just provides some extra high-level context.
Note that the crypto library isn't entirely new. ``lib/`` has contained some
crypto functions since 2005. Rather, it's just an approach that's been expanded
over time as it's been found to work well. It also largely just matches how the
kernel already does things elsewhere.
Scope and intended audience
===========================
The crypto library documentation is primarily meant for kernel developers who
need to use a particular cryptographic algorithm(s) in kernel code. For
example, "I just need to compute a SHA-256 hash." A secondary audience is
developers working on the crypto algorithm implementations themselves.
If you're looking for more general information about cryptography, like the
differences between the different crypto algorithms or how to select an
appropriate algorithm, you should refer to external sources which cover that
type of information much more comprehensively. If you need help selecting
algorithms for a new kernel feature that doesn't already have its algorithms
predefined, please reach out to ``linux-crypto@vger.kernel.org`` for advice.
Code organization
=================
- ``lib/crypto/*.c``: the crypto algorithm implementations
- ``lib/crypto/$(SRCARCH)/``: architecture-specific code for crypto algorithms.
It is here rather than somewhere in ``arch/`` partly because this allows
generic and architecture-optimized code to be easily built into a single
loadable module (when the algorithm is set to 'm' in the kconfig).
- ``lib/crypto/tests/``: KUnit tests for the crypto algorithms
- ``include/crypto/``: crypto headers, for both the crypto library and the
traditional crypto API
Generally, there is one kernel module per algorithm. Sometimes related
algorithms are grouped into one module. There is intentionally no common
framework, though there are some utility functions that multiple algorithms use.
Each algorithm module is controlled by a tristate kconfig symbol
``CRYPTO_LIB_$(ALGORITHM)``. As is the norm for library functions in the
kernel, these are hidden symbols which don't show up in the kconfig menu.
Instead, they are just selected by all the kconfig symbols that need them.
Many of the algorithms have multiple implementations: a generic implementation
and architecture-optimized implementation(s). Each module initialization
function, or initcall in the built-in case, automatically enables the best
implementation based on the available CPU features.
Note that the crypto library doesn't use the ``crypto/``,
``arch/$(SRCARCH)/crypto/``, or ``drivers/crypto/`` directories. These
directories are used by the traditional crypto API. When possible, algorithms
in the traditional crypto API are implemented by calls into the library.
Advantages
==========
Some of the advantages of the library over the traditional crypto API are:
- The library functions tend to be much easier to use. For example, a hash
value can be computed using only a single function call. Most of the library
functions always succeed and return void, eliminating the need to write
error-handling code. Most also accept standard virtual addresses, rather than
scatterlists which are difficult and less efficient to work with.
- The library functions are usually faster, especially for short inputs. They
call the crypto algorithms directly without inefficient indirect calls, memory
allocations, string parsing, lookups in an algorithm registry, and other
unnecessary API overhead. Architecture-optimized code is enabled by default.
- The library functions use standard link-time dependencies instead of
error-prone dynamic loading by name. There's no need for workarounds such as
forcing algorithms to be built-in or adding module soft dependencies.
- The library focuses on the approach that works the best on the vast majority
of systems: CPU-based implementations of the crypto algorithms, utilizing
on-CPU acceleration (such as AES instructions) when available.
- The library uses standard KUnit tests, rather than custom ad-hoc tests.
- The library tends to have higher assurance implementations of the crypto
algorithms. This is both due to its simpler design and because more of its
code is being regularly tested.
- The library supports features that don't fit into the rigid framework of the
traditional crypto API, for example interleaved hashing and XOFs.
When to use it
==============
In-kernel users should use the library (rather than the traditional crypto API)
whenever possible. Many subsystems have already been converted. It usually
simplifies their code significantly and improves performance.
Some kernel features allow userspace to provide an arbitrary string that selects
an arbitrary algorithm from the traditional crypto API by name. These features
generally will have to keep using the traditional crypto API for backwards
compatibility.
Note: new kernel features shouldn't support every algorithm, but rather make a
deliberate choice about what algorithm(s) to support. History has shown that
making a deliberate, thoughtful choice greatly simplifies code maintenance,
reduces the chance for mistakes (such as using an obsolete, insecure, or
inappropriate algorithm), and makes your feature easier to use.
Testing
=======
The crypto library uses standard KUnit tests. Like many of the kernel's other
KUnit tests, they are included in the set of tests that is run by
``tools/testing/kunit/kunit.py run --alltests``.
A ``.kunitconfig`` file is also provided to run just the crypto library tests.
For example, here's how to run them in user-mode Linux:
.. code-block:: sh
tools/testing/kunit/kunit.py run --kunitconfig=lib/crypto/
Many of the crypto algorithms have architecture-optimized implementations.
Testing those requires building an appropriate kernel and running the tests
either in QEMU or on appropriate hardware. Here's one example with QEMU:
.. code-block:: sh
tools/testing/kunit/kunit.py run --kunitconfig=lib/crypto/ --arch=arm64 --make_options LLVM=1
Depending on the code being tested, flags may need to be passed to QEMU to
emulate the correct type of hardware for the code to be reached.
Since correctness is essential in cryptographic code, new architecture-optimized
code is accepted only if it can be tested in QEMU.
Note: the crypto library also includes FIPS 140 self-tests. These are
lightweight, are designed specifically to meet FIPS 140 requirements, and exist
*only* to meet those requirements. Normal testing done by kernel developers and
integrators should use the much more comprehensive KUnit tests instead.
API documentation
=================
.. toctree::
:maxdepth: 2
libcrypto-blockcipher
libcrypto-hash
libcrypto-signature
libcrypto-utils
sha3
Documentation/crypto/sha3.rst
+2
View File
@@ -1,5 +1,7 @@
.. SPDX-License-Identifier: GPL-2.0-or-later
.. _sha3:
==========================
SHA-3 Algorithm Collection
==========================
MAINTAINERS
+8
View File
@@ -7303,6 +7303,14 @@ L: linux-cxl@vger.kernel.org
S: Supported
F: drivers/dax/
DEVICE DIRECT ACCESS (DAX) [fsdev_dax]
M: John Groves <jgroves@micron.com>
M: John Groves <John@Groves.net>
L: nvdimm@lists.linux.dev
L: linux-cxl@vger.kernel.org
S: Supported
F: drivers/dax/fsdev.c
DEVICE FREQUENCY (DEVFREQ)
M: MyungJoo Ham <myungjoo.ham@samsung.com>
M: Kyungmin Park <kyungmin.park@samsung.com>
arch/arm64/include/asm/pgtable.h
+3 -3
View File
@@ -1276,17 +1276,17 @@ static inline int pmdp_set_access_flags(struct vm_area_struct *vma,
#endif
#ifdef CONFIG_PAGE_TABLE_CHECK
static inline bool pte_user_accessible_page(pte_t pte, unsigned long addr)
static inline bool pte_user_accessible_page(struct mm_struct *mm, unsigned long addr, pte_t pte)
{
return pte_valid(pte) && (pte_user(pte) || pte_user_exec(pte));
}
static inline bool pmd_user_accessible_page(pmd_t pmd, unsigned long addr)
static inline bool pmd_user_accessible_page(struct mm_struct *mm, unsigned long addr, pmd_t pmd)
{
return pmd_valid(pmd) && !pmd_table(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd));
}
static inline bool pud_user_accessible_page(pud_t pud, unsigned long addr)
static inline bool pud_user_accessible_page(struct mm_struct *mm, unsigned long addr, pud_t pud)
{
return pud_valid(pud) && !pud_table(pud) && (pud_user(pud) || pud_user_exec(pud));
}
arch/powerpc/include/asm/book3s/32/pgtable.h
+1 -1
View File
@@ -438,7 +438,7 @@ static inline bool pte_access_permitted(pte_t pte, bool write)
return true;
}
static inline bool pte_user_accessible_page(pte_t pte, unsigned long addr)
static inline bool pte_user_accessible_page(struct mm_struct *mm, unsigned long addr, pte_t pte)
{
return pte_present(pte) && !is_kernel_addr(addr);
}
arch/powerpc/include/asm/book3s/64/pgtable.h
+5 -5
View File
@@ -549,7 +549,7 @@ static inline bool pte_access_permitted(pte_t pte, bool write)
return arch_pte_access_permitted(pte_val(pte), write, 0);
}
static inline bool pte_user_accessible_page(pte_t pte, unsigned long addr)
static inline bool pte_user_accessible_page(struct mm_struct *mm, unsigned long addr, pte_t pte)
{
return pte_present(pte) && pte_user(pte);
}
@@ -925,9 +925,9 @@ static inline bool pud_access_permitted(pud_t pud, bool write)
}
#define pud_user_accessible_page pud_user_accessible_page
static inline bool pud_user_accessible_page(pud_t pud, unsigned long addr)
static inline bool pud_user_accessible_page(struct mm_struct *mm, unsigned long addr, pud_t pud)
{
return pud_leaf(pud) && pte_user_accessible_page(pud_pte(pud), addr);
return pud_leaf(pud) && pte_user_accessible_page(mm, addr, pud_pte(pud));
}
#define __p4d_raw(x) ((p4d_t) { __pgd_raw(x) })
@@ -1096,9 +1096,9 @@ static inline bool pmd_access_permitted(pmd_t pmd, bool write)
}
#define pmd_user_accessible_page pmd_user_accessible_page
static inline bool pmd_user_accessible_page(pmd_t pmd, unsigned long addr)
static inline bool pmd_user_accessible_page(struct mm_struct *mm, unsigned long addr, pmd_t pmd)
{
return pmd_leaf(pmd) && pte_user_accessible_page(pmd_pte(pmd), addr);
return pmd_leaf(pmd) && pte_user_accessible_page(mm, addr, pmd_pte(pmd));
}
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
arch/powerpc/include/asm/nohash/pgtable.h
+1 -1
View File
@@ -249,7 +249,7 @@ static inline bool pte_access_permitted(pte_t pte, bool write)
return true;
}
static inline bool pte_user_accessible_page(pte_t pte, unsigned long addr)
static inline bool pte_user_accessible_page(struct mm_struct *mm, unsigned long addr, pte_t pte)
{
return pte_present(pte) && !is_kernel_addr(addr);
}
arch/powerpc/include/asm/pgtable.h
+2 -2
View File
@@ -213,11 +213,11 @@ static inline bool arch_supports_memmap_on_memory(unsigned long vmemmap_size)
#endif /* CONFIG_PPC64 */
#ifndef pmd_user_accessible_page
#define pmd_user_accessible_page(pmd, addr) false
#define pmd_user_accessible_page(mm, addr, pmd) false
#endif
#ifndef pud_user_accessible_page
#define pud_user_accessible_page(pud, addr) false
#define pud_user_accessible_page(mm, addr, pud) false
#endif
#endif /* __ASSEMBLER__ */
arch/riscv/include/asm/pgtable.h
+3 -3
View File
@@ -984,17 +984,17 @@ static inline void set_pud_at(struct mm_struct *mm, unsigned long addr,
}
#ifdef CONFIG_PAGE_TABLE_CHECK
static inline bool pte_user_accessible_page(pte_t pte, unsigned long addr)
static inline bool pte_user_accessible_page(struct mm_struct *mm, unsigned long addr, pte_t pte)
{
return pte_present(pte) && pte_user(pte);
}
static inline bool pmd_user_accessible_page(pmd_t pmd, unsigned long addr)
static inline bool pmd_user_accessible_page(struct mm_struct *mm, unsigned long addr, pmd_t pmd)
{
return pmd_leaf(pmd) && pmd_user(pmd);
}
static inline bool pud_user_accessible_page(pud_t pud, unsigned long addr)
static inline bool pud_user_accessible_page(struct mm_struct *mm, unsigned long addr, pud_t pud)
{
return pud_leaf(pud) && pud_user(pud);
}
arch/s390/Kconfig
+2 -1
View File
@@ -152,6 +152,7 @@ config S390
select ARCH_SUPPORTS_INT128 if CC_HAS_INT128 && CC_IS_CLANG
select ARCH_SUPPORTS_MSEAL_SYSTEM_MAPPINGS
select ARCH_SUPPORTS_NUMA_BALANCING
select ARCH_SUPPORTS_PAGE_TABLE_CHECK
select ARCH_SUPPORTS_PER_VMA_LOCK
select ARCH_USE_BUILTIN_BSWAP
select ARCH_USE_CMPXCHG_LOCKREF
@@ -1023,7 +1024,7 @@ config S390_KPROBES_SANITY_TEST
config S390_MODULES_SANITY_TEST
def_tristate n
depends on KUNIT
depends on KUNIT && m
default KUNIT_ALL_TESTS
prompt "Enable s390 specific modules tests"
select S390_MODULES_SANITY_TEST_HELPERS
arch/s390/boot/startup.c
+2 -1
View File
@@ -440,7 +440,8 @@ static unsigned long setup_kernel_memory_layout(unsigned long kernel_size)
max_mappable = max(ident_map_size, MAX_DCSS_ADDR);
max_mappable = min(max_mappable, vmemmap_start);
#ifdef CONFIG_RANDOMIZE_IDENTITY_BASE
__identity_base = round_down(vmemmap_start - max_mappable, rte_size);
if (kaslr_enabled())
__identity_base = round_down(vmemmap_start - max_mappable, rte_size);
#endif
boot_debug("identity map: 0x%016lx-0x%016lx\n", __identity_base,
__identity_base + ident_map_size);
arch/s390/configs/debug_defconfig
+2
View File
@@ -925,3 +925,5 @@ CONFIG_PERCPU_TEST=m
CONFIG_ATOMIC64_SELFTEST=y
CONFIG_TEST_BITOPS=m
CONFIG_TEST_BPF=m
CONFIG_PAGE_TABLE_CHECK=y
CONFIG_PAGE_TABLE_CHECK_ENFORCED=y
arch/s390/include/asm/pci.h
+4
View File
@@ -208,6 +208,10 @@ extern const struct attribute_group zpci_ident_attr_group;
&pfip_attr_group, \
&zpci_ident_attr_group,
extern const struct attribute_group zpci_slot_attr_group;
#define ARCH_PCI_SLOT_GROUPS (&zpci_slot_attr_group)
extern unsigned int s390_pci_force_floating __initdata;
extern unsigned int s390_pci_no_rid;
arch/s390/include/asm/percpu.h
+18
View File
@@ -12,6 +12,24 @@
*/
#define __my_cpu_offset get_lowcore()->percpu_offset
#define arch_raw_cpu_ptr(_ptr) \
({ \
unsigned long lc_percpu, tcp_ptr__; \
\
tcp_ptr__ = (__force unsigned long)(_ptr); \
lc_percpu = offsetof(struct lowcore, percpu_offset); \
asm_inline volatile( \
ALTERNATIVE("ag %[__ptr__],%[offzero](%%r0)\n", \
"ag %[__ptr__],%[offalt](%%r0)\n", \
ALT_FEATURE(MFEATURE_LOWCORE)) \
: [__ptr__] "+d" (tcp_ptr__) \
: [offzero] "i" (lc_percpu), \
[offalt] "i" (lc_percpu + LOWCORE_ALT_ADDRESS), \
"m" (((struct lowcore *)0)->percpu_offset) \
: "cc"); \
(TYPEOF_UNQUAL(*(_ptr)) __force __kernel *)tcp_ptr__; \
})
/*
* We use a compare-and-swap loop since that uses less cpu cycles than
* disabling and enabling interrupts like the generic variant would do.
arch/s390/include/asm/pgtable.h
+53 -7
View File
@@ -16,8 +16,10 @@
#include <linux/mm_types.h>
#include <linux/cpufeature.h>
#include <linux/page-flags.h>
#include <linux/page_table_check.h>
#include <linux/radix-tree.h>
#include <linux/atomic.h>
#include <linux/mmap_lock.h>
#include <asm/ctlreg.h>
#include <asm/bug.h>
#include <asm/page.h>
@@ -1190,6 +1192,7 @@ static inline pte_t ptep_get_and_clear(struct mm_struct *mm,
/* At this point the reference through the mapping is still present */
if (mm_is_protected(mm) && pte_present(res))
WARN_ON_ONCE(uv_convert_from_secure_pte(res));
page_table_check_pte_clear(mm, addr, res);
return res;
}
@@ -1208,6 +1211,7 @@ static inline pte_t ptep_clear_flush(struct vm_area_struct *vma,
/* At this point the reference through the mapping is still present */
if (mm_is_protected(vma->vm_mm) && pte_present(res))
WARN_ON_ONCE(uv_convert_from_secure_pte(res));
page_table_check_pte_clear(vma->vm_mm, addr, res);
return res;
}
@@ -1231,6 +1235,9 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm,
} else {
res = ptep_xchg_lazy(mm, addr, ptep, __pte(_PAGE_INVALID));
}
page_table_check_pte_clear(mm, addr, res);
/* Nothing to do */
if (!mm_is_protected(mm) || !pte_present(res))
return res;
@@ -1327,6 +1334,7 @@ static inline void set_ptes(struct mm_struct *mm, unsigned long addr,
{
if (pte_present(entry))
entry = clear_pte_bit(entry, __pgprot(_PAGE_UNUSED));
page_table_check_ptes_set(mm, addr, ptep, entry, nr);
for (;;) {
set_pte(ptep, entry);
if (--nr == 0)
@@ -1703,6 +1711,7 @@ static inline bool pmdp_clear_flush_young(struct vm_area_struct *vma,
static inline void set_pmd_at(struct mm_struct *mm, unsigned long addr,
pmd_t *pmdp, pmd_t entry)
{
page_table_check_pmd_set(mm, addr, pmdp, entry);
set_pmd(pmdp, entry);
}
@@ -1717,7 +1726,11 @@ static inline pmd_t pmd_mkhuge(pmd_t pmd)
static inline pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm,
unsigned long addr, pmd_t *pmdp)
{
return pmdp_xchg_direct(mm, addr, pmdp, __pmd(_SEGMENT_ENTRY_EMPTY));
pmd_t pmd;
pmd = pmdp_xchg_direct(mm, addr, pmdp, __pmd(_SEGMENT_ENTRY_EMPTY));
page_table_check_pmd_clear(mm, addr, pmd);
return pmd;
}
#define __HAVE_ARCH_PMDP_HUGE_GET_AND_CLEAR_FULL
@@ -1725,12 +1738,17 @@ static inline pmd_t pmdp_huge_get_and_clear_full(struct vm_area_struct *vma,
unsigned long addr,
pmd_t *pmdp, int full)
{
pmd_t pmd;
if (full) {
pmd_t pmd = *pmdp;
pmd = *pmdp;
set_pmd(pmdp, __pmd(_SEGMENT_ENTRY_EMPTY));
page_table_check_pmd_clear(vma->vm_mm, addr, pmd);
return pmd;
}
return pmdp_xchg_lazy(vma->vm_mm, addr, pmdp, __pmd(_SEGMENT_ENTRY_EMPTY));
pmd = pmdp_xchg_lazy(vma->vm_mm, addr, pmdp, __pmd(_SEGMENT_ENTRY_EMPTY));
page_table_check_pmd_clear(vma->vm_mm, addr, pmd);
return pmd;
}
#define __HAVE_ARCH_PMDP_HUGE_CLEAR_FLUSH
@@ -1744,11 +1762,16 @@ static inline pmd_t pmdp_huge_clear_flush(struct vm_area_struct *vma,
static inline pmd_t pmdp_invalidate(struct vm_area_struct *vma,
unsigned long addr, pmd_t *pmdp)
{
pmd_t pmd;
pmd_t pmd = *pmdp;
VM_WARN_ON_ONCE(!pmd_present(*pmdp));
pmd = __pmd(pmd_val(*pmdp) | _SEGMENT_ENTRY_INVALID);
return pmdp_xchg_direct(vma->vm_mm, addr, pmdp, pmd);
VM_WARN_ON_ONCE(!pmd_present(pmd));
pmd = set_pmd_bit(pmd, __pgprot(_SEGMENT_ENTRY_INVALID));
#ifdef CONFIG_PAGE_TABLE_CHECK
pmd = clear_pmd_bit(pmd, __pgprot(_SEGMENT_ENTRY_READ));
#endif
page_table_check_pmd_set(vma->vm_mm, addr, pmdp, pmd);
pmd = pmdp_xchg_direct(vma->vm_mm, addr, pmdp, pmd);
return pmd;
}
#define __HAVE_ARCH_PMDP_SET_WRPROTECT
@@ -1783,6 +1806,29 @@ static inline int has_transparent_hugepage(void)
}
#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
#ifdef CONFIG_PAGE_TABLE_CHECK
static inline bool pte_user_accessible_page(struct mm_struct *mm, unsigned long addr, pte_t pte)
{
VM_BUG_ON(mm == &init_mm);
return pte_present(pte);
}
static inline bool pmd_user_accessible_page(struct mm_struct *mm, unsigned long addr, pmd_t pmd)
{
VM_BUG_ON(mm == &init_mm);
return pmd_leaf(pmd) && (pmd_val(pmd) & _SEGMENT_ENTRY_READ);
}
static inline bool pud_user_accessible_page(struct mm_struct *mm, unsigned long addr, pud_t pud)
{
VM_BUG_ON(mm == &init_mm);
return pud_leaf(pud);
}
#endif
/*
* 64 bit swap entry format:
* A page-table entry has some bits we have to treat in a special way.
arch/s390/include/asm/setup.h
-1
View File
@@ -52,7 +52,6 @@ extern unsigned int zlib_dfltcc_support;
#define ZLIB_DFLTCC_INFLATE_ONLY 3
#define ZLIB_DFLTCC_FULL_DEBUG 4
extern unsigned long ident_map_size;
extern unsigned long max_mappable;
/* The Write Back bit position in the physaddr is given by the SLPC PCI */
arch/s390/pci/pci_sysfs.c
+20
View File
@@ -187,6 +187,17 @@ static ssize_t index_show(struct device *dev,
}
static DEVICE_ATTR_RO(index);
static ssize_t zpci_uid_slot_show(struct pci_slot *slot, char *buf)
{
struct zpci_dev *zdev = container_of(slot->hotplug, struct zpci_dev,
hotplug_slot);
return sysfs_emit(buf, "0x%x\n", zdev->uid);
}
static struct pci_slot_attribute zpci_slot_attr_uid =
__ATTR(uid, 0444, zpci_uid_slot_show, NULL);
static umode_t zpci_index_is_visible(struct kobject *kobj,
struct attribute *attr, int n)
{
@@ -243,6 +254,15 @@ const struct attribute_group pfip_attr_group = {
.attrs = pfip_attrs,
};
static struct attribute *zpci_slot_attrs[] = {
&zpci_slot_attr_uid.attr,
NULL,
};
const struct attribute_group zpci_slot_attr_group = {
.attrs = zpci_slot_attrs,
};
static struct attribute *clp_fw_attrs[] = {
&uid_checking_attr.attr,
NULL,
arch/x86/include/asm/pgtable.h
+3 -3
View File
@@ -1672,17 +1672,17 @@ static inline bool arch_has_hw_nonleaf_pmd_young(void)
#endif
#ifdef CONFIG_PAGE_TABLE_CHECK
static inline bool pte_user_accessible_page(pte_t pte, unsigned long addr)
static inline bool pte_user_accessible_page(struct mm_struct *mm, unsigned long addr, pte_t pte)
{
return (pte_val(pte) & _PAGE_PRESENT) && (pte_val(pte) & _PAGE_USER);
}
static inline bool pmd_user_accessible_page(pmd_t pmd, unsigned long addr)
static inline bool pmd_user_accessible_page(struct mm_struct *mm, unsigned long addr, pmd_t pmd)
{
return pmd_leaf(pmd) && (pmd_val(pmd) & _PAGE_PRESENT) && (pmd_val(pmd) & _PAGE_USER);
}
static inline bool pud_user_accessible_page(pud_t pud, unsigned long addr)
static inline bool pud_user_accessible_page(struct mm_struct *mm, unsigned long addr, pud_t pud)
{
return pud_leaf(pud) && (pud_val(pud) & _PAGE_PRESENT) && (pud_val(pud) & _PAGE_USER);
}
arch/x86/kernel/cpu/mshyperv.c
+13 -2
View File
@@ -237,8 +237,12 @@ void hv_remove_crash_handler(void)
#ifdef CONFIG_KEXEC_CORE
static void hv_machine_shutdown(void)
{
if (kexec_in_progress && hv_kexec_handler)
hv_kexec_handler();
if (kexec_in_progress) {
hv_stimer_global_cleanup();
if (hv_kexec_handler)
hv_kexec_handler();
}
/*
* Call hv_cpu_die() on all the CPUs, otherwise later the hypervisor
@@ -427,6 +431,10 @@ static void __init hv_smp_prepare_cpus(unsigned int max_cpus)
}
#ifdef CONFIG_X86_64
/* If AP LPs exist, we are in a kexec'd kernel and VPs already exist */
if (num_present_cpus() == 1 || hv_lp_exists(1))
return;
for_each_present_cpu(i) {
if (i == 0)
continue;
@@ -434,6 +442,9 @@ static void __init hv_smp_prepare_cpus(unsigned int max_cpus)
BUG_ON(ret);
}
ret = hv_call_notify_all_processors_started();
WARN_ON(ret);
for_each_present_cpu(i) {
if (i == 0)
continue;
arch/x86/kernel/kgdb.c
+5 -4
View File
@@ -407,10 +407,11 @@ static void kgdb_disable_hw_debug(struct pt_regs *regs)
* kgdb_roundup_cpus - Get other CPUs into a holding pattern
*
* On SMP systems, we need to get the attention of the other CPUs
* and get them be in a known state. This should do what is needed
* to get the other CPUs to call kgdb_wait(). Note that on some arches,
* the NMI approach is not used for rounding up all the CPUs. For example,
* in case of MIPS, smp_call_function() is used to roundup CPUs.
* and get them into a known state. This should do what is needed
* to get the other CPUs to call kgdb_handle_exception(). Note that
* on some arches, the NMI approach is not used for rounding up all
* the CPUs. For example, in case of MIPS, smp_call_function() is
* used to roundup CPUs.
*
* On non-SMP systems, this is not called.
*/
drivers/dax/Kconfig
+5
View File
@@ -65,6 +65,11 @@ config DEV_DAX_HMEM_DEVICES
depends on DEV_DAX_HMEM && DAX
def_bool y
config DEV_DAX_FSDEV
tristate
depends on DEV_DAX && FS_DAX
default DEV_DAX
config DEV_DAX_KMEM
tristate "KMEM DAX: map dax-devices as System-RAM"
default DEV_DAX
drivers/dax/Makefile
+2
View File
@@ -5,9 +5,11 @@ obj-$(CONFIG_DEV_DAX) += device_dax.o
obj-$(CONFIG_DEV_DAX_KMEM) += kmem.o
obj-$(CONFIG_DEV_DAX_PMEM) += dax_pmem.o
obj-$(CONFIG_DEV_DAX_CXL) += dax_cxl.o
obj-$(CONFIG_DEV_DAX_FSDEV) += fsdev_dax.o
dax-y := super.o
dax-y += bus.o
device_dax-y := device.o
dax_pmem-y := pmem.o
dax_cxl-y := cxl.o
fsdev_dax-y := fsdev.o
drivers/dax/bus.c
+20 -2
View File
@@ -40,8 +40,6 @@ static int dax_bus_uevent(const struct device *dev, struct kobj_uevent_env *env)
return add_uevent_var(env, "MODALIAS=" DAX_DEVICE_MODALIAS_FMT, 0);
}
#define to_dax_drv(__drv) container_of_const(__drv, struct dax_device_driver, drv)
static struct dax_id *__dax_match_id(const struct dax_device_driver *dax_drv,
const char *dev_name)
{
@@ -1431,6 +1429,26 @@ static const struct device_type dev_dax_type = {
.groups = dax_attribute_groups,
};
/* see "strong" declaration in tools/testing/nvdimm/dax-dev.c */
__weak phys_addr_t dax_pgoff_to_phys(struct dev_dax *dev_dax, pgoff_t pgoff,
unsigned long size)
{
for (int i = 0; i < dev_dax->nr_range; i++) {
struct dev_dax_range *dax_range = &dev_dax->ranges[i];
struct range *range = &dax_range->range;
phys_addr_t phys;
if (!in_range(pgoff, dax_range->pgoff, PHYS_PFN(range_len(range))))
continue;
phys = PFN_PHYS(pgoff - dax_range->pgoff) + range->start;
if (phys + size - 1 <= range->end)
return phys;
break;
}
return -1;
}
EXPORT_SYMBOL_GPL(dax_pgoff_to_phys);
static struct dev_dax *__devm_create_dev_dax(struct dev_dax_data *data)
{
struct dax_region *dax_region = data->dax_region;
drivers/dax/bus.h
+3
View File
@@ -33,6 +33,7 @@ struct dev_dax *devm_create_dev_dax(struct dev_dax_data *data);
enum dax_driver_type {
DAXDRV_KMEM_TYPE,
DAXDRV_DEVICE_TYPE,
DAXDRV_FSDEV_TYPE,
};
struct dax_device_driver {
@@ -43,6 +44,8 @@ struct dax_device_driver {
void (*remove)(struct dev_dax *dev);
};
#define to_dax_drv(__drv) container_of_const(__drv, struct dax_device_driver, drv)
int __dax_driver_register(struct dax_device_driver *dax_drv,
struct module *module, const char *mod_name);
#define dax_driver_register(driver) \
drivers/dax/dax-private.h
+4
View File
@@ -69,6 +69,8 @@ struct dev_dax_range {
* data while the device is activated in the driver.
* @region: parent region
* @dax_dev: core dax functionality
* @virt_addr: kva from memremap; used by fsdev_dax
* @cached_size: size of daxdev cached by fsdev_dax
* @align: alignment of this instance
* @target_node: effective numa node if dev_dax memory range is onlined
* @dyn_id: is this a dynamic or statically created instance
@@ -83,6 +85,8 @@ struct dev_dax_range {
struct dev_dax {
struct dax_region *region;
struct dax_device *dax_dev;
void *virt_addr;
u64 cached_size;
unsigned int align;
int target_node;
bool dyn_id;
drivers/dax/device.c
-23
View File
@@ -57,29 +57,6 @@ static int check_vma(struct dev_dax *dev_dax, struct vm_area_struct *vma,
vma->vm_file, func);
}
/* see "strong" declaration in tools/testing/nvdimm/dax-dev.c */
__weak phys_addr_t dax_pgoff_to_phys(struct dev_dax *dev_dax, pgoff_t pgoff,
unsigned long size)
{
int i;
for (i = 0; i < dev_dax->nr_range; i++) {
struct dev_dax_range *dax_range = &dev_dax->ranges[i];
struct range *range = &dax_range->range;
unsigned long long pgoff_end;
phys_addr_t phys;
pgoff_end = dax_range->pgoff + PHYS_PFN(range_len(range)) - 1;
if (pgoff < dax_range->pgoff || pgoff > pgoff_end)
continue;
phys = PFN_PHYS(pgoff - dax_range->pgoff) + range->start;
if (phys + size - 1 <= range->end)
return phys;
break;
}
return -1;
}
static void dax_set_mapping(struct vm_fault *vmf, unsigned long pfn,
unsigned long fault_size)
{
drivers/dax/fsdev.c
+349
View File
@@ -0,0 +1,349 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright(c) 2026 Micron Technology, Inc. */
#include <linux/memremap.h>
#include <linux/pagemap.h>
#include <linux/module.h>
#include <linux/device.h>
#include <linux/cdev.h>
#include <linux/slab.h>
#include <linux/dax.h>
#include <linux/uio.h>
#include <linux/fs.h>
#include <linux/mm.h>
#include "dax-private.h"
#include "bus.h"
/*
* FS-DAX compatible devdax driver
*
* Unlike drivers/dax/device.c which pre-initializes compound folios based
* on device alignment (via vmemmap_shift), this driver leaves folios
* uninitialized similar to pmem. This allows fs-dax filesystems like famfs
* to work without needing special handling for pre-initialized folios.
*
* Key differences from device.c:
* - pgmap type is MEMORY_DEVICE_FS_DAX (not MEMORY_DEVICE_GENERIC)
* - vmemmap_shift is NOT set (folios remain order-0)
* - fs-dax can dynamically create compound folios as needed
* - No mmap support - all access is through fs-dax/iomap
*/
static void fsdev_write_dax(void *addr, struct page *page,
unsigned int off, unsigned int len)
{
while (len) {
void *mem = kmap_local_page(page);
unsigned int chunk = min_t(unsigned int, len, PAGE_SIZE - off);
memcpy_flushcache(addr, mem + off, chunk);
kunmap_local(mem);
len -= chunk;
off = 0;
page++;
addr += chunk;
}
}
static long __fsdev_dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff,
long nr_pages, enum dax_access_mode mode, void **kaddr,
unsigned long *pfn)
{
struct dev_dax *dev_dax = dax_get_private(dax_dev);
size_t size = nr_pages << PAGE_SHIFT;
size_t offset = pgoff << PAGE_SHIFT;
void *virt_addr = dev_dax->virt_addr + offset;
phys_addr_t phys;
unsigned long local_pfn;
phys = dax_pgoff_to_phys(dev_dax, pgoff, size);
if (phys == -1) {
dev_dbg(&dev_dax->dev,
"pgoff (%#lx) out of range\n", pgoff);
return -EFAULT;
}
if (kaddr)
*kaddr = virt_addr;
local_pfn = PHYS_PFN(phys);
if (pfn)
*pfn = local_pfn;
/*
* Use cached_size which was computed at probe time. The size cannot
* change while the driver is bound (resize returns -EBUSY).
*/
return PHYS_PFN(min(size, dev_dax->cached_size - offset));
}
static int fsdev_dax_zero_page_range(struct dax_device *dax_dev,
pgoff_t pgoff, size_t nr_pages)
{
void *kaddr;
long rc;
WARN_ONCE(nr_pages > 1, "%s: nr_pages > 1\n", __func__);
rc = __fsdev_dax_direct_access(dax_dev, pgoff, 1, DAX_ACCESS, &kaddr, NULL);
if (rc < 0)
return rc;
fsdev_write_dax(kaddr, ZERO_PAGE(0), 0, PAGE_SIZE);
return 0;
}
static long fsdev_dax_direct_access(struct dax_device *dax_dev,
pgoff_t pgoff, long nr_pages, enum dax_access_mode mode,
void **kaddr, unsigned long *pfn)
{
return __fsdev_dax_direct_access(dax_dev, pgoff, nr_pages, mode,
kaddr, pfn);
}
static size_t fsdev_dax_recovery_write(struct dax_device *dax_dev, pgoff_t pgoff,
void *addr, size_t bytes, struct iov_iter *i)
{
return _copy_from_iter_flushcache(addr, bytes, i);
}
static const struct dax_operations dev_dax_ops = {
.direct_access = fsdev_dax_direct_access,
.zero_page_range = fsdev_dax_zero_page_range,
.recovery_write = fsdev_dax_recovery_write,
};
static void fsdev_cdev_del(void *cdev)
{
cdev_del(cdev);
}
static void fsdev_kill(void *dev_dax)
{
kill_dev_dax(dev_dax);
}
static void fsdev_clear_ops(void *data)
{
struct dev_dax *dev_dax = data;
dax_set_ops(dev_dax->dax_dev, NULL);
}
/*
* Page map operations for FS-DAX mode
* Similar to fsdax_pagemap_ops in drivers/nvdimm/pmem.c
*
* Note: folio_free callback is not needed for MEMORY_DEVICE_FS_DAX.
* The core mm code in free_zone_device_folio() handles the wake_up_var()
* directly for this memory type.
*/
static int fsdev_pagemap_memory_failure(struct dev_pagemap *pgmap,
unsigned long pfn, unsigned long nr_pages, int mf_flags)
{
struct dev_dax *dev_dax = pgmap->owner;
u64 offset = PFN_PHYS(pfn) - dev_dax->ranges[0].range.start;
u64 len = nr_pages << PAGE_SHIFT;
return dax_holder_notify_failure(dev_dax->dax_dev, offset,
len, mf_flags);
}
static const struct dev_pagemap_ops fsdev_pagemap_ops = {
.memory_failure = fsdev_pagemap_memory_failure,
};
/*
* Clear any stale folio state from pages in the given range.
* This is necessary because device_dax pre-initializes compound folios
* based on vmemmap_shift, and that state may persist after driver unbind.
* Since fsdev_dax uses MEMORY_DEVICE_FS_DAX without vmemmap_shift, fs-dax
* expects to find clean order-0 folios that it can build into compound
* folios on demand.
*
* At probe time, no filesystem should be mounted yet, so all mappings
* are stale and must be cleared along with compound state.
*/
static void fsdev_clear_folio_state(struct dev_dax *dev_dax)
{
for (int i = 0; i < dev_dax->nr_range; i++) {
struct range *range = &dev_dax->ranges[i].range;
unsigned long pfn = PHYS_PFN(range->start);
unsigned long end_pfn = PHYS_PFN(range->end) + 1;
while (pfn < end_pfn) {
struct folio *folio = pfn_folio(pfn);
int order = dax_folio_reset_order(folio);
pfn += 1UL << order;
}
}
}
static void fsdev_clear_folio_state_action(void *data)
{
fsdev_clear_folio_state(data);
}
static int fsdev_open(struct inode *inode, struct file *filp)
{
struct dax_device *dax_dev = inode_dax(inode);
struct dev_dax *dev_dax = dax_get_private(dax_dev);
filp->private_data = dev_dax;
return 0;
}
static int fsdev_release(struct inode *inode, struct file *filp)
{
return 0;
}
static const struct file_operations fsdev_fops = {
.llseek = noop_llseek,
.owner = THIS_MODULE,
.open = fsdev_open,
.release = fsdev_release,
};
static int fsdev_dax_probe(struct dev_dax *dev_dax)
{
struct dax_device *dax_dev = dev_dax->dax_dev;
struct device *dev = &dev_dax->dev;
struct dev_pagemap *pgmap;
struct inode *inode;
u64 data_offset = 0;
struct cdev *cdev;
void *addr;
int rc, i;
if (static_dev_dax(dev_dax)) {
if (dev_dax->nr_range > 1) {
dev_warn(dev, "static pgmap / multi-range device conflict\n");
return -EINVAL;
}
pgmap = dev_dax->pgmap;
} else {
size_t pgmap_size;
if (dev_dax->pgmap) {
dev_warn(dev, "dynamic-dax with pre-populated page map\n");
return -EINVAL;
}
pgmap_size = struct_size(pgmap, ranges, dev_dax->nr_range - 1);
pgmap = devm_kzalloc(dev, pgmap_size, GFP_KERNEL);
if (!pgmap)
return -ENOMEM;
pgmap->nr_range = dev_dax->nr_range;
dev_dax->pgmap = pgmap;
for (i = 0; i < dev_dax->nr_range; i++) {
struct range *range = &dev_dax->ranges[i].range;
pgmap->ranges[i] = *range;
}
}
for (i = 0; i < dev_dax->nr_range; i++) {
struct range *range = &dev_dax->ranges[i].range;
if (!devm_request_mem_region(dev, range->start,
range_len(range), dev_name(dev))) {
dev_warn(dev, "mapping%d: %#llx-%#llx could not reserve range\n",
i, range->start, range->end);
return -EBUSY;
}
}
/* Cache size now; it cannot change while driver is bound */
dev_dax->cached_size = 0;
for (i = 0; i < dev_dax->nr_range; i++)
dev_dax->cached_size += range_len(&dev_dax->ranges[i].range);
/*
* Use MEMORY_DEVICE_FS_DAX without setting vmemmap_shift, leaving
* folios at order-0. Unlike device.c (MEMORY_DEVICE_GENERIC), this
* lets fs-dax dynamically build compound folios as needed, similar
* to pmem behavior.
*/
pgmap->type = MEMORY_DEVICE_FS_DAX;
pgmap->ops = &fsdev_pagemap_ops;
pgmap->owner = dev_dax;
addr = devm_memremap_pages(dev, pgmap);
if (IS_ERR(addr))
return PTR_ERR(addr);
/*
* Clear any stale compound folio state left over from a previous
* driver (e.g., device_dax with vmemmap_shift). Also register this
* as a devm action so folio state is cleared on unbind, ensuring
* clean pages for subsequent drivers (e.g., kmem for system-ram).
*/
fsdev_clear_folio_state(dev_dax);
rc = devm_add_action_or_reset(dev, fsdev_clear_folio_state_action,
dev_dax);
if (rc)
return rc;
/* Detect whether the data is at a non-zero offset into the memory */
if (pgmap->range.start != dev_dax->ranges[0].range.start) {
u64 phys = dev_dax->ranges[0].range.start;
u64 pgmap_phys = dev_dax->pgmap[0].range.start;
if (!WARN_ON(pgmap_phys > phys))
data_offset = phys - pgmap_phys;
pr_debug("%s: offset detected phys=%llx pgmap_phys=%llx offset=%llx\n",
__func__, phys, pgmap_phys, data_offset);
}
dev_dax->virt_addr = addr + data_offset;
inode = dax_inode(dax_dev);
cdev = inode->i_cdev;
cdev_init(cdev, &fsdev_fops);
cdev->owner = dev->driver->owner;
cdev_set_parent(cdev, &dev->kobj);
rc = cdev_add(cdev, dev->devt, 1);
if (rc)
return rc;
rc = devm_add_action_or_reset(dev, fsdev_cdev_del, cdev);
if (rc)
return rc;
/* Set the dax operations for fs-dax access path */
rc = dax_set_ops(dax_dev, &dev_dax_ops);
if (rc)
return rc;
rc = devm_add_action_or_reset(dev, fsdev_clear_ops, dev_dax);
if (rc)
return rc;
run_dax(dax_dev);
return devm_add_action_or_reset(dev, fsdev_kill, dev_dax);
}
static struct dax_device_driver fsdev_dax_driver = {
.probe = fsdev_dax_probe,
.type = DAXDRV_FSDEV_TYPE,
};
static int __init dax_init(void)
{
return dax_driver_register(&fsdev_dax_driver);
}
static void __exit dax_exit(void)
{
dax_driver_unregister(&fsdev_dax_driver);
}
MODULE_AUTHOR("John Groves");
MODULE_DESCRIPTION("FS-DAX Device: fs-dax compatible devdax driver");
MODULE_LICENSE("GPL");
module_init(dax_init);
module_exit(dax_exit);
MODULE_ALIAS_DAX_DEVICE(0);
drivers/dax/super.c
+104 -3
View File
@@ -14,6 +14,7 @@
#include <linux/fs.h>
#include <linux/cacheinfo.h>
#include "dax-private.h"
#include "bus.h"
/**
* struct dax_device - anchor object for dax services
@@ -111,6 +112,10 @@ struct dax_device *fs_dax_get_by_bdev(struct block_device *bdev, u64 *start_off,
}
EXPORT_SYMBOL_GPL(fs_dax_get_by_bdev);
#endif /* CONFIG_BLOCK && CONFIG_FS_DAX */
#if IS_ENABLED(CONFIG_FS_DAX)
void fs_put_dax(struct dax_device *dax_dev, void *holder)
{
if (dax_dev && holder &&
@@ -119,7 +124,66 @@ void fs_put_dax(struct dax_device *dax_dev, void *holder)
put_dax(dax_dev);
}
EXPORT_SYMBOL_GPL(fs_put_dax);
#endif /* CONFIG_BLOCK && CONFIG_FS_DAX */
/**
* fs_dax_get() - get ownership of a devdax via holder/holder_ops
*
* fs-dax file systems call this function to prepare to use a devdax device for
* fsdax. This is like fs_dax_get_by_bdev(), but the caller already has struct
* dev_dax (and there is no bdev). The holder makes this exclusive.
*
* @dax_dev: dev to be prepared for fs-dax usage
* @holder: filesystem or mapped device inside the dax_device
* @hops: operations for the inner holder
*
* Returns: 0 on success, <0 on failure
*/
int fs_dax_get(struct dax_device *dax_dev, void *holder,
const struct dax_holder_operations *hops)
{
struct dev_dax *dev_dax;
struct dax_device_driver *dax_drv;
int id;
id = dax_read_lock();
if (!dax_dev || !dax_alive(dax_dev) || !igrab(&dax_dev->inode)) {
dax_read_unlock(id);
return -ENODEV;
}
dax_read_unlock(id);
/* Verify the device is bound to fsdev_dax driver */
dev_dax = dax_get_private(dax_dev);
if (!dev_dax) {
iput(&dax_dev->inode);
return -ENODEV;
}
device_lock(&dev_dax->dev);
if (!dev_dax->dev.driver) {
device_unlock(&dev_dax->dev);
iput(&dax_dev->inode);
return -ENODEV;
}
dax_drv = to_dax_drv(dev_dax->dev.driver);
if (dax_drv->type != DAXDRV_FSDEV_TYPE) {
device_unlock(&dev_dax->dev);
iput(&dax_dev->inode);
return -EOPNOTSUPP;
}
device_unlock(&dev_dax->dev);
if (cmpxchg(&dax_dev->holder_data, NULL, holder)) {
iput(&dax_dev->inode);
return -EBUSY;
}
dax_dev->holder_ops = hops;
return 0;
}
EXPORT_SYMBOL_GPL(fs_dax_get);
#endif /* CONFIG_FS_DAX */
enum dax_device_flags {
/* !alive + rcu grace period == no new operations / mappings */
@@ -157,6 +221,9 @@ long dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, long nr_pages,
if (!dax_alive(dax_dev))
return -ENXIO;
if (!dax_dev->ops)
return -EOPNOTSUPP;
if (nr_pages < 0)
return -EINVAL;
@@ -207,6 +274,10 @@ int dax_zero_page_range(struct dax_device *dax_dev, pgoff_t pgoff,
if (!dax_alive(dax_dev))
return -ENXIO;
if (!dax_dev->ops)
return -EOPNOTSUPP;
/*
* There are no callers that want to zero more than one page as of now.
* Once users are there, this check can be removed after the
@@ -223,7 +294,7 @@ EXPORT_SYMBOL_GPL(dax_zero_page_range);
size_t dax_recovery_write(struct dax_device *dax_dev, pgoff_t pgoff,
void *addr, size_t bytes, struct iov_iter *iter)
{
if (!dax_dev->ops->recovery_write)
if (!dax_dev->ops || !dax_dev->ops->recovery_write)
return 0;
return dax_dev->ops->recovery_write(dax_dev, pgoff, addr, bytes, iter);
}
@@ -307,6 +378,35 @@ void set_dax_nomc(struct dax_device *dax_dev)
}
EXPORT_SYMBOL_GPL(set_dax_nomc);
/**
* dax_set_ops - set the dax_operations for a dax_device
* @dax_dev: the dax_device to configure
* @ops: the operations to set (may be NULL to clear)
*
* This allows drivers to set the dax_operations after the dax_device
* has been allocated. This is needed when the device is created before
* the driver that needs specific ops is bound (e.g., fsdev_dax binding
* to a dev_dax created by hmem).
*
* When setting non-NULL ops, fails if ops are already set (returns -EBUSY).
* When clearing ops (NULL), always succeeds.
*
* Return: 0 on success, -EBUSY if ops already set
*/
int dax_set_ops(struct dax_device *dax_dev, const struct dax_operations *ops)
{
if (ops) {
/* Setting ops: fail if already set */
if (cmpxchg(&dax_dev->ops, NULL, ops) != NULL)
return -EBUSY;
} else {
/* Clearing ops: always allowed */
dax_dev->ops = NULL;
}
return 0;
}
EXPORT_SYMBOL_GPL(dax_set_ops);
bool dax_alive(struct dax_device *dax_dev)
{
lockdep_assert_held(&dax_srcu);
@@ -421,7 +521,7 @@ static int dax_set(struct inode *inode, void *data)
return 0;
}
static struct dax_device *dax_dev_get(dev_t devt)
struct dax_device *dax_dev_get(dev_t devt)
{
struct dax_device *dax_dev;
struct inode *inode;
@@ -444,6 +544,7 @@ static struct dax_device *dax_dev_get(dev_t devt)
return dax_dev;
}
EXPORT_SYMBOL_GPL(dax_dev_get);
struct dax_device *alloc_dax(void *private, const struct dax_operations *ops)
{
drivers/dma-buf/dma-buf.c
+1
View File
@@ -1355,6 +1355,7 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_attach_revocable, "DMA_BUF");
* Upon return importers may continue to access the DMA-buf memory. The caller
* must do two additional waits to ensure that the memory is no longer being
* accessed:
*
* 1) Until dma_resv_wait_timeout() retires fences the importer is allowed to
* fully access the memory.
* 2) Until the importer calls unmap it is allowed to speculatively
drivers/gpu/drm/amd/amdgpu/Kconfig
+17
View File
@@ -103,6 +103,23 @@ config DRM_AMDGPU_WERROR
Add -Werror to the build flags for amdgpu.ko.
Only enable this if you are warning code for amdgpu.ko.
config GCOV_PROFILE_AMDGPU
bool "Enable GCOV profiling on amdgpu"
depends on DRM_AMDGPU
depends on GCOV_KERNEL
default n
help
Enable GCOV profiling on the amdgpu driver for checking which
functions/lines are executed during testing. This adds compiler
instrumentation flags to all amdgpu source files, producing
.gcda/.gcno coverage data accessible via debugfs.
This increases the amdgpu module size by ~50% and adds ~2-5%
runtime overhead on GPU submission paths.
If unsure, say N.
source "drivers/gpu/drm/amd/acp/Kconfig"
source "drivers/gpu/drm/amd/display/Kconfig"
source "drivers/gpu/drm/amd/amdkfd/Kconfig"
drivers/gpu/drm/amd/amdgpu/Makefile
+4
View File
@@ -27,6 +27,10 @@ FULL_AMD_PATH=$(src)/..
DISPLAY_FOLDER_NAME=display
FULL_AMD_DISPLAY_PATH = $(FULL_AMD_PATH)/$(DISPLAY_FOLDER_NAME)
ifdef CONFIG_GCOV_PROFILE_AMDGPU
GCOV_PROFILE := y
endif
ccflags-y := -I$(FULL_AMD_PATH)/include/asic_reg \
-I$(FULL_AMD_PATH)/include \
-I$(FULL_AMD_PATH)/amdgpu \
drivers/gpu/drm/amd/amdgpu/aldebaran.c
+1 -1
View File
@@ -179,7 +179,7 @@ aldebaran_mode2_perform_reset(struct amdgpu_reset_control *reset_ctl,
list_for_each_entry(tmp_adev, reset_device_list, reset_list) {
/* For XGMI run all resets in parallel to speed up the process */
if (tmp_adev->gmc.xgmi.num_physical_nodes > 1) {
if (!queue_work(system_unbound_wq,
if (!queue_work(system_dfl_wq,
&tmp_adev->reset_cntl->reset_work))
r = -EALREADY;
} else
drivers/gpu/drm/amd/amdgpu/amdgpu.h
-5
View File
@@ -1045,11 +1045,6 @@ struct amdgpu_device {
struct amdgpu_mqd mqds[AMDGPU_HW_IP_NUM];
const struct amdgpu_userq_funcs *userq_funcs[AMDGPU_HW_IP_NUM];
/* xarray used to retrieve the user queue fence driver reference
* in the EOP interrupt handler to signal the particular user
* queue fence.
*/
struct xarray userq_xa;
/**
* @userq_doorbell_xa: Global user queue map (doorbell index → queue)
* Key: doorbell_index (unique global identifier for the queue)
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
+4 -1
View File
@@ -805,7 +805,10 @@ u64 amdgpu_amdkfd_xcp_memory_size(struct amdgpu_device *adev, int xcp_id)
} else {
tmp = adev->gmc.mem_partitions[mem_id].size;
}
do_div(tmp, adev->xcp_mgr->num_xcp_per_mem_partition);
if (adev->xcp_mgr->mem_alloc_mode == AMDGPU_PARTITION_MEM_CAPPING_EVEN)
do_div(tmp, adev->xcp_mgr->num_xcp_per_mem_partition);
return ALIGN_DOWN(tmp, PAGE_SIZE);
} else if (adev->apu_prefer_gtt) {
return (ttm_tt_pages_limit() << PAGE_SHIFT);
drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
+3 -3
View File
@@ -1685,9 +1685,9 @@ static int amdgpu_atombios_allocate_fb_scratch(struct amdgpu_device *adev)
(uint32_t)(ATOM_VRAM_BLOCK_SRIOV_MSG_SHARE_RESERVATION <<
ATOM_VRAM_OPERATION_FLAGS_SHIFT)) {
/* Firmware request VRAM reservation for SR-IOV */
adev->mman.fw_vram_usage_start_offset = (start_addr &
(~ATOM_VRAM_OPERATION_FLAGS_MASK)) << 10;
adev->mman.fw_vram_usage_size = size << 10;
amdgpu_ttm_init_vram_resv(adev, AMDGPU_RESV_FW_VRAM_USAGE,
(start_addr & (~ATOM_VRAM_OPERATION_FLAGS_MASK)) << 10,
size << 10, true);
/* Use the default scratch size */
usage_bytes = 0;
} else {
drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c
+9 -9
View File
@@ -120,9 +120,9 @@ static int amdgpu_atomfirmware_allocate_fb_v2_1(struct amdgpu_device *adev,
(u32)(ATOM_VRAM_BLOCK_SRIOV_MSG_SHARE_RESERVATION <<
ATOM_VRAM_OPERATION_FLAGS_SHIFT)) {
/* Firmware request VRAM reservation for SR-IOV */
adev->mman.fw_vram_usage_start_offset = (start_addr &
(~ATOM_VRAM_OPERATION_FLAGS_MASK)) << 10;
adev->mman.fw_vram_usage_size = fw_size << 10;
amdgpu_ttm_init_vram_resv(adev, AMDGPU_RESV_FW_VRAM_USAGE,
(start_addr & (~ATOM_VRAM_OPERATION_FLAGS_MASK)) << 10,
fw_size << 10, true);
/* Use the default scratch size */
*usage_bytes = 0;
} else {
@@ -152,18 +152,18 @@ static int amdgpu_atomfirmware_allocate_fb_v2_2(struct amdgpu_device *adev,
((fw_start_addr & (ATOM_VRAM_BLOCK_NEEDS_NO_RESERVATION <<
ATOM_VRAM_OPERATION_FLAGS_SHIFT)) == 0)) {
/* Firmware request VRAM reservation for SR-IOV */
adev->mman.fw_vram_usage_start_offset = (fw_start_addr &
(~ATOM_VRAM_OPERATION_FLAGS_MASK)) << 10;
adev->mman.fw_vram_usage_size = fw_size << 10;
amdgpu_ttm_init_vram_resv(adev, AMDGPU_RESV_FW_VRAM_USAGE,
(fw_start_addr & (~ATOM_VRAM_OPERATION_FLAGS_MASK)) << 10,
fw_size << 10, true);
}
if (amdgpu_sriov_vf(adev) &&
((drv_start_addr & (ATOM_VRAM_BLOCK_NEEDS_NO_RESERVATION <<
ATOM_VRAM_OPERATION_FLAGS_SHIFT)) == 0)) {
/* driver request VRAM reservation for SR-IOV */
adev->mman.drv_vram_usage_start_offset = (drv_start_addr &
(~ATOM_VRAM_OPERATION_FLAGS_MASK)) << 10;
adev->mman.drv_vram_usage_size = drv_size << 10;
amdgpu_ttm_init_vram_resv(adev, AMDGPU_RESV_DRV_VRAM_USAGE,
(drv_start_addr & (~ATOM_VRAM_OPERATION_FLAGS_MASK)) << 10,
drv_size << 10, true);
}
*usage_bytes = 0;
drivers/gpu/drm/amd/amdgpu/amdgpu_dev_coredump.c
+168 -17
View File
@@ -210,12 +210,24 @@ static void amdgpu_devcoredump_fw_info(struct amdgpu_device *adev,
static ssize_t
amdgpu_devcoredump_format(char *buffer, size_t count, struct amdgpu_coredump_info *coredump)
{
struct amdgpu_device *adev = coredump->adev;
struct drm_printer p;
struct drm_print_iterator iter;
struct amdgpu_vm_fault_info *fault_info;
struct amdgpu_bo_va_mapping *mapping;
struct amdgpu_ip_block *ip_block;
int ver;
struct amdgpu_res_cursor cursor;
struct amdgpu_bo *abo, *root;
uint64_t va_start, offset;
struct amdgpu_ring *ring;
struct amdgpu_vm *vm;
u32 *ib_content;
uint8_t *kptr;
int ver, i, j, r;
u32 ring_idx, off;
bool sizing_pass;
sizing_pass = buffer == NULL;
iter.data = buffer;
iter.offset = 0;
iter.remain = count;
@@ -303,23 +315,25 @@ amdgpu_devcoredump_format(char *buffer, size_t count, struct amdgpu_coredump_inf
/* Add ring buffer information */
drm_printf(&p, "Ring buffer information\n");
for (int i = 0; i < coredump->adev->num_rings; i++) {
int j = 0;
struct amdgpu_ring *ring = coredump->adev->rings[i];
if (coredump->num_rings) {
for (i = 0; i < coredump->num_rings; i++) {
ring_idx = coredump->rings[i].ring_index;
ring = coredump->adev->rings[ring_idx];
off = coredump->rings[i].offset;
drm_printf(&p, "ring name: %s\n", ring->name);
drm_printf(&p, "Rptr: 0x%llx Wptr: 0x%llx RB mask: %x\n",
amdgpu_ring_get_rptr(ring),
amdgpu_ring_get_wptr(ring),
ring->buf_mask);
drm_printf(&p, "Ring size in dwords: %d\n",
ring->ring_size / 4);
drm_printf(&p, "Ring contents\n");
drm_printf(&p, "Offset \t Value\n");
drm_printf(&p, "ring name: %s\n", ring->name);
drm_printf(&p, "Rptr: 0x%llx Wptr: 0x%llx RB mask: %x\n",
coredump->rings[i].rptr,
coredump->rings[i].wptr,
ring->buf_mask);
drm_printf(&p, "Ring size in dwords: %d\n",
ring->ring_size / 4);
drm_printf(&p, "Ring contents\n");
drm_printf(&p, "Offset \t Value\n");
while (j < ring->ring_size) {
drm_printf(&p, "0x%x \t 0x%x\n", j, ring->ring[j / 4]);
j += 4;
for (j = 0; j < ring->ring_size; j += 4)
drm_printf(&p, "0x%x \t 0x%x\n", j,
coredump->rings_dw[off + j / 4]);
}
}
@@ -328,6 +342,87 @@ amdgpu_devcoredump_format(char *buffer, size_t count, struct amdgpu_coredump_inf
else if (coredump->reset_vram_lost)
drm_printf(&p, "VRAM is lost due to GPU reset!\n");
if (coredump->num_ibs) {
/* Don't try to lookup the VM or map the BOs when calculating the
* size required to store the devcoredump.
*/
if (sizing_pass)
vm = NULL;
else
vm = amdgpu_vm_lock_by_pasid(adev, &root, coredump->pasid);
for (int i = 0; i < coredump->num_ibs && (sizing_pass || vm); i++) {
ib_content = kvmalloc_array(coredump->ibs[i].ib_size_dw, 4,
GFP_KERNEL);
if (!ib_content)
continue;
/* vm=NULL can only happen when 'sizing_pass' is true. Skip to the
* drm_printf() calls (ib_content doesn't need to be initialized
* as its content won't be written anywhere).
*/
if (!vm)
goto output_ib_content;
va_start = coredump->ibs[i].gpu_addr & AMDGPU_GMC_HOLE_MASK;
mapping = amdgpu_vm_bo_lookup_mapping(vm, va_start / AMDGPU_GPU_PAGE_SIZE);
if (!mapping)
goto free_ib_content;
offset = va_start - (mapping->start * AMDGPU_GPU_PAGE_SIZE);
abo = amdgpu_bo_ref(mapping->bo_va->base.bo);
r = amdgpu_bo_reserve(abo, false);
if (r)
goto free_ib_content;
if (abo->flags & AMDGPU_GEM_CREATE_NO_CPU_ACCESS) {
off = 0;
if (abo->tbo.resource->mem_type != TTM_PL_VRAM)
goto unreserve_abo;
amdgpu_res_first(abo->tbo.resource, offset,
coredump->ibs[i].ib_size_dw * 4,
&cursor);
while (cursor.remaining) {
amdgpu_device_mm_access(adev, cursor.start / 4,
&ib_content[off], cursor.size / 4,
false);
off += cursor.size;
amdgpu_res_next(&cursor, cursor.size);
}
} else {
r = ttm_bo_kmap(&abo->tbo, 0,
PFN_UP(abo->tbo.base.size),
&abo->kmap);
if (r)
goto unreserve_abo;
kptr = amdgpu_bo_kptr(abo);
kptr += offset;
memcpy(ib_content, kptr,
coredump->ibs[i].ib_size_dw * 4);
amdgpu_bo_kunmap(abo);
}
output_ib_content:
drm_printf(&p, "\nIB #%d 0x%llx %d dw\n",
i, coredump->ibs[i].gpu_addr, coredump->ibs[i].ib_size_dw);
for (int j = 0; j < coredump->ibs[i].ib_size_dw; j++)
drm_printf(&p, "0x%08x\n", ib_content[j]);
unreserve_abo:
if (vm)
amdgpu_bo_unreserve(abo);
free_ib_content:
kvfree(ib_content);
}
if (vm) {
amdgpu_bo_unreserve(root);
amdgpu_bo_unref(&root);
}
}
return count - iter.remain;
}
@@ -359,6 +454,8 @@ static void amdgpu_devcoredump_free(void *data)
struct amdgpu_coredump_info *coredump = data;
kvfree(coredump->formatted);
kvfree(coredump->rings);
kvfree(coredump->rings_dw);
kvfree(data);
}
@@ -395,18 +492,26 @@ void amdgpu_coredump(struct amdgpu_device *adev, bool skip_vram_check,
{
struct drm_device *dev = adev_to_drm(adev);
struct amdgpu_coredump_info *coredump;
size_t size = sizeof(*coredump);
struct drm_sched_job *s_job;
u64 total_ring_size, ring_count;
struct amdgpu_ring *ring;
int i, off, idx;
/* No need to generate a new coredump if there's one in progress already. */
if (work_pending(&adev->coredump_work))
return;
coredump = kzalloc_obj(*coredump, GFP_NOWAIT);
if (job && job->pasid)
size += sizeof(struct amdgpu_coredump_ib_info) * job->num_ibs;
coredump = kzalloc(size, GFP_NOWAIT);
if (!coredump)
return;
coredump->skip_vram_check = skip_vram_check;
coredump->reset_vram_lost = vram_lost;
coredump->pasid = job->pasid;
if (job && job->pasid) {
struct amdgpu_task_info *ti;
@@ -416,6 +521,11 @@ void amdgpu_coredump(struct amdgpu_device *adev, bool skip_vram_check,
coredump->reset_task_info = *ti;
amdgpu_vm_put_task_info(ti);
}
coredump->num_ibs = job->num_ibs;
for (i = 0; i < job->num_ibs; ++i) {
coredump->ibs[i].gpu_addr = job->ibs[i].gpu_addr;
coredump->ibs[i].ib_size_dw = job->ibs[i].length_dw;
}
}
if (job) {
@@ -423,6 +533,47 @@ void amdgpu_coredump(struct amdgpu_device *adev, bool skip_vram_check,
coredump->ring = to_amdgpu_ring(s_job->sched);
}
/* Dump ring content if memory allocation succeeds. */
ring_count = 0;
total_ring_size = 0;
for (i = 0; i < adev->num_rings; i++) {
ring = adev->rings[i];
/* Only dump rings with unsignalled fences. */
if (atomic_read(&ring->fence_drv.last_seq) == ring->fence_drv.sync_seq &&
coredump->ring != ring)
continue;
total_ring_size += ring->ring_size;
ring_count++;
}
coredump->rings_dw = kzalloc(total_ring_size, GFP_NOWAIT);
coredump->rings = kcalloc(ring_count, sizeof(struct amdgpu_coredump_ring), GFP_NOWAIT);
if (coredump->rings && coredump->rings_dw) {
for (i = 0, off = 0, idx = 0; i < adev->num_rings; i++) {
ring = adev->rings[i];
if (atomic_read(&ring->fence_drv.last_seq) == ring->fence_drv.sync_seq &&
coredump->ring != ring)
continue;
coredump->rings[idx].ring_index = ring->idx;
coredump->rings[idx].rptr = amdgpu_ring_get_rptr(ring);
coredump->rings[idx].wptr = amdgpu_ring_get_wptr(ring);
coredump->rings[idx].offset = off;
memcpy(&coredump->rings_dw[off], ring->ring, ring->ring_size);
off += ring->ring_size;
idx++;
}
coredump->num_rings = idx;
} else {
kvfree(coredump->rings_dw);
kvfree(coredump->rings);
coredump->rings_dw = NULL;
coredump->rings = NULL;
}
coredump->adev = adev;
ktime_get_ts64(&coredump->reset_time);
drivers/gpu/drm/amd/amdgpu/amdgpu_dev_coredump.h
+21
View File
@@ -31,6 +31,18 @@
#define AMDGPU_COREDUMP_VERSION "1"
struct amdgpu_coredump_ring {
u64 rptr;
u64 wptr;
u32 ring_index;
u32 offset;
};
struct amdgpu_coredump_ib_info {
uint64_t gpu_addr;
u32 ib_size_dw;
};
struct amdgpu_coredump_info {
struct amdgpu_device *adev;
struct amdgpu_task_info reset_task_info;
@@ -39,11 +51,20 @@ struct amdgpu_coredump_info {
bool skip_vram_check;
bool reset_vram_lost;
struct amdgpu_ring *ring;
struct amdgpu_coredump_ring *rings;
u32 *rings_dw;
u32 num_rings;
/* Readable form of coredevdump, generate once to speed up
* reading it (see drm_coredump_printer's documentation).
*/
ssize_t formatted_size;
char *formatted;
unsigned int pasid;
int num_ibs;
struct amdgpu_coredump_ib_info ibs[] __counted_by(num_ibs);
};
#endif
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+4 -6
View File
@@ -3757,15 +3757,13 @@ int amdgpu_device_init(struct amdgpu_device *adev,
spin_lock_init(&adev->virt.rlcg_reg_lock);
spin_lock_init(&adev->wb.lock);
xa_init_flags(&adev->userq_xa, XA_FLAGS_LOCK_IRQ);
INIT_LIST_HEAD(&adev->reset_list);
INIT_LIST_HEAD(&adev->ras_list);
INIT_LIST_HEAD(&adev->pm.od_kobj_list);
xa_init(&adev->userq_doorbell_xa);
xa_init_flags(&adev->userq_doorbell_xa, XA_FLAGS_LOCK_IRQ);
INIT_DELAYED_WORK(&adev->delayed_init_work,
amdgpu_device_delayed_init_work_handler);
@@ -4065,7 +4063,7 @@ fence_driver_init:
}
/* must succeed. */
amdgpu_ras_resume(adev);
queue_delayed_work(system_wq, &adev->delayed_init_work,
queue_delayed_work(system_dfl_wq, &adev->delayed_init_work,
msecs_to_jiffies(AMDGPU_RESUME_MS));
}
@@ -4630,7 +4628,7 @@ int amdgpu_device_resume(struct drm_device *dev, bool notify_clients)
if (r)
goto exit;
queue_delayed_work(system_wq, &adev->delayed_init_work,
queue_delayed_work(system_dfl_wq, &adev->delayed_init_work,
msecs_to_jiffies(AMDGPU_RESUME_MS));
exit:
if (amdgpu_sriov_vf(adev)) {
@@ -5339,7 +5337,7 @@ int amdgpu_do_asic_reset(struct list_head *device_list_handle,
list_for_each_entry(tmp_adev, device_list_handle, reset_list) {
/* For XGMI run all resets in parallel to speed up the process */
if (tmp_adev->gmc.xgmi.num_physical_nodes > 1) {
if (!queue_work(system_unbound_wq,
if (!queue_work(system_dfl_wq,
&tmp_adev->xgmi_reset_work))
r = -EALREADY;
} else
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+2 -5
View File
@@ -641,9 +641,7 @@ module_param_named(si_support, amdgpu_si_support, int, 0444);
* CIK (Sea Islands) are second generation GCN GPUs, supported by both
* drivers: radeon (old) and amdgpu (new). This parameter controls whether
* amdgpu should support CIK.
* By default:
* - CIK dedicated GPUs are supported by amdgpu.
* - CIK APUs are supported by radeon (except when radeon is not built).
* By default, CIK dedicated GPUs and APUs are supported by amdgpu.
* Only relevant when CONFIG_DRM_AMDGPU_CIK is enabled to build CIK support in amdgpu.
* See also radeon.cik_support which should be disabled when amdgpu.cik_support is
* enabled, and vice versa.
@@ -2323,8 +2321,6 @@ static bool amdgpu_support_enabled(struct device *dev,
case CHIP_BONAIRE:
case CHIP_HAWAII:
support_by_default = true;
fallthrough;
case CHIP_KAVERI:
case CHIP_KABINI:
case CHIP_MULLINS:
@@ -2332,6 +2328,7 @@ static bool amdgpu_support_enabled(struct device *dev,
param = "cik_support";
module_param = amdgpu_cik_support;
amdgpu_support_built = IS_ENABLED(CONFIG_DRM_AMDGPU_CIK);
support_by_default = true;
break;
default:
drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
+39
View File
@@ -1580,6 +1580,36 @@ static ssize_t amdgpu_gfx_set_compute_partition(struct device *dev,
return count;
}
static ssize_t compute_partition_mem_alloc_mode_show(struct device *dev,
struct device_attribute *addr,
char *buf)
{
struct drm_device *ddev = dev_get_drvdata(dev);
struct amdgpu_device *adev = drm_to_adev(ddev);
int mode = adev->xcp_mgr->mem_alloc_mode;
return sysfs_emit(buf, "%s\n",
amdgpu_gfx_compute_mem_alloc_mode_desc(mode));
}
static ssize_t compute_partition_mem_alloc_mode_store(struct device *dev,
struct device_attribute *addr,
const char *buf, size_t count)
{
struct drm_device *ddev = dev_get_drvdata(dev);
struct amdgpu_device *adev = drm_to_adev(ddev);
if (!strncasecmp("CAPPING", buf, strlen("CAPPING")))
adev->xcp_mgr->mem_alloc_mode = AMDGPU_PARTITION_MEM_CAPPING_EVEN;
else if (!strncasecmp("ALL", buf, strlen("ALL")))
adev->xcp_mgr->mem_alloc_mode = AMDGPU_PARTITION_MEM_ALLOC_ALL;
else
return -EINVAL;
return count;
}
static const char *xcp_desc[] = {
[AMDGPU_SPX_PARTITION_MODE] = "SPX",
[AMDGPU_DPX_PARTITION_MODE] = "DPX",
@@ -1935,6 +1965,10 @@ static DEVICE_ATTR(gfx_reset_mask, 0444,
static DEVICE_ATTR(compute_reset_mask, 0444,
amdgpu_gfx_get_compute_reset_mask, NULL);
static DEVICE_ATTR(compute_partition_mem_alloc_mode, 0644,
compute_partition_mem_alloc_mode_show,
compute_partition_mem_alloc_mode_store);
static int amdgpu_gfx_sysfs_xcp_init(struct amdgpu_device *adev)
{
struct amdgpu_xcp_mgr *xcp_mgr = adev->xcp_mgr;
@@ -1955,6 +1989,11 @@ static int amdgpu_gfx_sysfs_xcp_init(struct amdgpu_device *adev)
if (r)
return r;
r = device_create_file(adev->dev,
&dev_attr_compute_partition_mem_alloc_mode);
if (r)
return r;
if (xcp_switch_supported)
r = device_create_file(adev->dev,
&dev_attr_available_compute_partition);
drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
+17
View File
@@ -71,6 +71,11 @@ enum amdgpu_gfx_partition {
AMDGPU_AUTO_COMPUTE_PARTITION_MODE = -2,
};
enum amdgpu_gfx_partition_mem_alloc_mode {
AMDGPU_PARTITION_MEM_CAPPING_EVEN = 0,
AMDGPU_PARTITION_MEM_ALLOC_ALL = 1,
};
#define NUM_XCC(x) hweight16(x)
enum amdgpu_gfx_ras_mem_id_type {
@@ -677,4 +682,16 @@ static inline const char *amdgpu_gfx_compute_mode_desc(int mode)
}
}
static inline const char *amdgpu_gfx_compute_mem_alloc_mode_desc(int mode)
{
switch (mode) {
case AMDGPU_PARTITION_MEM_CAPPING_EVEN:
return "CAPPING";
case AMDGPU_PARTITION_MEM_ALLOC_ALL:
return "ALL";
default:
return "UNKNOWN";
}
}
#endif
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
+13 -10
View File
@@ -1033,17 +1033,17 @@ void amdgpu_gmc_set_vm_fault_masks(struct amdgpu_device *adev, int hub_type,
}
}
void amdgpu_gmc_get_vbios_allocations(struct amdgpu_device *adev)
void amdgpu_gmc_init_vga_resv_regions(struct amdgpu_device *adev)
{
unsigned size;
if (adev->gmc.is_app_apu)
return;
/*
* Some ASICs need to reserve a region of video memory to avoid access
* from driver
*/
adev->mman.stolen_reserved_offset = 0;
adev->mman.stolen_reserved_size = 0;
/*
* TODO:
* Currently there is a bug where some memory client outside
@@ -1060,8 +1060,8 @@ void amdgpu_gmc_get_vbios_allocations(struct amdgpu_device *adev)
*/
#ifdef CONFIG_X86
if (amdgpu_sriov_vf(adev) && hypervisor_is_type(X86_HYPER_MS_HYPERV)) {
adev->mman.stolen_reserved_offset = 0x500000;
adev->mman.stolen_reserved_size = 0x200000;
amdgpu_ttm_init_vram_resv(adev, AMDGPU_RESV_STOLEN_RESERVED,
0x500000, 0x200000, false);
}
#endif
break;
@@ -1099,11 +1099,14 @@ void amdgpu_gmc_get_vbios_allocations(struct amdgpu_device *adev)
size = 0;
if (size > AMDGPU_VBIOS_VGA_ALLOCATION) {
adev->mman.stolen_vga_size = AMDGPU_VBIOS_VGA_ALLOCATION;
adev->mman.stolen_extended_size = size - adev->mman.stolen_vga_size;
amdgpu_ttm_init_vram_resv(adev, AMDGPU_RESV_STOLEN_VGA,
0, AMDGPU_VBIOS_VGA_ALLOCATION, false);
amdgpu_ttm_init_vram_resv(adev, AMDGPU_RESV_STOLEN_EXTENDED,
AMDGPU_VBIOS_VGA_ALLOCATION,
size - AMDGPU_VBIOS_VGA_ALLOCATION, false);
} else {
adev->mman.stolen_vga_size = size;
adev->mman.stolen_extended_size = 0;
amdgpu_ttm_init_vram_resv(adev, AMDGPU_RESV_STOLEN_VGA,
0, size, false);
}
}
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
+1 -1
View File
@@ -456,7 +456,7 @@ extern void
amdgpu_gmc_set_vm_fault_masks(struct amdgpu_device *adev, int hub_type,
bool enable);
void amdgpu_gmc_get_vbios_allocations(struct amdgpu_device *adev);
void amdgpu_gmc_init_vga_resv_regions(struct amdgpu_device *adev);
void amdgpu_gmc_init_pdb0(struct amdgpu_device *adev);
uint64_t amdgpu_gmc_vram_mc2pa(struct amdgpu_device *adev, uint64_t mc_addr);
drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c
+19 -20
View File
@@ -22,7 +22,7 @@
*/
#include "amdgpu_ids.h"
#include <linux/idr.h>
#include <linux/xarray.h>
#include <linux/dma-fence-array.h>
@@ -40,8 +40,8 @@
* VMs are looked up from the PASID per amdgpu_device.
*/
static DEFINE_IDR(amdgpu_pasid_idr);
static DEFINE_SPINLOCK(amdgpu_pasid_idr_lock);
static DEFINE_XARRAY_FLAGS(amdgpu_pasid_xa, XA_FLAGS_LOCK_IRQ | XA_FLAGS_ALLOC1);
static u32 amdgpu_pasid_xa_next;
/* Helper to free pasid from a fence callback */
struct amdgpu_pasid_cb {
@@ -62,36 +62,37 @@ struct amdgpu_pasid_cb {
*/
int amdgpu_pasid_alloc(unsigned int bits)
{
int pasid;
u32 pasid;
int r;
if (bits == 0)
return -EINVAL;
spin_lock(&amdgpu_pasid_idr_lock);
/* TODO: Need to replace the idr with an xarry, and then
* handle the internal locking with ATOMIC safe paths.
*/
pasid = idr_alloc_cyclic(&amdgpu_pasid_idr, NULL, 1,
1U << bits, GFP_ATOMIC);
spin_unlock(&amdgpu_pasid_idr_lock);
if (pasid >= 0)
trace_amdgpu_pasid_allocated(pasid);
r = xa_alloc_cyclic_irq(&amdgpu_pasid_xa, &pasid, xa_mk_value(0),
XA_LIMIT(1, (1U << bits) - 1),
&amdgpu_pasid_xa_next, GFP_KERNEL);
if (r < 0)
return r;
trace_amdgpu_pasid_allocated(pasid);
return pasid;
}
/**
* amdgpu_pasid_free - Free a PASID
* @pasid: PASID to free
*
* Called in IRQ context.
*/
void amdgpu_pasid_free(u32 pasid)
{
unsigned long flags;
trace_amdgpu_pasid_freed(pasid);
spin_lock(&amdgpu_pasid_idr_lock);
idr_remove(&amdgpu_pasid_idr, pasid);
spin_unlock(&amdgpu_pasid_idr_lock);
xa_lock_irqsave(&amdgpu_pasid_xa, flags);
__xa_erase(&amdgpu_pasid_xa, pasid);
xa_unlock_irqrestore(&amdgpu_pasid_xa, flags);
}
static void amdgpu_pasid_free_cb(struct dma_fence *fence,
@@ -634,7 +635,5 @@ void amdgpu_vmid_mgr_fini(struct amdgpu_device *adev)
*/
void amdgpu_pasid_mgr_cleanup(void)
{
spin_lock(&amdgpu_pasid_idr_lock);
idr_destroy(&amdgpu_pasid_idr);
spin_unlock(&amdgpu_pasid_idr_lock);
xa_destroy(&amdgpu_pasid_xa);
}
drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
+7 -6
View File
@@ -1076,24 +1076,25 @@ int psp_update_fw_reservation(struct psp_context *psp)
return 0;
}
amdgpu_bo_free_kernel(&adev->mman.fw_reserved_memory, NULL, NULL);
amdgpu_ttm_unmark_vram_reserved(adev, AMDGPU_RESV_FW);
reserv_size = roundup(reserv_size, SZ_1M);
ret = amdgpu_bo_create_kernel_at(adev, reserv_addr, reserv_size, &adev->mman.fw_reserved_memory, NULL);
amdgpu_ttm_init_vram_resv(adev, AMDGPU_RESV_FW,
reserv_addr, reserv_size, false);
ret = amdgpu_ttm_mark_vram_reserved(adev, AMDGPU_RESV_FW);
if (ret) {
dev_err(adev->dev, "reserve fw region failed(%d)!\n", ret);
amdgpu_bo_free_kernel(&adev->mman.fw_reserved_memory, NULL, NULL);
return ret;
}
reserv_size_ext = roundup(reserv_size_ext, SZ_1M);
ret = amdgpu_bo_create_kernel_at(adev, reserv_addr_ext, reserv_size_ext,
&adev->mman.fw_reserved_memory_extend, NULL);
amdgpu_ttm_init_vram_resv(adev, AMDGPU_RESV_FW_EXTEND,
reserv_addr_ext, reserv_size_ext, false);
ret = amdgpu_ttm_mark_vram_reserved(adev, AMDGPU_RESV_FW_EXTEND);
if (ret) {
dev_err(adev->dev, "reserve extend fw region failed(%d)!\n", ret);
amdgpu_bo_free_kernel(&adev->mman.fw_reserved_memory_extend, NULL, NULL);
return ret;
}
drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
-1
View File
@@ -277,7 +277,6 @@ struct psp_memory_training_context {
/*vram offset of the c2p training data*/
u64 c2p_train_data_offset;
struct amdgpu_bo *c2p_bo;
enum psp_memory_training_init_flag init;
u32 training_cnt;
drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
+1 -1
View File
@@ -5726,7 +5726,7 @@ out:
static void amdgpu_ras_critical_region_init(struct amdgpu_device *adev)
{
amdgpu_ras_add_critical_region(adev, adev->mman.fw_reserved_memory);
amdgpu_ras_add_critical_region(adev, adev->mman.resv_region[AMDGPU_RESV_FW].bo);
}
static void amdgpu_ras_critical_region_fini(struct amdgpu_device *adev)
drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c
+1 -1
View File
@@ -116,7 +116,7 @@ static int amdgpu_reset_xgmi_reset_on_init_perform_reset(
/* Mode1 reset needs to be triggered on all devices together */
list_for_each_entry(tmp_adev, reset_device_list, reset_list) {
/* For XGMI run all resets in parallel to speed up the process */
if (!queue_work(system_unbound_wq, &tmp_adev->xgmi_reset_work))
if (!queue_work(system_dfl_wq, &tmp_adev->xgmi_reset_work))
r = -EALREADY;
if (r) {
dev_err(tmp_adev->dev,
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
+7 -4
View File
@@ -559,15 +559,18 @@ void amdgpu_debugfs_ring_init(struct amdgpu_device *adev,
int amdgpu_ring_init_mqd(struct amdgpu_ring *ring);
static inline u32 amdgpu_ib_get_value(struct amdgpu_ib *ib, int idx)
static inline u32 amdgpu_ib_get_value(struct amdgpu_ib *ib, uint32_t idx)
{
return ib->ptr[idx];
if (idx < ib->length_dw)
return ib->ptr[idx];
return 0;
}
static inline void amdgpu_ib_set_value(struct amdgpu_ib *ib, int idx,
static inline void amdgpu_ib_set_value(struct amdgpu_ib *ib, uint32_t idx,
uint32_t value)
{
ib->ptr[idx] = value;
if (idx < ib->length_dw)
ib->ptr[idx] = value;
}
int amdgpu_ib_get(struct amdgpu_device *adev, struct amdgpu_vm *vm,
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+157 -207
View File
@@ -1671,87 +1671,160 @@ static struct ttm_device_funcs amdgpu_bo_driver = {
.access_memory = &amdgpu_ttm_access_memory,
};
/*
* Firmware Reservation functions
*/
/**
* amdgpu_ttm_fw_reserve_vram_fini - free fw reserved vram
*
* @adev: amdgpu_device pointer
*
* free fw reserved vram if it has been reserved.
*/
static void amdgpu_ttm_fw_reserve_vram_fini(struct amdgpu_device *adev)
void amdgpu_ttm_init_vram_resv(struct amdgpu_device *adev,
enum amdgpu_resv_region_id id,
uint64_t offset, uint64_t size,
bool needs_cpu_map)
{
amdgpu_bo_free_kernel(&adev->mman.fw_vram_usage_reserved_bo,
NULL, &adev->mman.fw_vram_usage_va);
struct amdgpu_vram_resv *resv;
if (id >= AMDGPU_RESV_MAX)
return;
resv = &adev->mman.resv_region[id];
resv->offset = offset;
resv->size = size;
resv->needs_cpu_map = needs_cpu_map;
}
/*
* Driver Reservation functions
*/
/**
* amdgpu_ttm_drv_reserve_vram_fini - free drv reserved vram
*
* @adev: amdgpu_device pointer
*
* free drv reserved vram if it has been reserved.
*/
static void amdgpu_ttm_drv_reserve_vram_fini(struct amdgpu_device *adev)
static void amdgpu_ttm_init_fw_resv_region(struct amdgpu_device *adev)
{
amdgpu_bo_free_kernel(&adev->mman.drv_vram_usage_reserved_bo,
NULL,
&adev->mman.drv_vram_usage_va);
uint32_t reserve_size = 0;
if (!adev->discovery.reserve_tmr)
return;
/*
* Query reserved tmr size through atom firmwareinfo for Sienna_Cichlid and onwards for all
* the use cases (IP discovery/G6 memory training/profiling/diagnostic data.etc)
*
* Otherwise, fallback to legacy approach to check and reserve tmr block for ip
* discovery data and G6 memory training data respectively
*/
if (adev->bios)
reserve_size =
amdgpu_atomfirmware_get_fw_reserved_fb_size(adev);
if (!adev->bios &&
(amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(9, 4, 3) ||
amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(9, 4, 4) ||
amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(9, 5, 0)))
reserve_size = max(reserve_size, (uint32_t)280 << 20);
else if (!adev->bios &&
amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(12, 1, 0)) {
if (hweight32(adev->aid_mask) == 1)
reserve_size = max(reserve_size, (uint32_t)128 << 20);
else
reserve_size = max(reserve_size, (uint32_t)144 << 20);
} else if (!reserve_size)
reserve_size = DISCOVERY_TMR_OFFSET;
amdgpu_ttm_init_vram_resv(adev, AMDGPU_RESV_FW,
adev->gmc.real_vram_size - reserve_size,
reserve_size, false);
}
/**
* amdgpu_ttm_fw_reserve_vram_init - create bo vram reservation from fw
*
* @adev: amdgpu_device pointer
*
* create bo vram reservation from fw.
*/
static int amdgpu_ttm_fw_reserve_vram_init(struct amdgpu_device *adev)
static void amdgpu_ttm_init_mem_train_resv_region(struct amdgpu_device *adev)
{
uint64_t reserve_size;
uint64_t offset;
if (!adev->discovery.reserve_tmr)
return;
if (!adev->bios || amdgpu_sriov_vf(adev))
return;
if (!amdgpu_atomfirmware_mem_training_supported(adev))
return;
reserve_size = adev->mman.resv_region[AMDGPU_RESV_FW].size;
offset = ALIGN((adev->gmc.mc_vram_size - reserve_size - SZ_1M), SZ_1M);
amdgpu_ttm_init_vram_resv(adev, AMDGPU_RESV_MEM_TRAIN,
offset,
GDDR6_MEM_TRAINING_DATA_SIZE_IN_BYTES,
false);
}
static void amdgpu_ttm_init_vram_resv_regions(struct amdgpu_device *adev)
{
uint64_t vram_size = adev->gmc.visible_vram_size;
adev->mman.fw_vram_usage_va = NULL;
adev->mman.fw_vram_usage_reserved_bo = NULL;
/* Initialize memory reservations as required for VGA.
* This is used for VGA emulation and pre-OS scanout buffers to
* avoid display artifacts while transitioning between pre-OS
* and driver.
*/
amdgpu_gmc_init_vga_resv_regions(adev);
amdgpu_ttm_init_fw_resv_region(adev);
amdgpu_ttm_init_mem_train_resv_region(adev);
if (adev->mman.fw_vram_usage_size == 0 ||
adev->mman.fw_vram_usage_size > vram_size)
return 0;
if (adev->mman.resv_region[AMDGPU_RESV_FW_VRAM_USAGE].size > vram_size)
adev->mman.resv_region[AMDGPU_RESV_FW_VRAM_USAGE].size = 0;
return amdgpu_bo_create_kernel_at(adev,
adev->mman.fw_vram_usage_start_offset,
adev->mman.fw_vram_usage_size,
&adev->mman.fw_vram_usage_reserved_bo,
&adev->mman.fw_vram_usage_va);
if (adev->mman.resv_region[AMDGPU_RESV_DRV_VRAM_USAGE].size > vram_size)
adev->mman.resv_region[AMDGPU_RESV_DRV_VRAM_USAGE].size = 0;
}
/**
* amdgpu_ttm_drv_reserve_vram_init - create bo vram reservation from driver
*
* @adev: amdgpu_device pointer
*
* create bo vram reservation from drv.
*/
static int amdgpu_ttm_drv_reserve_vram_init(struct amdgpu_device *adev)
int amdgpu_ttm_mark_vram_reserved(struct amdgpu_device *adev,
enum amdgpu_resv_region_id id)
{
u64 vram_size = adev->gmc.visible_vram_size;
struct amdgpu_vram_resv *resv;
int ret;
adev->mman.drv_vram_usage_va = NULL;
adev->mman.drv_vram_usage_reserved_bo = NULL;
if (id >= AMDGPU_RESV_MAX)
return -EINVAL;
if (adev->mman.drv_vram_usage_size == 0 ||
adev->mman.drv_vram_usage_size > vram_size)
resv = &adev->mman.resv_region[id];
if (!resv->size)
return 0;
return amdgpu_bo_create_kernel_at(adev,
adev->mman.drv_vram_usage_start_offset,
adev->mman.drv_vram_usage_size,
&adev->mman.drv_vram_usage_reserved_bo,
&adev->mman.drv_vram_usage_va);
ret = amdgpu_bo_create_kernel_at(adev, resv->offset, resv->size,
&resv->bo,
resv->needs_cpu_map ? &resv->cpu_ptr : NULL);
if (ret) {
dev_err(adev->dev,
"reserve vram failed: id=%d offset=0x%llx size=0x%llx ret=%d\n",
id, resv->offset, resv->size, ret);
memset(resv, 0, sizeof(*resv));
}
return ret;
}
void amdgpu_ttm_unmark_vram_reserved(struct amdgpu_device *adev,
enum amdgpu_resv_region_id id)
{
struct amdgpu_vram_resv *resv;
if (id >= AMDGPU_RESV_MAX)
return;
resv = &adev->mman.resv_region[id];
if (!resv->bo)
return;
amdgpu_bo_free_kernel(&resv->bo, NULL,
resv->needs_cpu_map ? &resv->cpu_ptr : NULL);
memset(resv, 0, sizeof(*resv));
}
/*
* Reserve all regions with non-zero size. Regions whose info is not
* yet available (e.g., fw extended region) may still be reserved
* during runtime.
*/
static int amdgpu_ttm_alloc_vram_resv_regions(struct amdgpu_device *adev)
{
int i, r;
for (i = 0; i < AMDGPU_RESV_MAX; i++) {
r = amdgpu_ttm_mark_vram_reserved(adev, i);
if (r)
return r;
}
return 0;
}
/*
@@ -1770,25 +1843,23 @@ static int amdgpu_ttm_training_reserve_vram_fini(struct amdgpu_device *adev)
struct psp_memory_training_context *ctx = &adev->psp.mem_train_ctx;
ctx->init = PSP_MEM_TRAIN_NOT_SUPPORT;
amdgpu_bo_free_kernel(&ctx->c2p_bo, NULL, NULL);
ctx->c2p_bo = NULL;
amdgpu_ttm_unmark_vram_reserved(adev, AMDGPU_RESV_MEM_TRAIN);
return 0;
}
static void amdgpu_ttm_training_data_block_init(struct amdgpu_device *adev,
uint32_t reserve_size)
static void amdgpu_ttm_training_data_block_init(struct amdgpu_device *adev)
{
struct psp_memory_training_context *ctx = &adev->psp.mem_train_ctx;
struct amdgpu_vram_resv *resv =
&adev->mman.resv_region[AMDGPU_RESV_MEM_TRAIN];
memset(ctx, 0, sizeof(*ctx));
ctx->c2p_train_data_offset =
ALIGN((adev->gmc.mc_vram_size - reserve_size - SZ_1M), SZ_1M);
ctx->c2p_train_data_offset = resv->offset;
ctx->p2c_train_data_offset =
(adev->gmc.mc_vram_size - GDDR6_MEM_TRAINING_OFFSET);
ctx->train_data_size =
GDDR6_MEM_TRAINING_DATA_SIZE_IN_BYTES;
ctx->train_data_size = resv->size;
DRM_DEBUG("train_data_size:%llx,p2c_train_data_offset:%llx,c2p_train_data_offset:%llx.\n",
ctx->train_data_size,
@@ -1796,78 +1867,6 @@ static void amdgpu_ttm_training_data_block_init(struct amdgpu_device *adev,
ctx->c2p_train_data_offset);
}
/*
* reserve TMR memory at the top of VRAM which holds
* IP Discovery data and is protected by PSP.
*/
static int amdgpu_ttm_reserve_tmr(struct amdgpu_device *adev)
{
struct psp_memory_training_context *ctx = &adev->psp.mem_train_ctx;
bool mem_train_support = false;
uint32_t reserve_size = 0;
int ret;
if (adev->bios && !amdgpu_sriov_vf(adev)) {
if (amdgpu_atomfirmware_mem_training_supported(adev))
mem_train_support = true;
else
DRM_DEBUG("memory training does not support!\n");
}
/*
* Query reserved tmr size through atom firmwareinfo for Sienna_Cichlid and onwards for all
* the use cases (IP discovery/G6 memory training/profiling/diagnostic data.etc)
*
* Otherwise, fallback to legacy approach to check and reserve tmr block for ip
* discovery data and G6 memory training data respectively
*/
if (adev->bios)
reserve_size =
amdgpu_atomfirmware_get_fw_reserved_fb_size(adev);
if (!adev->bios &&
(amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(9, 4, 3) ||
amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(9, 4, 4) ||
amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(9, 5, 0)))
reserve_size = max(reserve_size, (uint32_t)280 << 20);
else if (!adev->bios &&
amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(12, 1, 0)) {
if (hweight32(adev->aid_mask) == 1)
reserve_size = max(reserve_size, (uint32_t)128 << 20);
else
reserve_size = max(reserve_size, (uint32_t)144 << 20);
} else if (!reserve_size)
reserve_size = DISCOVERY_TMR_OFFSET;
if (mem_train_support) {
/* reserve vram for mem train according to TMR location */
amdgpu_ttm_training_data_block_init(adev, reserve_size);
ret = amdgpu_bo_create_kernel_at(adev,
ctx->c2p_train_data_offset,
ctx->train_data_size,
&ctx->c2p_bo,
NULL);
if (ret) {
dev_err(adev->dev, "alloc c2p_bo failed(%d)!\n", ret);
amdgpu_ttm_training_reserve_vram_fini(adev);
return ret;
}
ctx->init = PSP_MEM_TRAIN_RESERVE_SUCCESS;
}
ret = amdgpu_bo_create_kernel_at(
adev, adev->gmc.real_vram_size - reserve_size, reserve_size,
&adev->mman.fw_reserved_memory, NULL);
if (ret) {
dev_err(adev->dev, "alloc tmr failed(%d)!\n", ret);
amdgpu_bo_free_kernel(&adev->mman.fw_reserved_memory, NULL,
NULL);
return ret;
}
return 0;
}
static int amdgpu_ttm_pools_init(struct amdgpu_device *adev)
{
int i;
@@ -2115,63 +2114,18 @@ int amdgpu_ttm_init(struct amdgpu_device *adev)
adev->gmc.visible_vram_size);
#endif
/*
*The reserved vram for firmware must be pinned to the specified
*place on the VRAM, so reserve it early.
*/
r = amdgpu_ttm_fw_reserve_vram_init(adev);
amdgpu_ttm_init_vram_resv_regions(adev);
r = amdgpu_ttm_alloc_vram_resv_regions(adev);
if (r)
return r;
/*
* The reserved VRAM for the driver must be pinned to a specific
* location in VRAM, so reserve it early.
*/
r = amdgpu_ttm_drv_reserve_vram_init(adev);
if (r)
return r;
if (adev->mman.resv_region[AMDGPU_RESV_MEM_TRAIN].size) {
struct psp_memory_training_context *ctx =
&adev->psp.mem_train_ctx;
/*
* only NAVI10 and later ASICs support IP discovery.
* If IP discovery is enabled, a block of memory should be
* reserved for it.
*/
if (adev->discovery.reserve_tmr) {
r = amdgpu_ttm_reserve_tmr(adev);
if (r)
return r;
}
/* allocate memory as required for VGA
* This is used for VGA emulation and pre-OS scanout buffers to
* avoid display artifacts while transitioning between pre-OS
* and driver.
*/
if (!adev->gmc.is_app_apu) {
r = amdgpu_bo_create_kernel_at(adev, 0,
adev->mman.stolen_vga_size,
&adev->mman.stolen_vga_memory,
NULL);
if (r)
return r;
r = amdgpu_bo_create_kernel_at(adev, adev->mman.stolen_vga_size,
adev->mman.stolen_extended_size,
&adev->mman.stolen_extended_memory,
NULL);
if (r)
return r;
r = amdgpu_bo_create_kernel_at(adev,
adev->mman.stolen_reserved_offset,
adev->mman.stolen_reserved_size,
&adev->mman.stolen_reserved_memory,
NULL);
if (r)
return r;
} else {
DRM_DEBUG_DRIVER("Skipped stolen memory reservation\n");
amdgpu_ttm_training_data_block_init(adev);
ctx->init = PSP_MEM_TRAIN_RESERVE_SUCCESS;
}
dev_info(adev->dev, " %uM of VRAM memory ready\n",
@@ -2284,23 +2238,19 @@ void amdgpu_ttm_fini(struct amdgpu_device *adev)
amdgpu_ttm_training_reserve_vram_fini(adev);
/* return the stolen vga memory back to VRAM */
if (!adev->gmc.is_app_apu) {
amdgpu_bo_free_kernel(&adev->mman.stolen_vga_memory, NULL, NULL);
amdgpu_bo_free_kernel(&adev->mman.stolen_extended_memory, NULL, NULL);
amdgpu_ttm_unmark_vram_reserved(adev, AMDGPU_RESV_STOLEN_VGA);
amdgpu_ttm_unmark_vram_reserved(adev, AMDGPU_RESV_STOLEN_EXTENDED);
/* return the FW reserved memory back to VRAM */
amdgpu_bo_free_kernel(&adev->mman.fw_reserved_memory, NULL,
NULL);
amdgpu_bo_free_kernel(&adev->mman.fw_reserved_memory_extend, NULL,
NULL);
if (adev->mman.stolen_reserved_size)
amdgpu_bo_free_kernel(&adev->mman.stolen_reserved_memory,
NULL, NULL);
amdgpu_ttm_unmark_vram_reserved(adev, AMDGPU_RESV_FW);
amdgpu_ttm_unmark_vram_reserved(adev, AMDGPU_RESV_FW_EXTEND);
amdgpu_ttm_unmark_vram_reserved(adev, AMDGPU_RESV_STOLEN_RESERVED);
}
amdgpu_bo_free_kernel(&adev->mman.sdma_access_bo, NULL,
&adev->mman.sdma_access_ptr);
amdgpu_ttm_free_mmio_remap_bo(adev);
amdgpu_ttm_fw_reserve_vram_fini(adev);
amdgpu_ttm_drv_reserve_vram_fini(adev);
amdgpu_ttm_unmark_vram_reserved(adev, AMDGPU_RESV_FW_VRAM_USAGE);
amdgpu_ttm_unmark_vram_reserved(adev, AMDGPU_RESV_DRV_VRAM_USAGE);
if (drm_dev_enter(adev_to_drm(adev), &idx)) {
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
+30 -23
View File
@@ -59,6 +59,26 @@ struct amdgpu_ttm_buffer_entity {
u64 gart_window_offs[2];
};
enum amdgpu_resv_region_id {
AMDGPU_RESV_STOLEN_VGA,
AMDGPU_RESV_STOLEN_EXTENDED,
AMDGPU_RESV_STOLEN_RESERVED,
AMDGPU_RESV_FW,
AMDGPU_RESV_FW_EXTEND,
AMDGPU_RESV_FW_VRAM_USAGE,
AMDGPU_RESV_DRV_VRAM_USAGE,
AMDGPU_RESV_MEM_TRAIN,
AMDGPU_RESV_MAX
};
struct amdgpu_vram_resv {
uint64_t offset;
uint64_t size;
struct amdgpu_bo *bo;
void *cpu_ptr;
bool needs_cpu_map;
};
struct amdgpu_mman {
struct ttm_device bdev;
struct ttm_pool *ttm_pools;
@@ -83,31 +103,9 @@ struct amdgpu_mman {
struct amdgpu_gtt_mgr gtt_mgr;
struct ttm_resource_manager preempt_mgr;
uint64_t stolen_vga_size;
struct amdgpu_bo *stolen_vga_memory;
uint64_t stolen_extended_size;
struct amdgpu_bo *stolen_extended_memory;
bool keep_stolen_vga_memory;
struct amdgpu_bo *stolen_reserved_memory;
uint64_t stolen_reserved_offset;
uint64_t stolen_reserved_size;
/* fw reserved memory */
struct amdgpu_bo *fw_reserved_memory;
struct amdgpu_bo *fw_reserved_memory_extend;
/* firmware VRAM reservation */
u64 fw_vram_usage_start_offset;
u64 fw_vram_usage_size;
struct amdgpu_bo *fw_vram_usage_reserved_bo;
void *fw_vram_usage_va;
/* driver VRAM reservation */
u64 drv_vram_usage_start_offset;
u64 drv_vram_usage_size;
struct amdgpu_bo *drv_vram_usage_reserved_bo;
void *drv_vram_usage_va;
struct amdgpu_vram_resv resv_region[AMDGPU_RESV_MAX];
/* PAGE_SIZE'd BO for process memory r/w over SDMA. */
struct amdgpu_bo *sdma_access_bo;
@@ -175,6 +173,15 @@ void amdgpu_vram_mgr_clear_reset_blocks(struct amdgpu_device *adev);
bool amdgpu_res_cpu_visible(struct amdgpu_device *adev,
struct ttm_resource *res);
void amdgpu_ttm_init_vram_resv(struct amdgpu_device *adev,
enum amdgpu_resv_region_id id,
uint64_t offset, uint64_t size,
bool needs_cpu_map);
int amdgpu_ttm_mark_vram_reserved(struct amdgpu_device *adev,
enum amdgpu_resv_region_id id);
void amdgpu_ttm_unmark_vram_reserved(struct amdgpu_device *adev,
enum amdgpu_resv_region_id id);
int amdgpu_ttm_init(struct amdgpu_device *adev);
void amdgpu_ttm_fini(struct amdgpu_device *adev);
void amdgpu_ttm_set_buffer_funcs_status(struct amdgpu_device *adev,
drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
+24 -57
View File
@@ -427,23 +427,14 @@ static int amdgpu_userq_map_helper(struct amdgpu_usermode_queue *queue)
return r;
}
static int amdgpu_userq_wait_for_last_fence(struct amdgpu_usermode_queue *queue)
static void amdgpu_userq_wait_for_last_fence(struct amdgpu_usermode_queue *queue)
{
struct amdgpu_userq_mgr *uq_mgr = queue->userq_mgr;
struct dma_fence *f = queue->last_fence;
int ret = 0;
if (f && !dma_fence_is_signaled(f)) {
ret = dma_fence_wait_timeout(f, true, MAX_SCHEDULE_TIMEOUT);
if (ret <= 0) {
drm_file_err(uq_mgr->file, "Timed out waiting for fence=%llu:%llu\n",
f->context, f->seqno);
queue->state = AMDGPU_USERQ_STATE_HUNG;
return -ETIME;
}
}
if (!f)
return;
return ret;
dma_fence_wait(f, false);
}
static void amdgpu_userq_cleanup(struct amdgpu_usermode_queue *queue)
@@ -458,9 +449,10 @@ static void amdgpu_userq_cleanup(struct amdgpu_usermode_queue *queue)
/* Drop the userq reference. */
amdgpu_userq_buffer_vas_list_cleanup(adev, queue);
uq_funcs->mqd_destroy(queue);
amdgpu_userq_fence_driver_free(queue);
/* Use interrupt-safe locking since IRQ handlers may access these XArrays */
xa_erase_irq(&adev->userq_doorbell_xa, queue->doorbell_index);
amdgpu_userq_fence_driver_free(queue);
queue->fence_drv = NULL;
queue->userq_mgr = NULL;
list_del(&queue->userq_va_list);
kfree(queue);
@@ -799,7 +791,7 @@ amdgpu_userq_create(struct drm_file *filp, union drm_amdgpu_userq *args)
queue->doorbell_index = index;
xa_init_flags(&queue->fence_drv_xa, XA_FLAGS_ALLOC);
r = amdgpu_userq_fence_driver_alloc(adev, queue);
r = amdgpu_userq_fence_driver_alloc(adev, &queue->fence_drv);
if (r) {
drm_file_err(uq_mgr->file, "Failed to alloc fence driver\n");
goto free_queue;
@@ -1023,7 +1015,8 @@ amdgpu_userq_restore_all(struct amdgpu_userq_mgr *uq_mgr)
mutex_unlock(&uq_mgr->userq_mutex);
if (ret)
drm_file_err(uq_mgr->file, "Failed to map all the queues\n");
drm_file_err(uq_mgr->file,
"Failed to map all the queues, restore failed ret=%d\n", ret);
return ret;
}
@@ -1230,13 +1223,11 @@ static void amdgpu_userq_restore_worker(struct work_struct *work)
ret = amdgpu_userq_vm_validate(uq_mgr);
if (ret) {
drm_file_err(uq_mgr->file, "Failed to validate BOs to restore\n");
drm_file_err(uq_mgr->file, "Failed to validate BOs to restore ret=%d\n", ret);
goto put_fence;
}
ret = amdgpu_userq_restore_all(uq_mgr);
if (ret)
drm_file_err(uq_mgr->file, "Failed to restore all queues\n");
amdgpu_userq_restore_all(uq_mgr);
put_fence:
dma_fence_put(ev_fence);
@@ -1258,7 +1249,8 @@ amdgpu_userq_evict_all(struct amdgpu_userq_mgr *uq_mgr)
}
if (ret)
drm_file_err(uq_mgr->file, "Couldn't unmap all the queues\n");
drm_file_err(uq_mgr->file,
"Couldn't unmap all the queues, eviction failed ret=%d\n", ret);
return ret;
}
@@ -1279,46 +1271,28 @@ void amdgpu_userq_reset_work(struct work_struct *work)
amdgpu_device_gpu_recover(adev, NULL, &reset_context);
}
static int
static void
amdgpu_userq_wait_for_signal(struct amdgpu_userq_mgr *uq_mgr)
{
struct amdgpu_usermode_queue *queue;
unsigned long queue_id;
int ret;
xa_for_each(&uq_mgr->userq_xa, queue_id, queue) {
struct dma_fence *f = queue->last_fence;
if (!f || dma_fence_is_signaled(f))
if (!f)
continue;
ret = dma_fence_wait_timeout(f, true, msecs_to_jiffies(100));
if (ret <= 0) {
drm_file_err(uq_mgr->file, "Timed out waiting for fence=%llu:%llu\n",
f->context, f->seqno);
return -ETIMEDOUT;
}
dma_fence_wait(f, false);
}
return 0;
}
void
amdgpu_userq_evict(struct amdgpu_userq_mgr *uq_mgr)
{
struct amdgpu_device *adev = uq_mgr->adev;
int ret;
/* Wait for any pending userqueue fence work to finish */
ret = amdgpu_userq_wait_for_signal(uq_mgr);
if (ret)
dev_err(adev->dev, "Not evicting userqueue, timeout waiting for work\n");
ret = amdgpu_userq_evict_all(uq_mgr);
if (ret)
dev_err(adev->dev, "Failed to evict userqueue\n");
amdgpu_userq_wait_for_signal(uq_mgr);
amdgpu_userq_evict_all(uq_mgr);
}
int amdgpu_userq_mgr_init(struct amdgpu_userq_mgr *userq_mgr, struct drm_file *file_priv,
@@ -1480,17 +1454,16 @@ int amdgpu_userq_start_sched_for_enforce_isolation(struct amdgpu_device *adev,
return ret;
}
int amdgpu_userq_gem_va_unmap_validate(struct amdgpu_device *adev,
struct amdgpu_bo_va_mapping *mapping,
uint64_t saddr)
void amdgpu_userq_gem_va_unmap_validate(struct amdgpu_device *adev,
struct amdgpu_bo_va_mapping *mapping,
uint64_t saddr)
{
u32 ip_mask = amdgpu_userq_get_supported_ip_mask(adev);
struct amdgpu_bo_va *bo_va = mapping->bo_va;
struct dma_resv *resv = bo_va->base.bo->tbo.base.resv;
int ret = 0;
if (!ip_mask)
return 0;
return;
dev_warn_once(adev->dev, "now unmapping a vital queue va:%llx\n", saddr);
/**
@@ -1501,14 +1474,8 @@ int amdgpu_userq_gem_va_unmap_validate(struct amdgpu_device *adev,
* unmap is only for one kind of userq VAs, so at this point suppose
* the eviction fence is always unsignaled.
*/
if (!dma_resv_test_signaled(resv, DMA_RESV_USAGE_BOOKKEEP)) {
ret = dma_resv_wait_timeout(resv, DMA_RESV_USAGE_BOOKKEEP, true,
MAX_SCHEDULE_TIMEOUT);
if (ret <= 0)
return -EBUSY;
}
return 0;
dma_resv_wait_timeout(resv, DMA_RESV_USAGE_BOOKKEEP,
false, MAX_SCHEDULE_TIMEOUT);
}
void amdgpu_userq_pre_reset(struct amdgpu_device *adev)
drivers/gpu/drm/amd/amdgpu/amdgpu_userq.h
+3 -3
View File
@@ -160,7 +160,7 @@ void amdgpu_userq_start_hang_detect_work(struct amdgpu_usermode_queue *queue);
int amdgpu_userq_input_va_validate(struct amdgpu_device *adev,
struct amdgpu_usermode_queue *queue,
u64 addr, u64 expected_size);
int amdgpu_userq_gem_va_unmap_validate(struct amdgpu_device *adev,
struct amdgpu_bo_va_mapping *mapping,
uint64_t saddr);
void amdgpu_userq_gem_va_unmap_validate(struct amdgpu_device *adev,
struct amdgpu_bo_va_mapping *mapping,
uint64_t saddr);
#endif
drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
+9 -23
View File
@@ -78,12 +78,15 @@ amdgpu_userq_fence_write(struct amdgpu_userq_fence_driver *fence_drv,
}
int amdgpu_userq_fence_driver_alloc(struct amdgpu_device *adev,
struct amdgpu_usermode_queue *userq)
struct amdgpu_userq_fence_driver **fence_drv_req)
{
struct amdgpu_userq_fence_driver *fence_drv;
unsigned long flags;
int r;
if (!fence_drv_req)
return -EINVAL;
*fence_drv_req = NULL;
fence_drv = kzalloc_obj(*fence_drv);
if (!fence_drv)
return -ENOMEM;
@@ -104,19 +107,10 @@ int amdgpu_userq_fence_driver_alloc(struct amdgpu_device *adev,
fence_drv->context = dma_fence_context_alloc(1);
get_task_comm(fence_drv->timeline_name, current);
xa_lock_irqsave(&adev->userq_xa, flags);
r = xa_err(__xa_store(&adev->userq_xa, userq->doorbell_index,
fence_drv, GFP_KERNEL));
xa_unlock_irqrestore(&adev->userq_xa, flags);
if (r)
goto free_seq64;
userq->fence_drv = fence_drv;
*fence_drv_req = fence_drv;
return 0;
free_seq64:
amdgpu_seq64_free(adev, fence_drv->va);
free_fence_drv:
kfree(fence_drv);
@@ -144,10 +138,10 @@ void
amdgpu_userq_fence_driver_free(struct amdgpu_usermode_queue *userq)
{
dma_fence_put(userq->last_fence);
userq->last_fence = NULL;
amdgpu_userq_walk_and_drop_fence_drv(&userq->fence_drv_xa);
xa_destroy(&userq->fence_drv_xa);
/* Drop the fence_drv reference held by user queue */
/* Drop the queue's ownership reference to fence_drv explicitly */
amdgpu_userq_fence_driver_put(userq->fence_drv);
}
@@ -187,11 +181,9 @@ void amdgpu_userq_fence_driver_destroy(struct kref *ref)
struct amdgpu_userq_fence_driver *fence_drv = container_of(ref,
struct amdgpu_userq_fence_driver,
refcount);
struct amdgpu_userq_fence_driver *xa_fence_drv;
struct amdgpu_device *adev = fence_drv->adev;
struct amdgpu_userq_fence *fence, *tmp;
struct xarray *xa = &adev->userq_xa;
unsigned long index, flags;
unsigned long flags;
struct dma_fence *f;
spin_lock_irqsave(&fence_drv->fence_list_lock, flags);
@@ -208,12 +200,6 @@ void amdgpu_userq_fence_driver_destroy(struct kref *ref)
}
spin_unlock_irqrestore(&fence_drv->fence_list_lock, flags);
xa_lock_irqsave(xa, flags);
xa_for_each(xa, index, xa_fence_drv)
if (xa_fence_drv == fence_drv)
__xa_erase(xa, index);
xa_unlock_irqrestore(xa, flags);
/* Free seq64 memory */
amdgpu_seq64_free(adev, fence_drv->va);
kfree(fence_drv);
drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.h
+1 -1
View File
@@ -64,7 +64,7 @@ void amdgpu_userq_fence_slab_fini(void);
void amdgpu_userq_fence_driver_get(struct amdgpu_userq_fence_driver *fence_drv);
void amdgpu_userq_fence_driver_put(struct amdgpu_userq_fence_driver *fence_drv);
int amdgpu_userq_fence_driver_alloc(struct amdgpu_device *adev,
struct amdgpu_usermode_queue *userq);
struct amdgpu_userq_fence_driver **fence_drv_req);
void amdgpu_userq_fence_driver_free(struct amdgpu_usermode_queue *userq);
void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_drv);
void amdgpu_userq_fence_driver_force_completion(struct amdgpu_usermode_queue *userq);
drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
+3
View File
@@ -680,6 +680,9 @@ static int amdgpu_vce_cs_reloc(struct amdgpu_cs_parser *p, struct amdgpu_ib *ib,
uint64_t addr;
int r;
if (lo >= ib->length_dw || hi >= ib->length_dw)
return -EINVAL;
if (index == 0xffffffff)
index = 0;
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
+26 -24
View File
@@ -437,12 +437,9 @@ static void amdgpu_virt_add_bad_page(struct amdgpu_device *adev,
struct eeprom_table_record bp;
uint64_t retired_page;
uint32_t bp_idx, bp_cnt;
void *vram_usage_va = NULL;
if (adev->mman.fw_vram_usage_va)
vram_usage_va = adev->mman.fw_vram_usage_va;
else
vram_usage_va = adev->mman.drv_vram_usage_va;
void *fw_va = adev->mman.resv_region[AMDGPU_RESV_FW_VRAM_USAGE].cpu_ptr;
void *drv_va = adev->mman.resv_region[AMDGPU_RESV_DRV_VRAM_USAGE].cpu_ptr;
void *vram_usage_va = fw_va ? fw_va : drv_va;
memset(&bp, 0, sizeof(bp));
@@ -710,15 +707,17 @@ void amdgpu_virt_fini_data_exchange(struct amdgpu_device *adev)
void amdgpu_virt_init_data_exchange(struct amdgpu_device *adev)
{
uint32_t *pfvf_data = NULL;
void *fw_va = adev->mman.resv_region[AMDGPU_RESV_FW_VRAM_USAGE].cpu_ptr;
void *drv_va = adev->mman.resv_region[AMDGPU_RESV_DRV_VRAM_USAGE].cpu_ptr;
adev->virt.fw_reserve.p_pf2vf = NULL;
adev->virt.fw_reserve.p_vf2pf = NULL;
adev->virt.vf2pf_update_interval_ms = 0;
adev->virt.vf2pf_update_retry_cnt = 0;
if (adev->mman.fw_vram_usage_va && adev->mman.drv_vram_usage_va) {
if (fw_va && drv_va) {
dev_warn(adev->dev, "Currently fw_vram and drv_vram should not have values at the same time!");
} else if (adev->mman.fw_vram_usage_va || adev->mman.drv_vram_usage_va) {
} else if (fw_va || drv_va) {
/* go through this logic in ip_init and reset to init workqueue*/
amdgpu_virt_exchange_data(adev);
@@ -763,41 +762,43 @@ void amdgpu_virt_exchange_data(struct amdgpu_device *adev)
uint64_t bp_block_offset = 0;
uint32_t bp_block_size = 0;
struct amd_sriov_msg_pf2vf_info *pf2vf_v2 = NULL;
void *fw_va = adev->mman.resv_region[AMDGPU_RESV_FW_VRAM_USAGE].cpu_ptr;
void *drv_va = adev->mman.resv_region[AMDGPU_RESV_DRV_VRAM_USAGE].cpu_ptr;
if (adev->mman.fw_vram_usage_va || adev->mman.drv_vram_usage_va) {
if (adev->mman.fw_vram_usage_va) {
if (fw_va || drv_va) {
if (fw_va) {
if (adev->virt.req_init_data_ver == GPU_CRIT_REGION_V2) {
adev->virt.fw_reserve.p_pf2vf =
(struct amd_sriov_msg_pf2vf_info_header *)
(adev->mman.fw_vram_usage_va +
(fw_va +
adev->virt.crit_regn_tbl[AMD_SRIOV_MSG_DATAEXCHANGE_TABLE_ID].offset);
adev->virt.fw_reserve.p_vf2pf =
(struct amd_sriov_msg_vf2pf_info_header *)
(adev->mman.fw_vram_usage_va +
(fw_va +
adev->virt.crit_regn_tbl[AMD_SRIOV_MSG_DATAEXCHANGE_TABLE_ID].offset +
(AMD_SRIOV_MSG_SIZE_KB << 10));
adev->virt.fw_reserve.ras_telemetry =
(adev->mman.fw_vram_usage_va +
(fw_va +
adev->virt.crit_regn_tbl[AMD_SRIOV_MSG_RAS_TELEMETRY_TABLE_ID].offset);
} else {
adev->virt.fw_reserve.p_pf2vf =
(struct amd_sriov_msg_pf2vf_info_header *)
(adev->mman.fw_vram_usage_va + (AMD_SRIOV_MSG_PF2VF_OFFSET_KB_V1 << 10));
(fw_va + (AMD_SRIOV_MSG_PF2VF_OFFSET_KB_V1 << 10));
adev->virt.fw_reserve.p_vf2pf =
(struct amd_sriov_msg_vf2pf_info_header *)
(adev->mman.fw_vram_usage_va + (AMD_SRIOV_MSG_VF2PF_OFFSET_KB_V1 << 10));
(fw_va + (AMD_SRIOV_MSG_VF2PF_OFFSET_KB_V1 << 10));
adev->virt.fw_reserve.ras_telemetry =
(adev->mman.fw_vram_usage_va + (AMD_SRIOV_MSG_RAS_TELEMETRY_OFFSET_KB_V1 << 10));
(fw_va + (AMD_SRIOV_MSG_RAS_TELEMETRY_OFFSET_KB_V1 << 10));
}
} else if (adev->mman.drv_vram_usage_va) {
} else if (drv_va) {
adev->virt.fw_reserve.p_pf2vf =
(struct amd_sriov_msg_pf2vf_info_header *)
(adev->mman.drv_vram_usage_va + (AMD_SRIOV_MSG_PF2VF_OFFSET_KB_V1 << 10));
(drv_va + (AMD_SRIOV_MSG_PF2VF_OFFSET_KB_V1 << 10));
adev->virt.fw_reserve.p_vf2pf =
(struct amd_sriov_msg_vf2pf_info_header *)
(adev->mman.drv_vram_usage_va + (AMD_SRIOV_MSG_VF2PF_OFFSET_KB_V1 << 10));
(drv_va + (AMD_SRIOV_MSG_VF2PF_OFFSET_KB_V1 << 10));
adev->virt.fw_reserve.ras_telemetry =
(adev->mman.drv_vram_usage_va + (AMD_SRIOV_MSG_RAS_TELEMETRY_OFFSET_KB_V1 << 10));
(drv_va + (AMD_SRIOV_MSG_RAS_TELEMETRY_OFFSET_KB_V1 << 10));
}
amdgpu_virt_read_pf2vf_data(adev);
@@ -1081,13 +1082,14 @@ int amdgpu_virt_init_critical_region(struct amdgpu_device *adev)
}
/* reserved memory starts from crit region base offset with the size of 5MB */
adev->mman.fw_vram_usage_start_offset = adev->virt.crit_regn.offset;
adev->mman.fw_vram_usage_size = adev->virt.crit_regn.size_kb << 10;
amdgpu_ttm_init_vram_resv(adev, AMDGPU_RESV_FW_VRAM_USAGE,
adev->virt.crit_regn.offset,
adev->virt.crit_regn.size_kb << 10, true);
dev_info(adev->dev,
"critical region v%d requested to reserve memory start at %08llx with %llu KB.\n",
init_data_hdr->version,
adev->mman.fw_vram_usage_start_offset,
adev->mman.fw_vram_usage_size >> 10);
adev->mman.resv_region[AMDGPU_RESV_FW_VRAM_USAGE].offset,
adev->mman.resv_region[AMDGPU_RESV_FW_VRAM_USAGE].size >> 10);
adev->virt.is_dynamic_crit_regn_enabled = true;
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+53 -36
View File
@@ -1978,7 +1978,6 @@ int amdgpu_vm_bo_unmap(struct amdgpu_device *adev,
struct amdgpu_bo_va_mapping *mapping;
struct amdgpu_vm *vm = bo_va->base.vm;
bool valid = true;
int r;
saddr /= AMDGPU_GPU_PAGE_SIZE;
@@ -2003,12 +2002,8 @@ int amdgpu_vm_bo_unmap(struct amdgpu_device *adev,
* during user requests GEM unmap IOCTL except for forcing the unmap
* from user space.
*/
if (unlikely(atomic_read(&bo_va->userq_va_mapped) > 0)) {
r = amdgpu_userq_gem_va_unmap_validate(adev, mapping, saddr);
if (unlikely(r == -EBUSY))
dev_warn_once(adev->dev,
"Attempt to unmap an active userq buffer\n");
}
if (unlikely(atomic_read(&bo_va->userq_va_mapped) > 0))
amdgpu_userq_gem_va_unmap_validate(adev, mapping, saddr);
list_del(&mapping->list);
amdgpu_vm_it_remove(mapping, &vm->va);
@@ -2954,6 +2949,50 @@ int amdgpu_vm_ioctl(struct drm_device *dev, void *data, struct drm_file *filp)
return 0;
}
/**
* amdgpu_vm_lock_by_pasid - return an amdgpu_vm and its root bo from a pasid, if possible.
* @adev: amdgpu device pointer
* @root: root BO of the VM
* @pasid: PASID of the VM
* The caller needs to unreserve and unref the root bo on success.
*/
struct amdgpu_vm *amdgpu_vm_lock_by_pasid(struct amdgpu_device *adev,
struct amdgpu_bo **root, u32 pasid)
{
unsigned long irqflags;
struct amdgpu_vm *vm;
int r;
xa_lock_irqsave(&adev->vm_manager.pasids, irqflags);
vm = xa_load(&adev->vm_manager.pasids, pasid);
*root = vm ? amdgpu_bo_ref(vm->root.bo) : NULL;
xa_unlock_irqrestore(&adev->vm_manager.pasids, irqflags);
if (!*root)
return NULL;
r = amdgpu_bo_reserve(*root, true);
if (r)
goto error_unref;
/* Double check that the VM still exists */
xa_lock_irqsave(&adev->vm_manager.pasids, irqflags);
vm = xa_load(&adev->vm_manager.pasids, pasid);
if (vm && vm->root.bo != *root)
vm = NULL;
xa_unlock_irqrestore(&adev->vm_manager.pasids, irqflags);
if (!vm)
goto error_unlock;
return vm;
error_unlock:
amdgpu_bo_unreserve(*root);
error_unref:
amdgpu_bo_unref(root);
return NULL;
}
/**
* amdgpu_vm_handle_fault - graceful handling of VM faults.
* @adev: amdgpu device pointer
@@ -2969,50 +3008,29 @@ int amdgpu_vm_ioctl(struct drm_device *dev, void *data, struct drm_file *filp)
* shouldn't be reported any more.
*/
bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,
u32 vmid, u32 node_id, uint64_t addr, uint64_t ts,
bool write_fault)
u32 vmid, u32 node_id, uint64_t addr,
uint64_t ts, bool write_fault)
{
bool is_compute_context = false;
struct amdgpu_bo *root;
unsigned long irqflags;
uint64_t value, flags;
struct amdgpu_vm *vm;
int r;
xa_lock_irqsave(&adev->vm_manager.pasids, irqflags);
vm = xa_load(&adev->vm_manager.pasids, pasid);
if (vm) {
root = amdgpu_bo_ref(vm->root.bo);
is_compute_context = vm->is_compute_context;
} else {
root = NULL;
}
xa_unlock_irqrestore(&adev->vm_manager.pasids, irqflags);
if (!root)
vm = amdgpu_vm_lock_by_pasid(adev, &root, pasid);
if (!vm)
return false;
is_compute_context = vm->is_compute_context;
if (is_compute_context && !svm_range_restore_pages(adev, pasid, vmid,
node_id, addr >> PAGE_SHIFT, ts, write_fault)) {
amdgpu_bo_unreserve(root);
amdgpu_bo_unref(&root);
return true;
}
addr /= AMDGPU_GPU_PAGE_SIZE;
r = amdgpu_bo_reserve(root, true);
if (r)
goto error_unref;
/* Double check that the VM still exists */
xa_lock_irqsave(&adev->vm_manager.pasids, irqflags);
vm = xa_load(&adev->vm_manager.pasids, pasid);
if (vm && vm->root.bo != root)
vm = NULL;
xa_unlock_irqrestore(&adev->vm_manager.pasids, irqflags);
if (!vm)
goto error_unlock;
flags = AMDGPU_PTE_VALID | AMDGPU_PTE_SNOOPED |
AMDGPU_PTE_SYSTEM;
@@ -3051,7 +3069,6 @@ error_unlock:
if (r < 0)
dev_err(adev->dev, "Can't handle page fault (%d)\n", r);
error_unref:
amdgpu_bo_unref(&root);
return false;
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
+3
View File
@@ -592,6 +592,9 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,
u32 vmid, u32 node_id, uint64_t addr, uint64_t ts,
bool write_fault);
struct amdgpu_vm *amdgpu_vm_lock_by_pasid(struct amdgpu_device *adev,
struct amdgpu_bo **root, u32 pasid);
void amdgpu_vm_set_task_info(struct amdgpu_vm *vm);
void amdgpu_vm_move_to_lru_tail(struct amdgpu_device *adev,
drivers/gpu/drm/amd/amdgpu/amdgpu_xcp.c
+1
View File
@@ -181,6 +181,7 @@ int amdgpu_xcp_init(struct amdgpu_xcp_mgr *xcp_mgr, int num_xcps, int mode)
}
xcp_mgr->num_xcps = num_xcps;
xcp_mgr->mem_alloc_mode = AMDGPU_PARTITION_MEM_CAPPING_EVEN;
amdgpu_xcp_update_partition_sched_list(adev);
return 0;
drivers/gpu/drm/amd/amdgpu/amdgpu_xcp.h
+2
View File
@@ -132,6 +132,8 @@ struct amdgpu_xcp_mgr {
struct amdgpu_xcp_cfg *xcp_cfg;
uint32_t supp_xcp_modes;
uint32_t avail_xcp_modes;
/* used to determin KFD memory alloc mode for each partition */
uint32_t mem_alloc_mode;
};
struct amdgpu_xcp_mgr_funcs {
drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
+5 -5
View File
@@ -6502,14 +6502,14 @@ static int gfx_v11_0_eop_irq(struct amdgpu_device *adev,
DRM_DEBUG("IH: CP EOP\n");
if (adev->enable_mes && doorbell_offset) {
struct amdgpu_userq_fence_driver *fence_drv = NULL;
struct xarray *xa = &adev->userq_xa;
struct amdgpu_usermode_queue *queue;
struct xarray *xa = &adev->userq_doorbell_xa;
unsigned long flags;
xa_lock_irqsave(xa, flags);
fence_drv = xa_load(xa, doorbell_offset);
if (fence_drv)
amdgpu_userq_fence_driver_process(fence_drv);
queue = xa_load(xa, doorbell_offset);
if (queue)
amdgpu_userq_fence_driver_process(queue->fence_drv);
xa_unlock_irqrestore(xa, flags);
} else {
me_id = (entry->ring_id & 0x0c) >> 2;
drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
+5 -5
View File
@@ -4854,14 +4854,14 @@ static int gfx_v12_0_eop_irq(struct amdgpu_device *adev,
DRM_DEBUG("IH: CP EOP\n");
if (adev->enable_mes && doorbell_offset) {
struct amdgpu_userq_fence_driver *fence_drv = NULL;
struct xarray *xa = &adev->userq_xa;
struct xarray *xa = &adev->userq_doorbell_xa;
struct amdgpu_usermode_queue *queue;
unsigned long flags;
xa_lock_irqsave(xa, flags);
fence_drv = xa_load(xa, doorbell_offset);
if (fence_drv)
amdgpu_userq_fence_driver_process(fence_drv);
queue = xa_load(xa, doorbell_offset);
if (queue)
amdgpu_userq_fence_driver_process(queue->fence_drv);
xa_unlock_irqrestore(xa, flags);
} else {
me_id = (entry->ring_id & 0x0c) >> 2;
drivers/gpu/drm/amd/amdgpu/gfx_v12_1.c
+6 -5
View File
@@ -3643,14 +3643,15 @@ static int gfx_v12_1_eop_irq(struct amdgpu_device *adev,
DRM_DEBUG("IH: CP EOP\n");
if (adev->enable_mes && doorbell_offset) {
struct amdgpu_userq_fence_driver *fence_drv = NULL;
struct xarray *xa = &adev->userq_xa;
struct xarray *xa = &adev->userq_doorbell_xa;
struct amdgpu_usermode_queue *queue;
unsigned long flags;
xa_lock_irqsave(xa, flags);
fence_drv = xa_load(xa, doorbell_offset);
if (fence_drv)
amdgpu_userq_fence_driver_process(fence_drv);
queue = xa_load(xa, doorbell_offset);
if (queue)
amdgpu_userq_fence_driver_process(queue->fence_drv);
xa_unlock_irqrestore(xa, flags);
} else {
me_id = (entry->ring_id & 0x0c) >> 2;
drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
-2
View File
@@ -860,8 +860,6 @@ static int gmc_v10_0_sw_init(struct amdgpu_ip_block *ip_block)
if (r)
return r;
amdgpu_gmc_get_vbios_allocations(adev);
/* Memory manager */
r = amdgpu_bo_init(adev);
if (r)
drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c
-2
View File
@@ -834,8 +834,6 @@ static int gmc_v11_0_sw_init(struct amdgpu_ip_block *ip_block)
if (r)
return r;
amdgpu_gmc_get_vbios_allocations(adev);
/* Memory manager */
r = amdgpu_bo_init(adev);
if (r)
drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
-2
View File
@@ -924,8 +924,6 @@ static int gmc_v12_0_sw_init(struct amdgpu_ip_block *ip_block)
if (r)
return r;
amdgpu_gmc_get_vbios_allocations(adev);
#ifdef HAVE_ACPI_DEV_GET_FIRST_MATCH_DEV
if (amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(12, 1, 0)) {
r = amdgpu_gmc_init_mem_ranges(adev);
drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
-2
View File
@@ -854,8 +854,6 @@ static int gmc_v6_0_sw_init(struct amdgpu_ip_block *ip_block)
if (r)
return r;
amdgpu_gmc_get_vbios_allocations(adev);
r = amdgpu_bo_init(adev);
if (r)
return r;
drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
-2
View File
@@ -1034,8 +1034,6 @@ static int gmc_v7_0_sw_init(struct amdgpu_ip_block *ip_block)
if (r)
return r;
amdgpu_gmc_get_vbios_allocations(adev);
/* Memory manager */
r = amdgpu_bo_init(adev);
if (r)
drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
-2
View File
@@ -1149,8 +1149,6 @@ static int gmc_v8_0_sw_init(struct amdgpu_ip_block *ip_block)
if (r)
return r;
amdgpu_gmc_get_vbios_allocations(adev);
/* Memory manager */
r = amdgpu_bo_init(adev);
if (r)
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
-2
View File
@@ -2010,8 +2010,6 @@ static int gmc_v9_0_sw_init(struct amdgpu_ip_block *ip_block)
if (r)
return r;
amdgpu_gmc_get_vbios_allocations(adev);
if (amdgpu_is_multi_aid(adev)) {
r = amdgpu_gmc_init_mem_ranges(adev);
if (r)
drivers/gpu/drm/amd/amdgpu/sdma_v6_0.c
+5 -5
View File
@@ -1662,16 +1662,16 @@ static int sdma_v6_0_process_fence_irq(struct amdgpu_device *adev,
u32 doorbell_offset = entry->src_data[0];
if (adev->enable_mes && doorbell_offset) {
struct amdgpu_userq_fence_driver *fence_drv = NULL;
struct xarray *xa = &adev->userq_xa;
struct amdgpu_usermode_queue *queue;
struct xarray *xa = &adev->userq_doorbell_xa;
unsigned long flags;
doorbell_offset >>= SDMA0_QUEUE0_DOORBELL_OFFSET__OFFSET__SHIFT;
xa_lock_irqsave(xa, flags);
fence_drv = xa_load(xa, doorbell_offset);
if (fence_drv)
amdgpu_userq_fence_driver_process(fence_drv);
queue = xa_load(xa, doorbell_offset);
if (queue)
amdgpu_userq_fence_driver_process(queue->fence_drv);
xa_unlock_irqrestore(xa, flags);
}
drivers/gpu/drm/amd/amdgpu/sdma_v7_0.c
+5 -5
View File
@@ -1594,16 +1594,16 @@ static int sdma_v7_0_process_fence_irq(struct amdgpu_device *adev,
u32 doorbell_offset = entry->src_data[0];
if (adev->enable_mes && doorbell_offset) {
struct amdgpu_userq_fence_driver *fence_drv = NULL;
struct xarray *xa = &adev->userq_xa;
struct xarray *xa = &adev->userq_doorbell_xa;
struct amdgpu_usermode_queue *queue;
unsigned long flags;
doorbell_offset >>= SDMA0_QUEUE0_DOORBELL_OFFSET__OFFSET__SHIFT;
xa_lock_irqsave(xa, flags);
fence_drv = xa_load(xa, doorbell_offset);
if (fence_drv)
amdgpu_userq_fence_driver_process(fence_drv);
queue = xa_load(xa, doorbell_offset);
if (queue)
amdgpu_userq_fence_driver_process(queue->fence_drv);
xa_unlock_irqrestore(xa, flags);
}
drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c
+19 -4
View File
@@ -1909,7 +1909,7 @@ static int vcn_v3_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job,
struct ttm_operation_ctx ctx = { false, false };
struct amdgpu_device *adev = p->adev;
struct amdgpu_bo_va_mapping *map;
uint32_t *msg, num_buffers;
uint32_t *msg, num_buffers, len_dw;
struct amdgpu_bo *bo;
uint64_t start, end;
unsigned int i;
@@ -1930,6 +1930,11 @@ static int vcn_v3_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job,
return -EINVAL;
}
if (end - addr < 16) {
DRM_ERROR("VCN messages must be at least 4 DWORDs!\n");
return -EINVAL;
}
bo->flags |= AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
amdgpu_bo_placement_from_domain(bo, bo->allowed_domains);
r = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx);
@@ -1946,8 +1951,8 @@ static int vcn_v3_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job,
msg = ptr + addr - start;
/* Check length */
if (msg[1] > end - addr) {
DRM_ERROR("VCN message header does not fit in BO!\n");
r = -EINVAL;
goto out;
}
@@ -1955,7 +1960,16 @@ static int vcn_v3_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job,
if (msg[3] != RDECODE_MSG_CREATE)
goto out;
len_dw = msg[1] / 4;
num_buffers = msg[2];
/* Verify that all indices fit within the claimed length. Each index is 4 DWORDs */
if (num_buffers > len_dw || 6 + num_buffers * 4 > len_dw) {
DRM_ERROR("VCN message has too many buffers!\n");
r = -EINVAL;
goto out;
}
for (i = 0, msg = &msg[6]; i < num_buffers; ++i, msg += 4) {
uint32_t offset, size, *create;
@@ -1965,14 +1979,15 @@ static int vcn_v3_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job,
offset = msg[1];
size = msg[2];
if (offset + size > end) {
if (size < 4 || offset + size > end - addr) {
DRM_ERROR("VCN message buffer exceeds BO bounds!\n");
r = -EINVAL;
goto out;
}
create = ptr + addr + offset - start;
/* H246, HEVC and VP9 can run on any instance */
/* H264, HEVC and VP9 can run on any instance */
if (create[0] == 0x7 || create[0] == 0x10 || create[0] == 0x11)
continue;
drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c
+30 -14
View File
@@ -1826,7 +1826,7 @@ static int vcn_v4_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job,
struct ttm_operation_ctx ctx = { false, false };
struct amdgpu_device *adev = p->adev;
struct amdgpu_bo_va_mapping *map;
uint32_t *msg, num_buffers;
uint32_t *msg, num_buffers, len_dw;
struct amdgpu_bo *bo;
uint64_t start, end;
unsigned int i;
@@ -1847,6 +1847,11 @@ static int vcn_v4_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job,
return -EINVAL;
}
if (end - addr < 16) {
DRM_ERROR("VCN messages must be at least 4 DWORDs!\n");
return -EINVAL;
}
bo->flags |= AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
amdgpu_bo_placement_from_domain(bo, bo->allowed_domains);
r = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx);
@@ -1863,8 +1868,8 @@ static int vcn_v4_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job,
msg = ptr + addr - start;
/* Check length */
if (msg[1] > end - addr) {
DRM_ERROR("VCN message header does not fit in BO!\n");
r = -EINVAL;
goto out;
}
@@ -1872,7 +1877,16 @@ static int vcn_v4_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job,
if (msg[3] != RDECODE_MSG_CREATE)
goto out;
len_dw = msg[1] / 4;
num_buffers = msg[2];
/* Verify that all indices fit within the claimed length. Each index is 4 DWORDs */
if (num_buffers > len_dw || 6 + num_buffers * 4 > len_dw) {
DRM_ERROR("VCN message has too many buffers!\n");
r = -EINVAL;
goto out;
}
for (i = 0, msg = &msg[6]; i < num_buffers; ++i, msg += 4) {
uint32_t offset, size, *create;
@@ -1882,7 +1896,8 @@ static int vcn_v4_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job,
offset = msg[1];
size = msg[2];
if (offset + size > end) {
if (size < 4 || offset + size > end - addr) {
DRM_ERROR("VCN message buffer exceeds BO bounds!\n");
r = -EINVAL;
goto out;
}
@@ -1913,9 +1928,10 @@ out:
static int vcn_v4_0_enc_find_ib_param(struct amdgpu_ib *ib, uint32_t id, int start)
{
int i;
uint32_t len;
for (i = start; i < ib->length_dw && ib->ptr[i] >= 8; i += ib->ptr[i] / 4) {
if (ib->ptr[i + 1] == id)
for (i = start; (len = amdgpu_ib_get_value(ib, i)) >= 8; i += len / 4) {
if (amdgpu_ib_get_value(ib, i + 1) == id)
return i;
}
return -1;
@@ -1926,8 +1942,6 @@ static int vcn_v4_0_ring_patch_cs_in_place(struct amdgpu_cs_parser *p,
struct amdgpu_ib *ib)
{
struct amdgpu_ring *ring = amdgpu_job_ring(job);
struct amdgpu_vcn_decode_buffer *decode_buffer;
uint64_t addr;
uint32_t val;
int idx = 0, sidx;
@@ -1938,20 +1952,22 @@ static int vcn_v4_0_ring_patch_cs_in_place(struct amdgpu_cs_parser *p,
while ((idx = vcn_v4_0_enc_find_ib_param(ib, RADEON_VCN_ENGINE_INFO, idx)) >= 0) {
val = amdgpu_ib_get_value(ib, idx + 2); /* RADEON_VCN_ENGINE_TYPE */
if (val == RADEON_VCN_ENGINE_TYPE_DECODE) {
decode_buffer = (struct amdgpu_vcn_decode_buffer *)&ib->ptr[idx + 6];
uint32_t valid_buf_flag = amdgpu_ib_get_value(ib, idx + 6);
uint64_t msg_buffer_addr;
if (!(decode_buffer->valid_buf_flag & 0x1))
if (!(valid_buf_flag & 0x1))
return 0;
addr = ((u64)decode_buffer->msg_buffer_address_hi) << 32 |
decode_buffer->msg_buffer_address_lo;
return vcn_v4_0_dec_msg(p, job, addr);
msg_buffer_addr = ((u64)amdgpu_ib_get_value(ib, idx + 7)) << 32 |
amdgpu_ib_get_value(ib, idx + 8);
return vcn_v4_0_dec_msg(p, job, msg_buffer_addr);
} else if (val == RADEON_VCN_ENGINE_TYPE_ENCODE) {
sidx = vcn_v4_0_enc_find_ib_param(ib, RENCODE_IB_PARAM_SESSION_INIT, idx);
if (sidx >= 0 && ib->ptr[sidx + 2] == RENCODE_ENCODE_STANDARD_AV1)
if (sidx >= 0 &&
amdgpu_ib_get_value(ib, sidx + 2) == RENCODE_ENCODE_STANDARD_AV1)
return vcn_v4_0_limit_sched(p, job);
}
idx += ib->ptr[idx] / 4;
idx += amdgpu_ib_get_value(ib, idx) / 4;
}
return 0;
}
drivers/gpu/drm/amd/amdkfd/kfd_process.c
+2 -1
View File
@@ -689,7 +689,8 @@ void kfd_procfs_del_queue(struct queue *q)
int kfd_process_create_wq(void)
{
if (!kfd_process_wq)
kfd_process_wq = alloc_workqueue("kfd_process_wq", 0, 0);
kfd_process_wq = alloc_workqueue("kfd_process_wq", WQ_UNBOUND,
0);
if (!kfd_restore_wq)
kfd_restore_wq = alloc_ordered_workqueue("kfd_restore_wq",
WQ_FREEZABLE);
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+72 -56
View File
@@ -572,7 +572,7 @@ static void schedule_dc_vmin_vmax(struct amdgpu_device *adev,
offload_work->stream = stream;
offload_work->adjust = adjust_copy;
queue_work(system_wq, &offload_work->work);
queue_work(system_percpu_wq, &offload_work->work);
}
static void dm_vupdate_high_irq(void *interrupt_params)
@@ -3833,6 +3833,66 @@ static struct drm_mode_config_helper_funcs amdgpu_dm_mode_config_helperfuncs = {
.atomic_commit_setup = amdgpu_dm_atomic_setup_commit,
};
#define DDC_MANUFACTURERNAME_SAMSUNG 0x2D4C
static void dm_set_panel_type(struct amdgpu_dm_connector *aconnector)
{
struct drm_connector *connector = &aconnector->base;
struct drm_display_info *display_info = &connector->display_info;
struct dc_link *link = aconnector->dc_link;
struct amdgpu_device *adev;
adev = drm_to_adev(connector->dev);
link->panel_type = PANEL_TYPE_NONE;
switch (display_info->amd_vsdb.panel_type) {
case AMD_VSDB_PANEL_TYPE_OLED:
link->panel_type = PANEL_TYPE_OLED;
break;
case AMD_VSDB_PANEL_TYPE_MINILED:
link->panel_type = PANEL_TYPE_MINILED;
break;
}
/* If VSDB didn't determine panel type, check DPCD ext caps */
if (link->panel_type == PANEL_TYPE_NONE) {
if (link->dpcd_sink_ext_caps.bits.miniled == 1)
link->panel_type = PANEL_TYPE_MINILED;
if (link->dpcd_sink_ext_caps.bits.oled == 1)
link->panel_type = PANEL_TYPE_OLED;
}
/*
* TODO: get panel type from DID2 that has device technology field
* to specify if it's OLED or not. But we need to wait for DID2
* support in DC and EDID parser to be able to use it here.
*/
if (link->panel_type == PANEL_TYPE_NONE) {
struct drm_amd_vsdb_info *vsdb = &display_info->amd_vsdb;
u32 lum1_max = vsdb->luminance_range1.max_luminance;
u32 lum2_max = vsdb->luminance_range2.max_luminance;
if (vsdb->version && link->local_sink &&
link->local_sink->edid_caps.manufacturer_id ==
DDC_MANUFACTURERNAME_SAMSUNG &&
lum1_max >= ((lum2_max * 3) / 2))
link->panel_type = PANEL_TYPE_MINILED;
}
if (link->panel_type == PANEL_TYPE_OLED)
drm_object_property_set_value(&connector->base,
adev_to_drm(adev)->mode_config.panel_type_property,
DRM_MODE_PANEL_TYPE_OLED);
else
drm_object_property_set_value(&connector->base,
adev_to_drm(adev)->mode_config.panel_type_property,
DRM_MODE_PANEL_TYPE_UNKNOWN);
drm_dbg_kms(aconnector->base.dev, "Panel type: %d\n", link->panel_type);
}
static void update_connector_ext_caps(struct amdgpu_dm_connector *aconnector)
{
const struct drm_panel_backlight_quirk *panel_backlight_quirk;
@@ -3854,10 +3914,6 @@ static void update_connector_ext_caps(struct amdgpu_dm_connector *aconnector)
caps->ext_caps = &aconnector->dc_link->dpcd_sink_ext_caps;
caps->aux_support = false;
drm_object_property_set_value(&conn_base->base,
adev_to_drm(adev)->mode_config.panel_type_property,
caps->ext_caps->bits.oled ? DRM_MODE_PANEL_TYPE_OLED : DRM_MODE_PANEL_TYPE_UNKNOWN);
if (caps->ext_caps->bits.oled == 1
/*
* ||
@@ -4031,6 +4087,7 @@ void amdgpu_dm_update_connector_after_detect(
amdgpu_dm_update_freesync_caps(connector, aconnector->drm_edid);
update_connector_ext_caps(aconnector);
dm_set_panel_type(aconnector);
} else {
hdmi_cec_unset_edid(aconnector);
drm_dp_cec_unset_edid(&aconnector->dm_dp_aux.aux);
@@ -4211,7 +4268,7 @@ static void handle_hpd_irq_helper(struct amdgpu_dm_connector *aconnector)
dc_sink_retain(aconnector->hdmi_prev_sink);
/* Schedule delayed detection. */
if (mod_delayed_work(system_wq,
if (mod_delayed_work(system_percpu_wq,
&aconnector->hdmi_hpd_debounce_work,
msecs_to_jiffies(aconnector->hdmi_hpd_debounce_delay_ms)))
drm_dbg_kms(dev, "HDMI HPD: Re-scheduled debounce work\n");
@@ -11142,8 +11199,8 @@ static void amdgpu_dm_atomic_commit_tail(struct drm_atomic_state *state)
if (!adev->in_suspend) {
/* return the stolen vga memory back to VRAM */
if (!adev->mman.keep_stolen_vga_memory)
amdgpu_bo_free_kernel(&adev->mman.stolen_vga_memory, NULL, NULL);
amdgpu_bo_free_kernel(&adev->mman.stolen_extended_memory, NULL, NULL);
amdgpu_ttm_unmark_vram_reserved(adev, AMDGPU_RESV_STOLEN_VGA);
amdgpu_ttm_unmark_vram_reserved(adev, AMDGPU_RESV_STOLEN_EXTENDED);
}
/*
@@ -13155,56 +13212,15 @@ static void parse_edid_displayid_vrr(struct drm_connector *connector,
}
}
static int parse_amd_vsdb(struct amdgpu_dm_connector *aconnector,
const struct edid *edid, struct amdgpu_hdmi_vsdb_info *vsdb_info)
static int get_amd_vsdb(struct amdgpu_dm_connector *aconnector,
struct amdgpu_hdmi_vsdb_info *vsdb_info)
{
u8 *edid_ext = NULL;
int i;
int j = 0;
int total_ext_block_len;
struct drm_connector *connector = &aconnector->base;
if (edid == NULL || edid->extensions == 0)
return -ENODEV;
vsdb_info->replay_mode = connector->display_info.amd_vsdb.replay_mode;
vsdb_info->amd_vsdb_version = connector->display_info.amd_vsdb.version;
/* Find DisplayID extension */
for (i = 0; i < edid->extensions; i++) {
edid_ext = (void *)(edid + (i + 1));
if (edid_ext[0] == DISPLAYID_EXT)
break;
}
total_ext_block_len = EDID_LENGTH * edid->extensions;
while (j < total_ext_block_len - sizeof(struct amd_vsdb_block)) {
struct amd_vsdb_block *amd_vsdb = (struct amd_vsdb_block *)&edid_ext[j];
unsigned int ieeeId = (amd_vsdb->ieee_id[2] << 16) | (amd_vsdb->ieee_id[1] << 8) | (amd_vsdb->ieee_id[0]);
if (ieeeId == HDMI_AMD_VENDOR_SPECIFIC_DATA_BLOCK_IEEE_REGISTRATION_ID &&
amd_vsdb->version == HDMI_AMD_VENDOR_SPECIFIC_DATA_BLOCK_VERSION_3) {
u8 panel_type;
vsdb_info->replay_mode = (amd_vsdb->feature_caps & AMD_VSDB_VERSION_3_FEATURECAP_REPLAYMODE) ? true : false;
vsdb_info->amd_vsdb_version = HDMI_AMD_VENDOR_SPECIFIC_DATA_BLOCK_VERSION_3;
drm_dbg_kms(aconnector->base.dev, "Panel supports Replay Mode: %d\n", vsdb_info->replay_mode);
panel_type = (amd_vsdb->color_space_eotf_support & AMD_VDSB_VERSION_3_PANEL_TYPE_MASK) >> AMD_VDSB_VERSION_3_PANEL_TYPE_SHIFT;
switch (panel_type) {
case AMD_VSDB_PANEL_TYPE_OLED:
aconnector->dc_link->panel_type = PANEL_TYPE_OLED;
break;
case AMD_VSDB_PANEL_TYPE_MINILED:
aconnector->dc_link->panel_type = PANEL_TYPE_MINILED;
break;
default:
aconnector->dc_link->panel_type = PANEL_TYPE_NONE;
break;
}
drm_dbg_kms(aconnector->base.dev, "Panel type: %d\n",
aconnector->dc_link->panel_type);
return true;
}
j++;
}
return false;
return connector->display_info.amd_vsdb.version != 0;
}
static int parse_hdmi_amd_vsdb(struct amdgpu_dm_connector *aconnector,
@@ -13307,7 +13323,7 @@ void amdgpu_dm_update_freesync_caps(struct drm_connector *connector,
freesync_capable = true;
}
parse_amd_vsdb(amdgpu_dm_connector, edid, &vsdb_info);
get_amd_vsdb(amdgpu_dm_connector, &vsdb_info);
if (vsdb_info.replay_mode) {
amdgpu_dm_connector->vsdb_info.replay_mode = vsdb_info.replay_mode;
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h
-14
View File
@@ -53,12 +53,6 @@
#define AMDGPU_DMUB_NOTIFICATION_MAX 8
#define HDMI_AMD_VENDOR_SPECIFIC_DATA_BLOCK_IEEE_REGISTRATION_ID 0x00001A
#define AMD_VSDB_VERSION_3_FEATURECAP_REPLAYMODE 0x40
#define AMD_VDSB_VERSION_3_PANEL_TYPE_MASK 0xC0
#define AMD_VDSB_VERSION_3_PANEL_TYPE_SHIFT 6
#define HDMI_AMD_VENDOR_SPECIFIC_DATA_BLOCK_VERSION_3 0x3
enum amd_vsdb_panel_type {
AMD_VSDB_PANEL_TYPE_DEFAULT = 0,
AMD_VSDB_PANEL_TYPE_MINILED,
@@ -97,14 +91,6 @@ struct dc_plane_state;
struct dmub_notification;
struct dmub_cmd_fused_request;
struct amd_vsdb_block {
unsigned char ieee_id[3];
unsigned char version;
unsigned char feature_caps;
unsigned char reserved[3];
unsigned char color_space_eotf_support;
};
struct common_irq_params {
struct amdgpu_device *adev;
enum dc_irq_source irq_src;
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c
+10 -11
View File
@@ -101,23 +101,22 @@ bool amdgpu_dm_crtc_vrr_active(const struct dm_crtc_state *dm_state)
/**
* amdgpu_dm_crtc_set_panel_sr_feature() - Manage panel self-refresh features.
*
* @vblank_work: is a pointer to a struct vblank_control_work object.
* @vblank_enabled: indicates whether the DRM vblank counter is currently
* enabled (true) or disabled (false).
* @allow_sr_entry: represents whether entry into the self-refresh mode is
* allowed (true) or not allowed (false).
* @dm: amdgpu display manager instance.
* @acrtc: CRTC whose panel self-refresh state is being updated.
* @stream: DC stream associated with @acrtc.
* @vblank_enabled: Whether the DRM vblank counter is currently enabled.
* @allow_sr_entry: Whether entry into self-refresh mode is allowed.
*
* The DRM vblank counter enable/disable action is used as the trigger to enable
* or disable various panel self-refresh features:
*
* Panel Replay and PSR SU
* - Enable when:
* - VRR is disabled
* - vblank counter is disabled
* - entry is allowed: usermode demonstrates an adequate number of fast
* commits)
* - CRC capture window isn't active
* - VRR is disabled
* - vblank counter is disabled
* - entry is allowed: usermode demonstrates an adequate number of fast
* commits
* - CRC capture window isn't active
* - Keep enabled even when vblank counter gets enabled
*
* PSR1
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_ism.c
+9
View File
@@ -35,6 +35,9 @@
/**
* dm_ism_next_state - Get next state based on current state and event
* @current_state: current ISM state
* @event: event being processed
* @next_state: place to store the next state
*
* This function defines the idle state management FSM. Invalid transitions
* are ignored and will not progress the FSM.
@@ -148,6 +151,11 @@ static uint64_t dm_ism_get_sso_delay(const struct amdgpu_dm_ism *ism,
/**
* dm_ism_get_idle_allow_delay - Calculate hysteresis-based idle allow delay
* @ism: ISM instance containing configuration, history, and current state
* @stream: display stream used to derive frame timing values for delay
*
* Calculates the delay before allowing idle optimizations based on recent
* idle history and the current stream timing.
*/
static uint64_t dm_ism_get_idle_allow_delay(const struct amdgpu_dm_ism *ism,
const struct dc_stream_state *stream)
@@ -212,6 +220,7 @@ static uint64_t dm_ism_get_idle_allow_delay(const struct amdgpu_dm_ism *ism,
/**
* dm_ism_insert_record - Insert a record into the circular history buffer
* @ism: ISM instance
*/
static void dm_ism_insert_record(struct amdgpu_dm_ism *ism)
{
drivers/gpu/drm/amd/display/amdgpu_dm/dc_fpu.c
+1 -1
View File
@@ -59,7 +59,7 @@ inline void dc_assert_fp_enabled(void)
}
/**
* dc_assert_fp_enabled - Check if FPU protection is enabled
* dc_is_fp_enabled - Check if FPU protection is enabled
*
* This function tells if the code is already under FPU protection or not. A
* function that works as an API for a set of FPU operations can use this
drivers/gpu/drm/amd/display/dc/core/dc_hw_sequencer.c
+3 -2
View File
@@ -23,6 +23,8 @@
*
*/
#include <linux/array_size.h>
#include "dm_services.h"
#include "core_types.h"
#include "timing_generator.h"
@@ -40,7 +42,6 @@
#include "dcn10/dcn10_hubbub.h"
#include "dce/dmub_hw_lock_mgr.h"
#define NUM_ELEMENTS(a) (sizeof(a) / sizeof((a)[0]))
#define MAX_NUM_MCACHE 8
/* used as index in array of black_color_format */
@@ -230,7 +231,7 @@ const uint16_t *find_color_matrix(enum dc_color_space color_space,
int i;
enum dc_color_space_type type;
const uint16_t *val = NULL;
int arr_size = NUM_ELEMENTS(output_csc_matrix);
int arr_size = ARRAY_SIZE(output_csc_matrix);
type = get_color_space_type(color_space);
for (i = 0; i < arr_size; i++)
drivers/gpu/drm/amd/display/dc/core/dc_resource.c
+1 -1
View File
@@ -5062,7 +5062,7 @@ void resource_build_bit_depth_reduction_params(struct dc_stream_state *stream,
option = DITHER_OPTION_SPATIAL8;
break;
case COLOR_DEPTH_101010:
option = DITHER_OPTION_TRUN10;
option = DITHER_OPTION_SPATIAL10;
break;
default:
option = DITHER_OPTION_DISABLE;
drivers/gpu/drm/amd/display/dc/dce/dce_clock_source.c
+3 -3
View File
@@ -23,6 +23,8 @@
*
*/
#include <linux/array_size.h>
#include "dm_services.h"
@@ -57,8 +59,6 @@
#define CALC_PLL_CLK_SRC_ERR_TOLERANCE 1
#define MAX_PLL_CALC_ERROR 0xFFFFFFFF
#define NUM_ELEMENTS(a) (sizeof(a) / sizeof((a)[0]))
static const struct spread_spectrum_data *get_ss_data_entry(
struct dce110_clk_src *clk_src,
enum signal_type signal,
@@ -1271,7 +1271,7 @@ const struct pixel_rate_range_table_entry *look_up_in_video_optimized_rate_tlb(
{
int i;
for (i = 0; i < NUM_ELEMENTS(video_optimized_pixel_rates); i++) {
for (i = 0; i < ARRAY_SIZE(video_optimized_pixel_rates); i++) {
const struct pixel_rate_range_table_entry *e = &video_optimized_pixel_rates[i];
if (e->range_min_khz <= pixel_rate_khz && pixel_rate_khz <= e->range_max_khz) {
drivers/gpu/drm/amd/display/dc/dpp/dcn10/dcn10_dpp_cm.c
-3
View File
@@ -49,9 +49,6 @@
#define FN(reg_name, field_name) \
dpp->tf_shift->field_name, dpp->tf_mask->field_name
#define NUM_ELEMENTS(a) (sizeof(a) / sizeof((a)[0]))
enum dcn10_coef_filter_type_sel {
SCL_COEF_LUMA_VERT_FILTER = 0,
SCL_COEF_LUMA_HORZ_FILTER = 1,
drivers/gpu/drm/amd/display/dc/dpp/dcn401/dcn401_dpp_cm.c
-3
View File
@@ -49,9 +49,6 @@
#define FN(reg_name, field_name) \
dpp->tf_shift->field_name, dpp->tf_mask->field_name
#define NUM_ELEMENTS(a) (sizeof(a) / sizeof((a)[0]))
enum dcn401_coef_filter_type_sel {
SCL_COEF_LUMA_VERT_FILTER = 0,
SCL_COEF_LUMA_HORZ_FILTER = 1,
drivers/gpu/drm/amd/display/dc/mpc/dcn20/dcn20_mpc.c
-2
View File
@@ -43,8 +43,6 @@
#define FN(reg_name, field_name) \
mpc20->mpc_shift->field_name, mpc20->mpc_mask->field_name
#define NUM_ELEMENTS(a) (sizeof(a) / sizeof((a)[0]))
void mpc2_update_blending(
struct mpc *mpc,
struct mpcc_blnd_cfg *blnd_cfg,
drivers/gpu/drm/amd/display/dc/mpc/dcn30/dcn30_mpc.c
-4
View File
@@ -40,10 +40,6 @@
#define FN(reg_name, field_name) \
mpc30->mpc_shift->field_name, mpc30->mpc_mask->field_name
#define NUM_ELEMENTS(a) (sizeof(a) / sizeof((a)[0]))
void mpc3_mpc_init(struct mpc *mpc)
{
struct dcn30_mpc *mpc30 = TO_DCN30_MPC(mpc);
drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
+17
View File
@@ -2164,4 +2164,21 @@ static inline void smu_feature_init(struct smu_context *smu, int feature_num)
smu_feature_list_clear_all(smu, SMU_FEATURE_LIST_ALLOWED);
}
/*
* smu_safe_u16_nn - Make u16 safe by filtering negative overflow errors
* @val: Input u16 value, may contain invalid negative overflows
*
* Convert u16 to non-negative value. Cast to s16 to detect negative values
* caused by calculation errors. Return 0 for negative errors, return
* original value if valid.
*
* Return: Valid u16 value or 0
*/
static inline u16 smu_safe_u16_nn(u16 val)
{
s16 tmp = (s16)val;
return tmp < 0 ? 0 : val;
}
#endif
drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c
+5 -5
View File
@@ -773,13 +773,13 @@ static int smu_v13_0_0_get_smu_metrics_data(struct smu_context *smu,
*value = metrics->AverageGfxclkFrequencyPreDs;
break;
case METRICS_AVERAGE_FCLK:
if (metrics->AverageUclkActivity <= SMU_13_0_0_BUSY_THRESHOLD)
if (smu_safe_u16_nn(metrics->AverageUclkActivity) <= SMU_13_0_0_BUSY_THRESHOLD)
*value = metrics->AverageFclkFrequencyPostDs;
else
*value = metrics->AverageFclkFrequencyPreDs;
break;
case METRICS_AVERAGE_UCLK:
if (metrics->AverageUclkActivity <= SMU_13_0_0_BUSY_THRESHOLD)
if (smu_safe_u16_nn(metrics->AverageUclkActivity) <= SMU_13_0_0_BUSY_THRESHOLD)
*value = metrics->AverageMemclkFrequencyPostDs;
else
*value = metrics->AverageMemclkFrequencyPreDs;
@@ -800,7 +800,7 @@ static int smu_v13_0_0_get_smu_metrics_data(struct smu_context *smu,
*value = metrics->AverageGfxActivity;
break;
case METRICS_AVERAGE_MEMACTIVITY:
*value = metrics->AverageUclkActivity;
*value = smu_safe_u16_nn(metrics->AverageUclkActivity);
break;
case METRICS_AVERAGE_VCNACTIVITY:
*value = max(metrics->Vcn0ActivityPercentage,
@@ -2085,7 +2085,7 @@ static ssize_t smu_v13_0_0_get_gpu_metrics(struct smu_context *smu,
metrics->AvgTemperature[TEMP_VR_MEM1]);
gpu_metrics->average_gfx_activity = metrics->AverageGfxActivity;
gpu_metrics->average_umc_activity = metrics->AverageUclkActivity;
gpu_metrics->average_umc_activity = smu_safe_u16_nn(metrics->AverageUclkActivity);
gpu_metrics->average_mm_activity = max(metrics->Vcn0ActivityPercentage,
metrics->Vcn1ActivityPercentage);
@@ -2102,7 +2102,7 @@ static ssize_t smu_v13_0_0_get_gpu_metrics(struct smu_context *smu,
else
gpu_metrics->average_gfxclk_frequency = metrics->AverageGfxclkFrequencyPreDs;
if (metrics->AverageUclkActivity <= SMU_13_0_0_BUSY_THRESHOLD)
if (smu_safe_u16_nn(metrics->AverageUclkActivity) <= SMU_13_0_0_BUSY_THRESHOLD)
gpu_metrics->average_uclk_frequency = metrics->AverageMemclkFrequencyPostDs;
else
gpu_metrics->average_uclk_frequency = metrics->AverageMemclkFrequencyPreDs;
drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_12_ppt.c
+7 -2
View File
@@ -479,9 +479,14 @@ static int smu_v13_0_12_get_system_metrics_table(struct smu_context *smu)
}
amdgpu_hdp_invalidate(smu->adev, NULL);
ret = smu_cmn_vram_cpy(smu, sys_table->cache.buffer,
table->cpu_addr,
smu_v13_0_12_get_system_metrics_size());
if (ret)
return ret;
smu_table_cache_update_time(sys_table, jiffies);
memcpy(sys_table->cache.buffer, table->cpu_addr,
smu_v13_0_12_get_system_metrics_size());
return 0;
}
drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_6_ppt.c
+12 -7
View File
@@ -778,7 +778,10 @@ int smu_v13_0_6_get_metrics_table(struct smu_context *smu, void *metrics_table,
}
amdgpu_hdp_invalidate(smu->adev, NULL);
memcpy(smu_table->metrics_table, table->cpu_addr, table_size);
ret = smu_cmn_vram_cpy(smu, smu_table->metrics_table,
table->cpu_addr, table_size);
if (ret)
return ret;
smu_table->metrics_time = jiffies;
}
@@ -857,9 +860,9 @@ int smu_v13_0_6_get_static_metrics_table(struct smu_context *smu)
}
amdgpu_hdp_invalidate(smu->adev, NULL);
memcpy(smu_table->metrics_table, table->cpu_addr, table_size);
return 0;
return smu_cmn_vram_cpy(smu, smu_table->metrics_table,
table->cpu_addr, table_size);
}
static void smu_v13_0_6_update_caps(struct smu_context *smu)
@@ -2404,13 +2407,15 @@ static int smu_v13_0_6_request_i2c_xfer(struct smu_context *smu,
table_size = smu_table->tables[SMU_TABLE_I2C_COMMANDS].size;
memcpy(table->cpu_addr, table_data, table_size);
ret = smu_cmn_vram_cpy(smu, table->cpu_addr, table_data, table_size);
if (ret)
return ret;
/* Flush hdp cache */
amdgpu_hdp_flush(adev, NULL);
ret = smu_cmn_send_smc_msg(smu, SMU_MSG_RequestI2cTransaction,
NULL);
return ret;
return smu_cmn_send_smc_msg(smu, SMU_MSG_RequestI2cTransaction,
NULL);
}
static int smu_v13_0_6_i2c_xfer(struct i2c_adapter *i2c_adap,
Loaded 100 of 187 files, more files were not shown because too many files have changed in this diff. Show more