tmakin/ack-tegra
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge 560b803067 ("Merge tag 'timers-core-2023-02-20' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip") into android-mainline

Browse Source 
Steps on the way to 6.3-rc1

Change-Id: Ied3a792e700181efe3e50c56904c62c9523de626
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
This commit is contained in:
Greg Kroah-Hartman
2023-03-03 13:27:17 +00:00
parent 95f33b0c4e 560b803067
commit f1af1c747b
86 changed files with 2295 additions and 822 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Documentation/admin-guide/kernel-parameters.txt
+11 -1
View File
@@ -5229,7 +5229,7 @@
rdt= [HW,X86,RDT]
Turn on/off individual RDT features. List is:
cmt, mbmtotal, mbmlocal, l3cat, l3cdp, l2cat, l2cdp,
mba.
mba, smba, bmec.
E.g. to turn on cmt and turn off mba use:
rdt=cmt,!mba
@@ -6377,6 +6377,16 @@
in situations with strict latency requirements (where
interruptions from clocksource watchdog are not
acceptable).
[x86] recalibrate: force recalibration against a HW timer
(HPET or PM timer) on systems whose TSC frequency was
obtained from HW or FW using either an MSR or CPUID(0x15).
Warn if the difference is more than 500 ppm.
[x86] watchdog: Use TSC as the watchdog clocksource with
which to check other HW timers (HPET or PM timer), but
only on systems where TSC has been deemed trustworthy.
This will be suppressed by an earlier tsc=nowatchdog and
can be overridden by a later tsc=nowatchdog. A console
message will flag any such suppression or overriding.
tsc_early_khz= [X86] Skip early TSC calibration and use the given
value instead. Useful when the early TSC frequency discovery
Documentation/devicetree/bindings/memory-controllers/xlnx,zynqmp-ocmc-1.0.yaml
+45
View File
@@ -0,0 +1,45 @@
# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
%YAML 1.2
---
$id: http://devicetree.org/schemas/memory-controllers/xlnx,zynqmp-ocmc-1.0.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: Xilinx Zynqmp OCM(On-Chip Memory) Controller
maintainers:
- Shubhrajyoti Datta <shubhrajyoti.datta@amd.com>
- Sai Krishna Potthuri <sai.krishna.potthuri@amd.com>
description: |
The OCM supports 64-bit wide ECC functionality to detect multi-bit errors
and recover from a single-bit memory fault.On a write, if all bytes are
being written, the ECC is generated and written into the ECC RAM along with
the write-data that is written into the data RAM. If one or more bytes are
not written, then the read operation results in an correctable error or
uncorrectable error.
properties:
compatible:
const: xlnx,zynqmp-ocmc-1.0
reg:
maxItems: 1
interrupts:
maxItems: 1
required:
- compatible
- reg
- interrupts
additionalProperties: false
examples:
- |
#include <dt-bindings/interrupt-controller/arm-gic.h>
memory-controller@ff960000 {
compatible = "xlnx,zynqmp-ocmc-1.0";
reg = <0xff960000 0x1000>;
interrupts = <GIC_SPI 10 IRQ_TYPE_LEVEL_HIGH>;
};
Documentation/devicetree/bindings/timer/mediatek,mtk-timer.txt
+1
View File
@@ -33,6 +33,7 @@ Required properties:
For those SoCs that use CPUX
* "mediatek,mt6795-systimer" for MT6795 compatible timers (CPUX)
* "mediatek,mt8365-systimer" for MT8365 compatible timers (CPUX)
- reg: Should contain location and length for timer register.
- clocks: Should contain system clock.
Documentation/devicetree/bindings/timer/riscv,timer.yaml
+52
View File
@@ -0,0 +1,52 @@
# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
%YAML 1.2
---
$id: http://devicetree.org/schemas/timer/riscv,timer.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: RISC-V timer
maintainers:
- Anup Patel <anup@brainfault.org>
description: |+
RISC-V platforms always have a RISC-V timer device for the supervisor-mode
based on the time CSR defined by the RISC-V privileged specification. The
timer interrupts of this device are configured using the RISC-V SBI Time
extension or the RISC-V Sstc extension.
The clock frequency of RISC-V timer device is specified via the
"timebase-frequency" DT property of "/cpus" DT node which is described
in Documentation/devicetree/bindings/riscv/cpus.yaml
properties:
compatible:
enum:
- riscv,timer
interrupts-extended:
minItems: 1
maxItems: 4096 # Should be enough?
riscv,timer-cannot-wake-cpu:
type: boolean
description:
If present, the timer interrupt cannot wake up the CPU from one or
more suspend/idle states.
additionalProperties: false
required:
- compatible
- interrupts-extended
examples:
- |
timer {
compatible = "riscv,timer";
interrupts-extended = <&cpu1intc 5>,
<&cpu2intc 5>,
<&cpu3intc 5>,
<&cpu4intc 5>;
};
...
Documentation/devicetree/bindings/timer/rockchip,rk-timer.yaml
+1
View File
@@ -17,6 +17,7 @@ properties:
- items:
- enum:
- rockchip,rv1108-timer
- rockchip,rv1126-timer
- rockchip,rk3036-timer
- rockchip,rk3128-timer
- rockchip,rk3188-timer
Documentation/devicetree/bindings/timer/sifive,clint.yaml
+8
View File
@@ -20,6 +20,10 @@ description:
property of "/cpus" DT node. The "timebase-frequency" DT property is
described in Documentation/devicetree/bindings/riscv/cpus.yaml
T-Head C906/C910 CPU cores include an implementation of CLINT too, however
their implementation lacks a memory-mapped MTIME register, thus not
compatible with SiFive ones.
properties:
compatible:
oneOf:
@@ -29,6 +33,10 @@ properties:
- starfive,jh7100-clint
- canaan,k210-clint
- const: sifive,clint0
- items:
- enum:
- allwinner,sun20i-d1-clint
- const: thead,c900-clint
- items:
- const: sifive,clint0
- const: riscv,clint0
Documentation/x86/resctrl.rst
+145 -2
View File
@@ -17,14 +17,21 @@ AMD refers to this feature as AMD Platform Quality of Service(AMD QoS).
This feature is enabled by the CONFIG_X86_CPU_RESCTRL and the x86 /proc/cpuinfo
flag bits:
============================================= ================================
=============================================== ================================
RDT (Resource Director Technology) Allocation "rdt_a"
CAT (Cache Allocation Technology) "cat_l3", "cat_l2"
CDP (Code and Data Prioritization) "cdp_l3", "cdp_l2"
CQM (Cache QoS Monitoring) "cqm_llc", "cqm_occup_llc"
MBM (Memory Bandwidth Monitoring) "cqm_mbm_total", "cqm_mbm_local"
MBA (Memory Bandwidth Allocation) "mba"
============================================= ================================
SMBA (Slow Memory Bandwidth Allocation) ""
BMEC (Bandwidth Monitoring Event Configuration) ""
=============================================== ================================
Historically, new features were made visible by default in /proc/cpuinfo. This
resulted in the feature flags becoming hard to parse by humans. Adding a new
flag to /proc/cpuinfo should be avoided if user space can obtain information
about the feature from resctrl's info directory.
To use the feature mount the file system::
@@ -161,6 +168,83 @@ with the following files:
"mon_features":
Lists the monitoring events if
monitoring is enabled for the resource.
Example::
# cat /sys/fs/resctrl/info/L3_MON/mon_features
llc_occupancy
mbm_total_bytes
mbm_local_bytes
If the system supports Bandwidth Monitoring Event
Configuration (BMEC), then the bandwidth events will
be configurable. The output will be::
# cat /sys/fs/resctrl/info/L3_MON/mon_features
llc_occupancy
mbm_total_bytes
mbm_total_bytes_config
mbm_local_bytes
mbm_local_bytes_config
"mbm_total_bytes_config", "mbm_local_bytes_config":
Read/write files containing the configuration for the mbm_total_bytes
and mbm_local_bytes events, respectively, when the Bandwidth
Monitoring Event Configuration (BMEC) feature is supported.
The event configuration settings are domain specific and affect
all the CPUs in the domain. When either event configuration is
changed, the bandwidth counters for all RMIDs of both events
(mbm_total_bytes as well as mbm_local_bytes) are cleared for that
domain. The next read for every RMID will report "Unavailable"
and subsequent reads will report the valid value.
Following are the types of events supported:
==== ========================================================
Bits Description
==== ========================================================
6 Dirty Victims from the QOS domain to all types of memory
5 Reads to slow memory in the non-local NUMA domain
4 Reads to slow memory in the local NUMA domain
3 Non-temporal writes to non-local NUMA domain
2 Non-temporal writes to local NUMA domain
1 Reads to memory in the non-local NUMA domain
0 Reads to memory in the local NUMA domain
==== ========================================================
By default, the mbm_total_bytes configuration is set to 0x7f to count
all the event types and the mbm_local_bytes configuration is set to
0x15 to count all the local memory events.
Examples:
* To view the current configuration::
::
# cat /sys/fs/resctrl/info/L3_MON/mbm_total_bytes_config
0=0x7f;1=0x7f;2=0x7f;3=0x7f
# cat /sys/fs/resctrl/info/L3_MON/mbm_local_bytes_config
0=0x15;1=0x15;3=0x15;4=0x15
* To change the mbm_total_bytes to count only reads on domain 0,
the bits 0, 1, 4 and 5 needs to be set, which is 110011b in binary
(in hexadecimal 0x33):
::
# echo "0=0x33" > /sys/fs/resctrl/info/L3_MON/mbm_total_bytes_config
# cat /sys/fs/resctrl/info/L3_MON/mbm_total_bytes_config
0=0x33;1=0x7f;2=0x7f;3=0x7f
* To change the mbm_local_bytes to count all the slow memory reads on
domain 0 and 1, the bits 4 and 5 needs to be set, which is 110000b
in binary (in hexadecimal 0x30):
::
# echo "0=0x30;1=0x30" > /sys/fs/resctrl/info/L3_MON/mbm_local_bytes_config
# cat /sys/fs/resctrl/info/L3_MON/mbm_local_bytes_config
0=0x30;1=0x30;3=0x15;4=0x15
"max_threshold_occupancy":
Read/write file provides the largest value (in
@@ -464,6 +548,25 @@ Memory bandwidth domain is L3 cache.
MB:<cache_id0>=bw_MBps0;<cache_id1>=bw_MBps1;...
Slow Memory Bandwidth Allocation (SMBA)
---------------------------------------
AMD hardware supports Slow Memory Bandwidth Allocation (SMBA).
CXL.memory is the only supported "slow" memory device. With the
support of SMBA, the hardware enables bandwidth allocation on
the slow memory devices. If there are multiple such devices in
the system, the throttling logic groups all the slow sources
together and applies the limit on them as a whole.
The presence of SMBA (with CXL.memory) is independent of slow memory
devices presence. If there are no such devices on the system, then
configuring SMBA will have no impact on the performance of the system.
The bandwidth domain for slow memory is L3 cache. Its schemata file
is formatted as:
::
SMBA:<cache_id0>=bandwidth0;<cache_id1>=bandwidth1;...
Reading/writing the schemata file
---------------------------------
Reading the schemata file will show the state of all resources
@@ -479,6 +582,46 @@ which you wish to change. E.g.
L3DATA:0=fffff;1=fffff;2=3c0;3=fffff
L3CODE:0=fffff;1=fffff;2=fffff;3=fffff
Reading/writing the schemata file (on AMD systems)
--------------------------------------------------
Reading the schemata file will show the current bandwidth limit on all
domains. The allocated resources are in multiples of one eighth GB/s.
When writing to the file, you need to specify what cache id you wish to
configure the bandwidth limit.
For example, to allocate 2GB/s limit on the first cache id:
::
# cat schemata
MB:0=2048;1=2048;2=2048;3=2048
L3:0=ffff;1=ffff;2=ffff;3=ffff
# echo "MB:1=16" > schemata
# cat schemata
MB:0=2048;1= 16;2=2048;3=2048
L3:0=ffff;1=ffff;2=ffff;3=ffff
Reading/writing the schemata file (on AMD systems) with SMBA feature
--------------------------------------------------------------------
Reading and writing the schemata file is the same as without SMBA in
above section.
For example, to allocate 8GB/s limit on the first cache id:
::
# cat schemata
SMBA:0=2048;1=2048;2=2048;3=2048
MB:0=2048;1=2048;2=2048;3=2048
L3:0=ffff;1=ffff;2=ffff;3=ffff
# echo "SMBA:1=64" > schemata
# cat schemata
SMBA:0=2048;1= 64;2=2048;3=2048
MB:0=2048;1=2048;2=2048;3=2048
L3:0=ffff;1=ffff;2=ffff;3=ffff
Cache Pseudo-Locking
====================
CAT enables a user to specify the amount of cache space that an
MAINTAINERS
+7
View File
@@ -22750,6 +22750,13 @@ F: Documentation/devicetree/bindings/dma/xilinx/xlnx,zynqmp-dpdma.yaml
F: drivers/dma/xilinx/xilinx_dpdma.c
F: include/dt-bindings/dma/xlnx-zynqmp-dpdma.h
XILINX ZYNQMP OCM EDAC DRIVER
M: Shubhrajyoti Datta <shubhrajyoti.datta@amd.com>
M: Sai Krishna Potthuri <sai.krishna.potthuri@amd.com>
S: Maintained
F: Documentation/devicetree/bindings/memory-controllers/xlnx,zynqmp-ocmc-1.0.yaml
F: drivers/edac/zynqmp_edac.c
XILINX ZYNQMP PSGTR PHY DRIVER
M: Anurag Kumar Vulisha <anurag.kumar.vulisha@xilinx.com>
M: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
arch/riscv/Kconfig
-1
View File
@@ -12,7 +12,6 @@ config 32BIT
config RISCV
def_bool y
select ARCH_CLOCKSOURCE_INIT
select ARCH_ENABLE_HUGEPAGE_MIGRATION if HUGETLB_PAGE && MIGRATION
select ARCH_ENABLE_SPLIT_PMD_PTLOCK if PGTABLE_LEVELS > 2
select ARCH_HAS_BINFMT_FLAT
arch/riscv/kernel/time.c
+2 -8
View File
@@ -5,6 +5,7 @@
*/
#include <linux/of_clk.h>
#include <linux/clockchips.h>
#include <linux/clocksource.h>
#include <linux/delay.h>
#include <asm/sbi.h>
@@ -29,13 +30,6 @@ void __init time_init(void)
of_clk_init(NULL);
timer_probe();
}
void clocksource_arch_init(struct clocksource *cs)
{
#ifdef CONFIG_GENERIC_GETTIMEOFDAY
cs->vdso_clock_mode = VDSO_CLOCKMODE_ARCHTIMER;
#else
cs->vdso_clock_mode = VDSO_CLOCKMODE_NONE;
#endif
tick_setup_hrtimer_broadcast();
}
arch/x86/Kconfig
+3 -3
View File
@@ -1502,7 +1502,7 @@ config X86_5LEVEL
depends on X86_64
help
5-level paging enables access to larger address space:
upto 128 PiB of virtual address space and 4 PiB of
up to 128 PiB of virtual address space and 4 PiB of
physical address space.
It will be supported by future Intel CPUs.
@@ -2609,8 +2609,8 @@ config CALL_THUNKS_DEBUG
a noisy dmesg about callthunks generation and call patching for
trouble shooting. The debug prints need to be enabled on the
kernel command line with 'debug-callthunks'.
Only enable this, when you are debugging call thunks as this
creates a noticable runtime overhead. If unsure say N.
Only enable this when you are debugging call thunks as this
creates a noticeable runtime overhead. If unsure say N.
config CPU_IBPB_ENTRY
bool "Enable IBPB on kernel entry"
arch/x86/entry/vdso/Makefile
+2 -1
View File
@@ -29,7 +29,7 @@ VDSO32-$(CONFIG_IA32_EMULATION) := y
# files to link into the vdso
vobjs-y := vdso-note.o vclock_gettime.o vgetcpu.o
vobjs32-y := vdso32/note.o vdso32/system_call.o vdso32/sigreturn.o
vobjs32-y += vdso32/vclock_gettime.o
vobjs32-y += vdso32/vclock_gettime.o vdso32/vgetcpu.o
vobjs-$(CONFIG_X86_SGX) += vsgx.o
# files to link into kernel
@@ -104,6 +104,7 @@ $(vobjs): KBUILD_AFLAGS += -DBUILD_VDSO
CFLAGS_REMOVE_vclock_gettime.o = -pg
CFLAGS_REMOVE_vdso32/vclock_gettime.o = -pg
CFLAGS_REMOVE_vgetcpu.o = -pg
CFLAGS_REMOVE_vdso32/vgetcpu.o = -pg
CFLAGS_REMOVE_vsgx.o = -pg
#
arch/x86/entry/vdso/vdso2c.h
+6
View File
@@ -179,6 +179,7 @@ static void BITSFUNC(go)(void *raw_addr, size_t raw_len,
fprintf(outfile, "/* AUTOMATICALLY GENERATED -- DO NOT EDIT */\n\n");
fprintf(outfile, "#include <linux/linkage.h>\n");
fprintf(outfile, "#include <linux/init.h>\n");
fprintf(outfile, "#include <asm/page_types.h>\n");
fprintf(outfile, "#include <asm/vdso.h>\n");
fprintf(outfile, "\n");
@@ -218,5 +219,10 @@ static void BITSFUNC(go)(void *raw_addr, size_t raw_len,
fprintf(outfile, "\t.sym_%s = %" PRIi64 ",\n",
required_syms[i].name, (int64_t)syms[i]);
}
fprintf(outfile, "};\n\n");
fprintf(outfile, "static __init int init_%s(void) {\n", image_name);
fprintf(outfile, "\treturn init_vdso_image(&%s);\n", image_name);
fprintf(outfile, "};\n");
fprintf(outfile, "subsys_initcall(init_%s);\n", image_name);
}
arch/x86/entry/vdso/vdso32-setup.c
-9
View File
@@ -51,17 +51,8 @@ __setup("vdso32=", vdso32_setup);
__setup_param("vdso=", vdso_setup, vdso32_setup, 0);
#endif
int __init sysenter_setup(void)
{
init_vdso_image(&vdso_image_32);
return 0;
}
#ifdef CONFIG_X86_64
subsys_initcall(sysenter_setup);
#ifdef CONFIG_SYSCTL
/* Register vsyscall32 into the ABI table */
#include <linux/sysctl.h>
arch/x86/entry/vdso/vdso32/fake_32bit_build.h
+25
View File
@@ -0,0 +1,25 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifdef CONFIG_X86_64
/*
* in case of a 32 bit VDSO for a 64 bit kernel fake a 32 bit kernel
* configuration
*/
#undef CONFIG_64BIT
#undef CONFIG_X86_64
#undef CONFIG_COMPAT
#undef CONFIG_PGTABLE_LEVELS
#undef CONFIG_ILLEGAL_POINTER_VALUE
#undef CONFIG_SPARSEMEM_VMEMMAP
#undef CONFIG_NR_CPUS
#undef CONFIG_PARAVIRT_XXL
#define CONFIG_X86_32 1
#define CONFIG_PGTABLE_LEVELS 2
#define CONFIG_PAGE_OFFSET 0
#define CONFIG_ILLEGAL_POINTER_VALUE 0
#define CONFIG_NR_CPUS 1
#define BUILD_VDSO32_64
#endif
arch/x86/entry/vdso/vdso32/vclock_gettime.c
+1 -26
View File
@@ -1,29 +1,4 @@
// SPDX-License-Identifier: GPL-2.0
#define BUILD_VDSO32
#ifdef CONFIG_X86_64
/*
* in case of a 32 bit VDSO for a 64 bit kernel fake a 32 bit kernel
* configuration
*/
#undef CONFIG_64BIT
#undef CONFIG_X86_64
#undef CONFIG_COMPAT
#undef CONFIG_PGTABLE_LEVELS
#undef CONFIG_ILLEGAL_POINTER_VALUE
#undef CONFIG_SPARSEMEM_VMEMMAP
#undef CONFIG_NR_CPUS
#undef CONFIG_PARAVIRT_XXL
#define CONFIG_X86_32 1
#define CONFIG_PGTABLE_LEVELS 2
#define CONFIG_PAGE_OFFSET 0
#define CONFIG_ILLEGAL_POINTER_VALUE 0
#define CONFIG_NR_CPUS 1
#define BUILD_VDSO32_64
#endif
#include "fake_32bit_build.h"
#include "../vclock_gettime.c"
arch/x86/entry/vdso/vdso32/vdso32.lds.S
+1
View File
@@ -28,6 +28,7 @@ VERSION
__vdso_time;
__vdso_clock_getres;
__vdso_clock_gettime64;
__vdso_getcpu;
};
LINUX_2.5 {
arch/x86/entry/vdso/vdso32/vgetcpu.c
+3
View File
@@ -0,0 +1,3 @@
// SPDX-License-Identifier: GPL-2.0
#include "fake_32bit_build.h"
#include "../vgetcpu.c"
arch/x86/entry/vdso/vgetcpu.c
+1 -2
View File
@@ -7,8 +7,7 @@
#include <linux/kernel.h>
#include <linux/getcpu.h>
#include <linux/time.h>
#include <asm/vgtod.h>
#include <asm/segment.h>
notrace long
__vdso_getcpu(unsigned *cpu, unsigned *node, struct getcpu_cache *unused)
arch/x86/entry/vdso/vma.c
+4 -15
View File
@@ -44,13 +44,16 @@ unsigned int vclocks_used __read_mostly;
unsigned int __read_mostly vdso64_enabled = 1;
#endif
void __init init_vdso_image(const struct vdso_image *image)
int __init init_vdso_image(const struct vdso_image *image)
{
BUILD_BUG_ON(VDSO_CLOCKMODE_MAX >= 32);
BUG_ON(image->size % PAGE_SIZE != 0);
apply_alternatives((struct alt_instr *)(image->data + image->alt),
(struct alt_instr *)(image->data + image->alt +
image->alt_len));
return 0;
}
static const struct vm_special_mapping vvar_mapping;
@@ -418,18 +421,4 @@ static __init int vdso_setup(char *s)
return 1;
}
__setup("vdso=", vdso_setup);
static int __init init_vdso(void)
{
BUILD_BUG_ON(VDSO_CLOCKMODE_MAX >= 32);
init_vdso_image(&vdso_image_64);
#ifdef CONFIG_X86_X32_ABI
init_vdso_image(&vdso_image_x32);
#endif
return 0;
}
subsys_initcall(init_vdso);
#endif /* CONFIG_X86_64 */
arch/x86/include/asm/alternative.h
+75 -55
View File
@@ -6,8 +6,10 @@
#include <linux/stringify.h>
#include <asm/asm.h>
#define ALTINSTR_FLAG_INV (1 << 15)
#define ALT_NOT(feat) ((feat) | ALTINSTR_FLAG_INV)
#define ALT_FLAGS_SHIFT 16
#define ALT_FLAG_NOT BIT(0)
#define ALT_NOT(feature) ((ALT_FLAG_NOT << ALT_FLAGS_SHIFT) | (feature))
#ifndef __ASSEMBLY__
@@ -59,10 +61,27 @@
".long 999b - .\n\t" \
".popsection\n\t"
/*
* The patching flags are part of the upper bits of the @ft_flags parameter when
* specifying them. The split is currently like this:
*
* [31... flags ...16][15... CPUID feature bit ...0]
*
* but since this is all hidden in the macros argument being split, those fields can be
* extended in the future to fit in a u64 or however the need arises.
*/
struct alt_instr {
s32 instr_offset; /* original instruction */
s32 repl_offset; /* offset to replacement instruction */
u16 cpuid; /* cpuid bit set for replacement */
union {
struct {
u32 cpuid: 16; /* CPUID bit set for replacement */
u32 flags: 16; /* patching control flags */
};
u32 ft_flags;
};
u8 instrlen; /* length of original instruction */
u8 replacementlen; /* length of new instruction */
} __packed;
@@ -182,10 +201,10 @@ static inline int alternatives_text_reserved(void *start, void *end)
" - (" alt_slen ")), 0x90\n" \
alt_end_marker ":\n"
#define ALTINSTR_ENTRY(feature, num) \
#define ALTINSTR_ENTRY(ft_flags, num) \
" .long 661b - .\n" /* label */ \
" .long " b_replacement(num)"f - .\n" /* new instruction */ \
" .word " __stringify(feature) "\n" /* feature bit */ \
" .4byte " __stringify(ft_flags) "\n" /* feature + flags */ \
" .byte " alt_total_slen "\n" /* source len */ \
" .byte " alt_rlen(num) "\n" /* replacement len */
@@ -194,20 +213,20 @@ static inline int alternatives_text_reserved(void *start, void *end)
b_replacement(num)":\n\t" newinstr "\n" e_replacement(num) ":\n"
/* alternative assembly primitive: */
#define ALTERNATIVE(oldinstr, newinstr, feature) \
#define ALTERNATIVE(oldinstr, newinstr, ft_flags) \
OLDINSTR(oldinstr, 1) \
".pushsection .altinstructions,\"a\"\n" \
ALTINSTR_ENTRY(feature, 1) \
ALTINSTR_ENTRY(ft_flags, 1) \
".popsection\n" \
".pushsection .altinstr_replacement, \"ax\"\n" \
ALTINSTR_REPLACEMENT(newinstr, 1) \
".popsection\n"
#define ALTERNATIVE_2(oldinstr, newinstr1, feature1, newinstr2, feature2)\
#define ALTERNATIVE_2(oldinstr, newinstr1, ft_flags1, newinstr2, ft_flags2) \
OLDINSTR_2(oldinstr, 1, 2) \
".pushsection .altinstructions,\"a\"\n" \
ALTINSTR_ENTRY(feature1, 1) \
ALTINSTR_ENTRY(feature2, 2) \
ALTINSTR_ENTRY(ft_flags1, 1) \
ALTINSTR_ENTRY(ft_flags2, 2) \
".popsection\n" \
".pushsection .altinstr_replacement, \"ax\"\n" \
ALTINSTR_REPLACEMENT(newinstr1, 1) \
@@ -215,21 +234,22 @@ static inline int alternatives_text_reserved(void *start, void *end)
".popsection\n"
/* If @feature is set, patch in @newinstr_yes, otherwise @newinstr_no. */
#define ALTERNATIVE_TERNARY(oldinstr, feature, newinstr_yes, newinstr_no) \
#define ALTERNATIVE_TERNARY(oldinstr, ft_flags, newinstr_yes, newinstr_no) \
ALTERNATIVE_2(oldinstr, newinstr_no, X86_FEATURE_ALWAYS, \
newinstr_yes, feature)
newinstr_yes, ft_flags)
#define ALTERNATIVE_3(oldinsn, newinsn1, feat1, newinsn2, feat2, newinsn3, feat3) \
OLDINSTR_3(oldinsn, 1, 2, 3) \
".pushsection .altinstructions,\"a\"\n" \
ALTINSTR_ENTRY(feat1, 1) \
ALTINSTR_ENTRY(feat2, 2) \
ALTINSTR_ENTRY(feat3, 3) \
".popsection\n" \
".pushsection .altinstr_replacement, \"ax\"\n" \
ALTINSTR_REPLACEMENT(newinsn1, 1) \
ALTINSTR_REPLACEMENT(newinsn2, 2) \
ALTINSTR_REPLACEMENT(newinsn3, 3) \
#define ALTERNATIVE_3(oldinsn, newinsn1, ft_flags1, newinsn2, ft_flags2, \
newinsn3, ft_flags3) \
OLDINSTR_3(oldinsn, 1, 2, 3) \
".pushsection .altinstructions,\"a\"\n" \
ALTINSTR_ENTRY(ft_flags1, 1) \
ALTINSTR_ENTRY(ft_flags2, 2) \
ALTINSTR_ENTRY(ft_flags3, 3) \
".popsection\n" \
".pushsection .altinstr_replacement, \"ax\"\n" \
ALTINSTR_REPLACEMENT(newinsn1, 1) \
ALTINSTR_REPLACEMENT(newinsn2, 2) \
ALTINSTR_REPLACEMENT(newinsn3, 3) \
".popsection\n"
/*
@@ -244,14 +264,14 @@ static inline int alternatives_text_reserved(void *start, void *end)
* For non barrier like inlines please define new variants
* without volatile and memory clobber.
*/
#define alternative(oldinstr, newinstr, feature) \
asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, feature) : : : "memory")
#define alternative(oldinstr, newinstr, ft_flags) \
asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, ft_flags) : : : "memory")
#define alternative_2(oldinstr, newinstr1, feature1, newinstr2, feature2) \
asm_inline volatile(ALTERNATIVE_2(oldinstr, newinstr1, feature1, newinstr2, feature2) ::: "memory")
#define alternative_2(oldinstr, newinstr1, ft_flags1, newinstr2, ft_flags2) \
asm_inline volatile(ALTERNATIVE_2(oldinstr, newinstr1, ft_flags1, newinstr2, ft_flags2) ::: "memory")
#define alternative_ternary(oldinstr, feature, newinstr_yes, newinstr_no) \
asm_inline volatile(ALTERNATIVE_TERNARY(oldinstr, feature, newinstr_yes, newinstr_no) ::: "memory")
#define alternative_ternary(oldinstr, ft_flags, newinstr_yes, newinstr_no) \
asm_inline volatile(ALTERNATIVE_TERNARY(oldinstr, ft_flags, newinstr_yes, newinstr_no) ::: "memory")
/*
* Alternative inline assembly with input.
@@ -261,8 +281,8 @@ static inline int alternatives_text_reserved(void *start, void *end)
* Argument numbers start with 1.
* Leaving an unused argument 0 to keep API compatibility.
*/
#define alternative_input(oldinstr, newinstr, feature, input...) \
asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, feature) \
#define alternative_input(oldinstr, newinstr, ft_flags, input...) \
asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, ft_flags) \
: : "i" (0), ## input)
/*
@@ -273,20 +293,20 @@ static inline int alternatives_text_reserved(void *start, void *end)
* Otherwise, if CPU has feature1, newinstr1 is used.
* Otherwise, oldinstr is used.
*/
#define alternative_input_2(oldinstr, newinstr1, feature1, newinstr2, \
feature2, input...) \
asm_inline volatile(ALTERNATIVE_2(oldinstr, newinstr1, feature1, \
newinstr2, feature2) \
#define alternative_input_2(oldinstr, newinstr1, ft_flags1, newinstr2, \
ft_flags2, input...) \
asm_inline volatile(ALTERNATIVE_2(oldinstr, newinstr1, ft_flags1, \
newinstr2, ft_flags2) \
: : "i" (0), ## input)
/* Like alternative_input, but with a single output argument */
#define alternative_io(oldinstr, newinstr, feature, output, input...) \
asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, feature) \
#define alternative_io(oldinstr, newinstr, ft_flags, output, input...) \
asm_inline volatile (ALTERNATIVE(oldinstr, newinstr, ft_flags) \
: output : "i" (0), ## input)
/* Like alternative_io, but for replacing a direct call with another one. */
#define alternative_call(oldfunc, newfunc, feature, output, input...) \
asm_inline volatile (ALTERNATIVE("call %P[old]", "call %P[new]", feature) \
#define alternative_call(oldfunc, newfunc, ft_flags, output, input...) \
asm_inline volatile (ALTERNATIVE("call %P[old]", "call %P[new]", ft_flags) \
: output : [old] "i" (oldfunc), [new] "i" (newfunc), ## input)
/*
@@ -295,10 +315,10 @@ static inline int alternatives_text_reserved(void *start, void *end)
* Otherwise, if CPU has feature1, function1 is used.
* Otherwise, old function is used.
*/
#define alternative_call_2(oldfunc, newfunc1, feature1, newfunc2, feature2, \
#define alternative_call_2(oldfunc, newfunc1, ft_flags1, newfunc2, ft_flags2, \
output, input...) \
asm_inline volatile (ALTERNATIVE_2("call %P[old]", "call %P[new1]", feature1,\
"call %P[new2]", feature2) \
asm_inline volatile (ALTERNATIVE_2("call %P[old]", "call %P[new1]", ft_flags1,\
"call %P[new2]", ft_flags2) \
: output, ASM_CALL_CONSTRAINT \
: [old] "i" (oldfunc), [new1] "i" (newfunc1), \
[new2] "i" (newfunc2), ## input)
@@ -347,10 +367,10 @@ static inline int alternatives_text_reserved(void *start, void *end)
* enough information for the alternatives patching code to patch an
* instruction. See apply_alternatives().
*/
.macro altinstruction_entry orig alt feature orig_len alt_len
.macro altinstr_entry orig alt ft_flags orig_len alt_len
.long \orig - .
.long \alt - .
.word \feature
.4byte \ft_flags
.byte \orig_len
.byte \alt_len
.endm
@@ -361,7 +381,7 @@ static inline int alternatives_text_reserved(void *start, void *end)
* @newinstr. ".skip" directive takes care of proper instruction padding
* in case @newinstr is longer than @oldinstr.
*/
.macro ALTERNATIVE oldinstr, newinstr, feature
.macro ALTERNATIVE oldinstr, newinstr, ft_flags
140:
\oldinstr
141:
@@ -369,7 +389,7 @@ static inline int alternatives_text_reserved(void *start, void *end)
142:
.pushsection .altinstructions,"a"
altinstruction_entry 140b,143f,\feature,142b-140b,144f-143f
altinstr_entry 140b,143f,\ft_flags,142b-140b,144f-143f
.popsection
.pushsection .altinstr_replacement,"ax"
@@ -399,7 +419,7 @@ static inline int alternatives_text_reserved(void *start, void *end)
* has @feature1, it replaces @oldinstr with @newinstr1. If CPU has
* @feature2, it replaces @oldinstr with @feature2.
*/
.macro ALTERNATIVE_2 oldinstr, newinstr1, feature1, newinstr2, feature2
.macro ALTERNATIVE_2 oldinstr, newinstr1, ft_flags1, newinstr2, ft_flags2
140:
\oldinstr
141:
@@ -408,8 +428,8 @@ static inline int alternatives_text_reserved(void *start, void *end)
142:
.pushsection .altinstructions,"a"
altinstruction_entry 140b,143f,\feature1,142b-140b,144f-143f
altinstruction_entry 140b,144f,\feature2,142b-140b,145f-144f
altinstr_entry 140b,143f,\ft_flags1,142b-140b,144f-143f
altinstr_entry 140b,144f,\ft_flags2,142b-140b,145f-144f
.popsection
.pushsection .altinstr_replacement,"ax"
@@ -421,7 +441,7 @@ static inline int alternatives_text_reserved(void *start, void *end)
.popsection
.endm
.macro ALTERNATIVE_3 oldinstr, newinstr1, feature1, newinstr2, feature2, newinstr3, feature3
.macro ALTERNATIVE_3 oldinstr, newinstr1, ft_flags1, newinstr2, ft_flags2, newinstr3, ft_flags3
140:
\oldinstr
141:
@@ -430,9 +450,9 @@ static inline int alternatives_text_reserved(void *start, void *end)
142:
.pushsection .altinstructions,"a"
altinstruction_entry 140b,143f,\feature1,142b-140b,144f-143f
altinstruction_entry 140b,144f,\feature2,142b-140b,145f-144f
altinstruction_entry 140b,145f,\feature3,142b-140b,146f-145f
altinstr_entry 140b,143f,\ft_flags1,142b-140b,144f-143f
altinstr_entry 140b,144f,\ft_flags2,142b-140b,145f-144f
altinstr_entry 140b,145f,\ft_flags3,142b-140b,146f-145f
.popsection
.pushsection .altinstr_replacement,"ax"
@@ -447,9 +467,9 @@ static inline int alternatives_text_reserved(void *start, void *end)
.endm
/* If @feature is set, patch in @newinstr_yes, otherwise @newinstr_no. */
#define ALTERNATIVE_TERNARY(oldinstr, feature, newinstr_yes, newinstr_no) \
#define ALTERNATIVE_TERNARY(oldinstr, ft_flags, newinstr_yes, newinstr_no) \
ALTERNATIVE_2 oldinstr, newinstr_no, X86_FEATURE_ALWAYS, \
newinstr_yes, feature
newinstr_yes, ft_flags
#endif /* __ASSEMBLY__ */
arch/x86/include/asm/cpufeatures.h
+2
View File
@@ -307,6 +307,8 @@
#define X86_FEATURE_SGX_EDECCSSA (11*32+18) /* "" SGX EDECCSSA user leaf function */
#define X86_FEATURE_CALL_DEPTH (11*32+19) /* "" Call depth tracking for RSB stuffing */
#define X86_FEATURE_MSR_TSX_CTRL (11*32+20) /* "" MSR IA32_TSX_CTRL (Intel) implemented */
#define X86_FEATURE_SMBA (11*32+21) /* "" Slow Memory Bandwidth Allocation */
#define X86_FEATURE_BMEC (11*32+22) /* "" Bandwidth Monitoring Event Configuration */
/* Intel-defined CPU features, CPUID level 0x00000007:1 (EAX), word 12 */
#define X86_FEATURE_AVX_VNNI (12*32+ 4) /* AVX VNNI instructions */
arch/x86/include/asm/mce.h
+3
View File
@@ -88,6 +88,9 @@
#define MCI_MISC_ADDR_MEM 3 /* memory address */
#define MCI_MISC_ADDR_GENERIC 7 /* generic */
/* MCi_ADDR register defines */
#define MCI_ADDR_PHYSADDR GENMASK_ULL(boot_cpu_data.x86_phys_bits - 1, 0)
/* CTL2 register defines */
#define MCI_CTL2_CMCI_EN BIT_ULL(30)
#define MCI_CTL2_CMCI_THRESHOLD_MASK 0x7fffULL
arch/x86/include/asm/microcode.h
+2 -2
View File
@@ -125,13 +125,13 @@ static inline unsigned int x86_cpuid_family(void)
#ifdef CONFIG_MICROCODE
extern void __init load_ucode_bsp(void);
extern void load_ucode_ap(void);
void reload_early_microcode(void);
void reload_early_microcode(unsigned int cpu);
extern bool initrd_gone;
void microcode_bsp_resume(void);
#else
static inline void __init load_ucode_bsp(void) { }
static inline void load_ucode_ap(void) { }
static inline void reload_early_microcode(void) { }
static inline void reload_early_microcode(unsigned int cpu) { }
static inline void microcode_bsp_resume(void) { }
#endif
arch/x86/include/asm/microcode_amd.h
+2 -2
View File
@@ -47,12 +47,12 @@ struct microcode_amd {
extern void __init load_ucode_amd_bsp(unsigned int family);
extern void load_ucode_amd_ap(unsigned int family);
extern int __init save_microcode_in_initrd_amd(unsigned int family);
void reload_ucode_amd(void);
void reload_ucode_amd(unsigned int cpu);
#else
static inline void __init load_ucode_amd_bsp(unsigned int family) {}
static inline void load_ucode_amd_ap(unsigned int family) {}
static inline int __init
save_microcode_in_initrd_amd(unsigned int family) { return -EINVAL; }
static inline void reload_ucode_amd(void) {}
static inline void reload_ucode_amd(unsigned int cpu) {}
#endif
#endif /* _ASM_X86_MICROCODE_AMD_H */
arch/x86/include/asm/msr-index.h
+2
View File
@@ -1084,6 +1084,8 @@
/* - AMD: */
#define MSR_IA32_MBA_BW_BASE 0xc0000200
#define MSR_IA32_SMBA_BW_BASE 0xc0000280
#define MSR_IA32_EVT_CFG_BASE 0xc0000400
/* MSR_IA32_VMX_MISC bits */
#define MSR_IA32_VMX_MISC_INTEL_PT (1ULL << 14)
arch/x86/include/asm/processor.h
+2 -2
View File
@@ -542,7 +542,6 @@ enum idle_boot_override {IDLE_NO_OVERRIDE=0, IDLE_HALT, IDLE_NOMWAIT,
IDLE_POLL};
extern void enable_sep_cpu(void);
extern int sysenter_setup(void);
/* Defined in head.S */
@@ -697,7 +696,8 @@ bool xen_set_default_idle(void);
#endif
void __noreturn stop_this_cpu(void *dummy);
void microcode_check(void);
void microcode_check(struct cpuinfo_x86 *prev_info);
void store_cpu_caps(struct cpuinfo_x86 *info);
enum l1tf_mitigations {
L1TF_MITIGATION_OFF,
arch/x86/include/asm/segment.h
+4 -4
View File
@@ -96,7 +96,7 @@
*
* 26 - ESPFIX small SS
* 27 - per-cpu [ offset to per-cpu data area ]
* 28 - unused
* 28 - VDSO getcpu
* 29 - unused
* 30 - unused
* 31 - TSS for double fault handler
@@ -119,6 +119,7 @@
#define GDT_ENTRY_ESPFIX_SS 26
#define GDT_ENTRY_PERCPU 27
#define GDT_ENTRY_CPUNODE 28
#define GDT_ENTRY_DOUBLEFAULT_TSS 31
@@ -159,6 +160,8 @@
# define __KERNEL_PERCPU 0
#endif
#define __CPUNODE_SEG (GDT_ENTRY_CPUNODE*8 + 3)
#else /* 64-bit: */
#include <asm/cache.h>
@@ -226,8 +229,6 @@
#define GDT_ENTRY_TLS_ENTRIES 3
#define TLS_SIZE (GDT_ENTRY_TLS_ENTRIES* 8)
#ifdef CONFIG_X86_64
/* Bit size and mask of CPU number stored in the per CPU data (and TSC_AUX) */
#define VDSO_CPUNODE_BITS 12
#define VDSO_CPUNODE_MASK 0xfff
@@ -265,7 +266,6 @@ static inline void vdso_read_cpunode(unsigned *cpu, unsigned *node)
}
#endif /* !__ASSEMBLY__ */
#endif /* CONFIG_X86_64 */
#ifdef __KERNEL__
arch/x86/include/asm/text-patching.h
+31
View File
@@ -184,6 +184,37 @@ void int3_emulate_ret(struct pt_regs *regs)
unsigned long ip = int3_emulate_pop(regs);
int3_emulate_jmp(regs, ip);
}
static __always_inline
void int3_emulate_jcc(struct pt_regs *regs, u8 cc, unsigned long ip, unsigned long disp)
{
static const unsigned long jcc_mask[6] = {
[0] = X86_EFLAGS_OF,
[1] = X86_EFLAGS_CF,
[2] = X86_EFLAGS_ZF,
[3] = X86_EFLAGS_CF | X86_EFLAGS_ZF,
[4] = X86_EFLAGS_SF,
[5] = X86_EFLAGS_PF,
};
bool invert = cc & 1;
bool match;
if (cc < 0xc) {
match = regs->flags & jcc_mask[cc >> 1];
} else {
match = ((regs->flags & X86_EFLAGS_SF) >> X86_EFLAGS_SF_BIT) ^
((regs->flags & X86_EFLAGS_OF) >> X86_EFLAGS_OF_BIT);
if (cc >= 0xe)
match = match || (regs->flags & X86_EFLAGS_ZF);
}
if ((match && !invert) || (!match && invert))
ip += disp;
int3_emulate_jmp(regs, ip);
}
#endif /* !CONFIG_UML_X86 */
#endif /* _ASM_X86_TEXT_PATCHING_H */
arch/x86/include/asm/time.h
+1
View File
@@ -8,6 +8,7 @@
extern void hpet_time_init(void);
extern void time_init(void);
extern bool pit_timer_init(void);
extern bool tsc_clocksource_watchdog_disabled(void);
extern struct clock_event_device *global_clock_event;
arch/x86/include/asm/vdso.h
+1 -1
View File
@@ -45,7 +45,7 @@ extern const struct vdso_image vdso_image_x32;
extern const struct vdso_image vdso_image_32;
#endif
extern void __init init_vdso_image(const struct vdso_image *image);
extern int __init init_vdso_image(const struct vdso_image *image);
extern int map_vdso_once(const struct vdso_image *image, unsigned long addr);
arch/x86/include/asm/vdso/gettimeofday.h
+2
View File
@@ -318,6 +318,8 @@ u64 vdso_calc_delta(u64 cycles, u64 last, u64 mask, u32 mult)
}
#define vdso_calc_delta vdso_calc_delta
int __vdso_clock_gettime64(clockid_t clock, struct __kernel_timespec *ts);
#endif /* !__ASSEMBLY__ */
#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
arch/x86/include/asm/vdso/processor.h
+4
View File
@@ -18,6 +18,10 @@ static __always_inline void cpu_relax(void)
rep_nop();
}
struct getcpu_cache;
notrace long __vdso_getcpu(unsigned *cpu, unsigned *node, struct getcpu_cache *unused);
#endif /* __ASSEMBLY__ */
#endif /* __ASM_VDSO_PROCESSOR_H */
arch/x86/kernel/alternative.c
+53 -23
View File
@@ -282,27 +282,25 @@ void __init_or_module noinline apply_alternatives(struct alt_instr *start,
*/
for (a = start; a < end; a++) {
int insn_buff_sz = 0;
/* Mask away "NOT" flag bit for feature to test. */
u16 feature = a->cpuid & ~ALTINSTR_FLAG_INV;
instr = (u8 *)&a->instr_offset + a->instr_offset;
replacement = (u8 *)&a->repl_offset + a->repl_offset;
BUG_ON(a->instrlen > sizeof(insn_buff));
BUG_ON(feature >= (NCAPINTS + NBUGINTS) * 32);
BUG_ON(a->cpuid >= (NCAPINTS + NBUGINTS) * 32);
/*
* Patch if either:
* - feature is present
* - feature not present but ALTINSTR_FLAG_INV is set to mean,
* - feature not present but ALT_FLAG_NOT is set to mean,
* patch if feature is *NOT* present.
*/
if (!boot_cpu_has(feature) == !(a->cpuid & ALTINSTR_FLAG_INV))
if (!boot_cpu_has(a->cpuid) == !(a->flags & ALT_FLAG_NOT))
goto next;
DPRINTK("feat: %s%d*32+%d, old: (%pS (%px) len: %d), repl: (%px, len: %d)",
(a->cpuid & ALTINSTR_FLAG_INV) ? "!" : "",
feature >> 5,
feature & 0x1f,
(a->flags & ALT_FLAG_NOT) ? "!" : "",
a->cpuid >> 5,
a->cpuid & 0x1f,
instr, instr, a->instrlen,
replacement, a->replacementlen);
@@ -340,6 +338,12 @@ next:
}
}
static inline bool is_jcc32(struct insn *insn)
{
/* Jcc.d32 second opcode byte is in the range: 0x80-0x8f */
return insn->opcode.bytes[0] == 0x0f && (insn->opcode.bytes[1] & 0xf0) == 0x80;
}
#if defined(CONFIG_RETPOLINE) && defined(CONFIG_OBJTOOL)
/*
@@ -378,12 +382,6 @@ static int emit_indirect(int op, int reg, u8 *bytes)
return i;
}
static inline bool is_jcc32(struct insn *insn)
{
/* Jcc.d32 second opcode byte is in the range: 0x80-0x8f */
return insn->opcode.bytes[0] == 0x0f && (insn->opcode.bytes[1] & 0xf0) == 0x80;
}
static int emit_call_track_retpoline(void *addr, struct insn *insn, int reg, u8 *bytes)
{
u8 op = insn->opcode.bytes[0];
@@ -1772,6 +1770,11 @@ void text_poke_sync(void)
on_each_cpu(do_sync_core, NULL, 1);
}
/*
* NOTE: crazy scheme to allow patching Jcc.d32 but not increase the size of
* this thing. When len == 6 everything is prefixed with 0x0f and we map
* opcode to Jcc.d8, using len to distinguish.
*/
struct text_poke_loc {
/* addr := _stext + rel_addr */
s32 rel_addr;
@@ -1893,6 +1896,10 @@ noinstr int poke_int3_handler(struct pt_regs *regs)
int3_emulate_jmp(regs, (long)ip + tp->disp);
break;
case 0x70 ... 0x7f: /* Jcc */
int3_emulate_jcc(regs, tp->opcode & 0xf, (long)ip, tp->disp);
break;
default:
BUG();
}
@@ -1966,16 +1973,26 @@ static void text_poke_bp_batch(struct text_poke_loc *tp, unsigned int nr_entries
* Second step: update all but the first byte of the patched range.
*/
for (do_sync = 0, i = 0; i < nr_entries; i++) {
u8 old[POKE_MAX_OPCODE_SIZE] = { tp[i].old, };
u8 old[POKE_MAX_OPCODE_SIZE+1] = { tp[i].old, };
u8 _new[POKE_MAX_OPCODE_SIZE+1];
const u8 *new = tp[i].text;
int len = tp[i].len;
if (len - INT3_INSN_SIZE > 0) {
memcpy(old + INT3_INSN_SIZE,
text_poke_addr(&tp[i]) + INT3_INSN_SIZE,
len - INT3_INSN_SIZE);
if (len == 6) {
_new[0] = 0x0f;
memcpy(_new + 1, new, 5);
new = _new;
}
text_poke(text_poke_addr(&tp[i]) + INT3_INSN_SIZE,
(const char *)tp[i].text + INT3_INSN_SIZE,
new + INT3_INSN_SIZE,
len - INT3_INSN_SIZE);
do_sync++;
}
@@ -2003,8 +2020,7 @@ static void text_poke_bp_batch(struct text_poke_loc *tp, unsigned int nr_entries
* The old instruction is recorded so that the event can be
* processed forwards or backwards.
*/
perf_event_text_poke(text_poke_addr(&tp[i]), old, len,
tp[i].text, len);
perf_event_text_poke(text_poke_addr(&tp[i]), old, len, new, len);
}
if (do_sync) {
@@ -2021,10 +2037,15 @@ static void text_poke_bp_batch(struct text_poke_loc *tp, unsigned int nr_entries
* replacing opcode.
*/
for (do_sync = 0, i = 0; i < nr_entries; i++) {
if (tp[i].text[0] == INT3_INSN_OPCODE)
u8 byte = tp[i].text[0];
if (tp[i].len == 6)
byte = 0x0f;
if (byte == INT3_INSN_OPCODE)
continue;
text_poke(text_poke_addr(&tp[i]), tp[i].text, INT3_INSN_SIZE);
text_poke(text_poke_addr(&tp[i]), &byte, INT3_INSN_SIZE);
do_sync++;
}
@@ -2042,9 +2063,11 @@ static void text_poke_loc_init(struct text_poke_loc *tp, void *addr,
const void *opcode, size_t len, const void *emulate)
{
struct insn insn;
int ret, i;
int ret, i = 0;
memcpy((void *)tp->text, opcode, len);
if (len == 6)
i = 1;
memcpy((void *)tp->text, opcode+i, len-i);
if (!emulate)
emulate = opcode;
@@ -2055,6 +2078,13 @@ static void text_poke_loc_init(struct text_poke_loc *tp, void *addr,
tp->len = len;
tp->opcode = insn.opcode.bytes[0];
if (is_jcc32(&insn)) {
/*
* Map Jcc.d32 onto Jcc.d8 and use len to distinguish.
*/
tp->opcode = insn.opcode.bytes[1] - 0x10;
}
switch (tp->opcode) {
case RET_INSN_OPCODE:
case JMP32_INSN_OPCODE:
@@ -2071,7 +2101,6 @@ static void text_poke_loc_init(struct text_poke_loc *tp, void *addr,
BUG_ON(len != insn.length);
}
switch (tp->opcode) {
case INT3_INSN_OPCODE:
case RET_INSN_OPCODE:
@@ -2080,6 +2109,7 @@ static void text_poke_loc_init(struct text_poke_loc *tp, void *addr,
case CALL_INSN_OPCODE:
case JMP32_INSN_OPCODE:
case JMP8_INSN_OPCODE:
case 0x70 ... 0x7f: /* Jcc */
tp->disp = insn.immediate.value;
break;
arch/x86/kernel/cpu/cacheinfo.c
+1 -4
View File
@@ -734,7 +734,7 @@ void init_hygon_cacheinfo(struct cpuinfo_x86 *c)
void init_intel_cacheinfo(struct cpuinfo_x86 *c)
{
/* Cache sizes */
unsigned int trace = 0, l1i = 0, l1d = 0, l2 = 0, l3 = 0;
unsigned int l1i = 0, l1d = 0, l2 = 0, l3 = 0;
unsigned int new_l1d = 0, new_l1i = 0; /* Cache sizes from cpuid(4) */
unsigned int new_l2 = 0, new_l3 = 0, i; /* Cache sizes from cpuid(4) */
unsigned int l2_id = 0, l3_id = 0, num_threads_sharing, index_msb;
@@ -835,9 +835,6 @@ void init_intel_cacheinfo(struct cpuinfo_x86 *c)
case LVL_3:
l3 += cache_table[k].size;
break;
case LVL_TRACE:
trace += cache_table[k].size;
break;
}
break;
arch/x86/kernel/cpu/common.c
+32 -20
View File
@@ -567,7 +567,7 @@ static __init int setup_disable_pku(char *arg)
return 1;
}
__setup("nopku", setup_disable_pku);
#endif /* CONFIG_X86_64 */
#endif
#ifdef CONFIG_X86_KERNEL_IBT
@@ -1958,7 +1958,6 @@ void __init identify_boot_cpu(void)
if (HAS_KERNEL_IBT && cpu_feature_enabled(X86_FEATURE_IBT))
pr_info("CET detected: Indirect Branch Tracking enabled\n");
#ifdef CONFIG_X86_32
sysenter_setup();
enable_sep_cpu();
#endif
cpu_detect_tlb(&boot_cpu_data);
@@ -2130,7 +2129,6 @@ static void wait_for_master_cpu(int cpu)
#endif
}
#ifdef CONFIG_X86_64
static inline void setup_getcpu(int cpu)
{
unsigned long cpudata = vdso_encode_cpunode(cpu, early_cpu_to_node(cpu));
@@ -2152,6 +2150,7 @@ static inline void setup_getcpu(int cpu)
write_gdt_entry(get_cpu_gdt_rw(cpu), GDT_ENTRY_CPUNODE, &d, DESCTYPE_S);
}
#ifdef CONFIG_X86_64
static inline void ucode_cpu_init(int cpu)
{
if (cpu)
@@ -2171,8 +2170,6 @@ static inline void tss_setup_ist(struct tss_struct *tss)
#else /* CONFIG_X86_64 */
static inline void setup_getcpu(int cpu) { }
static inline void ucode_cpu_init(int cpu)
{
show_ucode_info_early();
@@ -2302,30 +2299,45 @@ void cpu_init_secondary(void)
#endif
#ifdef CONFIG_MICROCODE_LATE_LOADING
/*
/**
* store_cpu_caps() - Store a snapshot of CPU capabilities
* @curr_info: Pointer where to store it
*
* Returns: None
*/
void store_cpu_caps(struct cpuinfo_x86 *curr_info)
{
/* Reload CPUID max function as it might've changed. */
curr_info->cpuid_level = cpuid_eax(0);
/* Copy all capability leafs and pick up the synthetic ones. */
memcpy(&curr_info->x86_capability, &boot_cpu_data.x86_capability,
sizeof(curr_info->x86_capability));
/* Get the hardware CPUID leafs */
get_cpu_cap(curr_info);
}
/**
* microcode_check() - Check if any CPU capabilities changed after an update.
* @prev_info: CPU capabilities stored before an update.
*
* The microcode loader calls this upon late microcode load to recheck features,
* only when microcode has been updated. Caller holds microcode_mutex and CPU
* hotplug lock.
*
* Return: None
*/
void microcode_check(void)
void microcode_check(struct cpuinfo_x86 *prev_info)
{
struct cpuinfo_x86 info;
struct cpuinfo_x86 curr_info;
perf_check_microcode();
/* Reload CPUID max function as it might've changed. */
info.cpuid_level = cpuid_eax(0);
store_cpu_caps(&curr_info);
/*
* Copy all capability leafs to pick up the synthetic ones so that
* memcmp() below doesn't fail on that. The ones coming from CPUID will
* get overwritten in get_cpu_cap().
*/
memcpy(&info.x86_capability, &boot_cpu_data.x86_capability, sizeof(info.x86_capability));
get_cpu_cap(&info);
if (!memcmp(&info.x86_capability, &boot_cpu_data.x86_capability, sizeof(info.x86_capability)))
if (!memcmp(&prev_info->x86_capability, &curr_info.x86_capability,
sizeof(prev_info->x86_capability)))
return;
pr_warn("x86/CPU: CPU features have changed after loading microcode, but might not take effect.\n");
arch/x86/kernel/cpu/cpuid-deps.c
+2
View File
@@ -68,6 +68,8 @@ static const struct cpuid_dep cpuid_deps[] = {
{ X86_FEATURE_CQM_OCCUP_LLC, X86_FEATURE_CQM_LLC },
{ X86_FEATURE_CQM_MBM_TOTAL, X86_FEATURE_CQM_LLC },
{ X86_FEATURE_CQM_MBM_LOCAL, X86_FEATURE_CQM_LLC },
{ X86_FEATURE_BMEC, X86_FEATURE_CQM_MBM_TOTAL },
{ X86_FEATURE_BMEC, X86_FEATURE_CQM_MBM_LOCAL },
{ X86_FEATURE_AVX512_BF16, X86_FEATURE_AVX512VL },
{ X86_FEATURE_AVX512_FP16, X86_FEATURE_AVX512BW },
{ X86_FEATURE_ENQCMD, X86_FEATURE_XSAVES },
arch/x86/kernel/cpu/mce/amd.c
+3 -9
View File
@@ -306,6 +306,8 @@ static void smca_configure(unsigned int bank, unsigned int cpu)
if ((low & BIT(5)) && !((high >> 5) & 0x3))
high |= BIT(5);
this_cpu_ptr(mce_banks_array)[bank].lsb_in_status = !!(low & BIT(8));
wrmsr(smca_config, low, high);
}
@@ -736,15 +738,7 @@ static void __log_error(unsigned int bank, u64 status, u64 addr, u64 misc)
if (m.status & MCI_STATUS_ADDRV) {
m.addr = addr;
/*
* Extract [55:<lsb>] where lsb is the least significant
* *valid* bit of the address bits.
*/
if (mce_flags.smca) {
u8 lsb = (m.addr >> 56) & 0x3f;
m.addr &= GENMASK_ULL(55, lsb);
}
smca_extract_err_addr(&m);
}
if (mce_flags.smca) {
arch/x86/kernel/cpu/mce/core.c
+11 -21
View File
@@ -67,13 +67,7 @@ DEFINE_PER_CPU(unsigned, mce_exception_count);
DEFINE_PER_CPU_READ_MOSTLY(unsigned int, mce_num_banks);
struct mce_bank {
u64 ctl; /* subevents to enable */
__u64 init : 1, /* initialise bank? */
__reserved_1 : 63;
};
static DEFINE_PER_CPU_READ_MOSTLY(struct mce_bank[MAX_NR_BANKS], mce_banks_array);
DEFINE_PER_CPU_READ_MOSTLY(struct mce_bank[MAX_NR_BANKS], mce_banks_array);
#define ATTR_LEN 16
/* One object for each MCE bank, shared by all CPUs */
@@ -579,7 +573,7 @@ static int uc_decode_notifier(struct notifier_block *nb, unsigned long val,
mce->severity != MCE_DEFERRED_SEVERITY)
return NOTIFY_DONE;
pfn = mce->addr >> PAGE_SHIFT;
pfn = (mce->addr & MCI_ADDR_PHYSADDR) >> PAGE_SHIFT;
if (!memory_failure(pfn, 0)) {
set_mce_nospec(pfn);
mce->kflags |= MCE_HANDLED_UC;
@@ -633,15 +627,7 @@ static noinstr void mce_read_aux(struct mce *m, int i)
m->addr <<= shift;
}
/*
* Extract [55:<lsb>] where lsb is the least significant
* *valid* bit of the address bits.
*/
if (mce_flags.smca) {
u8 lsb = (m->addr >> 56) & 0x3f;
m->addr &= GENMASK_ULL(55, lsb);
}
smca_extract_err_addr(m);
}
if (mce_flags.smca) {
@@ -1308,6 +1294,7 @@ static void kill_me_maybe(struct callback_head *cb)
{
struct task_struct *p = container_of(cb, struct task_struct, mce_kill_me);
int flags = MF_ACTION_REQUIRED;
unsigned long pfn;
int ret;
p->mce_count = 0;
@@ -1316,9 +1303,10 @@ static void kill_me_maybe(struct callback_head *cb)
if (!p->mce_ripv)
flags |= MF_MUST_KILL;
ret = memory_failure(p->mce_addr >> PAGE_SHIFT, flags);
pfn = (p->mce_addr & MCI_ADDR_PHYSADDR) >> PAGE_SHIFT;
ret = memory_failure(pfn, flags);
if (!ret) {
set_mce_nospec(p->mce_addr >> PAGE_SHIFT);
set_mce_nospec(pfn);
sync_core();
return;
}
@@ -1340,11 +1328,13 @@ static void kill_me_maybe(struct callback_head *cb)
static void kill_me_never(struct callback_head *cb)
{
struct task_struct *p = container_of(cb, struct task_struct, mce_kill_me);
unsigned long pfn;
p->mce_count = 0;
pr_err("Kernel accessed poison in user space at %llx\n", p->mce_addr);
if (!memory_failure(p->mce_addr >> PAGE_SHIFT, 0))
set_mce_nospec(p->mce_addr >> PAGE_SHIFT);
pfn = (p->mce_addr & MCI_ADDR_PHYSADDR) >> PAGE_SHIFT;
if (!memory_failure(pfn, 0))
set_mce_nospec(pfn);
}
static void queue_task_work(struct mce *m, char *msg, void (*func)(struct callback_head *))
arch/x86/kernel/cpu/mce/internal.h
+44
View File
@@ -177,6 +177,24 @@ struct mce_vendor_flags {
extern struct mce_vendor_flags mce_flags;
struct mce_bank {
/* subevents to enable */
u64 ctl;
/* initialise bank? */
__u64 init : 1,
/*
* (AMD) MCA_CONFIG[McaLsbInStatusSupported]: When set, this bit indicates
* the LSB field is found in MCA_STATUS and not in MCA_ADDR.
*/
lsb_in_status : 1,
__reserved_1 : 62;
};
DECLARE_PER_CPU_READ_MOSTLY(struct mce_bank[MAX_NR_BANKS], mce_banks_array);
enum mca_msr {
MCA_CTL,
MCA_STATUS,
@@ -189,8 +207,34 @@ extern bool filter_mce(struct mce *m);
#ifdef CONFIG_X86_MCE_AMD
extern bool amd_filter_mce(struct mce *m);
/*
* If MCA_CONFIG[McaLsbInStatusSupported] is set, extract ErrAddr in bits
* [56:0] of MCA_STATUS, else in bits [55:0] of MCA_ADDR.
*/
static __always_inline void smca_extract_err_addr(struct mce *m)
{
u8 lsb;
if (!mce_flags.smca)
return;
if (this_cpu_ptr(mce_banks_array)[m->bank].lsb_in_status) {
lsb = (m->status >> 24) & 0x3f;
m->addr &= GENMASK_ULL(56, lsb);
return;
}
lsb = (m->addr >> 56) & 0x3f;
m->addr &= GENMASK_ULL(55, lsb);
}
#else
static inline bool amd_filter_mce(struct mce *m) { return false; }
static inline void smca_extract_err_addr(struct mce *m) { }
#endif
#ifdef CONFIG_X86_ANCIENT_MCE
arch/x86/kernel/cpu/microcode/amd.c
+37 -39
View File
@@ -55,7 +55,9 @@ struct cont_desc {
};
static u32 ucode_new_rev;
static u8 amd_ucode_patch[PATCH_MAX_SIZE];
/* One blob per node. */
static u8 amd_ucode_patch[MAX_NUMNODES][PATCH_MAX_SIZE];
/*
* Microcode patch container file is prepended to the initrd in cpio
@@ -330,8 +332,9 @@ static size_t parse_container(u8 *ucode, size_t size, struct cont_desc *desc)
ret = verify_patch(x86_family(desc->cpuid_1_eax), buf, size, &patch_size, true);
if (ret < 0) {
/*
* Patch verification failed, skip to the next
* container, if there's one:
* Patch verification failed, skip to the next container, if
* there is one. Before exit, check whether that container has
* found a patch already. If so, use it.
*/
goto out;
} else if (ret > 0) {
@@ -350,6 +353,7 @@ skip:
size -= patch_size + SECTION_HDR_SIZE;
}
out:
/*
* If we have found a patch (desc->mc), it means we're looking at the
* container which has a patch for this CPU so return 0 to mean, @ucode
@@ -364,7 +368,6 @@ skip:
return 0;
}
out:
return orig_size - size;
}
@@ -414,8 +417,7 @@ static int __apply_microcode_amd(struct microcode_amd *mc)
*
* Returns true if container found (sets @desc), false otherwise.
*/
static bool
apply_microcode_early_amd(u32 cpuid_1_eax, void *ucode, size_t size, bool save_patch)
static bool early_apply_microcode(u32 cpuid_1_eax, void *ucode, size_t size, bool save_patch)
{
struct cont_desc desc = { 0 };
u8 (*patch)[PATCH_MAX_SIZE];
@@ -428,7 +430,7 @@ apply_microcode_early_amd(u32 cpuid_1_eax, void *ucode, size_t size, bool save_p
patch = (u8 (*)[PATCH_MAX_SIZE])__pa_nodebug(&amd_ucode_patch);
#else
new_rev = &ucode_new_rev;
patch = &amd_ucode_patch;
patch = &amd_ucode_patch[0];
#endif
desc.cpuid_1_eax = cpuid_1_eax;
@@ -481,7 +483,7 @@ static bool get_builtin_microcode(struct cpio_data *cp, unsigned int family)
return false;
}
static void __load_ucode_amd(unsigned int cpuid_1_eax, struct cpio_data *ret)
static void find_blobs_in_containers(unsigned int cpuid_1_eax, struct cpio_data *ret)
{
struct ucode_cpu_info *uci;
struct cpio_data cp;
@@ -511,11 +513,11 @@ void __init load_ucode_amd_bsp(unsigned int cpuid_1_eax)
{
struct cpio_data cp = { };
__load_ucode_amd(cpuid_1_eax, &cp);
find_blobs_in_containers(cpuid_1_eax, &cp);
if (!(cp.data && cp.size))
return;
apply_microcode_early_amd(cpuid_1_eax, cp.data, cp.size, true);
early_apply_microcode(cpuid_1_eax, cp.data, cp.size, true);
}
void load_ucode_amd_ap(unsigned int cpuid_1_eax)
@@ -546,15 +548,14 @@ void load_ucode_amd_ap(unsigned int cpuid_1_eax)
}
}
__load_ucode_amd(cpuid_1_eax, &cp);
find_blobs_in_containers(cpuid_1_eax, &cp);
if (!(cp.data && cp.size))
return;
apply_microcode_early_amd(cpuid_1_eax, cp.data, cp.size, false);
early_apply_microcode(cpuid_1_eax, cp.data, cp.size, false);
}
static enum ucode_state
load_microcode_amd(bool save, u8 family, const u8 *data, size_t size);
static enum ucode_state load_microcode_amd(u8 family, const u8 *data, size_t size);
int __init save_microcode_in_initrd_amd(unsigned int cpuid_1_eax)
{
@@ -572,19 +573,19 @@ int __init save_microcode_in_initrd_amd(unsigned int cpuid_1_eax)
if (!desc.mc)
return -EINVAL;
ret = load_microcode_amd(true, x86_family(cpuid_1_eax), desc.data, desc.size);
ret = load_microcode_amd(x86_family(cpuid_1_eax), desc.data, desc.size);
if (ret > UCODE_UPDATED)
return -EINVAL;
return 0;
}
void reload_ucode_amd(void)
void reload_ucode_amd(unsigned int cpu)
{
struct microcode_amd *mc;
u32 rev, dummy __always_unused;
struct microcode_amd *mc;
mc = (struct microcode_amd *)amd_ucode_patch;
mc = (struct microcode_amd *)amd_ucode_patch[cpu_to_node(cpu)];
rdmsr(MSR_AMD64_PATCH_LEVEL, rev, dummy);
@@ -816,6 +817,7 @@ static int verify_and_add_patch(u8 family, u8 *fw, unsigned int leftover,
return 0;
}
/* Scan the blob in @data and add microcode patches to the cache. */
static enum ucode_state __load_microcode_amd(u8 family, const u8 *data,
size_t size)
{
@@ -850,9 +852,10 @@ static enum ucode_state __load_microcode_amd(u8 family, const u8 *data,
return UCODE_OK;
}
static enum ucode_state
load_microcode_amd(bool save, u8 family, const u8 *data, size_t size)
static enum ucode_state load_microcode_amd(u8 family, const u8 *data, size_t size)
{
struct cpuinfo_x86 *c;
unsigned int nid, cpu;
struct ucode_patch *p;
enum ucode_state ret;
@@ -865,23 +868,23 @@ load_microcode_amd(bool save, u8 family, const u8 *data, size_t size)
return ret;
}
p = find_patch(0);
if (!p) {
return ret;
} else {
if (boot_cpu_data.microcode >= p->patch_id)
return ret;
for_each_node(nid) {
cpu = cpumask_first(cpumask_of_node(nid));
c = &cpu_data(cpu);
p = find_patch(cpu);
if (!p)
continue;
if (c->microcode >= p->patch_id)
continue;
ret = UCODE_NEW;
memset(&amd_ucode_patch[nid], 0, PATCH_MAX_SIZE);
memcpy(&amd_ucode_patch[nid], p->data, min_t(u32, p->size, PATCH_MAX_SIZE));
}
/* save BSP's matching patch for early load */
if (!save)
return ret;
memset(amd_ucode_patch, 0, PATCH_MAX_SIZE);
memcpy(amd_ucode_patch, p->data, min_t(u32, p->size, PATCH_MAX_SIZE));
return ret;
}
@@ -905,14 +908,9 @@ static enum ucode_state request_microcode_amd(int cpu, struct device *device)
{
char fw_name[36] = "amd-ucode/microcode_amd.bin";
struct cpuinfo_x86 *c = &cpu_data(cpu);
bool bsp = c->cpu_index == boot_cpu_data.cpu_index;
enum ucode_state ret = UCODE_NFOUND;
const struct firmware *fw;
/* reload ucode container only on the boot cpu */
if (!bsp)
return UCODE_OK;
if (c->x86 >= 0x15)
snprintf(fw_name, sizeof(fw_name), "amd-ucode/microcode_amd_fam%.2xh.bin", c->x86);
@@ -925,7 +923,7 @@ static enum ucode_state request_microcode_amd(int cpu, struct device *device)
if (!verify_container(fw->data, fw->size, false))
goto fw_release;
ret = load_microcode_amd(bsp, c->x86, fw->data, fw->size);
ret = load_microcode_amd(c->x86, fw->data, fw->size);
fw_release:
release_firmware(fw);
arch/x86/kernel/cpu/microcode/core.c
+26 -19
View File
@@ -298,7 +298,7 @@ struct cpio_data find_microcode_in_initrd(const char *path, bool use_pa)
#endif
}
void reload_early_microcode(void)
void reload_early_microcode(unsigned int cpu)
{
int vendor, family;
@@ -312,7 +312,7 @@ void reload_early_microcode(void)
break;
case X86_VENDOR_AMD:
if (family >= 0x10)
reload_ucode_amd();
reload_ucode_amd(cpu);
break;
default:
break;
@@ -409,10 +409,10 @@ static int __reload_late(void *info)
goto wait_for_siblings;
if (err >= UCODE_NFOUND) {
if (err == UCODE_ERROR)
if (err == UCODE_ERROR) {
pr_warn("Error reloading microcode on CPU %d\n", cpu);
ret = -1;
ret = -1;
}
}
wait_for_siblings:
@@ -438,6 +438,7 @@ wait_for_siblings:
static int microcode_reload_late(void)
{
int old = boot_cpu_data.microcode, ret;
struct cpuinfo_x86 prev_info;
pr_err("Attempting late microcode loading - it is dangerous and taints the kernel.\n");
pr_err("You should switch to early loading, if possible.\n");
@@ -445,12 +446,21 @@ static int microcode_reload_late(void)
atomic_set(&late_cpus_in, 0);
atomic_set(&late_cpus_out, 0);
ret = stop_machine_cpuslocked(__reload_late, NULL, cpu_online_mask);
if (ret == 0)
microcode_check();
/*
* Take a snapshot before the microcode update in order to compare and
* check whether any bits changed after an update.
*/
store_cpu_caps(&prev_info);
pr_info("Reload completed, microcode revision: 0x%x -> 0x%x\n",
old, boot_cpu_data.microcode);
ret = stop_machine_cpuslocked(__reload_late, NULL, cpu_online_mask);
if (!ret) {
pr_info("Reload succeeded, microcode revision: 0x%x -> 0x%x\n",
old, boot_cpu_data.microcode);
microcode_check(&prev_info);
} else {
pr_info("Reload failed, current microcode revision: 0x%x\n",
boot_cpu_data.microcode);
}
return ret;
}
@@ -465,11 +475,8 @@ static ssize_t reload_store(struct device *dev,
ssize_t ret = 0;
ret = kstrtoul(buf, 0, &val);
if (ret)
return ret;
if (val != 1)
return size;
if (ret || val != 1)
return -EINVAL;
cpus_read_lock();
@@ -507,7 +514,7 @@ static ssize_t version_show(struct device *dev,
return sprintf(buf, "0x%x\n", uci->cpu_sig.rev);
}
static ssize_t pf_show(struct device *dev,
static ssize_t processor_flags_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct ucode_cpu_info *uci = ucode_cpu_info + dev->id;
@@ -515,8 +522,8 @@ static ssize_t pf_show(struct device *dev,
return sprintf(buf, "0x%x\n", uci->cpu_sig.pf);
}
static DEVICE_ATTR(version, 0444, version_show, NULL);
static DEVICE_ATTR(processor_flags, 0444, pf_show, NULL);
static DEVICE_ATTR_RO(version);
static DEVICE_ATTR_RO(processor_flags);
static struct attribute *mc_default_attrs[] = {
&dev_attr_version.attr,
@@ -557,7 +564,7 @@ void microcode_bsp_resume(void)
if (uci->mc)
microcode_ops->apply_microcode(cpu);
else
reload_early_microcode();
reload_early_microcode(cpu);
}
static struct syscore_ops mc_syscore_ops = {
arch/x86/kernel/cpu/microcode/intel.c
+18 -26
View File
@@ -305,14 +305,11 @@ static bool load_builtin_intel_microcode(struct cpio_data *cp)
return false;
}
/*
* Print ucode update info.
*/
static void
print_ucode_info(struct ucode_cpu_info *uci, unsigned int date)
static void print_ucode_info(int old_rev, int new_rev, unsigned int date)
{
pr_info_once("microcode updated early to revision 0x%x, date = %04x-%02x-%02x\n",
uci->cpu_sig.rev,
pr_info_once("updated early: 0x%x -> 0x%x, date = %04x-%02x-%02x\n",
old_rev,
new_rev,
date & 0xffff,
date >> 24,
(date >> 16) & 0xff);
@@ -322,6 +319,7 @@ print_ucode_info(struct ucode_cpu_info *uci, unsigned int date)
static int delay_ucode_info;
static int current_mc_date;
static int early_old_rev;
/*
* Print early updated ucode info after printk works. This is delayed info dump.
@@ -332,7 +330,7 @@ void show_ucode_info_early(void)
if (delay_ucode_info) {
intel_cpu_collect_info(&uci);
print_ucode_info(&uci, current_mc_date);
print_ucode_info(early_old_rev, uci.cpu_sig.rev, current_mc_date);
delay_ucode_info = 0;
}
}
@@ -341,40 +339,32 @@ void show_ucode_info_early(void)
* At this point, we can not call printk() yet. Delay printing microcode info in
* show_ucode_info_early() until printk() works.
*/
static void print_ucode(struct ucode_cpu_info *uci)
static void print_ucode(int old_rev, int new_rev, int date)
{
struct microcode_intel *mc;
int *delay_ucode_info_p;
int *current_mc_date_p;
mc = uci->mc;
if (!mc)
return;
int *early_old_rev_p;
delay_ucode_info_p = (int *)__pa_nodebug(&delay_ucode_info);
current_mc_date_p = (int *)__pa_nodebug(&current_mc_date);
early_old_rev_p = (int *)__pa_nodebug(&early_old_rev);
*delay_ucode_info_p = 1;
*current_mc_date_p = mc->hdr.date;
*current_mc_date_p = date;
*early_old_rev_p = old_rev;
}
#else
static inline void print_ucode(struct ucode_cpu_info *uci)
static inline void print_ucode(int old_rev, int new_rev, int date)
{
struct microcode_intel *mc;
mc = uci->mc;
if (!mc)
return;
print_ucode_info(uci, mc->hdr.date);
print_ucode_info(old_rev, new_rev, date);
}
#endif
static int apply_microcode_early(struct ucode_cpu_info *uci, bool early)
{
struct microcode_intel *mc;
u32 rev;
u32 rev, old_rev;
mc = uci->mc;
if (!mc)
@@ -391,6 +381,8 @@ static int apply_microcode_early(struct ucode_cpu_info *uci, bool early)
return UCODE_OK;
}
old_rev = rev;
/*
* Writeback and invalidate caches before updating microcode to avoid
* internal issues depending on what the microcode is updating.
@@ -407,9 +399,9 @@ static int apply_microcode_early(struct ucode_cpu_info *uci, bool early)
uci->cpu_sig.rev = rev;
if (early)
print_ucode(uci);
print_ucode(old_rev, uci->cpu_sig.rev, mc->hdr.date);
else
print_ucode_info(uci, mc->hdr.date);
print_ucode_info(old_rev, uci->cpu_sig.rev, mc->hdr.date);
return 0;
}
arch/x86/kernel/cpu/resctrl/core.c
+51 -3
View File
@@ -100,6 +100,18 @@ struct rdt_hw_resource rdt_resources_all[] = {
.fflags = RFTYPE_RES_MB,
},
},
[RDT_RESOURCE_SMBA] =
{
.r_resctrl = {
.rid = RDT_RESOURCE_SMBA,
.name = "SMBA",
.cache_level = 3,
.domains = domain_init(RDT_RESOURCE_SMBA),
.parse_ctrlval = parse_bw,
.format_str = "%d=%*u",
.fflags = RFTYPE_RES_MB,
},
},
};
/*
@@ -150,6 +162,13 @@ bool is_mba_sc(struct rdt_resource *r)
if (!r)
return rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl.membw.mba_sc;
/*
* The software controller support is only applicable to MBA resource.
* Make sure to check for resource type.
*/
if (r->rid != RDT_RESOURCE_MBA)
return false;
return r->membw.mba_sc;
}
@@ -213,9 +232,15 @@ static bool __rdt_get_mem_config_amd(struct rdt_resource *r)
struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
union cpuid_0x10_3_eax eax;
union cpuid_0x10_x_edx edx;
u32 ebx, ecx;
u32 ebx, ecx, subleaf;
cpuid_count(0x80000020, 1, &eax.full, &ebx, &ecx, &edx.full);
/*
* Query CPUID_Fn80000020_EDX_x01 for MBA and
* CPUID_Fn80000020_EDX_x02 for SMBA
*/
subleaf = (r->rid == RDT_RESOURCE_SMBA) ? 2 : 1;
cpuid_count(0x80000020, subleaf, &eax.full, &ebx, &ecx, &edx.full);
hw_res->num_closid = edx.split.cos_max + 1;
r->default_ctrl = MAX_MBA_BW_AMD;
@@ -647,6 +672,8 @@ enum {
RDT_FLAG_L2_CAT,
RDT_FLAG_L2_CDP,
RDT_FLAG_MBA,
RDT_FLAG_SMBA,
RDT_FLAG_BMEC,
};
#define RDT_OPT(idx, n, f) \
@@ -670,6 +697,8 @@ static struct rdt_options rdt_options[] __initdata = {
RDT_OPT(RDT_FLAG_L2_CAT, "l2cat", X86_FEATURE_CAT_L2),
RDT_OPT(RDT_FLAG_L2_CDP, "l2cdp", X86_FEATURE_CDP_L2),
RDT_OPT(RDT_FLAG_MBA, "mba", X86_FEATURE_MBA),
RDT_OPT(RDT_FLAG_SMBA, "smba", X86_FEATURE_SMBA),
RDT_OPT(RDT_FLAG_BMEC, "bmec", X86_FEATURE_BMEC),
};
#define NUM_RDT_OPTIONS ARRAY_SIZE(rdt_options)
@@ -699,7 +728,7 @@ static int __init set_rdt_options(char *str)
}
__setup("rdt", set_rdt_options);
static bool __init rdt_cpu_has(int flag)
bool __init rdt_cpu_has(int flag)
{
bool ret = boot_cpu_has(flag);
struct rdt_options *o;
@@ -734,6 +763,19 @@ static __init bool get_mem_config(void)
return false;
}
static __init bool get_slow_mem_config(void)
{
struct rdt_hw_resource *hw_res = &rdt_resources_all[RDT_RESOURCE_SMBA];
if (!rdt_cpu_has(X86_FEATURE_SMBA))
return false;
if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD)
return __rdt_get_mem_config_amd(&hw_res->r_resctrl);
return false;
}
static __init bool get_rdt_alloc_resources(void)
{
struct rdt_resource *r;
@@ -764,6 +806,9 @@ static __init bool get_rdt_alloc_resources(void)
if (get_mem_config())
ret = true;
if (get_slow_mem_config())
ret = true;
return ret;
}
@@ -853,6 +898,9 @@ static __init void rdt_init_res_defs_amd(void)
} else if (r->rid == RDT_RESOURCE_MBA) {
hw_res->msr_base = MSR_IA32_MBA_BW_BASE;
hw_res->msr_update = mba_wrmsr_amd;
} else if (r->rid == RDT_RESOURCE_SMBA) {
hw_res->msr_base = MSR_IA32_SMBA_BW_BASE;
hw_res->msr_update = mba_wrmsr_amd;
}
}
}
arch/x86/kernel/cpu/resctrl/ctrlmondata.c
+4 -9
View File
@@ -209,7 +209,7 @@ static int parse_line(char *line, struct resctrl_schema *s,
unsigned long dom_id;
if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP &&
r->rid == RDT_RESOURCE_MBA) {
(r->rid == RDT_RESOURCE_MBA || r->rid == RDT_RESOURCE_SMBA)) {
rdt_last_cmd_puts("Cannot pseudo-lock MBA resource\n");
return -EINVAL;
}
@@ -310,7 +310,6 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)
enum resctrl_conf_type t;
cpumask_var_t cpu_mask;
struct rdt_domain *d;
int cpu;
u32 idx;
if (!zalloc_cpumask_var(&cpu_mask, GFP_KERNEL))
@@ -341,13 +340,9 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)
if (cpumask_empty(cpu_mask))
goto done;
cpu = get_cpu();
/* Update resource control msr on this CPU if it's in cpu_mask. */
if (cpumask_test_cpu(cpu, cpu_mask))
rdt_ctrl_update(&msr_param);
/* Update resource control msr on other CPUs. */
smp_call_function_many(cpu_mask, rdt_ctrl_update, &msr_param, 1);
put_cpu();
/* Update resource control msr on all the CPUs. */
on_each_cpu_mask(cpu_mask, rdt_ctrl_update, &msr_param, 1);
done:
free_cpumask_var(cpu_mask);
arch/x86/kernel/cpu/resctrl/internal.h
+28
View File
@@ -30,6 +30,29 @@
*/
#define MBM_CNTR_WIDTH_OFFSET_MAX (62 - MBM_CNTR_WIDTH_BASE)
/* Reads to Local DRAM Memory */
#define READS_TO_LOCAL_MEM BIT(0)
/* Reads to Remote DRAM Memory */
#define READS_TO_REMOTE_MEM BIT(1)
/* Non-Temporal Writes to Local Memory */
#define NON_TEMP_WRITE_TO_LOCAL_MEM BIT(2)
/* Non-Temporal Writes to Remote Memory */
#define NON_TEMP_WRITE_TO_REMOTE_MEM BIT(3)
/* Reads to Local Memory the system identifies as "Slow Memory" */
#define READS_TO_LOCAL_S_MEM BIT(4)
/* Reads to Remote Memory the system identifies as "Slow Memory" */
#define READS_TO_REMOTE_S_MEM BIT(5)
/* Dirty Victims to All Types of Memory */
#define DIRTY_VICTIMS_TO_ALL_MEM BIT(6)
/* Max event bits supported */
#define MAX_EVT_CONFIG_BITS GENMASK(6, 0)
struct rdt_fs_context {
struct kernfs_fs_context kfc;
@@ -52,11 +75,13 @@ DECLARE_STATIC_KEY_FALSE(rdt_mon_enable_key);
* struct mon_evt - Entry in the event list of a resource
* @evtid: event id
* @name: name of the event
* @configurable: true if the event is configurable
* @list: entry in &rdt_resource->evt_list
*/
struct mon_evt {
enum resctrl_event_id evtid;
char *name;
bool configurable;
struct list_head list;
};
@@ -409,6 +434,7 @@ enum resctrl_res_level {
RDT_RESOURCE_L3,
RDT_RESOURCE_L2,
RDT_RESOURCE_MBA,
RDT_RESOURCE_SMBA,
/* Must be the last */
RDT_NUM_RESOURCES,
@@ -511,6 +537,7 @@ void closid_free(int closid);
int alloc_rmid(void);
void free_rmid(u32 rmid);
int rdt_get_mon_l3_config(struct rdt_resource *r);
bool __init rdt_cpu_has(int flag);
void mon_event_count(void *info);
int rdtgroup_mondata_show(struct seq_file *m, void *arg);
void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
@@ -527,5 +554,6 @@ bool has_busy_rmid(struct rdt_resource *r, struct rdt_domain *d);
void __check_limbo(struct rdt_domain *d, bool force_free);
void rdt_domain_reconfigure_cdp(struct rdt_resource *r);
void __init thread_throttle_mode_init(void);
void __init mbm_config_rftype_init(const char *config);
#endif /* _ASM_X86_RESCTRL_INTERNAL_H */
arch/x86/kernel/cpu/resctrl/monitor.c
+29 -1
View File
@@ -204,6 +204,23 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_domain *d,
}
}
/*
* Assumes that hardware counters are also reset and thus that there is
* no need to record initial non-zero counts.
*/
void resctrl_arch_reset_rmid_all(struct rdt_resource *r, struct rdt_domain *d)
{
struct rdt_hw_domain *hw_dom = resctrl_to_arch_dom(d);
if (is_mbm_total_enabled())
memset(hw_dom->arch_mbm_total, 0,
sizeof(*hw_dom->arch_mbm_total) * r->num_rmid);
if (is_mbm_local_enabled())
memset(hw_dom->arch_mbm_local, 0,
sizeof(*hw_dom->arch_mbm_local) * r->num_rmid);
}
static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width)
{
u64 shift = 64 - width, chunks;
@@ -763,7 +780,7 @@ static void l3_mon_evt_init(struct rdt_resource *r)
list_add_tail(&mbm_local_event.list, &r->evt_list);
}
int rdt_get_mon_l3_config(struct rdt_resource *r)
int __init rdt_get_mon_l3_config(struct rdt_resource *r)
{
unsigned int mbm_offset = boot_cpu_data.x86_cache_mbm_width_offset;
struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
@@ -800,6 +817,17 @@ int rdt_get_mon_l3_config(struct rdt_resource *r)
if (ret)
return ret;
if (rdt_cpu_has(X86_FEATURE_BMEC)) {
if (rdt_cpu_has(X86_FEATURE_CQM_MBM_TOTAL)) {
mbm_total_event.configurable = true;
mbm_config_rftype_init("mbm_total_bytes_config");
}
if (rdt_cpu_has(X86_FEATURE_CQM_MBM_LOCAL)) {
mbm_local_event.configurable = true;
mbm_config_rftype_init("mbm_local_bytes_config");
}
}
l3_mon_evt_init(r);
r->mon_capable = true;
arch/x86/kernel/cpu/resctrl/rdtgroup.c
+282 -25
View File
@@ -325,12 +325,7 @@ static void update_cpu_closid_rmid(void *info)
static void
update_closid_rmid(const struct cpumask *cpu_mask, struct rdtgroup *r)
{
int cpu = get_cpu();
if (cpumask_test_cpu(cpu, cpu_mask))
update_cpu_closid_rmid(r);
smp_call_function_many(cpu_mask, update_cpu_closid_rmid, r, 1);
put_cpu();
on_each_cpu_mask(cpu_mask, update_cpu_closid_rmid, r, 1);
}
static int cpus_mon_write(struct rdtgroup *rdtgrp, cpumask_var_t newmask,
@@ -1003,8 +998,11 @@ static int rdt_mon_features_show(struct kernfs_open_file *of,
struct rdt_resource *r = of->kn->parent->priv;
struct mon_evt *mevt;
list_for_each_entry(mevt, &r->evt_list, list)
list_for_each_entry(mevt, &r->evt_list, list) {
seq_printf(seq, "%s\n", mevt->name);
if (mevt->configurable)
seq_printf(seq, "%s_config\n", mevt->name);
}
return 0;
}
@@ -1215,7 +1213,7 @@ static bool rdtgroup_mode_test_exclusive(struct rdtgroup *rdtgrp)
list_for_each_entry(s, &resctrl_schema_all, list) {
r = s->res;
if (r->rid == RDT_RESOURCE_MBA)
if (r->rid == RDT_RESOURCE_MBA || r->rid == RDT_RESOURCE_SMBA)
continue;
has_cache = true;
list_for_each_entry(d, &r->domains, list) {
@@ -1404,7 +1402,8 @@ static int rdtgroup_size_show(struct kernfs_open_file *of,
ctrl = resctrl_arch_get_config(r, d,
closid,
type);
if (r->rid == RDT_RESOURCE_MBA)
if (r->rid == RDT_RESOURCE_MBA ||
r->rid == RDT_RESOURCE_SMBA)
size = ctrl;
else
size = rdtgroup_cbm_to_size(r, d, ctrl);
@@ -1421,6 +1420,248 @@ out:
return ret;
}
struct mon_config_info {
u32 evtid;
u32 mon_config;
};
#define INVALID_CONFIG_INDEX UINT_MAX
/**
* mon_event_config_index_get - get the hardware index for the
* configurable event
* @evtid: event id.
*
* Return: 0 for evtid == QOS_L3_MBM_TOTAL_EVENT_ID
* 1 for evtid == QOS_L3_MBM_LOCAL_EVENT_ID
* INVALID_CONFIG_INDEX for invalid evtid
*/
static inline unsigned int mon_event_config_index_get(u32 evtid)
{
switch (evtid) {
case QOS_L3_MBM_TOTAL_EVENT_ID:
return 0;
case QOS_L3_MBM_LOCAL_EVENT_ID:
return 1;
default:
/* Should never reach here */
return INVALID_CONFIG_INDEX;
}
}
static void mon_event_config_read(void *info)
{
struct mon_config_info *mon_info = info;
unsigned int index;
u64 msrval;
index = mon_event_config_index_get(mon_info->evtid);
if (index == INVALID_CONFIG_INDEX) {
pr_warn_once("Invalid event id %d\n", mon_info->evtid);
return;
}
rdmsrl(MSR_IA32_EVT_CFG_BASE + index, msrval);
/* Report only the valid event configuration bits */
mon_info->mon_config = msrval & MAX_EVT_CONFIG_BITS;
}
static void mondata_config_read(struct rdt_domain *d, struct mon_config_info *mon_info)
{
smp_call_function_any(&d->cpu_mask, mon_event_config_read, mon_info, 1);
}
static int mbm_config_show(struct seq_file *s, struct rdt_resource *r, u32 evtid)
{
struct mon_config_info mon_info = {0};
struct rdt_domain *dom;
bool sep = false;
mutex_lock(&rdtgroup_mutex);
list_for_each_entry(dom, &r->domains, list) {
if (sep)
seq_puts(s, ";");
memset(&mon_info, 0, sizeof(struct mon_config_info));
mon_info.evtid = evtid;
mondata_config_read(dom, &mon_info);
seq_printf(s, "%d=0x%02x", dom->id, mon_info.mon_config);
sep = true;
}
seq_puts(s, "\n");
mutex_unlock(&rdtgroup_mutex);
return 0;
}
static int mbm_total_bytes_config_show(struct kernfs_open_file *of,
struct seq_file *seq, void *v)
{
struct rdt_resource *r = of->kn->parent->priv;
mbm_config_show(seq, r, QOS_L3_MBM_TOTAL_EVENT_ID);
return 0;
}
static int mbm_local_bytes_config_show(struct kernfs_open_file *of,
struct seq_file *seq, void *v)
{
struct rdt_resource *r = of->kn->parent->priv;
mbm_config_show(seq, r, QOS_L3_MBM_LOCAL_EVENT_ID);
return 0;
}
static void mon_event_config_write(void *info)
{
struct mon_config_info *mon_info = info;
unsigned int index;
index = mon_event_config_index_get(mon_info->evtid);
if (index == INVALID_CONFIG_INDEX) {
pr_warn_once("Invalid event id %d\n", mon_info->evtid);
return;
}
wrmsr(MSR_IA32_EVT_CFG_BASE + index, mon_info->mon_config, 0);
}
static int mbm_config_write_domain(struct rdt_resource *r,
struct rdt_domain *d, u32 evtid, u32 val)
{
struct mon_config_info mon_info = {0};
int ret = 0;
/* mon_config cannot be more than the supported set of events */
if (val > MAX_EVT_CONFIG_BITS) {
rdt_last_cmd_puts("Invalid event configuration\n");
return -EINVAL;
}
/*
* Read the current config value first. If both are the same then
* no need to write it again.
*/
mon_info.evtid = evtid;
mondata_config_read(d, &mon_info);
if (mon_info.mon_config == val)
goto out;
mon_info.mon_config = val;
/*
* Update MSR_IA32_EVT_CFG_BASE MSR on one of the CPUs in the
* domain. The MSRs offset from MSR MSR_IA32_EVT_CFG_BASE
* are scoped at the domain level. Writing any of these MSRs
* on one CPU is observed by all the CPUs in the domain.
*/
smp_call_function_any(&d->cpu_mask, mon_event_config_write,
&mon_info, 1);
/*
* When an Event Configuration is changed, the bandwidth counters
* for all RMIDs and Events will be cleared by the hardware. The
* hardware also sets MSR_IA32_QM_CTR.Unavailable (bit 62) for
* every RMID on the next read to any event for every RMID.
* Subsequent reads will have MSR_IA32_QM_CTR.Unavailable (bit 62)
* cleared while it is tracked by the hardware. Clear the
* mbm_local and mbm_total counts for all the RMIDs.
*/
resctrl_arch_reset_rmid_all(r, d);
out:
return ret;
}
static int mon_config_write(struct rdt_resource *r, char *tok, u32 evtid)
{
char *dom_str = NULL, *id_str;
unsigned long dom_id, val;
struct rdt_domain *d;
int ret = 0;
next:
if (!tok || tok[0] == '\0')
return 0;
/* Start processing the strings for each domain */
dom_str = strim(strsep(&tok, ";"));
id_str = strsep(&dom_str, "=");
if (!id_str || kstrtoul(id_str, 10, &dom_id)) {
rdt_last_cmd_puts("Missing '=' or non-numeric domain id\n");
return -EINVAL;
}
if (!dom_str || kstrtoul(dom_str, 16, &val)) {
rdt_last_cmd_puts("Non-numeric event configuration value\n");
return -EINVAL;
}
list_for_each_entry(d, &r->domains, list) {
if (d->id == dom_id) {
ret = mbm_config_write_domain(r, d, evtid, val);
if (ret)
return -EINVAL;
goto next;
}
}
return -EINVAL;
}
static ssize_t mbm_total_bytes_config_write(struct kernfs_open_file *of,
char *buf, size_t nbytes,
loff_t off)
{
struct rdt_resource *r = of->kn->parent->priv;
int ret;
/* Valid input requires a trailing newline */
if (nbytes == 0 || buf[nbytes - 1] != '\n')
return -EINVAL;
mutex_lock(&rdtgroup_mutex);
rdt_last_cmd_clear();
buf[nbytes - 1] = '\0';
ret = mon_config_write(r, buf, QOS_L3_MBM_TOTAL_EVENT_ID);
mutex_unlock(&rdtgroup_mutex);
return ret ?: nbytes;
}
static ssize_t mbm_local_bytes_config_write(struct kernfs_open_file *of,
char *buf, size_t nbytes,
loff_t off)
{
struct rdt_resource *r = of->kn->parent->priv;
int ret;
/* Valid input requires a trailing newline */
if (nbytes == 0 || buf[nbytes - 1] != '\n')
return -EINVAL;
mutex_lock(&rdtgroup_mutex);
rdt_last_cmd_clear();
buf[nbytes - 1] = '\0';
ret = mon_config_write(r, buf, QOS_L3_MBM_LOCAL_EVENT_ID);
mutex_unlock(&rdtgroup_mutex);
return ret ?: nbytes;
}
/* rdtgroup information files for one cache resource. */
static struct rftype res_common_files[] = {
{
@@ -1519,6 +1760,20 @@ static struct rftype res_common_files[] = {
.seq_show = max_threshold_occ_show,
.fflags = RF_MON_INFO | RFTYPE_RES_CACHE,
},
{
.name = "mbm_total_bytes_config",
.mode = 0644,
.kf_ops = &rdtgroup_kf_single_ops,
.seq_show = mbm_total_bytes_config_show,
.write = mbm_total_bytes_config_write,
},
{
.name = "mbm_local_bytes_config",
.mode = 0644,
.kf_ops = &rdtgroup_kf_single_ops,
.seq_show = mbm_local_bytes_config_show,
.write = mbm_local_bytes_config_write,
},
{
.name = "cpus",
.mode = 0644,
@@ -1625,6 +1880,15 @@ void __init thread_throttle_mode_init(void)
rft->fflags = RF_CTRL_INFO | RFTYPE_RES_MB;
}
void __init mbm_config_rftype_init(const char *config)
{
struct rftype *rft;
rft = rdtgroup_get_rftype_by_name(config);
if (rft)
rft->fflags = RF_MON_INFO | RFTYPE_RES_CACHE;
}
/**
* rdtgroup_kn_mode_restrict - Restrict user access to named resctrl file
* @r: The resource group with which the file is associated.
@@ -1866,13 +2130,9 @@ static int set_cache_qos_cfg(int level, bool enable)
/* Pick one CPU from each domain instance to update MSR */
cpumask_set_cpu(cpumask_any(&d->cpu_mask), cpu_mask);
}
cpu = get_cpu();
/* Update QOS_CFG MSR on this cpu if it's in cpu_mask. */
if (cpumask_test_cpu(cpu, cpu_mask))
update(&enable);
/* Update QOS_CFG MSR on all other cpus in cpu_mask. */
smp_call_function_many(cpu_mask, update, &enable, 1);
put_cpu();
/* Update QOS_CFG MSR on all the CPUs in cpu_mask */
on_each_cpu_mask(cpu_mask, update, &enable, 1);
free_cpumask_var(cpu_mask);
@@ -2349,7 +2609,7 @@ static int reset_all_ctrls(struct rdt_resource *r)
struct msr_param msr_param;
cpumask_var_t cpu_mask;
struct rdt_domain *d;
int i, cpu;
int i;
if (!zalloc_cpumask_var(&cpu_mask, GFP_KERNEL))
return -ENOMEM;
@@ -2370,13 +2630,9 @@ static int reset_all_ctrls(struct rdt_resource *r)
for (i = 0; i < hw_res->num_closid; i++)
hw_dom->ctrl_val[i] = r->default_ctrl;
}
cpu = get_cpu();
/* Update CBM on this cpu if it's in cpu_mask. */
if (cpumask_test_cpu(cpu, cpu_mask))
rdt_ctrl_update(&msr_param);
/* Update CBM on all other cpus in cpu_mask. */
smp_call_function_many(cpu_mask, rdt_ctrl_update, &msr_param, 1);
put_cpu();
/* Update CBM on all the CPUs in cpu_mask */
on_each_cpu_mask(cpu_mask, rdt_ctrl_update, &msr_param, 1);
free_cpumask_var(cpu_mask);
@@ -2855,7 +3111,8 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)
list_for_each_entry(s, &resctrl_schema_all, list) {
r = s->res;
if (r->rid == RDT_RESOURCE_MBA) {
if (r->rid == RDT_RESOURCE_MBA ||
r->rid == RDT_RESOURCE_SMBA) {
rdtgroup_init_mba(r, rdtgrp->closid);
if (is_mba_sc(r))
continue;
arch/x86/kernel/cpu/scattered.c
+2
View File
@@ -45,6 +45,8 @@ static const struct cpuid_bit cpuid_bits[] = {
{ X86_FEATURE_CPB, CPUID_EDX, 9, 0x80000007, 0 },
{ X86_FEATURE_PROC_FEEDBACK, CPUID_EDX, 11, 0x80000007, 0 },
{ X86_FEATURE_MBA, CPUID_EBX, 6, 0x80000008, 0 },
{ X86_FEATURE_SMBA, CPUID_EBX, 2, 0x80000020, 0 },
{ X86_FEATURE_BMEC, CPUID_EBX, 3, 0x80000020, 0 },
{ X86_FEATURE_PERFMON_V2, CPUID_EAX, 0, 0x80000022, 0 },
{ X86_FEATURE_AMD_LBR_V2, CPUID_EAX, 1, 0x80000022, 0 },
{ 0, 0, 0, 0, 0 }
arch/x86/kernel/e820.c
+3 -3
View File
@@ -53,7 +53,7 @@
*
* Once the E820 map has been converted to the standard Linux memory layout
* information its role stops - modifying it has no effect and does not get
* re-propagated. So itsmain role is a temporary bootstrap storage of firmware
* re-propagated. So its main role is a temporary bootstrap storage of firmware
* specific memory layout data during early bootup.
*/
static struct e820_table e820_table_init __initdata;
@@ -395,7 +395,7 @@ int __init e820__update_table(struct e820_table *table)
/* Continue building up new map based on this information: */
if (current_type != last_type || e820_nomerge(current_type)) {
if (last_type != 0) {
if (last_type) {
new_entries[new_nr_entries].size = change_point[chg_idx]->addr - last_addr;
/* Move forward only if the new size was non-zero: */
if (new_entries[new_nr_entries].size != 0)
@@ -403,7 +403,7 @@ int __init e820__update_table(struct e820_table *table)
if (++new_nr_entries >= max_nr_entries)
break;
}
if (current_type != 0) {
if (current_type) {
new_entries[new_nr_entries].addr = change_point[chg_idx]->addr;
new_entries[new_nr_entries].type = current_type;
last_addr = change_point[chg_idx]->addr;
arch/x86/kernel/hpet.c
+2
View File
@@ -1091,6 +1091,8 @@ int __init hpet_enable(void)
if (!hpet_counting())
goto out_nohpet;
if (tsc_clocksource_watchdog_disabled())
clocksource_hpet.flags |= CLOCK_SOURCE_MUST_VERIFY;
clocksource_register_hz(&clocksource_hpet, (u32)hpet_freq);
if (id & HPET_ID_LEGSUP) {
arch/x86/kernel/kprobes/core.c
+8 -30
View File
@@ -464,50 +464,26 @@ static void kprobe_emulate_call(struct kprobe *p, struct pt_regs *regs)
}
NOKPROBE_SYMBOL(kprobe_emulate_call);
static nokprobe_inline
void __kprobe_emulate_jmp(struct kprobe *p, struct pt_regs *regs, bool cond)
static void kprobe_emulate_jmp(struct kprobe *p, struct pt_regs *regs)
{
unsigned long ip = regs->ip - INT3_INSN_SIZE + p->ainsn.size;
if (cond)
ip += p->ainsn.rel32;
ip += p->ainsn.rel32;
int3_emulate_jmp(regs, ip);
}
static void kprobe_emulate_jmp(struct kprobe *p, struct pt_regs *regs)
{
__kprobe_emulate_jmp(p, regs, true);
}
NOKPROBE_SYMBOL(kprobe_emulate_jmp);
static const unsigned long jcc_mask[6] = {
[0] = X86_EFLAGS_OF,
[1] = X86_EFLAGS_CF,
[2] = X86_EFLAGS_ZF,
[3] = X86_EFLAGS_CF | X86_EFLAGS_ZF,
[4] = X86_EFLAGS_SF,
[5] = X86_EFLAGS_PF,
};
static void kprobe_emulate_jcc(struct kprobe *p, struct pt_regs *regs)
{
bool invert = p->ainsn.jcc.type & 1;
bool match;
unsigned long ip = regs->ip - INT3_INSN_SIZE + p->ainsn.size;
if (p->ainsn.jcc.type < 0xc) {
match = regs->flags & jcc_mask[p->ainsn.jcc.type >> 1];
} else {
match = ((regs->flags & X86_EFLAGS_SF) >> X86_EFLAGS_SF_BIT) ^
((regs->flags & X86_EFLAGS_OF) >> X86_EFLAGS_OF_BIT);
if (p->ainsn.jcc.type >= 0xe)
match = match || (regs->flags & X86_EFLAGS_ZF);
}
__kprobe_emulate_jmp(p, regs, (match && !invert) || (!match && invert));
int3_emulate_jcc(regs, p->ainsn.jcc.type, ip, p->ainsn.rel32);
}
NOKPROBE_SYMBOL(kprobe_emulate_jcc);
static void kprobe_emulate_loop(struct kprobe *p, struct pt_regs *regs)
{
unsigned long ip = regs->ip - INT3_INSN_SIZE + p->ainsn.size;
bool match;
if (p->ainsn.loop.type != 3) { /* LOOP* */
@@ -535,7 +511,9 @@ static void kprobe_emulate_loop(struct kprobe *p, struct pt_regs *regs)
else if (p->ainsn.loop.type == 1) /* LOOPE */
match = match && (regs->flags & X86_EFLAGS_ZF);
__kprobe_emulate_jmp(p, regs, match);
if (match)
ip += p->ainsn.rel32;
int3_emulate_jmp(regs, ip);
}
NOKPROBE_SYMBOL(kprobe_emulate_loop);
arch/x86/kernel/setup.c
+2 -8
View File
@@ -114,11 +114,6 @@ static struct resource bss_resource = {
#ifdef CONFIG_X86_32
/* CPU data as detected by the assembly code in head_32.S */
struct cpuinfo_x86 new_cpu_data;
/* Common CPU data for all CPUs */
struct cpuinfo_x86 boot_cpu_data __read_mostly;
EXPORT_SYMBOL(boot_cpu_data);
unsigned int def_to_bigsmp;
struct apm_info apm_info;
@@ -132,11 +127,10 @@ EXPORT_SYMBOL(ist_info);
struct ist_info ist_info;
#endif
#else
struct cpuinfo_x86 boot_cpu_data __read_mostly;
EXPORT_SYMBOL(boot_cpu_data);
#endif
struct cpuinfo_x86 boot_cpu_data __read_mostly;
EXPORT_SYMBOL(boot_cpu_data);
#if !defined(CONFIG_X86_PAE) || defined(CONFIG_X86_64)
__visible unsigned long mmu_cr4_features __ro_after_init;
arch/x86/kernel/signal.c
+1 -1
View File
@@ -360,7 +360,7 @@ static bool strict_sigaltstack_size __ro_after_init = false;
static int __init strict_sas_size(char *arg)
{
return kstrtobool(arg, &strict_sigaltstack_size);
return kstrtobool(arg, &strict_sigaltstack_size) == 0;
}
__setup("strict_sas_size", strict_sas_size);
arch/x86/kernel/smpboot.c
+1 -1
View File
@@ -1833,7 +1833,7 @@ void native_play_dead(void)
play_dead_common();
tboot_shutdown(TB_SHUTDOWN_WFS);
mwait_play_dead(); /* Only returns on failure */
mwait_play_dead();
if (cpuidle_play_dead())
hlt_play_dead();
}
arch/x86/kernel/static_call.c
+47 -3
View File
@@ -9,6 +9,7 @@ enum insn_type {
NOP = 1, /* site cond-call */
JMP = 2, /* tramp / site tail-call */
RET = 3, /* tramp / site cond-tail-call */
JCC = 4,
};
/*
@@ -25,12 +26,40 @@ static const u8 xor5rax[] = { 0x2e, 0x2e, 0x2e, 0x31, 0xc0 };
static const u8 retinsn[] = { RET_INSN_OPCODE, 0xcc, 0xcc, 0xcc, 0xcc };
static u8 __is_Jcc(u8 *insn) /* Jcc.d32 */
{
u8 ret = 0;
if (insn[0] == 0x0f) {
u8 tmp = insn[1];
if ((tmp & 0xf0) == 0x80)
ret = tmp;
}
return ret;
}
extern void __static_call_return(void);
asm (".global __static_call_return\n\t"
".type __static_call_return, @function\n\t"
ASM_FUNC_ALIGN "\n\t"
"__static_call_return:\n\t"
ANNOTATE_NOENDBR
ANNOTATE_RETPOLINE_SAFE
"ret; int3\n\t"
".size __static_call_return, . - __static_call_return \n\t");
static void __ref __static_call_transform(void *insn, enum insn_type type,
void *func, bool modinit)
{
const void *emulate = NULL;
int size = CALL_INSN_SIZE;
const void *code;
u8 op, buf[6];
if ((type == JMP || type == RET) && (op = __is_Jcc(insn)))
type = JCC;
switch (type) {
case CALL:
@@ -57,6 +86,20 @@ static void __ref __static_call_transform(void *insn, enum insn_type type,
else
code = &retinsn;
break;
case JCC:
if (!func) {
func = __static_call_return;
if (cpu_feature_enabled(X86_FEATURE_RETHUNK))
func = x86_return_thunk;
}
buf[0] = 0x0f;
__text_gen_insn(buf+1, op, insn+1, func, 5);
code = buf;
size = 6;
break;
}
if (memcmp(insn, code, size) == 0)
@@ -68,9 +111,9 @@ static void __ref __static_call_transform(void *insn, enum insn_type type,
text_poke_bp(insn, code, size, emulate);
}
static void __static_call_validate(void *insn, bool tail, bool tramp)
static void __static_call_validate(u8 *insn, bool tail, bool tramp)
{
u8 opcode = *(u8 *)insn;
u8 opcode = insn[0];
if (tramp && memcmp(insn+5, tramp_ud, 3)) {
pr_err("trampoline signature fail");
@@ -79,7 +122,8 @@ static void __static_call_validate(void *insn, bool tail, bool tramp)
if (tail) {
if (opcode == JMP32_INSN_OPCODE ||
opcode == RET_INSN_OPCODE)
opcode == RET_INSN_OPCODE ||
__is_Jcc(insn))
return;
} else {
if (opcode == CALL_INSN_OPCODE ||
arch/x86/kernel/tsc.c
+56 -12
View File
@@ -48,6 +48,8 @@ static DEFINE_STATIC_KEY_FALSE(__use_tsc);
int tsc_clocksource_reliable;
static int __read_mostly tsc_force_recalibrate;
static u32 art_to_tsc_numerator;
static u32 art_to_tsc_denominator;
static u64 art_to_tsc_offset;
@@ -291,6 +293,7 @@ __setup("notsc", notsc_setup);
static int no_sched_irq_time;
static int no_tsc_watchdog;
static int tsc_as_watchdog;
static int __init tsc_setup(char *str)
{
@@ -300,8 +303,22 @@ static int __init tsc_setup(char *str)
no_sched_irq_time = 1;
if (!strcmp(str, "unstable"))
mark_tsc_unstable("boot parameter");
if (!strcmp(str, "nowatchdog"))
if (!strcmp(str, "nowatchdog")) {
no_tsc_watchdog = 1;
if (tsc_as_watchdog)
pr_alert("%s: Overriding earlier tsc=watchdog with tsc=nowatchdog\n",
__func__);
tsc_as_watchdog = 0;
}
if (!strcmp(str, "recalibrate"))
tsc_force_recalibrate = 1;
if (!strcmp(str, "watchdog")) {
if (no_tsc_watchdog)
pr_alert("%s: tsc=watchdog overridden by earlier tsc=nowatchdog\n",
__func__);
else
tsc_as_watchdog = 1;
}
return 1;
}
@@ -911,8 +928,7 @@ void recalibrate_cpu_khz(void)
cpu_khz_old, cpu_khz);
#endif
}
EXPORT_SYMBOL(recalibrate_cpu_khz);
EXPORT_SYMBOL_GPL(recalibrate_cpu_khz);
static unsigned long long cyc2ns_suspend;
@@ -1185,6 +1201,12 @@ static void __init tsc_disable_clocksource_watchdog(void)
clocksource_tsc.flags &= ~CLOCK_SOURCE_MUST_VERIFY;
}
bool tsc_clocksource_watchdog_disabled(void)
{
return !(clocksource_tsc.flags & CLOCK_SOURCE_MUST_VERIFY) &&
tsc_as_watchdog && !no_tsc_watchdog;
}
static void __init check_system_tsc_reliable(void)
{
#if defined(CONFIG_MGEODEGX1) || defined(CONFIG_MGEODE_LX) || defined(CONFIG_X86_GENERIC)
@@ -1373,6 +1395,25 @@ restart:
else
freq = calc_pmtimer_ref(delta, ref_start, ref_stop);
/* Will hit this only if tsc_force_recalibrate has been set */
if (boot_cpu_has(X86_FEATURE_TSC_KNOWN_FREQ)) {
/* Warn if the deviation exceeds 500 ppm */
if (abs(tsc_khz - freq) > (tsc_khz >> 11)) {
pr_warn("Warning: TSC freq calibrated by CPUID/MSR differs from what is calibrated by HW timer, please check with vendor!!\n");
pr_info("Previous calibrated TSC freq:\t %lu.%03lu MHz\n",
(unsigned long)tsc_khz / 1000,
(unsigned long)tsc_khz % 1000);
}
pr_info("TSC freq recalibrated by [%s]:\t %lu.%03lu MHz\n",
hpet ? "HPET" : "PM_TIMER",
(unsigned long)freq / 1000,
(unsigned long)freq % 1000);
return;
}
/* Make sure we're within 1% */
if (abs(tsc_khz - freq) > tsc_khz/100)
goto out;
@@ -1406,8 +1447,10 @@ static int __init init_tsc_clocksource(void)
if (!boot_cpu_has(X86_FEATURE_TSC) || !tsc_khz)
return 0;
if (tsc_unstable)
goto unreg;
if (tsc_unstable) {
clocksource_unregister(&clocksource_tsc_early);
return 0;
}
if (boot_cpu_has(X86_FEATURE_NONSTOP_TSC_S3))
clocksource_tsc.flags |= CLOCK_SOURCE_SUSPEND_NONSTOP;
@@ -1420,9 +1463,10 @@ static int __init init_tsc_clocksource(void)
if (boot_cpu_has(X86_FEATURE_ART))
art_related_clocksource = &clocksource_tsc;
clocksource_register_khz(&clocksource_tsc, tsc_khz);
unreg:
clocksource_unregister(&clocksource_tsc_early);
return 0;
if (!tsc_force_recalibrate)
return 0;
}
schedule_delayed_work(&tsc_irqwork, 0);
@@ -1509,6 +1553,11 @@ void __init tsc_early_init(void)
void __init tsc_init(void)
{
if (!cpu_feature_enabled(X86_FEATURE_TSC)) {
setup_clear_cpu_cap(X86_FEATURE_TSC_DEADLINE_TIMER);
return;
}
/*
* native_calibrate_cpu_early can only calibrate using methods that are
* available early in boot.
@@ -1516,11 +1565,6 @@ void __init tsc_init(void)
if (x86_platform.calibrate_cpu == native_calibrate_cpu_early)
x86_platform.calibrate_cpu = native_calibrate_cpu;
if (!boot_cpu_has(X86_FEATURE_TSC)) {
setup_clear_cpu_cap(X86_FEATURE_TSC_DEADLINE_TIMER);
return;
}
if (!tsc_khz) {
/* We failed to determine frequencies earlier, try again */
if (!determine_cpu_tsc_frequencies(false)) {
drivers/clocksource/Kconfig
+1 -1
View File
@@ -706,7 +706,7 @@ config INGENIC_OST
config MICROCHIP_PIT64B
bool "Microchip PIT64B support"
depends on OF || COMPILE_TEST
depends on OF && ARM
select TIMER_OF
help
This option enables Microchip PIT64B timer for Atmel
drivers/clocksource/acpi_pm.c
+4 -2
View File
@@ -23,6 +23,7 @@
#include <linux/pci.h>
#include <linux/delay.h>
#include <asm/io.h>
#include <asm/time.h>
/*
* The I/O port the PMTMR resides at.
@@ -210,8 +211,9 @@ static int __init init_acpi_pm_clocksource(void)
return -ENODEV;
}
return clocksource_register_hz(&clocksource_acpi_pm,
PMTMR_TICKS_PER_SEC);
if (tsc_clocksource_watchdog_disabled())
clocksource_acpi_pm.flags |= CLOCK_SOURCE_MUST_VERIFY;
return clocksource_register_hz(&clocksource_acpi_pm, PMTMR_TICKS_PER_SEC);
}
/* We use fs_initcall because we want the PCI fixups to have run
drivers/clocksource/em_sti.c
+1 -6
View File
@@ -333,11 +333,6 @@ static int em_sti_probe(struct platform_device *pdev)
return 0;
}
static int em_sti_remove(struct platform_device *pdev)
{
return -EBUSY; /* cannot unregister clockevent and clocksource */
}
static const struct of_device_id em_sti_dt_ids[] = {
{ .compatible = "renesas,em-sti", },
{},
@@ -346,10 +341,10 @@ MODULE_DEVICE_TABLE(of, em_sti_dt_ids);
static struct platform_driver em_sti_device_driver = {
.probe = em_sti_probe,
.remove = em_sti_remove,
.driver = {
.name = "em_sti",
.of_match_table = em_sti_dt_ids,
.suppress_bind_attrs = true,
}
};
drivers/clocksource/sh_cmt.c
+1 -6
View File
@@ -1145,17 +1145,12 @@ static int sh_cmt_probe(struct platform_device *pdev)
return 0;
}
static int sh_cmt_remove(struct platform_device *pdev)
{
return -EBUSY; /* cannot unregister clockevent and clocksource */
}
static struct platform_driver sh_cmt_device_driver = {
.probe = sh_cmt_probe,
.remove = sh_cmt_remove,
.driver = {
.name = "sh_cmt",
.of_match_table = of_match_ptr(sh_cmt_of_table),
.suppress_bind_attrs = true,
},
.id_table = sh_cmt_id_table,
};
drivers/clocksource/sh_tmu.c
+1 -6
View File
@@ -632,11 +632,6 @@ static int sh_tmu_probe(struct platform_device *pdev)
return 0;
}
static int sh_tmu_remove(struct platform_device *pdev)
{
return -EBUSY; /* cannot unregister clockevent and clocksource */
}
static const struct platform_device_id sh_tmu_id_table[] = {
{ "sh-tmu", SH_TMU },
{ "sh-tmu-sh3", SH_TMU_SH3 },
@@ -652,10 +647,10 @@ MODULE_DEVICE_TABLE(of, sh_tmu_of_table);
static struct platform_driver sh_tmu_device_driver = {
.probe = sh_tmu_probe,
.remove = sh_tmu_remove,
.driver = {
.name = "sh_tmu",
.of_match_table = of_match_ptr(sh_tmu_of_table),
.suppress_bind_attrs = true,
},
.id_table = sh_tmu_id_table,
};
drivers/clocksource/timer-microchip-pit64b.c
+12
View File
@@ -9,6 +9,7 @@
#include <linux/clk.h>
#include <linux/clockchips.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/of_address.h>
#include <linux/of_irq.h>
@@ -92,6 +93,8 @@ struct mchp_pit64b_clksrc {
static void __iomem *mchp_pit64b_cs_base;
/* Default cycles for clockevent timer. */
static u64 mchp_pit64b_ce_cycles;
/* Delay timer. */
static struct delay_timer mchp_pit64b_dt;
static inline u64 mchp_pit64b_cnt_read(void __iomem *base)
{
@@ -169,6 +172,11 @@ static u64 notrace mchp_pit64b_sched_read_clk(void)
return mchp_pit64b_cnt_read(mchp_pit64b_cs_base);
}
static unsigned long notrace mchp_pit64b_dt_read(void)
{
return mchp_pit64b_cnt_read(mchp_pit64b_cs_base);
}
static int mchp_pit64b_clkevt_shutdown(struct clock_event_device *cedev)
{
struct mchp_pit64b_timer *timer = clkevt_to_mchp_pit64b_timer(cedev);
@@ -376,6 +384,10 @@ static int __init mchp_pit64b_init_clksrc(struct mchp_pit64b_timer *timer,
sched_clock_register(mchp_pit64b_sched_read_clk, 64, clk_rate);
mchp_pit64b_dt.read_current_timer = mchp_pit64b_dt_read;
mchp_pit64b_dt.freq = clk_rate;
register_current_timer_delay(&mchp_pit64b_dt);
return 0;
}
drivers/clocksource/timer-riscv.c
+21 -6
View File
@@ -28,6 +28,7 @@
#include <asm/timex.h>
static DEFINE_STATIC_KEY_FALSE(riscv_sstc_available);
static bool riscv_timer_cannot_wake_cpu;
static int riscv_clock_next_event(unsigned long delta,
struct clock_event_device *ce)
@@ -73,10 +74,15 @@ static u64 notrace riscv_sched_clock(void)
static struct clocksource riscv_clocksource = {
.name = "riscv_clocksource",
.rating = 300,
.rating = 400,
.mask = CLOCKSOURCE_MASK(64),
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
.read = riscv_clocksource_rdtime,
#if IS_ENABLED(CONFIG_GENERIC_GETTIMEOFDAY)
.vdso_clock_mode = VDSO_CLOCKMODE_ARCHTIMER,
#else
.vdso_clock_mode = VDSO_CLOCKMODE_NONE,
#endif
};
static int riscv_timer_starting_cpu(unsigned int cpu)
@@ -85,6 +91,8 @@ static int riscv_timer_starting_cpu(unsigned int cpu)
ce->cpumask = cpumask_of(cpu);
ce->irq = riscv_clock_event_irq;
if (riscv_timer_cannot_wake_cpu)
ce->features |= CLOCK_EVT_FEAT_C3STOP;
clockevents_config_and_register(ce, riscv_timebase, 100, 0x7fffffff);
enable_percpu_irq(riscv_clock_event_irq,
@@ -139,6 +147,13 @@ static int __init riscv_timer_init_dt(struct device_node *n)
if (cpuid != smp_processor_id())
return 0;
child = of_find_compatible_node(NULL, NULL, "riscv,timer");
if (child) {
riscv_timer_cannot_wake_cpu = of_property_read_bool(child,
"riscv,timer-cannot-wake-cpu");
of_node_put(child);
}
domain = NULL;
child = of_get_compatible_child(n, "riscv,cpu-intc");
if (!child) {
@@ -177,6 +192,11 @@ static int __init riscv_timer_init_dt(struct device_node *n)
return error;
}
if (riscv_isa_extension_available(NULL, SSTC)) {
pr_info("Timer interrupt in S-mode is available via sstc extension\n");
static_branch_enable(&riscv_sstc_available);
}
error = cpuhp_setup_state(CPUHP_AP_RISCV_TIMER_STARTING,
"clockevents/riscv/timer:starting",
riscv_timer_starting_cpu, riscv_timer_dying_cpu);
@@ -184,11 +204,6 @@ static int __init riscv_timer_init_dt(struct device_node *n)
pr_err("cpu hp setup state failed for RISCV timer [%d]\n",
error);
if (riscv_isa_extension_available(NULL, SSTC)) {
pr_info("Timer interrupt in S-mode is available via sstc extension\n");
static_branch_enable(&riscv_sstc_available);
}
return error;
}
drivers/clocksource/timer-sun4i.c
+2 -1
View File
@@ -144,7 +144,8 @@ static struct timer_of to = {
.clkevt = {
.name = "sun4i_tick",
.rating = 350,
.features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT,
.features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT |
CLOCK_EVT_FEAT_DYNIRQ,
.set_state_shutdown = sun4i_clkevt_shutdown,
.set_state_periodic = sun4i_clkevt_set_periodic,
.set_state_oneshot = sun4i_clkevt_set_oneshot,
drivers/edac/Kconfig
+8
View File
@@ -542,4 +542,12 @@ config EDAC_DMC520
Support for error detection and correction on the
SoCs with ARM DMC-520 DRAM controller.
config EDAC_ZYNQMP
tristate "Xilinx ZynqMP OCM Controller"
depends on ARCH_ZYNQMP || COMPILE_TEST
help
This driver supports error detection and correction for the
Xilinx ZynqMP OCM (On Chip Memory) controller. It can also be
built as a module. In that case it will be called zynqmp_edac.
endif # EDAC
drivers/edac/Makefile
+1
View File
@@ -84,3 +84,4 @@ obj-$(CONFIG_EDAC_QCOM) += qcom_edac.o
obj-$(CONFIG_EDAC_ASPEED) += aspeed_edac.o
obj-$(CONFIG_EDAC_BLUEFIELD) += bluefield_edac.o
obj-$(CONFIG_EDAC_DMC520) += dmc520_edac.o
obj-$(CONFIG_EDAC_ZYNQMP) += zynqmp_edac.o
drivers/edac/amd64_edac.c
+17 -200
View File
@@ -182,21 +182,6 @@ static inline int amd64_read_dct_pci_cfg(struct amd64_pvt *pvt, u8 dct,
* other archs, we might not have access to the caches directly.
*/
static inline void __f17h_set_scrubval(struct amd64_pvt *pvt, u32 scrubval)
{
/*
* Fam17h supports scrub values between 0x5 and 0x14. Also, the values
* are shifted down by 0x5, so scrubval 0x5 is written to the register
* as 0x0, scrubval 0x6 as 0x1, etc.
*/
if (scrubval >= 0x5 && scrubval <= 0x14) {
scrubval -= 0x5;
pci_write_bits32(pvt->F6, F17H_SCR_LIMIT_ADDR, scrubval, 0xF);
pci_write_bits32(pvt->F6, F17H_SCR_BASE_ADDR, 1, 0x1);
} else {
pci_write_bits32(pvt->F6, F17H_SCR_BASE_ADDR, 0, 0x1);
}
}
/*
* Scan the scrub rate mapping table for a close or matching bandwidth value to
* issue. If requested is too big, then use last maximum value found.
@@ -229,9 +214,7 @@ static int __set_scrub_rate(struct amd64_pvt *pvt, u32 new_bw, u32 min_rate)
scrubval = scrubrates[i].scrubval;
if (pvt->umc) {
__f17h_set_scrubval(pvt, scrubval);
} else if (pvt->fam == 0x15 && pvt->model == 0x60) {
if (pvt->fam == 0x15 && pvt->model == 0x60) {
f15h_select_dct(pvt, 0);
pci_write_bits32(pvt->F2, F15H_M60H_SCRCTRL, scrubval, 0x001F);
f15h_select_dct(pvt, 1);
@@ -271,16 +254,7 @@ static int get_scrub_rate(struct mem_ctl_info *mci)
int i, retval = -EINVAL;
u32 scrubval = 0;
if (pvt->umc) {
amd64_read_pci_cfg(pvt->F6, F17H_SCR_BASE_ADDR, &scrubval);
if (scrubval & BIT(0)) {
amd64_read_pci_cfg(pvt->F6, F17H_SCR_LIMIT_ADDR, &scrubval);
scrubval &= 0xF;
scrubval += 0x5;
} else {
scrubval = 0;
}
} else if (pvt->fam == 0x15) {
if (pvt->fam == 0x15) {
/* Erratum #505 */
if (pvt->model < 0x10)
f15h_select_dct(pvt, 0);
@@ -1454,9 +1428,6 @@ static void __dump_misc_regs_df(struct amd64_pvt *pvt)
debug_display_dimm_sizes_df(pvt, i);
}
edac_dbg(1, "F0x104 (DRAM Hole Address): 0x%08x, base: 0x%08x\n",
pvt->dhar, dhar_base(pvt));
}
/* Display and decode various NB registers for debug purposes. */
@@ -1491,6 +1462,8 @@ static void __dump_misc_regs(struct amd64_pvt *pvt)
/* Only if NOT ganged does dclr1 have valid info */
if (!dct_ganging_enabled(pvt))
debug_dump_dramcfg_low(pvt, pvt->dclr1, 1);
edac_dbg(1, " DramHoleValid: %s\n", dhar_valid(pvt) ? "yes" : "no");
}
/* Display and decode various NB registers for debug purposes. */
@@ -1501,8 +1474,6 @@ static void dump_misc_regs(struct amd64_pvt *pvt)
else
__dump_misc_regs(pvt);
edac_dbg(1, " DramHoleValid: %s\n", dhar_valid(pvt) ? "yes" : "no");
amd64_info("using x%u syndromes.\n", pvt->ecc_sym_sz);
}
@@ -1732,24 +1703,6 @@ ddr3:
pvt->dram_type = (pvt->dclr0 & BIT(16)) ? MEM_DDR3 : MEM_RDDR3;
}
/* Get the number of DCT channels the memory controller is using. */
static int k8_early_channel_count(struct amd64_pvt *pvt)
{
int flag;
if (pvt->ext_model >= K8_REV_F)
/* RevF (NPT) and later */
flag = pvt->dclr0 & WIDTH_128;
else
/* RevE and earlier */
flag = pvt->dclr0 & REVE_WIDTH_128;
/* not used */
pvt->dclr1 = 0;
return (flag) ? 2 : 1;
}
/* On F10h and later ErrAddr is MC4_ADDR[47:1] */
static u64 get_error_address(struct amd64_pvt *pvt, struct mce *m)
{
@@ -2001,69 +1954,6 @@ static int k8_dbam_to_chip_select(struct amd64_pvt *pvt, u8 dct,
}
}
/*
* Get the number of DCT channels in use.
*
* Return:
* number of Memory Channels in operation
* Pass back:
* contents of the DCL0_LOW register
*/
static int f1x_early_channel_count(struct amd64_pvt *pvt)
{
int i, j, channels = 0;
/* On F10h, if we are in 128 bit mode, then we are using 2 channels */
if (pvt->fam == 0x10 && (pvt->dclr0 & WIDTH_128))
return 2;
/*
* Need to check if in unganged mode: In such, there are 2 channels,
* but they are not in 128 bit mode and thus the above 'dclr0' status
* bit will be OFF.
*
* Need to check DCT0[0] and DCT1[0] to see if only one of them has
* their CSEnable bit on. If so, then SINGLE DIMM case.
*/
edac_dbg(0, "Data width is not 128 bits - need more decoding\n");
/*
* Check DRAM Bank Address Mapping values for each DIMM to see if there
* is more than just one DIMM present in unganged mode. Need to check
* both controllers since DIMMs can be placed in either one.
*/
for (i = 0; i < 2; i++) {
u32 dbam = (i ? pvt->dbam1 : pvt->dbam0);
for (j = 0; j < 4; j++) {
if (DBAM_DIMM(j, dbam) > 0) {
channels++;
break;
}
}
}
if (channels > 2)
channels = 2;
amd64_info("MCT channel count: %d\n", channels);
return channels;
}
static int f17_early_channel_count(struct amd64_pvt *pvt)
{
int i, channels = 0;
/* SDP Control bit 31 (SdpInit) is clear for unused UMC channels */
for_each_umc(i)
channels += !!(pvt->umc[i].sdp_ctrl & UMC_SDP_INIT);
amd64_info("MCT channel count: %d\n", channels);
return channels;
}
static int ddr3_cs_size(unsigned i, bool dct_width)
{
unsigned shift = 0;
@@ -2858,7 +2748,6 @@ static struct amd64_family_type family_types[] = {
.f2_id = PCI_DEVICE_ID_AMD_K8_NB_MEMCTL,
.max_mcs = 2,
.ops = {
.early_channel_count = k8_early_channel_count,
.map_sysaddr_to_csrow = k8_map_sysaddr_to_csrow,
.dbam_to_cs = k8_dbam_to_chip_select,
}
@@ -2869,7 +2758,6 @@ static struct amd64_family_type family_types[] = {
.f2_id = PCI_DEVICE_ID_AMD_10H_NB_DRAM,
.max_mcs = 2,
.ops = {
.early_channel_count = f1x_early_channel_count,
.map_sysaddr_to_csrow = f1x_map_sysaddr_to_csrow,
.dbam_to_cs = f10_dbam_to_chip_select,
}
@@ -2880,7 +2768,6 @@ static struct amd64_family_type family_types[] = {
.f2_id = PCI_DEVICE_ID_AMD_15H_NB_F2,
.max_mcs = 2,
.ops = {
.early_channel_count = f1x_early_channel_count,
.map_sysaddr_to_csrow = f1x_map_sysaddr_to_csrow,
.dbam_to_cs = f15_dbam_to_chip_select,
}
@@ -2891,7 +2778,6 @@ static struct amd64_family_type family_types[] = {
.f2_id = PCI_DEVICE_ID_AMD_15H_M30H_NB_F2,
.max_mcs = 2,
.ops = {
.early_channel_count = f1x_early_channel_count,
.map_sysaddr_to_csrow = f1x_map_sysaddr_to_csrow,
.dbam_to_cs = f16_dbam_to_chip_select,
}
@@ -2902,7 +2788,6 @@ static struct amd64_family_type family_types[] = {
.f2_id = PCI_DEVICE_ID_AMD_15H_M60H_NB_F2,
.max_mcs = 2,
.ops = {
.early_channel_count = f1x_early_channel_count,
.map_sysaddr_to_csrow = f1x_map_sysaddr_to_csrow,
.dbam_to_cs = f15_m60h_dbam_to_chip_select,
}
@@ -2913,7 +2798,6 @@ static struct amd64_family_type family_types[] = {
.f2_id = PCI_DEVICE_ID_AMD_16H_NB_F2,
.max_mcs = 2,
.ops = {
.early_channel_count = f1x_early_channel_count,
.map_sysaddr_to_csrow = f1x_map_sysaddr_to_csrow,
.dbam_to_cs = f16_dbam_to_chip_select,
}
@@ -2924,89 +2808,64 @@ static struct amd64_family_type family_types[] = {
.f2_id = PCI_DEVICE_ID_AMD_16H_M30H_NB_F2,
.max_mcs = 2,
.ops = {
.early_channel_count = f1x_early_channel_count,
.map_sysaddr_to_csrow = f1x_map_sysaddr_to_csrow,
.dbam_to_cs = f16_dbam_to_chip_select,
}
},
[F17_CPUS] = {
.ctl_name = "F17h",
.f0_id = PCI_DEVICE_ID_AMD_17H_DF_F0,
.f6_id = PCI_DEVICE_ID_AMD_17H_DF_F6,
.max_mcs = 2,
.ops = {
.early_channel_count = f17_early_channel_count,
.dbam_to_cs = f17_addr_mask_to_cs_size,
}
},
[F17_M10H_CPUS] = {
.ctl_name = "F17h_M10h",
.f0_id = PCI_DEVICE_ID_AMD_17H_M10H_DF_F0,
.f6_id = PCI_DEVICE_ID_AMD_17H_M10H_DF_F6,
.max_mcs = 2,
.ops = {
.early_channel_count = f17_early_channel_count,
.dbam_to_cs = f17_addr_mask_to_cs_size,
}
},
[F17_M30H_CPUS] = {
.ctl_name = "F17h_M30h",
.f0_id = PCI_DEVICE_ID_AMD_17H_M30H_DF_F0,
.f6_id = PCI_DEVICE_ID_AMD_17H_M30H_DF_F6,
.max_mcs = 8,
.ops = {
.early_channel_count = f17_early_channel_count,
.dbam_to_cs = f17_addr_mask_to_cs_size,
}
},
[F17_M60H_CPUS] = {
.ctl_name = "F17h_M60h",
.f0_id = PCI_DEVICE_ID_AMD_17H_M60H_DF_F0,
.f6_id = PCI_DEVICE_ID_AMD_17H_M60H_DF_F6,
.max_mcs = 2,
.ops = {
.early_channel_count = f17_early_channel_count,
.dbam_to_cs = f17_addr_mask_to_cs_size,
}
},
[F17_M70H_CPUS] = {
.ctl_name = "F17h_M70h",
.f0_id = PCI_DEVICE_ID_AMD_17H_M70H_DF_F0,
.f6_id = PCI_DEVICE_ID_AMD_17H_M70H_DF_F6,
.max_mcs = 2,
.ops = {
.early_channel_count = f17_early_channel_count,
.dbam_to_cs = f17_addr_mask_to_cs_size,
}
},
[F19_CPUS] = {
.ctl_name = "F19h",
.f0_id = PCI_DEVICE_ID_AMD_19H_DF_F0,
.f6_id = PCI_DEVICE_ID_AMD_19H_DF_F6,
.max_mcs = 8,
.ops = {
.early_channel_count = f17_early_channel_count,
.dbam_to_cs = f17_addr_mask_to_cs_size,
}
},
[F19_M10H_CPUS] = {
.ctl_name = "F19h_M10h",
.f0_id = PCI_DEVICE_ID_AMD_19H_M10H_DF_F0,
.f6_id = PCI_DEVICE_ID_AMD_19H_M10H_DF_F6,
.max_mcs = 12,
.flags.zn_regs_v2 = 1,
.ops = {
.early_channel_count = f17_early_channel_count,
.dbam_to_cs = f17_addr_mask_to_cs_size,
}
},
[F19_M50H_CPUS] = {
.ctl_name = "F19h_M50h",
.f0_id = PCI_DEVICE_ID_AMD_19H_M50H_DF_F0,
.f6_id = PCI_DEVICE_ID_AMD_19H_M50H_DF_F6,
.max_mcs = 2,
.ops = {
.early_channel_count = f17_early_channel_count,
.dbam_to_cs = f17_addr_mask_to_cs_size,
}
},
@@ -3316,36 +3175,12 @@ log_error:
/*
* Use pvt->F3 which contains the F3 CPU PCI device to get the related
* F1 (AddrMap) and F2 (Dct) devices. Return negative value on error.
* Reserve F0 and F6 on systems with a UMC.
*/
static int
reserve_mc_sibling_devs(struct amd64_pvt *pvt, u16 pci_id1, u16 pci_id2)
{
if (pvt->umc) {
pvt->F0 = pci_get_related_function(pvt->F3->vendor, pci_id1, pvt->F3);
if (!pvt->F0) {
edac_dbg(1, "F0 not found, device 0x%x\n", pci_id1);
return -ENODEV;
}
pvt->F6 = pci_get_related_function(pvt->F3->vendor, pci_id2, pvt->F3);
if (!pvt->F6) {
pci_dev_put(pvt->F0);
pvt->F0 = NULL;
edac_dbg(1, "F6 not found: device 0x%x\n", pci_id2);
return -ENODEV;
}
if (!pci_ctl_dev)
pci_ctl_dev = &pvt->F0->dev;
edac_dbg(1, "F0: %s\n", pci_name(pvt->F0));
edac_dbg(1, "F3: %s\n", pci_name(pvt->F3));
edac_dbg(1, "F6: %s\n", pci_name(pvt->F6));
if (pvt->umc)
return 0;
}
/* Reserve the ADDRESS MAP Device */
pvt->F1 = pci_get_related_function(pvt->F3->vendor, pci_id1, pvt->F3);
@@ -3377,8 +3212,7 @@ reserve_mc_sibling_devs(struct amd64_pvt *pvt, u16 pci_id1, u16 pci_id2)
static void free_mc_sibling_devs(struct amd64_pvt *pvt)
{
if (pvt->umc) {
pci_dev_put(pvt->F0);
pci_dev_put(pvt->F6);
return;
} else {
pci_dev_put(pvt->F1);
pci_dev_put(pvt->F2);
@@ -3468,7 +3302,6 @@ static void read_mc_regs(struct amd64_pvt *pvt)
if (pvt->umc) {
__read_mc_regs_df(pvt);
amd64_read_pci_cfg(pvt->F0, DF_DHAR, &pvt->dhar);
goto skip;
}
@@ -3691,7 +3524,7 @@ static int init_csrows(struct mem_ctl_info *mci)
: EDAC_SECDED;
}
for (j = 0; j < pvt->channel_count; j++) {
for (j = 0; j < fam_type->max_mcs; j++) {
dimm = csrow->channels[j]->dimm;
dimm->mtype = pvt->dram_type;
dimm->edac_mode = edac_mode;
@@ -3967,6 +3800,9 @@ static void setup_mci_misc_attrs(struct mem_ctl_info *mci)
mci->dev_name = pci_name(pvt->F3);
mci->ctl_page_to_phys = NULL;
if (pvt->fam >= 0x17)
return;
/* memory scrubber interface */
mci->set_sdram_scrub_rate = set_scrub_rate;
mci->get_sdram_scrub_rate = get_scrub_rate;
@@ -4092,16 +3928,13 @@ static const struct attribute_group *amd64_edac_attr_groups[] = {
static int hw_info_get(struct amd64_pvt *pvt)
{
u16 pci_id1, pci_id2;
u16 pci_id1 = 0, pci_id2 = 0;
int ret;
if (pvt->fam >= 0x17) {
pvt->umc = kcalloc(fam_type->max_mcs, sizeof(struct amd64_umc), GFP_KERNEL);
if (!pvt->umc)
return -ENOMEM;
pci_id1 = fam_type->f0_id;
pci_id2 = fam_type->f6_id;
} else {
pci_id1 = fam_type->f1_id;
pci_id2 = fam_type->f2_id;
@@ -4118,7 +3951,7 @@ static int hw_info_get(struct amd64_pvt *pvt)
static void hw_info_put(struct amd64_pvt *pvt)
{
if (pvt->F0 || pvt->F1)
if (pvt->F1)
free_mc_sibling_devs(pvt);
kfree(pvt->umc);
@@ -4128,28 +3961,12 @@ static int init_one_instance(struct amd64_pvt *pvt)
{
struct mem_ctl_info *mci = NULL;
struct edac_mc_layer layers[2];
int ret = -EINVAL;
int ret = -ENOMEM;
/*
* We need to determine how many memory channels there are. Then use
* that information for calculating the size of the dynamic instance
* tables in the 'mci' structure.
*/
pvt->channel_count = pvt->ops->early_channel_count(pvt);
if (pvt->channel_count < 0)
return ret;
ret = -ENOMEM;
layers[0].type = EDAC_MC_LAYER_CHIP_SELECT;
layers[0].size = pvt->csels[0].b_cnt;
layers[0].is_virt_csrow = true;
layers[1].type = EDAC_MC_LAYER_CHANNEL;
/*
* Always allocate two channels since we can have setups with DIMMs on
* only one channel. Also, this simplifies handling later for the price
* of a couple of KBs tops.
*/
layers[1].size = fam_type->max_mcs;
layers[1].is_virt_csrow = false;
@@ -4370,12 +4187,12 @@ static int __init amd64_edac_init(void)
}
/* register stuff with EDAC MCE */
if (boot_cpu_data.x86 >= 0x17)
if (boot_cpu_data.x86 >= 0x17) {
amd_register_ecc_decoder(decode_umc_error);
else
} else {
amd_register_ecc_decoder(decode_bus_error);
setup_pci_device();
setup_pci_device();
}
#ifdef CONFIG_X86_32
amd64_err("%s on 32-bit is unsupported. USE AT YOUR OWN RISK!\n", EDAC_MOD_STR);
drivers/edac/amd64_edac.h
+2 -22
View File
@@ -114,22 +114,6 @@
#define PCI_DEVICE_ID_AMD_16H_NB_F2 0x1532
#define PCI_DEVICE_ID_AMD_16H_M30H_NB_F1 0x1581
#define PCI_DEVICE_ID_AMD_16H_M30H_NB_F2 0x1582
#define PCI_DEVICE_ID_AMD_17H_DF_F0 0x1460
#define PCI_DEVICE_ID_AMD_17H_DF_F6 0x1466
#define PCI_DEVICE_ID_AMD_17H_M10H_DF_F0 0x15e8
#define PCI_DEVICE_ID_AMD_17H_M10H_DF_F6 0x15ee
#define PCI_DEVICE_ID_AMD_17H_M30H_DF_F0 0x1490
#define PCI_DEVICE_ID_AMD_17H_M30H_DF_F6 0x1496
#define PCI_DEVICE_ID_AMD_17H_M60H_DF_F0 0x1448
#define PCI_DEVICE_ID_AMD_17H_M60H_DF_F6 0x144e
#define PCI_DEVICE_ID_AMD_17H_M70H_DF_F0 0x1440
#define PCI_DEVICE_ID_AMD_17H_M70H_DF_F6 0x1446
#define PCI_DEVICE_ID_AMD_19H_DF_F0 0x1650
#define PCI_DEVICE_ID_AMD_19H_DF_F6 0x1656
#define PCI_DEVICE_ID_AMD_19H_M10H_DF_F0 0x14ad
#define PCI_DEVICE_ID_AMD_19H_M10H_DF_F6 0x14b3
#define PCI_DEVICE_ID_AMD_19H_M50H_DF_F0 0x166a
#define PCI_DEVICE_ID_AMD_19H_M50H_DF_F6 0x1670
/*
* Function 1 - Address Map
@@ -215,8 +199,6 @@
#define DCT_SEL_HI 0x114
#define F15H_M60H_SCRCTRL 0x1C8
#define F17H_SCR_BASE_ADDR 0x48
#define F17H_SCR_LIMIT_ADDR 0x4C
/*
* Function 3 - Misc Control
@@ -356,7 +338,7 @@ struct amd64_pvt {
struct low_ops *ops;
/* pci_device handles which we utilize */
struct pci_dev *F0, *F1, *F2, *F3, *F6;
struct pci_dev *F1, *F2, *F3;
u16 mc_node_id; /* MC index of this MC node */
u8 fam; /* CPU family */
@@ -364,7 +346,6 @@ struct amd64_pvt {
u8 stepping; /* ... stepping */
int ext_model; /* extended model value of this node */
int channel_count;
/* Raw registers */
u32 dclr0; /* DRAM Configuration Low DCT0 reg */
@@ -484,7 +465,6 @@ struct ecc_settings {
* functions and per device encoding/decoding logic.
*/
struct low_ops {
int (*early_channel_count) (struct amd64_pvt *pvt);
void (*map_sysaddr_to_csrow) (struct mem_ctl_info *mci, u64 sys_addr,
struct err_info *);
int (*dbam_to_cs) (struct amd64_pvt *pvt, u8 dct,
@@ -503,7 +483,7 @@ struct amd64_family_flags {
struct amd64_family_type {
const char *ctl_name;
u16 f0_id, f1_id, f2_id, f6_id;
u16 f1_id, f2_id;
/* Maximum number of memory controllers per die/node. */
u8 max_mcs;
struct amd64_family_flags flags;
drivers/edac/i10nm_base.c
+369 -90
View File
@@ -13,7 +13,7 @@
#include "edac_module.h"
#include "skx_common.h"
#define I10NM_REVISION "v0.0.5"
#define I10NM_REVISION "v0.0.6"
#define EDAC_MOD_STR "i10nm_edac"
/* Debug macros */
@@ -22,25 +22,34 @@
#define I10NM_GET_SCK_BAR(d, reg) \
pci_read_config_dword((d)->uracu, 0xd0, &(reg))
#define I10NM_GET_IMC_BAR(d, i, reg) \
pci_read_config_dword((d)->uracu, 0xd8 + (i) * 4, &(reg))
#define I10NM_GET_IMC_BAR(d, i, reg) \
pci_read_config_dword((d)->uracu, \
(res_cfg->type == GNR ? 0xd4 : 0xd8) + (i) * 4, &(reg))
#define I10NM_GET_SAD(d, offset, i, reg)\
pci_read_config_dword((d)->sad_all, (offset) + (i) * 8, &(reg))
pci_read_config_dword((d)->sad_all, (offset) + (i) * \
(res_cfg->type == GNR ? 12 : 8), &(reg))
#define I10NM_GET_HBM_IMC_BAR(d, reg) \
pci_read_config_dword((d)->uracu, 0xd4, &(reg))
#define I10NM_GET_CAPID3_CFG(d, reg) \
pci_read_config_dword((d)->pcu_cr3, 0x90, &(reg))
pci_read_config_dword((d)->pcu_cr3, \
res_cfg->type == GNR ? 0x290 : 0x90, &(reg))
#define I10NM_GET_CAPID5_CFG(d, reg) \
pci_read_config_dword((d)->pcu_cr3, \
res_cfg->type == GNR ? 0x298 : 0x98, &(reg))
#define I10NM_GET_DIMMMTR(m, i, j) \
readl((m)->mbase + ((m)->hbm_mc ? 0x80c : 0x2080c) + \
readl((m)->mbase + ((m)->hbm_mc ? 0x80c : \
(res_cfg->type == GNR ? 0xc0c : 0x2080c)) + \
(i) * (m)->chan_mmio_sz + (j) * 4)
#define I10NM_GET_MCDDRTCFG(m, i) \
readl((m)->mbase + ((m)->hbm_mc ? 0x970 : 0x20970) + \
(i) * (m)->chan_mmio_sz)
#define I10NM_GET_MCMTR(m, i) \
readl((m)->mbase + ((m)->hbm_mc ? 0xef8 : 0x20ef8) + \
readl((m)->mbase + ((m)->hbm_mc ? 0xef8 : \
(res_cfg->type == GNR ? 0xaf8 : 0x20ef8)) + \
(i) * (m)->chan_mmio_sz)
#define I10NM_GET_AMAP(m, i) \
readl((m)->mbase + ((m)->hbm_mc ? 0x814 : 0x20814) + \
readl((m)->mbase + ((m)->hbm_mc ? 0x814 : \
(res_cfg->type == GNR ? 0xc14 : 0x20814)) + \
(i) * (m)->chan_mmio_sz)
#define I10NM_GET_REG32(m, i, offset) \
readl((m)->mbase + (i) * (m)->chan_mmio_sz + (offset))
@@ -56,7 +65,10 @@
#define I10NM_GET_HBM_IMC_MMIO_OFFSET(reg) \
((GET_BITFIELD(reg, 0, 10) << 12) + 0x140000)
#define I10NM_GNR_IMC_MMIO_OFFSET 0x24c000
#define I10NM_GNR_IMC_MMIO_SIZE 0x4000
#define I10NM_HBM_IMC_MMIO_SIZE 0x9000
#define I10NM_DDR_IMC_CH_CNT(reg) GET_BITFIELD(reg, 21, 24)
#define I10NM_IS_HBM_PRESENT(reg) GET_BITFIELD(reg, 27, 30)
#define I10NM_IS_HBM_IMC(reg) GET_BITFIELD(reg, 29, 29)
@@ -148,35 +160,47 @@ static void __enable_retry_rd_err_log(struct skx_imc *imc, int chan, bool enable
static void enable_retry_rd_err_log(bool enable)
{
int i, j, imc_num, chan_num;
struct skx_imc *imc;
struct skx_dev *d;
int i, j;
edac_dbg(2, "\n");
list_for_each_entry(d, i10nm_edac_list, list)
for (i = 0; i < I10NM_NUM_IMC; i++) {
list_for_each_entry(d, i10nm_edac_list, list) {
imc_num = res_cfg->ddr_imc_num;
chan_num = res_cfg->ddr_chan_num;
for (i = 0; i < imc_num; i++) {
imc = &d->imc[i];
if (!imc->mbase)
continue;
for (j = 0; j < I10NM_NUM_CHANNELS; j++) {
if (imc->hbm_mc) {
__enable_retry_rd_err_log(imc, j, enable,
res_cfg->offsets_scrub_hbm0,
res_cfg->offsets_demand_hbm0,
NULL);
__enable_retry_rd_err_log(imc, j, enable,
res_cfg->offsets_scrub_hbm1,
res_cfg->offsets_demand_hbm1,
NULL);
} else {
__enable_retry_rd_err_log(imc, j, enable,
res_cfg->offsets_scrub,
res_cfg->offsets_demand,
res_cfg->offsets_demand2);
}
for (j = 0; j < chan_num; j++)
__enable_retry_rd_err_log(imc, j, enable,
res_cfg->offsets_scrub,
res_cfg->offsets_demand,
res_cfg->offsets_demand2);
}
imc_num += res_cfg->hbm_imc_num;
chan_num = res_cfg->hbm_chan_num;
for (; i < imc_num; i++) {
imc = &d->imc[i];
if (!imc->mbase || !imc->hbm_mc)
continue;
for (j = 0; j < chan_num; j++) {
__enable_retry_rd_err_log(imc, j, enable,
res_cfg->offsets_scrub_hbm0,
res_cfg->offsets_demand_hbm0,
NULL);
__enable_retry_rd_err_log(imc, j, enable,
res_cfg->offsets_scrub_hbm1,
res_cfg->offsets_demand_hbm1,
NULL);
}
}
}
}
@@ -311,6 +335,79 @@ static struct pci_dev *pci_get_dev_wrapper(int dom, unsigned int bus,
return pdev;
}
/**
* i10nm_get_imc_num() - Get the number of present DDR memory controllers.
*
* @cfg : The pointer to the structure of EDAC resource configurations.
*
* For Granite Rapids CPUs, the number of present DDR memory controllers read
* at runtime overwrites the value statically configured in @cfg->ddr_imc_num.
* For other CPUs, the number of present DDR memory controllers is statically
* configured in @cfg->ddr_imc_num.
*
* RETURNS : 0 on success, < 0 on failure.
*/
static int i10nm_get_imc_num(struct res_config *cfg)
{
int n, imc_num, chan_num = 0;
struct skx_dev *d;
u32 reg;
list_for_each_entry(d, i10nm_edac_list, list) {
d->pcu_cr3 = pci_get_dev_wrapper(d->seg, d->bus[res_cfg->pcu_cr3_bdf.bus],
res_cfg->pcu_cr3_bdf.dev,
res_cfg->pcu_cr3_bdf.fun);
if (!d->pcu_cr3)
continue;
if (I10NM_GET_CAPID5_CFG(d, reg))
continue;
n = I10NM_DDR_IMC_CH_CNT(reg);
if (!chan_num) {
chan_num = n;
edac_dbg(2, "Get DDR CH number: %d\n", chan_num);
} else if (chan_num != n) {
i10nm_printk(KERN_NOTICE, "Get DDR CH numbers: %d, %d\n", chan_num, n);
}
}
switch (cfg->type) {
case GNR:
/*
* One channel per DDR memory controller for Granite Rapids CPUs.
*/
imc_num = chan_num;
if (!imc_num) {
i10nm_printk(KERN_ERR, "Invalid DDR MC number\n");
return -ENODEV;
}
if (imc_num > I10NM_NUM_DDR_IMC) {
i10nm_printk(KERN_ERR, "Need to make I10NM_NUM_DDR_IMC >= %d\n", imc_num);
return -EINVAL;
}
if (cfg->ddr_imc_num != imc_num) {
/*
* Store the number of present DDR memory controllers.
*/
cfg->ddr_imc_num = imc_num;
edac_dbg(2, "Set DDR MC number: %d", imc_num);
}
return 0;
default:
/*
* For other CPUs, the number of present DDR memory controllers
* is statically pre-configured in cfg->ddr_imc_num.
*/
return 0;
}
}
static bool i10nm_check_2lm(struct res_config *cfg)
{
struct skx_dev *d;
@@ -318,9 +415,9 @@ static bool i10nm_check_2lm(struct res_config *cfg)
int i;
list_for_each_entry(d, i10nm_edac_list, list) {
d->sad_all = pci_get_dev_wrapper(d->seg, d->bus[1],
PCI_SLOT(cfg->sad_all_devfn),
PCI_FUNC(cfg->sad_all_devfn));
d->sad_all = pci_get_dev_wrapper(d->seg, d->bus[res_cfg->sad_all_bdf.bus],
res_cfg->sad_all_bdf.dev,
res_cfg->sad_all_bdf.fun);
if (!d->sad_all)
continue;
@@ -337,20 +434,39 @@ static bool i10nm_check_2lm(struct res_config *cfg)
}
/*
* Check whether the error comes from DDRT by ICX/Tremont model specific error code.
* Refer to SDM vol3B 16.11.3 Intel IMC MC error codes for IA32_MCi_STATUS.
* Check whether the error comes from DDRT by ICX/Tremont/SPR model specific error code.
* Refer to SDM vol3B 17.11.3/17.13.2 Intel IMC MC error codes for IA32_MCi_STATUS.
*/
static bool i10nm_mscod_is_ddrt(u32 mscod)
{
switch (mscod) {
case 0x0106: case 0x0107:
case 0x0800: case 0x0804:
case 0x0806 ... 0x0808:
case 0x080a ... 0x080e:
case 0x0810: case 0x0811:
case 0x0816: case 0x081e:
case 0x081f:
return true;
switch (res_cfg->type) {
case I10NM:
switch (mscod) {
case 0x0106: case 0x0107:
case 0x0800: case 0x0804:
case 0x0806 ... 0x0808:
case 0x080a ... 0x080e:
case 0x0810: case 0x0811:
case 0x0816: case 0x081e:
case 0x081f:
return true;
}
break;
case SPR:
switch (mscod) {
case 0x0800: case 0x0804:
case 0x0806 ... 0x0808:
case 0x080a ... 0x080e:
case 0x0810: case 0x0811:
case 0x0816: case 0x081e:
case 0x081f:
return true;
}
break;
default:
return false;
}
return false;
@@ -358,6 +474,7 @@ static bool i10nm_mscod_is_ddrt(u32 mscod)
static bool i10nm_mc_decode_available(struct mce *mce)
{
#define ICX_IMCx_CHy 0x06666000
u8 bank;
if (!decoding_via_mca || mem_cfg_2lm)
@@ -371,21 +488,26 @@ static bool i10nm_mc_decode_available(struct mce *mce)
switch (res_cfg->type) {
case I10NM:
if (bank < 13 || bank > 26)
/* Check whether the bank is one of {13,14,17,18,21,22,25,26} */
if (!(ICX_IMCx_CHy & (1 << bank)))
return false;
/* DDRT errors can't be decoded from MCA bank registers */
if (MCI_MISC_ECC_MODE(mce->misc) == MCI_MISC_ECC_DDRT)
break;
case SPR:
if (bank < 13 || bank > 20)
return false;
if (i10nm_mscod_is_ddrt(MCI_STATUS_MSCOD(mce->status)))
return false;
/* Check whether one of {13,14,17,18,21,22,25,26} */
return ((bank - 13) & BIT(1)) == 0;
break;
default:
return false;
}
/* DDRT errors can't be decoded from MCA bank registers */
if (MCI_MISC_ECC_MODE(mce->misc) == MCI_MISC_ECC_DDRT)
return false;
if (i10nm_mscod_is_ddrt(MCI_STATUS_MSCOD(mce->status)))
return false;
return true;
}
static bool i10nm_mc_decode(struct decoded_addr *res)
@@ -407,9 +529,29 @@ static bool i10nm_mc_decode(struct decoded_addr *res)
switch (res_cfg->type) {
case I10NM:
bank = m->bank - 13;
res->imc = bank / 4;
res->channel = bank % 2;
bank = m->bank - 13;
res->imc = bank / 4;
res->channel = bank % 2;
res->column = GET_BITFIELD(m->misc, 9, 18) << 2;
res->row = GET_BITFIELD(m->misc, 19, 39);
res->bank_group = GET_BITFIELD(m->misc, 40, 41);
res->bank_address = GET_BITFIELD(m->misc, 42, 43);
res->bank_group |= GET_BITFIELD(m->misc, 44, 44) << 2;
res->rank = GET_BITFIELD(m->misc, 56, 58);
res->dimm = res->rank >> 2;
res->rank = res->rank % 4;
break;
case SPR:
bank = m->bank - 13;
res->imc = bank / 2;
res->channel = bank % 2;
res->column = GET_BITFIELD(m->misc, 9, 18) << 2;
res->row = GET_BITFIELD(m->misc, 19, 36);
res->bank_group = GET_BITFIELD(m->misc, 37, 38);
res->bank_address = GET_BITFIELD(m->misc, 39, 40);
res->bank_group |= GET_BITFIELD(m->misc, 41, 41) << 2;
res->rank = GET_BITFIELD(m->misc, 57, 57);
res->dimm = GET_BITFIELD(m->misc, 58, 58);
break;
default:
return false;
@@ -421,18 +563,101 @@ static bool i10nm_mc_decode(struct decoded_addr *res)
return false;
}
res->column = GET_BITFIELD(m->misc, 9, 18) << 2;
res->row = GET_BITFIELD(m->misc, 19, 39);
res->bank_group = GET_BITFIELD(m->misc, 40, 41);
res->bank_address = GET_BITFIELD(m->misc, 42, 43);
res->bank_group |= GET_BITFIELD(m->misc, 44, 44) << 2;
res->rank = GET_BITFIELD(m->misc, 56, 58);
res->dimm = res->rank >> 2;
res->rank = res->rank % 4;
return true;
}
/**
* get_gnr_mdev() - Get the PCI device of the @logical_idx-th DDR memory controller.
*
* @d : The pointer to the structure of CPU socket EDAC device.
* @logical_idx : The logical index of the present memory controller (0 ~ max present MC# - 1).
* @physical_idx : To store the corresponding physical index of @logical_idx.
*
* RETURNS : The PCI device of the @logical_idx-th DDR memory controller, NULL on failure.
*/
static struct pci_dev *get_gnr_mdev(struct skx_dev *d, int logical_idx, int *physical_idx)
{
#define GNR_MAX_IMC_PCI_CNT 28
struct pci_dev *mdev;
int i, logical = 0;
/*
* Detect present memory controllers from { PCI device: 8-5, function 7-1 }
*/
for (i = 0; i < GNR_MAX_IMC_PCI_CNT; i++) {
mdev = pci_get_dev_wrapper(d->seg,
d->bus[res_cfg->ddr_mdev_bdf.bus],
res_cfg->ddr_mdev_bdf.dev + i / 7,
res_cfg->ddr_mdev_bdf.fun + i % 7);
if (mdev) {
if (logical == logical_idx) {
*physical_idx = i;
return mdev;
}
pci_dev_put(mdev);
logical++;
}
}
return NULL;
}
/**
* get_ddr_munit() - Get the resource of the i-th DDR memory controller.
*
* @d : The pointer to the structure of CPU socket EDAC device.
* @i : The index of the CPU socket relative DDR memory controller.
* @offset : To store the MMIO offset of the i-th DDR memory controller.
* @size : To store the MMIO size of the i-th DDR memory controller.
*
* RETURNS : The PCI device of the i-th DDR memory controller, NULL on failure.
*/
static struct pci_dev *get_ddr_munit(struct skx_dev *d, int i, u32 *offset, unsigned long *size)
{
struct pci_dev *mdev;
int physical_idx;
u32 reg;
switch (res_cfg->type) {
case GNR:
if (I10NM_GET_IMC_BAR(d, 0, reg)) {
i10nm_printk(KERN_ERR, "Failed to get mc0 bar\n");
return NULL;
}
mdev = get_gnr_mdev(d, i, &physical_idx);
if (!mdev)
return NULL;
*offset = I10NM_GET_IMC_MMIO_OFFSET(reg) +
I10NM_GNR_IMC_MMIO_OFFSET +
physical_idx * I10NM_GNR_IMC_MMIO_SIZE;
*size = I10NM_GNR_IMC_MMIO_SIZE;
break;
default:
if (I10NM_GET_IMC_BAR(d, i, reg)) {
i10nm_printk(KERN_ERR, "Failed to get mc%d bar\n", i);
return NULL;
}
mdev = pci_get_dev_wrapper(d->seg,
d->bus[res_cfg->ddr_mdev_bdf.bus],
res_cfg->ddr_mdev_bdf.dev + i,
res_cfg->ddr_mdev_bdf.fun);
if (!mdev)
return NULL;
*offset = I10NM_GET_IMC_MMIO_OFFSET(reg);
*size = I10NM_GET_IMC_MMIO_SIZE(reg);
}
return mdev;
}
static int i10nm_get_ddr_munits(void)
{
struct pci_dev *mdev;
@@ -444,11 +669,15 @@ static int i10nm_get_ddr_munits(void)
u64 base;
list_for_each_entry(d, i10nm_edac_list, list) {
d->util_all = pci_get_dev_wrapper(d->seg, d->bus[1], 29, 1);
d->util_all = pci_get_dev_wrapper(d->seg, d->bus[res_cfg->util_all_bdf.bus],
res_cfg->util_all_bdf.dev,
res_cfg->util_all_bdf.fun);
if (!d->util_all)
return -ENODEV;
d->uracu = pci_get_dev_wrapper(d->seg, d->bus[0], 0, 1);
d->uracu = pci_get_dev_wrapper(d->seg, d->bus[res_cfg->uracu_bdf.bus],
res_cfg->uracu_bdf.dev,
res_cfg->uracu_bdf.fun);
if (!d->uracu)
return -ENODEV;
@@ -461,9 +690,9 @@ static int i10nm_get_ddr_munits(void)
edac_dbg(2, "socket%d mmio base 0x%llx (reg 0x%x)\n",
j++, base, reg);
for (i = 0; i < I10NM_NUM_DDR_IMC; i++) {
mdev = pci_get_dev_wrapper(d->seg, d->bus[0],
12 + i, 0);
for (i = 0; i < res_cfg->ddr_imc_num; i++) {
mdev = get_ddr_munit(d, i, &off, &size);
if (i == 0 && !mdev) {
i10nm_printk(KERN_ERR, "No IMC found\n");
return -ENODEV;
@@ -473,13 +702,6 @@ static int i10nm_get_ddr_munits(void)
d->imc[i].mdev = mdev;
if (I10NM_GET_IMC_BAR(d, i, reg)) {
i10nm_printk(KERN_ERR, "Failed to get mc bar\n");
return -ENODEV;
}
off = I10NM_GET_IMC_MMIO_OFFSET(reg);
size = I10NM_GET_IMC_MMIO_SIZE(reg);
edac_dbg(2, "mc%d mmio base 0x%llx size 0x%lx (reg 0x%x)\n",
i, base + off, size, reg);
@@ -519,7 +741,6 @@ static int i10nm_get_hbm_munits(void)
u64 base;
list_for_each_entry(d, i10nm_edac_list, list) {
d->pcu_cr3 = pci_get_dev_wrapper(d->seg, d->bus[1], 30, 3);
if (!d->pcu_cr3)
return -ENODEV;
@@ -540,11 +761,13 @@ static int i10nm_get_hbm_munits(void)
}
base += I10NM_GET_HBM_IMC_MMIO_OFFSET(reg);
lmc = I10NM_NUM_DDR_IMC;
lmc = res_cfg->ddr_imc_num;
for (i = 0; i < res_cfg->hbm_imc_num; i++) {
mdev = pci_get_dev_wrapper(d->seg, d->bus[res_cfg->hbm_mdev_bdf.bus],
res_cfg->hbm_mdev_bdf.dev + i / 4,
res_cfg->hbm_mdev_bdf.fun + i % 4);
for (i = 0; i < I10NM_NUM_HBM_IMC; i++) {
mdev = pci_get_dev_wrapper(d->seg, d->bus[0],
12 + i / 4, 1 + i % 4);
if (i == 0 && !mdev) {
i10nm_printk(KERN_ERR, "No hbm mc found\n");
return -ENODEV;
@@ -594,8 +817,16 @@ static struct res_config i10nm_cfg0 = {
.type = I10NM,
.decs_did = 0x3452,
.busno_cfg_offset = 0xcc,
.ddr_imc_num = 4,
.ddr_chan_num = 2,
.ddr_dimm_num = 2,
.ddr_chan_mmio_sz = 0x4000,
.sad_all_devfn = PCI_DEVFN(29, 0),
.sad_all_bdf = {1, 29, 0},
.pcu_cr3_bdf = {1, 30, 3},
.util_all_bdf = {1, 29, 1},
.uracu_bdf = {0, 0, 1},
.ddr_mdev_bdf = {0, 12, 0},
.hbm_mdev_bdf = {0, 12, 1},
.sad_all_offset = 0x108,
.offsets_scrub = offsets_scrub_icx,
.offsets_demand = offsets_demand_icx,
@@ -605,8 +836,16 @@ static struct res_config i10nm_cfg1 = {
.type = I10NM,
.decs_did = 0x3452,
.busno_cfg_offset = 0xd0,
.ddr_imc_num = 4,
.ddr_chan_num = 2,
.ddr_dimm_num = 2,
.ddr_chan_mmio_sz = 0x4000,
.sad_all_devfn = PCI_DEVFN(29, 0),
.sad_all_bdf = {1, 29, 0},
.pcu_cr3_bdf = {1, 30, 3},
.util_all_bdf = {1, 29, 1},
.uracu_bdf = {0, 0, 1},
.ddr_mdev_bdf = {0, 12, 0},
.hbm_mdev_bdf = {0, 12, 1},
.sad_all_offset = 0x108,
.offsets_scrub = offsets_scrub_icx,
.offsets_demand = offsets_demand_icx,
@@ -616,10 +855,21 @@ static struct res_config spr_cfg = {
.type = SPR,
.decs_did = 0x3252,
.busno_cfg_offset = 0xd0,
.ddr_imc_num = 4,
.ddr_chan_num = 2,
.ddr_dimm_num = 2,
.hbm_imc_num = 16,
.hbm_chan_num = 2,
.hbm_dimm_num = 1,
.ddr_chan_mmio_sz = 0x8000,
.hbm_chan_mmio_sz = 0x4000,
.support_ddr5 = true,
.sad_all_devfn = PCI_DEVFN(10, 0),
.sad_all_bdf = {1, 10, 0},
.pcu_cr3_bdf = {1, 30, 3},
.util_all_bdf = {1, 29, 1},
.uracu_bdf = {0, 0, 1},
.ddr_mdev_bdf = {0, 12, 0},
.hbm_mdev_bdf = {0, 12, 1},
.sad_all_offset = 0x300,
.offsets_scrub = offsets_scrub_spr,
.offsets_scrub_hbm0 = offsets_scrub_spr_hbm0,
@@ -630,6 +880,23 @@ static struct res_config spr_cfg = {
.offsets_demand_hbm1 = offsets_demand_spr_hbm1,
};
static struct res_config gnr_cfg = {
.type = GNR,
.decs_did = 0x3252,
.busno_cfg_offset = 0xd0,
.ddr_imc_num = 12,
.ddr_chan_num = 1,
.ddr_dimm_num = 2,
.ddr_chan_mmio_sz = 0x4000,
.support_ddr5 = true,
.sad_all_bdf = {0, 13, 0},
.pcu_cr3_bdf = {0, 5, 0},
.util_all_bdf = {0, 13, 1},
.uracu_bdf = {0, 0, 1},
.ddr_mdev_bdf = {0, 5, 1},
.sad_all_offset = 0x300,
};
static const struct x86_cpu_id i10nm_cpuids[] = {
X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS(ATOM_TREMONT_D, X86_STEPPINGS(0x0, 0x3), &i10nm_cfg0),
X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS(ATOM_TREMONT_D, X86_STEPPINGS(0x4, 0xf), &i10nm_cfg1),
@@ -637,6 +904,8 @@ static const struct x86_cpu_id i10nm_cpuids[] = {
X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS(ICELAKE_X, X86_STEPPINGS(0x4, 0xf), &i10nm_cfg1),
X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS(ICELAKE_D, X86_STEPPINGS(0x0, 0xf), &i10nm_cfg1),
X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS(SAPPHIRERAPIDS_X, X86_STEPPINGS(0x0, 0xf), &spr_cfg),
X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS(EMERALDRAPIDS_X, X86_STEPPINGS(0x0, 0xf), &spr_cfg),
X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS(GRANITERAPIDS_X, X86_STEPPINGS(0x0, 0xf), &gnr_cfg),
{}
};
MODULE_DEVICE_TABLE(x86cpu, i10nm_cpuids);
@@ -656,7 +925,7 @@ static int i10nm_get_dimm_config(struct mem_ctl_info *mci,
{
struct skx_pvt *pvt = mci->pvt_info;
struct skx_imc *imc = pvt->imc;
u32 mtr, amap, mcddrtcfg;
u32 mtr, amap, mcddrtcfg = 0;
struct dimm_info *dimm;
int i, j, ndimms;
@@ -666,7 +935,10 @@ static int i10nm_get_dimm_config(struct mem_ctl_info *mci,
ndimms = 0;
amap = I10NM_GET_AMAP(imc, i);
mcddrtcfg = I10NM_GET_MCDDRTCFG(imc, i);
if (res_cfg->type != GNR)
mcddrtcfg = I10NM_GET_MCDDRTCFG(imc, i);
for (j = 0; j < imc->num_dimms; j++) {
dimm = edac_get_dimm(mci, i, j, 0);
mtr = I10NM_GET_DIMMMTR(imc, i, j);
@@ -752,6 +1024,7 @@ static int __init i10nm_init(void)
struct skx_dev *d;
int rc, i, off[3] = {0xd0, 0xc8, 0xcc};
u64 tolm, tohm;
int imc_num;
edac_dbg(2, "\n");
@@ -784,6 +1057,10 @@ static int __init i10nm_init(void)
return -ENODEV;
}
rc = i10nm_get_imc_num(cfg);
if (rc < 0)
goto fail;
mem_cfg_2lm = i10nm_check_2lm(cfg);
skx_set_mem_cfg(mem_cfg_2lm);
@@ -792,6 +1069,8 @@ static int __init i10nm_init(void)
if (i10nm_get_hbm_munits() && rc)
goto fail;
imc_num = res_cfg->ddr_imc_num + res_cfg->hbm_imc_num;
list_for_each_entry(d, i10nm_edac_list, list) {
rc = skx_get_src_id(d, 0xf8, &src_id);
if (rc < 0)
@@ -802,7 +1081,7 @@ static int __init i10nm_init(void)
goto fail;
edac_dbg(2, "src_id = %d node_id = %d\n", src_id, node_id);
for (i = 0; i < I10NM_NUM_IMC; i++) {
for (i = 0; i < imc_num; i++) {
if (!d->imc[i].mdev)
continue;
@@ -812,12 +1091,12 @@ static int __init i10nm_init(void)
d->imc[i].node_id = node_id;
if (d->imc[i].hbm_mc) {
d->imc[i].chan_mmio_sz = cfg->hbm_chan_mmio_sz;
d->imc[i].num_channels = I10NM_NUM_HBM_CHANNELS;
d->imc[i].num_dimms = I10NM_NUM_HBM_DIMMS;
d->imc[i].num_channels = cfg->hbm_chan_num;
d->imc[i].num_dimms = cfg->hbm_dimm_num;
} else {
d->imc[i].chan_mmio_sz = cfg->ddr_chan_mmio_sz;
d->imc[i].num_channels = I10NM_NUM_DDR_CHANNELS;
d->imc[i].num_dimms = I10NM_NUM_DDR_DIMMS;
d->imc[i].num_channels = cfg->ddr_chan_num;
d->imc[i].num_dimms = cfg->ddr_dimm_num;
}
rc = skx_register_mci(&d->imc[i], d->imc[i].mdev,
drivers/edac/qcom_edac.c
+7
View File
@@ -396,12 +396,19 @@ static int qcom_llcc_edac_remove(struct platform_device *pdev)
return 0;
}
static const struct platform_device_id qcom_llcc_edac_id_table[] = {
{ .name = "qcom_llcc_edac" },
{}
};
MODULE_DEVICE_TABLE(platform, qcom_llcc_edac_id_table);
static struct platform_driver qcom_llcc_edac_driver = {
.probe = qcom_llcc_edac_probe,
.remove = qcom_llcc_edac_remove,
.driver = {
.name = "qcom_llcc_edac",
},
.id_table = qcom_llcc_edac_id_table,
};
module_platform_driver(qcom_llcc_edac_driver);
drivers/edac/skx_common.c
+34 -44
View File
@@ -560,44 +560,28 @@ static void skx_mce_output_error(struct mem_ctl_info *mci,
tp_event = HW_EVENT_ERR_CORRECTED;
}
/*
* According to Intel Architecture spec vol 3B,
* Table 15-10 "IA32_MCi_Status [15:0] Compound Error Code Encoding"
* memory errors should fit one of these masks:
* 000f 0000 1mmm cccc (binary)
* 000f 0010 1mmm cccc (binary) [RAM used as cache]
* where:
* f = Correction Report Filtering Bit. If 1, subsequent errors
* won't be shown
* mmm = error type
* cccc = channel
* If the mask doesn't match, report an error to the parsing logic
*/
if (!((errcode & 0xef80) == 0x80 || (errcode & 0xef80) == 0x280)) {
optype = "Can't parse: it is not a mem";
} else {
switch (optypenum) {
case 0:
optype = "generic undef request error";
break;
case 1:
optype = "memory read error";
break;
case 2:
optype = "memory write error";
break;
case 3:
optype = "addr/cmd error";
break;
case 4:
optype = "memory scrubbing error";
scrub_err = true;
break;
default:
optype = "reserved";
break;
}
switch (optypenum) {
case 0:
optype = "generic undef request error";
break;
case 1:
optype = "memory read error";
break;
case 2:
optype = "memory write error";
break;
case 3:
optype = "addr/cmd error";
break;
case 4:
optype = "memory scrubbing error";
scrub_err = true;
break;
default:
optype = "reserved";
break;
}
if (res->decoded_by_adxl) {
len = snprintf(skx_msg, MSG_SIZE, "%s%s err_code:0x%04x:0x%04x %s",
overflow ? " OVERFLOW" : "",
@@ -632,12 +616,18 @@ static bool skx_error_in_1st_level_mem(const struct mce *m)
if (!skx_mem_cfg_2lm)
return false;
errcode = GET_BITFIELD(m->status, 0, 15);
errcode = GET_BITFIELD(m->status, 0, 15) & MCACOD_MEM_ERR_MASK;
if ((errcode & 0xef80) != 0x280)
return false;
return errcode == MCACOD_EXT_MEM_ERR;
}
return true;
static bool skx_error_in_mem(const struct mce *m)
{
u32 errcode;
errcode = GET_BITFIELD(m->status, 0, 15) & MCACOD_MEM_ERR_MASK;
return (errcode == MCACOD_MEM_CTL_ERR || errcode == MCACOD_EXT_MEM_ERR);
}
int skx_mce_check_error(struct notifier_block *nb, unsigned long val,
@@ -651,13 +641,13 @@ int skx_mce_check_error(struct notifier_block *nb, unsigned long val,
if (mce->kflags & MCE_HANDLED_CEC)
return NOTIFY_DONE;
/* ignore unless this is memory related with an address */
if ((mce->status & 0xefff) >> 7 != 1 || !(mce->status & MCI_STATUS_ADDRV))
/* Ignore unless this is memory related with an address */
if (!skx_error_in_mem(mce) || !(mce->status & MCI_STATUS_ADDRV))
return NOTIFY_DONE;
memset(&res, 0, sizeof(res));
res.mce = mce;
res.addr = mce->addr;
res.addr = mce->addr & MCI_ADDR_PHYSADDR;
/* Try driver decoder first */
if (!(driver_decode && driver_decode(&res))) {
drivers/edac/skx_common.h
+57 -4
View File
@@ -33,7 +33,7 @@
#define SKX_NUM_CHANNELS 3 /* Channels per memory controller */
#define SKX_NUM_DIMMS 2 /* Max DIMMS per channel */
#define I10NM_NUM_DDR_IMC 4
#define I10NM_NUM_DDR_IMC 12
#define I10NM_NUM_DDR_CHANNELS 2
#define I10NM_NUM_DDR_DIMMS 2
@@ -56,6 +56,30 @@
#define MCI_MISC_ECC_MODE(m) (((m) >> 59) & 15)
#define MCI_MISC_ECC_DDRT 8 /* read from DDRT */
/*
* According to Intel Architecture spec vol 3B,
* Table 15-10 "IA32_MCi_Status [15:0] Compound Error Code Encoding"
* memory errors should fit one of these masks:
* 000f 0000 1mmm cccc (binary)
* 000f 0010 1mmm cccc (binary) [RAM used as cache]
* where:
* f = Correction Report Filtering Bit. If 1, subsequent errors
* won't be shown
* mmm = error type
* cccc = channel
*/
#define MCACOD_MEM_ERR_MASK 0xef80
/*
* Errors from either the memory of the 1-level memory system or the
* 2nd level memory (the slow "far" memory) of the 2-level memory system.
*/
#define MCACOD_MEM_CTL_ERR 0x80
/*
* Errors from the 1st level memory (the fast "near" memory as cache)
* of the 2-level memory system.
*/
#define MCACOD_EXT_MEM_ERR 0x280
/*
* Each cpu socket contains some pci devices that provide global
* information, and also some that are local to each of the two
@@ -105,7 +129,8 @@ struct skx_pvt {
enum type {
SKX,
I10NM,
SPR
SPR,
GNR
};
enum {
@@ -149,19 +174,47 @@ struct decoded_addr {
bool decoded_by_adxl;
};
struct pci_bdf {
u32 bus : 8;
u32 dev : 5;
u32 fun : 3;
};
struct res_config {
enum type type;
/* Configuration agent device ID */
unsigned int decs_did;
/* Default bus number configuration register offset */
int busno_cfg_offset;
/* DDR memory controllers per socket */
int ddr_imc_num;
/* DDR channels per DDR memory controller */
int ddr_chan_num;
/* DDR DIMMs per DDR memory channel */
int ddr_dimm_num;
/* Per DDR channel memory-mapped I/O size */
int ddr_chan_mmio_sz;
/* HBM memory controllers per socket */
int hbm_imc_num;
/* HBM channels per HBM memory controller */
int hbm_chan_num;
/* HBM DIMMs per HBM memory channel */
int hbm_dimm_num;
/* Per HBM channel memory-mapped I/O size */
int hbm_chan_mmio_sz;
bool support_ddr5;
/* SAD device number and function number */
unsigned int sad_all_devfn;
/* SAD device BDF */
struct pci_bdf sad_all_bdf;
/* PCU device BDF */
struct pci_bdf pcu_cr3_bdf;
/* UTIL device BDF */
struct pci_bdf util_all_bdf;
/* URACU device BDF */
struct pci_bdf uracu_bdf;
/* DDR mdev device BDF */
struct pci_bdf ddr_mdev_bdf;
/* HBM mdev device BDF */
struct pci_bdf hbm_mdev_bdf;
int sad_all_offset;
/* Offsets of retry_rd_err_log registers */
u32 *offsets_scrub;
drivers/edac/zynqmp_edac.c
+467
View File
@@ -0,0 +1,467 @@
// SPDX-License-Identifier: GPL-2.0
/*
* Xilinx ZynqMP OCM ECC Driver
*
* Copyright (C) 2022 Advanced Micro Devices, Inc.
*/
#include <linux/edac.h>
#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_platform.h>
#include <linux/platform_device.h>
#include "edac_module.h"
#define ZYNQMP_OCM_EDAC_MSG_SIZE 256
#define ZYNQMP_OCM_EDAC_STRING "zynqmp_ocm"
/* Error/Interrupt registers */
#define ERR_CTRL_OFST 0x0
#define OCM_ISR_OFST 0x04
#define OCM_IMR_OFST 0x08
#define OCM_IEN_OFST 0x0C
#define OCM_IDS_OFST 0x10
/* ECC control register */
#define ECC_CTRL_OFST 0x14
/* Correctable error info registers */
#define CE_FFA_OFST 0x1C
#define CE_FFD0_OFST 0x20
#define CE_FFD1_OFST 0x24
#define CE_FFD2_OFST 0x28
#define CE_FFD3_OFST 0x2C
#define CE_FFE_OFST 0x30
/* Uncorrectable error info registers */
#define UE_FFA_OFST 0x34
#define UE_FFD0_OFST 0x38
#define UE_FFD1_OFST 0x3C
#define UE_FFD2_OFST 0x40
#define UE_FFD3_OFST 0x44
#define UE_FFE_OFST 0x48
/* ECC control register bit field definitions */
#define ECC_CTRL_CLR_CE_ERR 0x40
#define ECC_CTRL_CLR_UE_ERR 0x80
/* Fault injection data and count registers */
#define OCM_FID0_OFST 0x4C
#define OCM_FID1_OFST 0x50
#define OCM_FID2_OFST 0x54
#define OCM_FID3_OFST 0x58
#define OCM_FIC_OFST 0x74
#define UE_MAX_BITPOS_LOWER 31
#define UE_MIN_BITPOS_UPPER 32
#define UE_MAX_BITPOS_UPPER 63
/* Interrupt masks */
#define OCM_CEINTR_MASK BIT(6)
#define OCM_UEINTR_MASK BIT(7)
#define OCM_ECC_ENABLE_MASK BIT(0)
#define OCM_FICOUNT_MASK GENMASK(23, 0)
#define OCM_NUM_UE_BITPOS 2
#define OCM_BASEVAL 0xFFFC0000
#define EDAC_DEVICE "ZynqMP-OCM"
/**
* struct ecc_error_info - ECC error log information
* @addr: Fault generated at this address
* @fault_lo: Generated fault data (lower 32-bit)
* @fault_hi: Generated fault data (upper 32-bit)
*/
struct ecc_error_info {
u32 addr;
u32 fault_lo;
u32 fault_hi;
};
/**
* struct ecc_status - ECC status information to report
* @ce_cnt: Correctable error count
* @ue_cnt: Uncorrectable error count
* @ceinfo: Correctable error log information
* @ueinfo: Uncorrectable error log information
*/
struct ecc_status {
u32 ce_cnt;
u32 ue_cnt;
struct ecc_error_info ceinfo;
struct ecc_error_info ueinfo;
};
/**
* struct edac_priv - OCM private instance data
* @baseaddr: Base address of the OCM
* @message: Buffer for framing the event specific info
* @stat: ECC status information
* @ce_cnt: Correctable Error count
* @ue_cnt: Uncorrectable Error count
* @debugfs_dir: Directory entry for debugfs
* @ce_bitpos: Bit position for Correctable Error
* @ue_bitpos: Array to store UnCorrectable Error bit positions
* @fault_injection_cnt: Fault Injection Counter value
*/
struct edac_priv {
void __iomem *baseaddr;
char message[ZYNQMP_OCM_EDAC_MSG_SIZE];
struct ecc_status stat;
u32 ce_cnt;
u32 ue_cnt;
#ifdef CONFIG_EDAC_DEBUG
struct dentry *debugfs_dir;
u8 ce_bitpos;
u8 ue_bitpos[OCM_NUM_UE_BITPOS];
u32 fault_injection_cnt;
#endif
};
/**
* get_error_info - Get the current ECC error info
* @base: Pointer to the base address of the OCM
* @p: Pointer to the OCM ECC status structure
* @mask: Status register mask value
*
* Determines there is any ECC error or not
*
*/
static void get_error_info(void __iomem *base, struct ecc_status *p, int mask)
{
if (mask & OCM_CEINTR_MASK) {
p->ce_cnt++;
p->ceinfo.fault_lo = readl(base + CE_FFD0_OFST);
p->ceinfo.fault_hi = readl(base + CE_FFD1_OFST);
p->ceinfo.addr = (OCM_BASEVAL | readl(base + CE_FFA_OFST));
writel(ECC_CTRL_CLR_CE_ERR, base + OCM_ISR_OFST);
} else if (mask & OCM_UEINTR_MASK) {
p->ue_cnt++;
p->ueinfo.fault_lo = readl(base + UE_FFD0_OFST);
p->ueinfo.fault_hi = readl(base + UE_FFD1_OFST);
p->ueinfo.addr = (OCM_BASEVAL | readl(base + UE_FFA_OFST));
writel(ECC_CTRL_CLR_UE_ERR, base + OCM_ISR_OFST);
}
}
/**
* handle_error - Handle error types CE and UE
* @dci: Pointer to the EDAC device instance
* @p: Pointer to the OCM ECC status structure
*
* Handles correctable and uncorrectable errors.
*/
static void handle_error(struct edac_device_ctl_info *dci, struct ecc_status *p)
{
struct edac_priv *priv = dci->pvt_info;
struct ecc_error_info *pinf;
if (p->ce_cnt) {
pinf = &p->ceinfo;
snprintf(priv->message, ZYNQMP_OCM_EDAC_MSG_SIZE,
"\nOCM ECC error type :%s\nAddr: [0x%x]\nFault Data[0x%08x%08x]",
"CE", pinf->addr, pinf->fault_hi, pinf->fault_lo);
edac_device_handle_ce(dci, 0, 0, priv->message);
}
if (p->ue_cnt) {
pinf = &p->ueinfo;
snprintf(priv->message, ZYNQMP_OCM_EDAC_MSG_SIZE,
"\nOCM ECC error type :%s\nAddr: [0x%x]\nFault Data[0x%08x%08x]",
"UE", pinf->addr, pinf->fault_hi, pinf->fault_lo);
edac_device_handle_ue(dci, 0, 0, priv->message);
}
memset(p, 0, sizeof(*p));
}
/**
* intr_handler - ISR routine
* @irq: irq number
* @dev_id: device id pointer
*
* Return: IRQ_NONE, if CE/UE interrupt not set or IRQ_HANDLED otherwise
*/
static irqreturn_t intr_handler(int irq, void *dev_id)
{
struct edac_device_ctl_info *dci = dev_id;
struct edac_priv *priv = dci->pvt_info;
int regval;
regval = readl(priv->baseaddr + OCM_ISR_OFST);
if (!(regval & (OCM_CEINTR_MASK | OCM_UEINTR_MASK))) {
WARN_ONCE(1, "Unhandled IRQ%d, ISR: 0x%x", irq, regval);
return IRQ_NONE;
}
get_error_info(priv->baseaddr, &priv->stat, regval);
priv->ce_cnt += priv->stat.ce_cnt;
priv->ue_cnt += priv->stat.ue_cnt;
handle_error(dci, &priv->stat);
return IRQ_HANDLED;
}
/**
* get_eccstate - Return the ECC status
* @base: Pointer to the OCM base address
*
* Get the ECC enable/disable status
*
* Return: ECC status 0/1.
*/
static bool get_eccstate(void __iomem *base)
{
return readl(base + ECC_CTRL_OFST) & OCM_ECC_ENABLE_MASK;
}
#ifdef CONFIG_EDAC_DEBUG
/**
* write_fault_count - write fault injection count
* @priv: Pointer to the EDAC private struct
*
* Update the fault injection count register, once the counter reaches
* zero, it injects errors
*/
static void write_fault_count(struct edac_priv *priv)
{
u32 ficount = priv->fault_injection_cnt;
if (ficount & ~OCM_FICOUNT_MASK) {
ficount &= OCM_FICOUNT_MASK;
edac_printk(KERN_INFO, EDAC_DEVICE,
"Fault injection count value truncated to %d\n", ficount);
}
writel(ficount, priv->baseaddr + OCM_FIC_OFST);
}
/*
* To get the Correctable Error injected, the following steps are needed:
* - Setup the optional Fault Injection Count:
* echo <fault_count val> > /sys/kernel/debug/edac/ocm/inject_fault_count
* - Write the Correctable Error bit position value:
* echo <bit_pos val> > /sys/kernel/debug/edac/ocm/inject_ce_bitpos
*/
static ssize_t inject_ce_write(struct file *file, const char __user *data,
size_t count, loff_t *ppos)
{
struct edac_device_ctl_info *edac_dev = file->private_data;
struct edac_priv *priv = edac_dev->pvt_info;
int ret;
if (!data)
return -EFAULT;
ret = kstrtou8_from_user(data, count, 0, &priv->ce_bitpos);
if (ret)
return ret;
if (priv->ce_bitpos > UE_MAX_BITPOS_UPPER)
return -EINVAL;
if (priv->ce_bitpos <= UE_MAX_BITPOS_LOWER) {
writel(BIT(priv->ce_bitpos), priv->baseaddr + OCM_FID0_OFST);
writel(0, priv->baseaddr + OCM_FID1_OFST);
} else {
writel(BIT(priv->ce_bitpos - UE_MIN_BITPOS_UPPER),
priv->baseaddr + OCM_FID1_OFST);
writel(0, priv->baseaddr + OCM_FID0_OFST);
}
write_fault_count(priv);
return count;
}
static const struct file_operations inject_ce_fops = {
.open = simple_open,
.write = inject_ce_write,
.llseek = generic_file_llseek,
};
/*
* To get the Uncorrectable Error injected, the following steps are needed:
* - Setup the optional Fault Injection Count:
* echo <fault_count val> > /sys/kernel/debug/edac/ocm/inject_fault_count
* - Write the Uncorrectable Error bit position values:
* echo <bit_pos0 val>,<bit_pos1 val> > /sys/kernel/debug/edac/ocm/inject_ue_bitpos
*/
static ssize_t inject_ue_write(struct file *file, const char __user *data,
size_t count, loff_t *ppos)
{
struct edac_device_ctl_info *edac_dev = file->private_data;
struct edac_priv *priv = edac_dev->pvt_info;
char buf[6], *pbuf, *token[2];
u64 ue_bitpos;
int i, ret;
u8 len;
if (!data)
return -EFAULT;
len = min_t(size_t, count, sizeof(buf));
if (copy_from_user(buf, data, len))
return -EFAULT;
buf[len] = '\0';
pbuf = &buf[0];
for (i = 0; i < OCM_NUM_UE_BITPOS; i++)
token[i] = strsep(&pbuf, ",");
ret = kstrtou8(token[0], 0, &priv->ue_bitpos[0]);
if (ret)
return ret;
ret = kstrtou8(token[1], 0, &priv->ue_bitpos[1]);
if (ret)
return ret;
if (priv->ue_bitpos[0] > UE_MAX_BITPOS_UPPER ||
priv->ue_bitpos[1] > UE_MAX_BITPOS_UPPER)
return -EINVAL;
if (priv->ue_bitpos[0] == priv->ue_bitpos[1]) {
edac_printk(KERN_ERR, EDAC_DEVICE, "Bit positions should not be equal\n");
return -EINVAL;
}
ue_bitpos = BIT(priv->ue_bitpos[0]) | BIT(priv->ue_bitpos[1]);
writel((u32)ue_bitpos, priv->baseaddr + OCM_FID0_OFST);
writel((u32)(ue_bitpos >> 32), priv->baseaddr + OCM_FID1_OFST);
write_fault_count(priv);
return count;
}
static const struct file_operations inject_ue_fops = {
.open = simple_open,
.write = inject_ue_write,
.llseek = generic_file_llseek,
};
static void setup_debugfs(struct edac_device_ctl_info *edac_dev)
{
struct edac_priv *priv = edac_dev->pvt_info;
priv->debugfs_dir = edac_debugfs_create_dir("ocm");
if (!priv->debugfs_dir)
return;
edac_debugfs_create_x32("inject_fault_count", 0644, priv->debugfs_dir,
&priv->fault_injection_cnt);
edac_debugfs_create_file("inject_ue_bitpos", 0644, priv->debugfs_dir,
edac_dev, &inject_ue_fops);
edac_debugfs_create_file("inject_ce_bitpos", 0644, priv->debugfs_dir,
edac_dev, &inject_ce_fops);
}
#endif
static int edac_probe(struct platform_device *pdev)
{
struct edac_device_ctl_info *dci;
struct edac_priv *priv;
void __iomem *baseaddr;
struct resource *res;
int irq, ret;
baseaddr = devm_platform_get_and_ioremap_resource(pdev, 0, &res);
if (IS_ERR(baseaddr))
return PTR_ERR(baseaddr);
if (!get_eccstate(baseaddr)) {
edac_printk(KERN_INFO, EDAC_DEVICE, "ECC not enabled\n");
return -ENXIO;
}
dci = edac_device_alloc_ctl_info(sizeof(*priv), ZYNQMP_OCM_EDAC_STRING,
1, ZYNQMP_OCM_EDAC_STRING, 1, 0, NULL, 0,
edac_device_alloc_index());
if (!dci)
return -ENOMEM;
priv = dci->pvt_info;
platform_set_drvdata(pdev, dci);
dci->dev = &pdev->dev;
priv->baseaddr = baseaddr;
dci->mod_name = pdev->dev.driver->name;
dci->ctl_name = ZYNQMP_OCM_EDAC_STRING;
dci->dev_name = dev_name(&pdev->dev);
irq = platform_get_irq(pdev, 0);
if (irq < 0) {
ret = irq;
goto free_dev_ctl;
}
ret = devm_request_irq(&pdev->dev, irq, intr_handler, 0,
dev_name(&pdev->dev), dci);
if (ret) {
edac_printk(KERN_ERR, EDAC_DEVICE, "Failed to request Irq\n");
goto free_dev_ctl;
}
/* Enable UE, CE interrupts */
writel((OCM_CEINTR_MASK | OCM_UEINTR_MASK), priv->baseaddr + OCM_IEN_OFST);
#ifdef CONFIG_EDAC_DEBUG
setup_debugfs(dci);
#endif
ret = edac_device_add_device(dci);
if (ret)
goto free_dev_ctl;
return 0;
free_dev_ctl:
edac_device_free_ctl_info(dci);
return ret;
}
static int edac_remove(struct platform_device *pdev)
{
struct edac_device_ctl_info *dci = platform_get_drvdata(pdev);
struct edac_priv *priv = dci->pvt_info;
/* Disable UE, CE interrupts */
writel((OCM_CEINTR_MASK | OCM_UEINTR_MASK), priv->baseaddr + OCM_IDS_OFST);
#ifdef CONFIG_EDAC_DEBUG
debugfs_remove_recursive(priv->debugfs_dir);
#endif
edac_device_del_device(&pdev->dev);
edac_device_free_ctl_info(dci);
return 0;
}
static const struct of_device_id zynqmp_ocm_edac_match[] = {
{ .compatible = "xlnx,zynqmp-ocmc-1.0"},
{ /* end of table */ }
};
MODULE_DEVICE_TABLE(of, zynqmp_ocm_edac_match);
static struct platform_driver zynqmp_ocm_edac_driver = {
.driver = {
.name = "zynqmp-ocm-edac",
.of_match_table = zynqmp_ocm_edac_match,
},
.probe = edac_probe,
.remove = edac_remove,
};
module_platform_driver(zynqmp_ocm_edac_driver);
MODULE_AUTHOR("Advanced Micro Devices, Inc");
MODULE_DESCRIPTION("Xilinx ZynqMP OCM ECC driver");
MODULE_LICENSE("GPL");
include/linux/bits.h
-1
View File
@@ -6,7 +6,6 @@
#include <vdso/bits.h>
#include <asm/bitsperlong.h>
#define BIT_ULL(nr) (ULL(1) << (nr))
#define BIT_MASK(nr) (UL(1) << ((nr) % BITS_PER_LONG))
#define BIT_WORD(nr) ((nr) / BITS_PER_LONG)
#define BIT_ULL_MASK(nr) (ULL(1) << ((nr) % BITS_PER_LONG_LONG))
include/linux/resctrl.h
+11
View File
@@ -250,6 +250,17 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,
void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_domain *d,
u32 rmid, enum resctrl_event_id eventid);
/**
* resctrl_arch_reset_rmid_all() - Reset all private state associated with
* all rmids and eventids.
* @r: The resctrl resource.
* @d: The domain for which all architectural counter state will
* be cleared.
*
* This can be called from any CPU.
*/
void resctrl_arch_reset_rmid_all(struct rdt_resource *r, struct rdt_domain *d);
extern unsigned int resctrl_rmid_realloc_threshold;
extern unsigned int resctrl_rmid_realloc_limit;
include/vdso/bits.h
+1
View File
@@ -5,5 +5,6 @@
#include <vdso/const.h>
#define BIT(nr) (UL(1) << (nr))
#define BIT_ULL(nr) (ULL(1) << (nr))
#endif /* __VDSO_BITS_H */
kernel/time/Kconfig
+5 -1
View File
@@ -200,10 +200,14 @@ config CLOCKSOURCE_WATCHDOG_MAX_SKEW_US
int "Clocksource watchdog maximum allowable skew (in μs)"
depends on CLOCKSOURCE_WATCHDOG
range 50 1000
default 100
default 125
help
Specify the maximum amount of allowable watchdog skew in
microseconds before reporting the clocksource to be unstable.
The default is based on a half-second clocksource watchdog
interval and NTP's maximum frequency drift of 500 parts
per million. If the clocksource is good enough for NTP,
it is good enough for the clocksource watchdog!
endmenu
endif
kernel/time/clocksource.c
+51 -21
View File
@@ -95,6 +95,11 @@ static char override_name[CS_NAME_LEN];
static int finished_booting;
static u64 suspend_start;
/*
* Interval: 0.5sec.
*/
#define WATCHDOG_INTERVAL (HZ >> 1)
/*
* Threshold: 0.0312s, when doubled: 0.0625s.
* Also a default for cs->uncertainty_margin when registering clocks.
@@ -106,11 +111,14 @@ static u64 suspend_start;
* clocksource surrounding a read of the clocksource being validated.
* This delay could be due to SMIs, NMIs, or to VCPU preemptions. Used as
* a lower bound for cs->uncertainty_margin values when registering clocks.
*
* The default of 500 parts per million is based on NTP's limits.
* If a clocksource is good enough for NTP, it is good enough for us!
*/
#ifdef CONFIG_CLOCKSOURCE_WATCHDOG_MAX_SKEW_US
#define MAX_SKEW_USEC CONFIG_CLOCKSOURCE_WATCHDOG_MAX_SKEW_US
#else
#define MAX_SKEW_USEC 100
#define MAX_SKEW_USEC (125 * WATCHDOG_INTERVAL / HZ)
#endif
#define WATCHDOG_MAX_SKEW (MAX_SKEW_USEC * NSEC_PER_USEC)
@@ -140,11 +148,6 @@ static inline void clocksource_watchdog_unlock(unsigned long *flags)
static int clocksource_watchdog_kthread(void *data);
static void __clocksource_change_rating(struct clocksource *cs, int rating);
/*
* Interval: 0.5sec.
*/
#define WATCHDOG_INTERVAL (HZ >> 1)
static void clocksource_watchdog_work(struct work_struct *work)
{
/*
@@ -257,8 +260,8 @@ static enum wd_read_status cs_watchdog_read(struct clocksource *cs, u64 *csnow,
goto skip_test;
}
pr_warn("timekeeping watchdog on CPU%d: %s read-back delay of %lldns, attempt %d, marking unstable\n",
smp_processor_id(), watchdog->name, wd_delay, nretries);
pr_warn("timekeeping watchdog on CPU%d: wd-%s-wd excessive read-back delay of %lldns vs. limit of %ldns, wd-wd read-back delay only %lldns, attempt %d, marking %s unstable\n",
smp_processor_id(), cs->name, wd_delay, WATCHDOG_MAX_SKEW, wd_seq_delay, nretries, cs->name);
return WD_READ_UNSTABLE;
skip_test:
@@ -384,6 +387,15 @@ void clocksource_verify_percpu(struct clocksource *cs)
}
EXPORT_SYMBOL_GPL(clocksource_verify_percpu);
static inline void clocksource_reset_watchdog(void)
{
struct clocksource *cs;
list_for_each_entry(cs, &watchdog_list, wd_list)
cs->flags &= ~CLOCK_SOURCE_WATCHDOG;
}
static void clocksource_watchdog(struct timer_list *unused)
{
u64 csnow, wdnow, cslast, wdlast, delta;
@@ -391,6 +403,7 @@ static void clocksource_watchdog(struct timer_list *unused)
int64_t wd_nsec, cs_nsec;
struct clocksource *cs;
enum wd_read_status read_ret;
unsigned long extra_wait = 0;
u32 md;
spin_lock(&watchdog_lock);
@@ -410,13 +423,30 @@ static void clocksource_watchdog(struct timer_list *unused)
read_ret = cs_watchdog_read(cs, &csnow, &wdnow);
if (read_ret != WD_READ_SUCCESS) {
if (read_ret == WD_READ_UNSTABLE)
/* Clock readout unreliable, so give it up. */
__clocksource_unstable(cs);
if (read_ret == WD_READ_UNSTABLE) {
/* Clock readout unreliable, so give it up. */
__clocksource_unstable(cs);
continue;
}
/*
* When WD_READ_SKIP is returned, it means the system is likely
* under very heavy load, where the latency of reading
* watchdog/clocksource is very big, and affect the accuracy of
* watchdog check. So give system some space and suspend the
* watchdog check for 5 minutes.
*/
if (read_ret == WD_READ_SKIP) {
/*
* As the watchdog timer will be suspended, and
* cs->last could keep unchanged for 5 minutes, reset
* the counters.
*/
clocksource_reset_watchdog();
extra_wait = HZ * 300;
break;
}
/* Clocksource initialized ? */
if (!(cs->flags & CLOCK_SOURCE_WATCHDOG) ||
atomic_read(&watchdog_reset_pending)) {
@@ -443,12 +473,20 @@ static void clocksource_watchdog(struct timer_list *unused)
/* Check the deviation from the watchdog clocksource. */
md = cs->uncertainty_margin + watchdog->uncertainty_margin;
if (abs(cs_nsec - wd_nsec) > md) {
u64 cs_wd_msec;
u64 wd_msec;
u32 wd_rem;
pr_warn("timekeeping watchdog on CPU%d: Marking clocksource '%s' as unstable because the skew is too large:\n",
smp_processor_id(), cs->name);
pr_warn(" '%s' wd_nsec: %lld wd_now: %llx wd_last: %llx mask: %llx\n",
watchdog->name, wd_nsec, wdnow, wdlast, watchdog->mask);
pr_warn(" '%s' cs_nsec: %lld cs_now: %llx cs_last: %llx mask: %llx\n",
cs->name, cs_nsec, csnow, cslast, cs->mask);
cs_wd_msec = div_u64_rem(cs_nsec - wd_nsec, 1000U * 1000U, &wd_rem);
wd_msec = div_u64_rem(wd_nsec, 1000U * 1000U, &wd_rem);
pr_warn(" Clocksource '%s' skewed %lld ns (%lld ms) over watchdog '%s' interval of %lld ns (%lld ms)\n",
cs->name, cs_nsec - wd_nsec, cs_wd_msec, watchdog->name, wd_nsec, wd_msec);
if (curr_clocksource == cs)
pr_warn(" '%s' is current clocksource.\n", cs->name);
else if (curr_clocksource)
@@ -512,7 +550,7 @@ static void clocksource_watchdog(struct timer_list *unused)
* pair clocksource_stop_watchdog() clocksource_start_watchdog().
*/
if (!timer_pending(&watchdog_timer)) {
watchdog_timer.expires += WATCHDOG_INTERVAL;
watchdog_timer.expires += WATCHDOG_INTERVAL + extra_wait;
add_timer_on(&watchdog_timer, next_cpu);
}
out:
@@ -537,14 +575,6 @@ static inline void clocksource_stop_watchdog(void)
watchdog_running = 0;
}
static inline void clocksource_reset_watchdog(void)
{
struct clocksource *cs;
list_for_each_entry(cs, &watchdog_list, wd_list)
cs->flags &= ~CLOCK_SOURCE_WATCHDOG;
}
static void clocksource_resume_watchdog(void)
{
atomic_inc(&watchdog_reset_pending);
kernel/time/hrtimer.c
+14 -4
View File
@@ -2089,7 +2089,7 @@ long hrtimer_nanosleep(ktime_t rqtp, const enum hrtimer_mode mode,
u64 slack;
slack = current->timer_slack_ns;
if (dl_task(current) || rt_task(current))
if (rt_task(current))
slack = 0;
hrtimer_init_sleeper_on_stack(&t, clockid, mode);
@@ -2126,6 +2126,7 @@ SYSCALL_DEFINE2(nanosleep, struct __kernel_timespec __user *, rqtp,
if (!timespec64_valid(&tu))
return -EINVAL;
current->restart_block.fn = do_no_restart_syscall;
current->restart_block.nanosleep.type = rmtp ? TT_NATIVE : TT_NONE;
current->restart_block.nanosleep.rmtp = rmtp;
return hrtimer_nanosleep(timespec64_to_ktime(tu), HRTIMER_MODE_REL,
@@ -2147,6 +2148,7 @@ SYSCALL_DEFINE2(nanosleep_time32, struct old_timespec32 __user *, rqtp,
if (!timespec64_valid(&tu))
return -EINVAL;
current->restart_block.fn = do_no_restart_syscall;
current->restart_block.nanosleep.type = rmtp ? TT_COMPAT : TT_NONE;
current->restart_block.nanosleep.compat_rmtp = rmtp;
return hrtimer_nanosleep(timespec64_to_ktime(tu), HRTIMER_MODE_REL,
@@ -2270,7 +2272,7 @@ void __init hrtimers_init(void)
/**
* schedule_hrtimeout_range_clock - sleep until timeout
* @expires: timeout value (ktime_t)
* @delta: slack in expires timeout (ktime_t)
* @delta: slack in expires timeout (ktime_t) for SCHED_OTHER tasks
* @mode: timer mode
* @clock_id: timer clock to be used
*/
@@ -2297,6 +2299,13 @@ schedule_hrtimeout_range_clock(ktime_t *expires, u64 delta,
return -EINTR;
}
/*
* Override any slack passed by the user if under
* rt contraints.
*/
if (rt_task(current))
delta = 0;
hrtimer_init_sleeper_on_stack(&t, clock_id, mode);
hrtimer_set_expires_range_ns(&t.timer, *expires, delta);
hrtimer_sleeper_start_expires(&t, mode);
@@ -2316,7 +2325,7 @@ EXPORT_SYMBOL_GPL(schedule_hrtimeout_range_clock);
/**
* schedule_hrtimeout_range - sleep until timeout
* @expires: timeout value (ktime_t)
* @delta: slack in expires timeout (ktime_t)
* @delta: slack in expires timeout (ktime_t) for SCHED_OTHER tasks
* @mode: timer mode
*
* Make the current task sleep until the given expiry time has
@@ -2324,7 +2333,8 @@ EXPORT_SYMBOL_GPL(schedule_hrtimeout_range_clock);
* the current task state has been set (see set_current_state()).
*
* The @delta argument gives the kernel the freedom to schedule the
* actual wakeup to a time that is both power and performance friendly.
* actual wakeup to a time that is both power and performance friendly
* for regular (non RT/DL) tasks.
* The kernel give the normal best effort behavior for "@expires+@delta",
* but may decide to fire the timer earlier, but no earlier than @expires.
*
kernel/time/posix-cpu-timers.c
+6 -7
View File
@@ -243,13 +243,12 @@ static void proc_sample_cputime_atomic(struct task_cputime_atomic *at,
*/
static inline void __update_gt_cputime(atomic64_t *cputime, u64 sum_cputime)
{
u64 curr_cputime;
retry:
curr_cputime = atomic64_read(cputime);
if (sum_cputime > curr_cputime) {
if (atomic64_cmpxchg(cputime, curr_cputime, sum_cputime) != curr_cputime)
goto retry;
}
u64 curr_cputime = atomic64_read(cputime);
do {
if (sum_cputime <= curr_cputime)
return;
} while (!atomic64_try_cmpxchg(cputime, &curr_cputime, sum_cputime));
}
static void update_gt_cputime(struct task_cputime_atomic *cputime_atomic,
kernel/time/posix-stubs.c
+2
View File
@@ -147,6 +147,7 @@ SYSCALL_DEFINE4(clock_nanosleep, const clockid_t, which_clock, int, flags,
return -EINVAL;
if (flags & TIMER_ABSTIME)
rmtp = NULL;
current->restart_block.fn = do_no_restart_syscall;
current->restart_block.nanosleep.type = rmtp ? TT_NATIVE : TT_NONE;
current->restart_block.nanosleep.rmtp = rmtp;
texp = timespec64_to_ktime(t);
@@ -240,6 +241,7 @@ SYSCALL_DEFINE4(clock_nanosleep_time32, clockid_t, which_clock, int, flags,
return -EINVAL;
if (flags & TIMER_ABSTIME)
rmtp = NULL;
current->restart_block.fn = do_no_restart_syscall;
current->restart_block.nanosleep.type = rmtp ? TT_COMPAT : TT_NONE;
current->restart_block.nanosleep.compat_rmtp = rmtp;
texp = timespec64_to_ktime(t);
kernel/time/posix-timers.c
+2
View File
@@ -1270,6 +1270,7 @@ SYSCALL_DEFINE4(clock_nanosleep, const clockid_t, which_clock, int, flags,
return -EINVAL;
if (flags & TIMER_ABSTIME)
rmtp = NULL;
current->restart_block.fn = do_no_restart_syscall;
current->restart_block.nanosleep.type = rmtp ? TT_NATIVE : TT_NONE;
current->restart_block.nanosleep.rmtp = rmtp;
@@ -1297,6 +1298,7 @@ SYSCALL_DEFINE4(clock_nanosleep_time32, clockid_t, which_clock, int, flags,
return -EINVAL;
if (flags & TIMER_ABSTIME)
rmtp = NULL;
current->restart_block.fn = do_no_restart_syscall;
current->restart_block.nanosleep.type = rmtp ? TT_COMPAT : TT_NONE;
current->restart_block.nanosleep.compat_rmtp = rmtp;
kernel/time/test_udelay.c
+1 -1
View File
@@ -149,7 +149,7 @@ module_init(udelay_test_init);
static void __exit udelay_test_exit(void)
{
mutex_lock(&udelay_test_lock);
debugfs_remove(debugfs_lookup(DEBUGFS_FILENAME, NULL));
debugfs_lookup_and_remove(DEBUGFS_FILENAME, NULL);
mutex_unlock(&udelay_test_lock);
}
tools/objtool/arch/x86/include/arch/special.h
+3 -3
View File
@@ -11,11 +11,11 @@
#define JUMP_NEW_OFFSET 4
#define JUMP_KEY_OFFSET 8
#define ALT_ENTRY_SIZE 12
#define ALT_ENTRY_SIZE 14
#define ALT_ORIG_OFFSET 0
#define ALT_NEW_OFFSET 4
#define ALT_FEATURE_OFFSET 8
#define ALT_ORIG_LEN_OFFSET 10
#define ALT_NEW_LEN_OFFSET 11
#define ALT_ORIG_LEN_OFFSET 12
#define ALT_NEW_LEN_OFFSET 13
#endif /* _X86_ARCH_SPECIAL_H */
tools/testing/selftests/x86/test_vsyscall.c
+2 -5
View File
@@ -92,11 +92,8 @@ static void init_vdso(void)
printf("[WARN]\tfailed to find time in vDSO\n");
vdso_getcpu = (getcpu_t)dlsym(vdso, "__vdso_getcpu");
if (!vdso_getcpu) {
/* getcpu() was never wired up in the 32-bit vDSO. */
printf("[%s]\tfailed to find getcpu in vDSO\n",
sizeof(long) == 8 ? "WARN" : "NOTE");
}
if (!vdso_getcpu)
printf("[WARN]\tfailed to find getcpu in vDSO\n");
}
static int init_vsys(void)