BugLink: https://bugs.launchpad.net/bugs/2111953
commit e397a603e49cc7c7c113fad9f55a09637f290c34 upstream.
Replace lz4c with lz4 for kernel image compression.
Although lz4 and lz4c are functionally similar, lz4c has been deprecated
upstream since 2018. Since as early as Ubuntu 16.04 and Fedora 25, lz4
and lz4c have been packaged together, making it safe to update the
requirement from lz4c to lz4.
Consequently, some distributions and build systems, such as OpenEmbedded,
have fully transitioned to using lz4. OpenEmbedded core adopted this
change in commit fe167e082cbd ("bitbake.conf: require lz4 instead of
lz4c"), causing compatibility issues when building the mainline kernel
in the latest OpenEmbedded environment, as seen in the errors below.
This change also updates the LZ4 compression commands to make it backward
compatible by replacing stdin and stdout with the '-' option, due to some
unclear reason, the stdout keyword does not work for lz4 and '-' works for
both. In addition, this modifies the legacy '-c1' with '-9' which is also
compatible with both. This fixes the mainline kernel build failures with
the latest master OpenEmbedded builds associated with the mentioned
compatibility issues.
LZ4 arch/arm/boot/compressed/piggy_data
/bin/sh: 1: lz4c: not found
...
...
ERROR: oe_runmake failed
Link: https://github.com/lz4/lz4/pull/553
Suggested-by: Francesco Dolcini <francesco.dolcini@toradex.com>
Signed-off-by: Parth Pancholi <parth.pancholi@toradex.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Cc: Salvatore Bonaccorso <carnil@debian.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Manuel Diewald <manuel.diewald@canonical.com>
Signed-off-by: Mehmet Basaran <mehmet.basaran@canonical.com>
BugLink: https://bugs.launchpad.net/bugs/2073788
commit aba091547ef6159d52471f42a3ef531b7b660ed8 upstream.
There is an issue in clang's ThinLTO caching (enabled for the kernel via
'--thinlto-cache-dir') with .incbin, which the kernel occasionally uses
to include data within the kernel, such as the .config file for
/proc/config.gz. For example, when changing the .config and rebuilding
vmlinux, the copy of .config in vmlinux does not match the copy of
.config in the build folder:
$ echo 'CONFIG_LTO_NONE=n
CONFIG_LTO_CLANG_THIN=y
CONFIG_IKCONFIG=y
CONFIG_HEADERS_INSTALL=y' >kernel/configs/repro.config
$ make -skj"$(nproc)" ARCH=x86_64 LLVM=1 clean defconfig repro.config vmlinux
...
$ grep CONFIG_HEADERS_INSTALL .config
CONFIG_HEADERS_INSTALL=y
$ scripts/extract-ikconfig vmlinux | grep CONFIG_HEADERS_INSTALL
CONFIG_HEADERS_INSTALL=y
$ scripts/config -d HEADERS_INSTALL
$ make -kj"$(nproc)" ARCH=x86_64 LLVM=1 vmlinux
...
UPD kernel/config_data
GZIP kernel/config_data.gz
CC kernel/configs.o
...
LD vmlinux
...
$ grep CONFIG_HEADERS_INSTALL .config
# CONFIG_HEADERS_INSTALL is not set
$ scripts/extract-ikconfig vmlinux | grep CONFIG_HEADERS_INSTALL
CONFIG_HEADERS_INSTALL=y
Without '--thinlto-cache-dir' or when using full LTO, this issue does
not occur.
Benchmarking incremental builds on a few different machines with and
without the cache shows a 20% increase in incremental build time without
the cache when measured by touching init/main.c and running 'make all'.
ARCH=arm64 defconfig + CONFIG_LTO_CLANG_THIN=y on an arm64 host:
Benchmark 1: With ThinLTO cache
Time (mean ± σ): 56.347 s ± 0.163 s [User: 83.768 s, System: 24.661 s]
Range (min … max): 56.109 s … 56.594 s 10 runs
Benchmark 2: Without ThinLTO cache
Time (mean ± σ): 67.740 s ± 0.479 s [User: 718.458 s, System: 31.797 s]
Range (min … max): 67.059 s … 68.556 s 10 runs
Summary
With ThinLTO cache ran
1.20 ± 0.01 times faster than Without ThinLTO cache
ARCH=x86_64 defconfig + CONFIG_LTO_CLANG_THIN=y on an x86_64 host:
Benchmark 1: With ThinLTO cache
Time (mean ± σ): 85.772 s ± 0.252 s [User: 91.505 s, System: 8.408 s]
Range (min … max): 85.447 s … 86.244 s 10 runs
Benchmark 2: Without ThinLTO cache
Time (mean ± σ): 103.833 s ± 0.288 s [User: 232.058 s, System: 8.569 s]
Range (min … max): 103.286 s … 104.124 s 10 runs
Summary
With ThinLTO cache ran
1.21 ± 0.00 times faster than Without ThinLTO cache
While it is unfortunate to take this performance improvement off the
table, correctness is more important. If/when this is fixed in LLVM, it
can potentially be brought back in a conditional manner. Alternatively,
a developer can just disable LTO if doing incremental compiles quickly
is important, as a full compile cycle can still take over a minute even
with the cache and it is unlikely that LTO will result in functional
differences for a kernel change.
Cc: stable@vger.kernel.org
Fixes: dc5723b02e ("kbuild: add support for Clang LTO")
Reported-by: Yifan Hong <elsk@google.com>
Closes: https://github.com/ClangBuiltLinux/linux/issues/2021
Reported-by: Masami Hiramatsu <mhiramat@kernel.org>
Closes: https://lore.kernel.org/r/20220327115526.cc4b0ff55fc53c97683c3e4d@kernel.org/
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Portia Stephens <portia.stephens@canonical.com>
Signed-off-by: Roxana Nicolescu <roxana.nicolescu@canonical.com>
rsync has been removed from the i386 archives starting with lunar,
causing build errors when generating kernel packages for this
architecture.
Replace rsync with tar for now, until we decide to drop i386 completely.
Signed-off-by: Andrea Righi <andrea.righi@canonical.com>
If you use only remote build servers such as launchpad, cbd, etc.
you may not care about this, but if you run 'make mrproper', the
debian directory is entirely removed because it is a generated
directory for the upstream kernel.
I do not want to lose the debian directory by accident when I am
debugging something on my local machine.
Signed-off-by: Masahiro Yamada <masahiro.yamada@canonical.com>
Signed-off-by: Andrea Righi <andrea.righi@canonical.com>
When building out-of-tree (which we do for package builds), the compiler
emits the following warning:
cc1: warning: ubuntu/include: No such file or directory [-Wmissing-include-dirs]
Fix that by (always) using the absolute path of the include directory.
Signed-off-by: Juerg Haefliger <juergh@canonical.com>
Acked-by: Dimitri John Ledkov <dimitri.ledkov@canonical.com>
Acked-by: Tim Gardner <tim.gardner@canonical.com>
Signed-off-by: Paolo Pisati <paolo.pisati@canonical.com>
Pull Kbuild fixes from Masahiro Yamada:
- Fix UML build with clang-18 and newer
- Avoid using the alias attribute in host programs
- Replace tabs with spaces when followed by conditionals for future GNU
Make versions
- Fix rpm-pkg for the systemd-provided kernel-install tool
- Fix the undefined behavior in Kconfig for a 'int' symbol used in a
conditional
* tag 'kbuild-fixes-v6.8' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild:
kconfig: initialize sym->curr.tri to 'no' for all symbol types again
kbuild: rpm-pkg: simplify installkernel %post
kbuild: Replace tabs with spaces when followed by conditionals
modpost: avoid using the alias attribute
kbuild: fix W= flags in the help message
modpost: Add '.ltext' and '.ltext.*' to TEXT_SECTIONS
um: Fix adding '-no-pie' for clang
kbuild: defconf: use SRCARCH to find merged configs
It seems that we have finished addressing all the remaining
issues regarding -Wstringop-overflow. So, we are now in good
shape to enable this compiler option globally.
Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Pull Kbuild updates from Masahiro Yamada:
- Make Kconfig parse the input .config more precisely
- Support W=c and W=e options for Kconfig
- Set Kconfig int/hex symbols to zero if the 'default' property is
missing
- Add .editorconfig
- Add scripts/git.orderFile
- Add a script to detect backward-incompatible changes in UAPI headers
- Resolve the symlink passed to O= option properly
- Use the user-supplied mtime for all files in the builtin initramfs,
which provides better reproducible builds
- Fix the direct execution of debian/rules for Debian package builds
- Use build ID instead of the .gnu_debuglink section for the Debian dbg
package
* tag 'kbuild-v6.8' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (53 commits)
kbuild: deb-pkg: use debian/<package> for tmpdir
kbuild: deb-pkg: move 'make headers' to build-arch
kbuild: deb-pkg: do not search for 'scripts' directory under arch/
kbuild: deb-pkg: use build ID instead of debug link for dbg package
kbuild: deb-pkg: use more debhelper commands in builddeb
kbuild: deb-pkg: remove unneeded '-f $srctree/Makefile' in debian/rules
kbuild: deb-pkg: allow to run debian/rules from output directory
kbuild: deb-pkg: set DEB_* variables if debian/rules is directly executed
kbuild: deb-pkg: squash scripts/package/deb-build-option to debian/rules
kbuild: deb-pkg: factor out common Make options in debian/rules
kbuild: deb-pkg: hard-code Build-Depends
kbuild: deb-pkg: split debian/copyright from the mkdebian script
gen_init_cpio: Apply mtime supplied by user to all file types
kbuild: resolve symlinks for O= properly
docs: dev-tools: Add UAPI checker documentation
check-uapi: Introduce check-uapi.sh
scripts: Introduce a default git.orderFile
kconfig: WERROR unmet symbol dependency
Add .editorconfig file for basic formatting
kconfig: Use KCONFIG_CONFIG instead of .config
...
Pull non-MM updates from Andrew Morton:
"Quite a lot of kexec work this time around. Many singleton patches in
many places. The notable patch series are:
- nilfs2 folio conversion from Matthew Wilcox in 'nilfs2: Folio
conversions for file paths'.
- Additional nilfs2 folio conversion from Ryusuke Konishi in 'nilfs2:
Folio conversions for directory paths'.
- IA64 remnant removal in Heiko Carstens's 'Remove unused code after
IA-64 removal'.
- Arnd Bergmann has enabled the -Wmissing-prototypes warning
everywhere in 'Treewide: enable -Wmissing-prototypes'. This had
some followup fixes:
- Nathan Chancellor has cleaned up the hexagon build in the series
'hexagon: Fix up instances of -Wmissing-prototypes'.
- Nathan also addressed some s390 warnings in 's390: A couple of
fixes for -Wmissing-prototypes'.
- Arnd Bergmann addresses the same warnings for MIPS in his series
'mips: address -Wmissing-prototypes warnings'.
- Baoquan He has made kexec_file operate in a top-down-fitting manner
similar to kexec_load in the series 'kexec_file: Load kernel at top
of system RAM if required'
- Baoquan He has also added the self-explanatory 'kexec_file: print
out debugging message if required'.
- Some checkstack maintenance work from Tiezhu Yang in the series
'Modify some code about checkstack'.
- Douglas Anderson has disentangled the watchdog code's logging when
multiple reports are occurring simultaneously. The series is
'watchdog: Better handling of concurrent lockups'.
- Yuntao Wang has contributed some maintenance work on the crash code
in 'crash: Some cleanups and fixes'"
* tag 'mm-nonmm-stable-2024-01-09-10-33' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (157 commits)
crash_core: fix and simplify the logic of crash_exclude_mem_range()
x86/crash: use SZ_1M macro instead of hardcoded value
x86/crash: remove the unused image parameter from prepare_elf_headers()
kdump: remove redundant DEFAULT_CRASH_KERNEL_LOW_SIZE
scripts/decode_stacktrace.sh: strip unexpected CR from lines
watchdog: if panicking and we dumped everything, don't re-enable dumping
watchdog/hardlockup: use printk_cpu_sync_get_irqsave() to serialize reporting
watchdog/softlockup: use printk_cpu_sync_get_irqsave() to serialize reporting
watchdog/hardlockup: adopt softlockup logic avoiding double-dumps
kexec_core: fix the assignment to kimage->control_page
x86/kexec: fix incorrect end address passed to kernel_ident_mapping_init()
lib/trace_readwrite.c:: replace asm-generic/io with linux/io
nilfs2: cpfile: fix some kernel-doc warnings
stacktrace: fix kernel-doc typo
scripts/checkstack.pl: fix no space expression between sp and offset
x86/kexec: fix incorrect argument passed to kexec_dprintk()
x86/kexec: use pr_err() instead of kexec_dprintk() when an error occurs
nilfs2: add missing set_freezable() for freezable kthread
kernel: relay: remove relay_file_splice_read dead code, doesn't work
docs: submit-checklist: remove all of "make namespacecheck"
...
Currently, Kbuild follows the logical chain of directories for the O=
option, just like 'cd' (or 'realpath --logical') does.
Example:
$ mkdir -p /tmp/a /tmp/x/y
$ ln -s /tmp/x/y /tmp/a/b
$ realpath /tmp/a/b/..
/tmp/x
$ realpath --logical /tmp/a/b/..
/tmp/a
$ make O=/tmp/a/b/.. defconfig
make[1]: Entering directory '/tmp/a'
[snip]
make[1]: Leaving directory '/tmp/a'
'make O=/tmp/a/b/.. defconfig' creates the kernel configuration in
/tmp/a instead of /tmp/x despite /tmp/a/b/.. resolves to /tmp/x.
This is because Kbuild internally uses the 'cd ... && pwd' for the
path resolution, but this behavior is not predictable for users.
Additionally, it is not consistent with how the Kbuild handles the
M= option or GNU Make works with 'make -C /tmp/a/b/..'.
Using the physical directory structure for the O= option seems more
reasonable.
The comment says "expand a shell special character '~'", but it has
already been expanded to the home directory in the command line.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <n.schier@avm.de>
The checkstack script omits all functions with a stack usage of less than
100 bytes. However the script already has support for a parameter which
allows to override the default, but it cannot be set with
$ make checkstack
Add a MINSTACKSIZE parameter which allows to change the default. This might
be useful in order to print the stack usage of all functions, or only those
with large stack usage:
$ make checkstack MINSTACKSIZE=0
$ make checkstack MINSTACKSIZE=800
Link: https://lkml.kernel.org/r/20231120183719.2188479-4-hca@linux.ibm.com
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Cc: Maninder Singh <maninder1.s@samsung.com>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: Vaneet Narang <v.narang@samsung.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
KCONFIG_WARN_UNKNOWN_SYMBOLS=1 and KCONFIG_WERROR=1 are descriptive
and suitable in scripting, but typing them from the command line can
be tedious.
Associate them with KBUILD_EXTRA_WARN (and the W= shorthand).
Support a new letter 'c' to enable extra checks in Kconfig. You can
still manage compiler warnings (W=1) and Kconfig warnings (W=c)
independently.
Reuse the letter 'e' to turn Kconfig warnings into errors.
As usual, you can combine multiple letters in KCONFIG_EXTRA_WARN.
$ KCONFIG_WARN_UNKNOWN_SYMBOLS=1 KCONFIG_WERROR=1 make defconfig
can be shortened to:
$ KBUILD_EXTRA_WARN=ce make defconfig
or, even shorter:
$ make W=ce defconfig
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
The rpm-pkg and deb-pkg targets have transitioned to using 'git archive'
for tarball creation.
Although the old cmd_src_tar is still used by snap-pkg, there is no need
to pack and unpack a tarball solely for passing the source to snapcraft.
Instead, you can use 'source-type: local' to tell the source location to
snapcraft.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Pull Kbuild updates from Masahiro Yamada:
- Implement the binary search in modpost for faster symbol lookup
- Respect HOSTCC when linking host programs written in Rust
- Change the binrpm-pkg target to generate kernel-devel RPM package
- Fix endianness issues for tee and ishtp MODULE_DEVICE_TABLE
- Unify vdso_install rules
- Remove unused __memexit* annotations
- Eliminate stale whitelisting for __devinit/__devexit from modpost
- Enable dummy-tools to handle the -fpatchable-function-entry flag
- Add 'userldlibs' syntax
* tag 'kbuild-v6.7' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (30 commits)
kbuild: support 'userldlibs' syntax
kbuild: dummy-tools: pretend we understand -fpatchable-function-entry
kbuild: Correct missing architecture-specific hyphens
modpost: squash ALL_{INIT,EXIT}_TEXT_SECTIONS to ALL_TEXT_SECTIONS
modpost: merge sectioncheck table entries regarding init/exit sections
modpost: use ALL_INIT_SECTIONS for the section check from DATA_SECTIONS
modpost: disallow the combination of EXPORT_SYMBOL and __meminit*
modpost: remove EXIT_SECTIONS macro
modpost: remove MEM_INIT_SECTIONS macro
modpost: remove more symbol patterns from the section check whitelist
modpost: disallow *driver to reference .meminit* sections
linux/init: remove __memexit* annotations
modpost: remove ALL_EXIT_DATA_SECTIONS macro
kbuild: simplify cmd_ld_multi_m
kbuild: avoid too many execution of scripts/pahole-flags.sh
kbuild: remove ARCH_POSTLINK from module builds
kbuild: unify no-compiler-targets and no-sync-config-targets
kbuild: unify vdso_install rules
docs: kbuild: add INSTALL_DTBS_PATH
UML: remove unused cmd_vdso_install
...
Parth Pancholi
Nathan Chancellor
Greg Kroah-Hartman
Sasha Levin
Andrea Righi
Masahiro Yamada
Juerg Haefliger
Andy Whitcroft
Leann Ogasawara
Linus Torvalds
Masahiro Yamada
Dmitry Goncharov
Gustavo A. R. Silva
Heiko Carstens