[PATCH 0/6] ci: run tests and static analysis on allyesconfig

[PATCH 0/6] ci: run tests and static analysis on allyesconfig

[Ahmad Fatoum]

With a one-liner fix remaining[1], sandbox now builds allyesconfig sans
SDL/libftdi to completion. Let's seize the opportunity and add it to CI,
so we easily detect breakage around allyesconfig in future.

This series also switches the recently added Coverity and CodeQL Github
Actions to use allyesconfig to extend coverage to non-sandbox related
drivers as well.

[1]: https://lore.barebox.org/barebox/20250312-rpmb-v1-7-0f213382a3f3@pengutronix.de/T/#m4e6fcf916dd9461bbef25a93439ad8fdb36c8217

Ahmad Fatoum (6):
  sandbox: use CROSS_PKG_CONFIG when compiling sandbox arch code
  sandbox: hide sdl/libftdi symbols if libraries not found
  MAKEALL: fix check_pipe_status usage
  MAKEALL: remove use of alias in script
  ci: pytest: run test suite on sandbox allyesconfig
  ci: run static analysis on allyesconfig

 .github/workflows/codeql.yml              |  2 +-
 .github/workflows/coverity.sh             |  2 +-
 .github/workflows/test-labgrid-pytest.yml |  4 ++++
 MAKEALL                                   | 14 +++++++-------
 arch/sandbox/Kconfig                      |  6 ++++++
 arch/sandbox/Makefile                     |  4 ++--
 arch/sandbox/os/Makefile                  |  4 ++--
 drivers/gpio/Kconfig                      |  2 +-
 drivers/sound/Kconfig                     |  2 +-
 drivers/video/Kconfig                     |  2 +-
 test/sandbox/allyesconfig.yaml            | 17 +++++++++++++++++
 11 files changed, 43 insertions(+), 16 deletions(-)
 create mode 100644 test/sandbox/allyesconfig.yaml

-- 
2.39.5

[PATCH 1/6] sandbox: use CROSS_PKG_CONFIG when compiling sandbox arch code

[Ahmad Fatoum]

sandbox only means that we run under Linux as host operating system
(as opposed to before). It can still be cross compiled and indeed that's
what barebox-tools in OE-core does to build the target tools.

OE-core doesn't yet build barebox for sandbox itself (only the scripts),
but nonetheless, if it did, SDL and libftdi need to be usable with the
cross toolchain, not with the host toolchain, so let's use
CROSS_PKG_CONFIG accordingly.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 arch/sandbox/Makefile    | 4 ++--
 arch/sandbox/os/Makefile | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/sandbox/Makefile b/arch/sandbox/Makefile
index 157e856c59b1..dbdf8a6e8117 100644
--- a/arch/sandbox/Makefile
+++ b/arch/sandbox/Makefile
@@ -62,11 +62,11 @@ archprepare: maketools
 PHONY += maketools
 
 ifeq ($(CONFIG_SDL),y)
-SDL_LIBS := $(shell $(PKG_CONFIG) sdl2 --libs)
+SDL_LIBS := $(shell $(CROSS_PKG_CONFIG) sdl2 --libs)
 endif
 
 ifeq ($(CONFIG_GPIO_LIBFTDI1),y)
-FTDI1_LIBS := $(shell $(PKG_CONFIG) libftdi1 --libs)
+FTDI1_LIBS := $(shell $(CROSS_PKG_CONFIG) libftdi1 --libs)
 endif
 
 ifeq ($(CONFIG_ASAN),y)
diff --git a/arch/sandbox/os/Makefile b/arch/sandbox/os/Makefile
index 7a76fb0290e8..c77702bb0c1d 100644
--- a/arch/sandbox/os/Makefile
+++ b/arch/sandbox/os/Makefile
@@ -23,8 +23,8 @@ endif
 obj-y = common.o tap.o setjmp.o
 obj-$(CONFIG_MALLOC_LIBC) += libc_malloc.o
 
-CFLAGS_sdl.o = $(shell $(PKG_CONFIG) sdl2 --cflags)
+CFLAGS_sdl.o = $(shell $(CROSS_PKG_CONFIG) sdl2 --cflags)
 obj-$(CONFIG_SDL) += sdl.o
 
-CFLAGS_ftdi.o = $(shell $(PKG_CONFIG) libftdi1 --cflags)
+CFLAGS_ftdi.o = $(shell $(CROSS_PKG_CONFIG) libftdi1 --cflags)
 obj-$(CONFIG_GPIO_LIBFTDI1) += ftdi.o
-- 
2.39.5

[PATCH 2/6] sandbox: hide sdl/libftdi symbols if libraries not found

[Ahmad Fatoum]

To make an allyesconfig build work out of the box on sandbox, let's hide
the SDL and libftdi GPIO drivers unless the libraries are already
installed.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 arch/sandbox/Kconfig  | 6 ++++++
 drivers/gpio/Kconfig  | 2 +-
 drivers/sound/Kconfig | 2 +-
 drivers/video/Kconfig | 2 +-
 4 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/arch/sandbox/Kconfig b/arch/sandbox/Kconfig
index 0357414a3f79..c404f73a9970 100644
--- a/arch/sandbox/Kconfig
+++ b/arch/sandbox/Kconfig
@@ -81,6 +81,12 @@ config CMD_SANDBOX_CPUINFO
 	help
 	  Say yes here to get a dummy cpuinfo command.
 
+config HAVE_LIBSDL2
+	def_bool $(success,$(CROSS_PKG_CONFIG) --exists sdl2)
+
+config HAVE_LIBFTDI
+	def_bool $(success,$(CROSS_PKG_CONFIG) --exists libftdi1)
+
 config SDL
 	bool
 
diff --git a/drivers/gpio/Kconfig b/drivers/gpio/Kconfig
index dd95e8906b39..7caa1aa96eb3 100644
--- a/drivers/gpio/Kconfig
+++ b/drivers/gpio/Kconfig
@@ -229,7 +229,7 @@ config GPIO_STARFIVE
 
 config GPIO_LIBFTDI1
 	bool "libftdi1 driver"
-	depends on SANDBOX
+	depends on HAVE_LIBFTDI1
 
 config GPIO_ZYNQ
 	tristate "Xilinx Zynq GPIO support"
diff --git a/drivers/sound/Kconfig b/drivers/sound/Kconfig
index bf6f715200e0..bc695fed2cd4 100644
--- a/drivers/sound/Kconfig
+++ b/drivers/sound/Kconfig
@@ -11,7 +11,7 @@ if SOUND
 
 config SOUND_SDL
 	bool "SDL sound driver for sandbox"
-	depends on SANDBOX && OFDEVICE
+	depends on HAVE_LIBSDL2 && OFDEVICE
 	select SDL
 
 config PWM_BEEPER
diff --git a/drivers/video/Kconfig b/drivers/video/Kconfig
index 0539e2d453da..ef19948219f3 100644
--- a/drivers/video/Kconfig
+++ b/drivers/video/Kconfig
@@ -70,7 +70,7 @@ config DRIVER_VIDEO_STM32_LTDC
 
 config DRIVER_VIDEO_SDL
 	bool "SDL framebuffer driver"
-	depends on SANDBOX
+	depends on HAVE_LIBSDL2
 	select SDL
 
 config DRIVER_VIDEO_PXA
-- 
2.39.5

[PATCH 3/6] MAKEALL: fix check_pipe_status usage

[Ahmad Fatoum]

When no logdir is specified, there is no shell pipeline to check the
status of and thus check_pipe_status will not check the status of the
just run command.

Fix this by placing check_pipe_status directly after the pipelines in
question.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 MAKEALL | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/MAKEALL b/MAKEALL
index a9409fbf7b07..85b71e2e75c1 100755
--- a/MAKEALL
+++ b/MAKEALL
@@ -101,9 +101,12 @@ with_logs_collected() {
 
 	if [ -n "${logdir}" ]; then
 		"$@" 2>&1 > "${log_report}" | tee "${log_err}"
+		check_pipe_status
 	else
 		"$@"
 	fi
+
+	return $?
 }
 
 report() {
@@ -169,7 +172,6 @@ do_build_defconfig() {
 	done
 	with_logs_collected ${MAKE} $silent_flag olddefconfig
 
-	check_pipe_status
 	configure_result="$?"
 
 	report "Configure: "
@@ -178,8 +180,6 @@ do_build_defconfig() {
 		report "OK     \n"
 
 		with_logs_collected ${MAKE} $silent_flag ${TARGET}
-
-		check_pipe_status
 		compile_result="$?"
 
 		report "Compile: " ${defconfig}
@@ -230,8 +230,6 @@ do_test_defconfig() {
 	local err=0
 
 	LG_BUILDDIR=$BUILDDIR with_logs_collected pytest --lg-env $yaml "$@"
-
-	check_pipe_status
 	compile_result="$?"
 
 	report "Test: " ${yaml}
-- 
2.39.5

[PATCH 4/6] MAKEALL: remove use of alias in script

[Ahmad Fatoum]

Aliases are meant for interactive shell and not expanded by default in
scripts. As we are going to call pytest in one location only, just turn
it into a variable.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 MAKEALL | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/MAKEALL b/MAKEALL
index 85b71e2e75c1..d47f0204f1f1 100755
--- a/MAKEALL
+++ b/MAKEALL
@@ -219,7 +219,9 @@ do_build_defconfig() {
 }
 
 if command -v labgrid-pytest >/dev/null; then
-	alias pytest=labgrid-pytest
+	pytest=labgrid-pytest
+else
+	pytest=pytest
 fi
 
 do_test_defconfig() {
@@ -229,7 +231,7 @@ do_test_defconfig() {
 	local step_time_start=$(date +%s)
 	local err=0
 
-	LG_BUILDDIR=$BUILDDIR with_logs_collected pytest --lg-env $yaml "$@"
+	LG_BUILDDIR=$BUILDDIR with_logs_collected $pytest --lg-env $yaml "$@"
 	compile_result="$?"
 
 	report "Test: " ${yaml}
-- 
2.39.5

[PATCH 5/6] ci: pytest: run test suite on sandbox allyesconfig

[Ahmad Fatoum]

Building allyesconfig in CI allows us to catch early compile-time
breakage when changes are done treewide. Running pytest over it
will ensure that compile-tested drivers don't break normal startup
when unused.

Additionally, allyesconfig also enables ASAN/UBSAN, which is a nice side
effect. A future change would be adding KASAN for other architectures as
well.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 .github/workflows/test-labgrid-pytest.yml |  4 ++++
 test/sandbox/allyesconfig.yaml            | 17 +++++++++++++++++
 2 files changed, 21 insertions(+)
 create mode 100644 test/sandbox/allyesconfig.yaml

diff --git a/.github/workflows/test-labgrid-pytest.yml b/.github/workflows/test-labgrid-pytest.yml
index 2a9f7a53aced..c43649e85fec 100644
--- a/.github/workflows/test-labgrid-pytest.yml
+++ b/.github/workflows/test-labgrid-pytest.yml
@@ -55,6 +55,10 @@ jobs:
             lgenv: 'test/sandbox/sandbox_defconfig.yaml'
             defconfig: sandbox_defconfig
 
+          - ARCH: sandbox
+            lgenv: 'test/sandbox/allyesconfig.yaml'
+            defconfig: sandbox_defconfig
+
     steps:
     - name: Checkout code
       uses: actions/checkout@v4
diff --git a/test/sandbox/allyesconfig.yaml b/test/sandbox/allyesconfig.yaml
new file mode 100644
index 000000000000..76b995127651
--- /dev/null
+++ b/test/sandbox/allyesconfig.yaml
@@ -0,0 +1,17 @@
+targets:
+  main:
+    drivers:
+      ExternalConsoleDriver:
+        cmd: !template "$LG_BUILDDIR/barebox"
+      ExternalPowerDriver:
+        cmd_on: 'true'
+        cmd_off: 'true'
+      BareboxDriver:
+        prompt: "barebox@[^:]+:[^ ]+ "
+      BareboxTestStrategy: {}
+    runner:
+      kconfig_add:
+        - CONFIG_CONSOLE_DISABLE_INPUT=n
+        - CONFIG_MALLOC_LIBC=y # for ASAN integration
+imports:
+  -  ../strategy.py
-- 
2.39.5

[PATCH 6/6] ci: run static analysis on allyesconfig

[Ahmad Fatoum]

Now that allyesconfig builds again for sandbox, let's switch the
coverity and CodeQL workflows to use it.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 .github/workflows/codeql.yml  | 2 +-
 .github/workflows/coverity.sh | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml
index ca4b1f611d53..767c49c4beeb 100644
--- a/.github/workflows/codeql.yml
+++ b/.github/workflows/codeql.yml
@@ -37,7 +37,7 @@ jobs:
     - name: Build C Code
       shell: bash
       run: |
-        make ARCH=sandbox sandbox_defconfig
+        make ARCH=sandbox allyesconfig
         make -j$(nproc) ARCH=sandbox
 
     - name: Perform CodeQL Analysis
diff --git a/.github/workflows/coverity.sh b/.github/workflows/coverity.sh
index ee4a481503c7..eb6cbfe93fa7 100755
--- a/.github/workflows/coverity.sh
+++ b/.github/workflows/coverity.sh
@@ -35,7 +35,7 @@ function run_coverity {
 
     version="$(make bareboxversion)-g${GITHUB_SHA:0:10}"
 
-    make ARCH=sandbox sandbox_defconfig
+    make ARCH=sandbox allyesconfig
     COVERITY_UNSUPPORTED=1 "$tool_dir/bin/cov-build" --dir "$results_dir" sh -c "make -j$(nproc) ARCH=sandbox"
     "$tool_dir/bin/cov-import-scm" --dir "$results_dir" --scm git --log "$results_dir/scm_log.txt"
 
-- 
2.39.5

Re: [PATCH 0/6] ci: run tests and static analysis on allyesconfig

[Sascha Hauer]

On Wed, 26 Mar 2025 08:59:13 +0100, Ahmad Fatoum wrote:
> With a one-liner fix remaining[1], sandbox now builds allyesconfig sans
> SDL/libftdi to completion. Let's seize the opportunity and add it to CI,
> so we easily detect breakage around allyesconfig in future.
> 
> This series also switches the recently added Coverity and CodeQL Github
> Actions to use allyesconfig to extend coverage to non-sandbox related
> drivers as well.
> 
> [...]

Applied, thanks!

[1/6] sandbox: use CROSS_PKG_CONFIG when compiling sandbox arch code
      https://git.pengutronix.de/cgit/barebox/commit/?id=1026d6df3db8 (link may not be stable)
[2/6] sandbox: hide sdl/libftdi symbols if libraries not found
      https://git.pengutronix.de/cgit/barebox/commit/?id=3763b996fac0 (link may not be stable)
[3/6] MAKEALL: fix check_pipe_status usage
      https://git.pengutronix.de/cgit/barebox/commit/?id=b88a1d166575 (link may not be stable)
[4/6] MAKEALL: remove use of alias in script
      https://git.pengutronix.de/cgit/barebox/commit/?id=61f240e31659 (link may not be stable)
[5/6] ci: pytest: run test suite on sandbox allyesconfig
      https://git.pengutronix.de/cgit/barebox/commit/?id=18bf3d40253c (link may not be stable)
[6/6] ci: run static analysis on allyesconfig
      https://git.pengutronix.de/cgit/barebox/commit/?id=a886cc803dfa (link may not be stable)

Best regards,
-- 
Sascha Hauer <s.hauer@pengutronix.de>