From: Ahmad Fatoum <a.fatoum@pengutronix.de>
To: barebox@lists.infradead.org
Cc: rcz@pengutronix.de
Subject: [PATCH 00/17] add barebox in-tree testing infrastructure
Date: Mon, 12 Apr 2021 09:16:34 +0200 [thread overview]
Message-ID: <20210412071651.8769-1-a.fatoum@pengutronix.de> (raw)
Commit 10948f86a3c5 ("watchdog: fix watchdog_set_timeout breakage for
drivers like imxwd") fixes an issue caught by a labgrid test suite
exercising functionality of a barebox release.
This could've been caught sooner if these tests were to run as part
of an automated _continuous_ testing process while sitting in next.
This series lays the ground work for that:
- It commits to tree, machine readable YAML specification on how to
build, emulate and test a number of defconfigs
- It adds the glue required to configure tuxmake to build according
to these configuration files
- It extends the build system to allow calling labgrid-pytest
on these files from within the tree
- It adds a very basic self-test framework to allow for unit-testing
barebox functionality
- It adds a command line interface to tie this all together. Examples:
./test/emulate.pl virtio@efi_defconfig --emulate -- -device ?
./test/emulate.pl virtio@efi_defconfig --test -- --lg-log
The use of labgrid-pytest QEMUDriver's allows us in future the flexibility
to share tests between virtualized and physical targets. For now only
QEMUDriver-enabled targets are considered.
To demonstrate usage, this series imports the Linux printf selftest, which
is compiled into barebox and verified to run without errors by pytest.
Unless there are major issues, I'd appreciate if this series could get
merged first with changes happening gradually. This would allow writing
unit tests for new functionality in parallel.
Ahmad Fatoum (17):
kbuild: add install target
kbuild: add ARCH={i386, x86_64} as aliases for x86
kbuild: add ARCH=um alias for sandbox
MIPS: qemu-malta: generate swapped image as part of multi-image build
MIPS: qemu-malta: replace board code with syscon-reboot
MIPS: configs: qemu-malta: enable Virt I/O
openrisc: set default KBUILD_IMAGE
Documentation: boards: RISC-V: update TinyEMU support
test: add basic barebox self-test infrastructure
test: self: port Linux printf kselftest
test: add labgrid configs for some emulated targets
test: add first sample tests
test: support running labgrid-pytest from out-of-tree build directory
test: add emulate.pl, a runner for barebox on emulated targets
test: self: run selftests as part of the pytest suite
test: add bthread test
[RFC] test: run ./test/emulate.pl in Github action
.github/workflows/build-configs.yaml | 41 ++
.github/workflows/vm-tests.yaml | 71 +++
.gitignore | 2 +
Documentation/boards/emulated.rst | 70 +++
Documentation/boards/mips/qemu-malta.rst | 16 +-
Documentation/boards/riscv.rst | 20 +-
Documentation/boards/riscv/barebox-virt32.cfg | 7 +
Documentation/boards/riscv/barebox-virt64.cfg | 7 +
Kconfig | 1 +
Makefile | 52 +-
arch/mips/configs/qemu-malta_defconfig | 12 +-
arch/mips/dts/qemu-malta.dts | 13 +
arch/mips/mach-malta/Kconfig | 1 +
arch/mips/mach-malta/Makefile | 1 -
arch/mips/mach-malta/include/mach/hardware.h | 6 -
arch/mips/mach-malta/reset.c | 31 --
arch/openrisc/Makefile | 2 +
commands/Makefile | 1 +
commands/selftest.c | 88 ++++
images/.gitignore | 1 +
images/Makefile | 7 +-
images/Makefile.malta | 8 +-
include/bselftest.h | 70 +++
include/stdlib.h | 5 +
scripts/Makefile | 1 +
scripts/bswap.c | 83 +++
test/.gitignore | 1 +
test/Kconfig | 8 +
test/Makefile | 1 +
test/__init__.py | 0
test/arm/a15@vexpress_defconfig.yaml | 25 +
test/arm/a9@vexpress_defconfig.yaml | 25 +
test/arm/qemu_virt64_defconfig.yaml | 28 +
test/arm/vexpress_defconfig.yaml | 1 +
test/arm/virt@vexpress_defconfig.yaml | 26 +
test/conftest.py | 21 +
test/emulate.pl | 483 ++++++++++++++++++
test/kconfig/base.cfg | 4 +
test/kconfig/full.cfg | 2 +
test/kconfig/no-deps.cfg | 3 +
test/kconfig/virtio-pci.cfg | 6 +
test/mips/be@qemu-malta_defconfig.yaml | 26 +
test/mips/le@qemu-malta_defconfig.yaml | 29 ++
test/mips/qemu-malta_defconfig.yaml | 1 +
test/openrisc/generic_defconfig.yaml | 25 +
test/py/__init__.py | 0
test/py/helper.py | 33 ++
test/py/test_bselftests.py | 8 +
test/py/test_bthread.py | 23 +
test/py/test_shell.py | 16 +
test/riscv/qemu@virt32_defconfig.yaml | 29 ++
test/riscv/qemu@virt64_defconfig.yaml | 29 ++
test/riscv/tinyemu@virt32_defconfig.yaml | 26 +
test/riscv/tinyemu@virt64_defconfig.yaml | 26 +
test/riscv/virt32_defconfig.yaml | 1 +
test/riscv/virt64_defconfig.yaml | 1 +
test/sandbox/sandbox_defconfig.yaml | 14 +
test/self/Kconfig | 39 ++
test/self/Makefile | 4 +
test/self/core.c | 39 ++
test/self/printf.c | 302 +++++++++++
test/x86/efi_defconfig.yaml | 1 +
test/x86/pc@efi_defconfig.yaml | 33 ++
test/x86/q35@efi_defconfig.yaml | 34 ++
test/x86/virtio@efi_defconfig.yaml | 34 ++
65 files changed, 1960 insertions(+), 64 deletions(-)
create mode 100644 .github/workflows/build-configs.yaml
create mode 100644 .github/workflows/vm-tests.yaml
create mode 100644 Documentation/boards/emulated.rst
create mode 100644 Documentation/boards/riscv/barebox-virt32.cfg
create mode 100644 Documentation/boards/riscv/barebox-virt64.cfg
delete mode 100644 arch/mips/mach-malta/reset.c
create mode 100644 commands/selftest.c
create mode 100644 include/bselftest.h
create mode 100644 scripts/bswap.c
create mode 100644 test/.gitignore
create mode 100644 test/Kconfig
create mode 100644 test/Makefile
create mode 100644 test/__init__.py
create mode 100644 test/arm/a15@vexpress_defconfig.yaml
create mode 100644 test/arm/a9@vexpress_defconfig.yaml
create mode 100644 test/arm/qemu_virt64_defconfig.yaml
create mode 120000 test/arm/vexpress_defconfig.yaml
create mode 100644 test/arm/virt@vexpress_defconfig.yaml
create mode 100644 test/conftest.py
create mode 100755 test/emulate.pl
create mode 100644 test/kconfig/base.cfg
create mode 100644 test/kconfig/full.cfg
create mode 100644 test/kconfig/no-deps.cfg
create mode 100644 test/kconfig/virtio-pci.cfg
create mode 100644 test/mips/be@qemu-malta_defconfig.yaml
create mode 100644 test/mips/le@qemu-malta_defconfig.yaml
create mode 120000 test/mips/qemu-malta_defconfig.yaml
create mode 100644 test/openrisc/generic_defconfig.yaml
create mode 100644 test/py/__init__.py
create mode 100644 test/py/helper.py
create mode 100644 test/py/test_bselftests.py
create mode 100644 test/py/test_bthread.py
create mode 100644 test/py/test_shell.py
create mode 100644 test/riscv/qemu@virt32_defconfig.yaml
create mode 100644 test/riscv/qemu@virt64_defconfig.yaml
create mode 100644 test/riscv/tinyemu@virt32_defconfig.yaml
create mode 100644 test/riscv/tinyemu@virt64_defconfig.yaml
create mode 120000 test/riscv/virt32_defconfig.yaml
create mode 120000 test/riscv/virt64_defconfig.yaml
create mode 100644 test/sandbox/sandbox_defconfig.yaml
create mode 100644 test/self/Kconfig
create mode 100644 test/self/Makefile
create mode 100644 test/self/core.c
create mode 100644 test/self/printf.c
create mode 120000 test/x86/efi_defconfig.yaml
create mode 100644 test/x86/pc@efi_defconfig.yaml
create mode 100644 test/x86/q35@efi_defconfig.yaml
create mode 100644 test/x86/virtio@efi_defconfig.yaml
--
2.29.2
_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox
next reply other threads:[~2021-04-12 7:19 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-12 7:16 Ahmad Fatoum [this message]
2021-04-12 7:16 ` [PATCH 01/17] kbuild: add install target Ahmad Fatoum
2021-04-12 7:16 ` [PATCH 02/17] kbuild: add ARCH={i386, x86_64} as aliases for x86 Ahmad Fatoum
2021-04-12 7:16 ` [PATCH 03/17] kbuild: add ARCH=um alias for sandbox Ahmad Fatoum
2021-04-12 7:16 ` [PATCH 04/17] MIPS: qemu-malta: generate swapped image as part of multi-image build Ahmad Fatoum
2021-04-12 9:27 ` Jules Maselbas
2021-04-12 9:45 ` Ahmad Fatoum
2021-04-12 7:16 ` [PATCH 05/17] MIPS: qemu-malta: replace board code with syscon-reboot Ahmad Fatoum
2021-04-12 7:16 ` [PATCH 06/17] MIPS: configs: qemu-malta: enable Virt I/O Ahmad Fatoum
2021-04-12 7:16 ` [PATCH 07/17] openrisc: set default KBUILD_IMAGE Ahmad Fatoum
2021-04-12 7:16 ` [PATCH 08/17] Documentation: boards: RISC-V: update TinyEMU support Ahmad Fatoum
2021-04-12 7:16 ` [PATCH 09/17] test: add basic barebox self-test infrastructure Ahmad Fatoum
2021-04-12 10:12 ` Jules Maselbas
2021-04-12 7:16 ` [PATCH 10/17] test: self: port Linux printf kselftest Ahmad Fatoum
2021-04-12 7:16 ` [PATCH 11/17] test: add labgrid configs for some emulated targets Ahmad Fatoum
2021-04-12 7:16 ` [PATCH 12/17] test: add first sample tests Ahmad Fatoum
2021-04-12 7:16 ` [PATCH 13/17] test: support running labgrid-pytest from out-of-tree build directory Ahmad Fatoum
2021-04-12 7:16 ` [PATCH 14/17] test: add emulate.pl, a runner for barebox on emulated targets Ahmad Fatoum
2021-04-12 7:16 ` [PATCH 15/17] test: self: run selftests as part of the pytest suite Ahmad Fatoum
2021-04-12 7:16 ` [PATCH 16/17] test: add bthread test Ahmad Fatoum
2021-04-12 7:16 ` [PATCH 17/17] [RFC] test: run ./test/emulate.pl in Github action Ahmad Fatoum
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210412071651.8769-1-a.fatoum@pengutronix.de \
--to=a.fatoum@pengutronix.de \
--cc=barebox@lists.infradead.org \
--cc=rcz@pengutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox