[PATCH 00/10] MAKEALL updates

[PATCH 00/10] MAKEALL updates

[Sascha Hauer]

This series puts some love into the existing MAKEALL script. Some bugs
are fixes, pick the correct 32/64 ARM toolchain based on the config
file, support applying Kconfig fragments before compilation and some
output beautification.

Sascha

Sascha Hauer (10):
  MAKEALL: Configure before setting up CROSS_COMPILE
  MAKEALL: Select arm64 toolchain for 64bit ARM configs
  MAKEALL: order configs alphabetically
  MAKEALL: Do not print sizes
  MAKEALL: Do not trap
  MAKEALL: fix printing number of compiled configs
  MAKEALL: support Kconfig fragments
  MAKEALL: rename variables
  MAKEALL: separate errors and warnings
  MAKEALL: allow multiple defconfigs as arguments

 MAKEALL | 88 +++++++++++++++++++++++++++++++++++----------------------
 1 file changed, 54 insertions(+), 34 deletions(-)

-- 
2.39.2

[PATCH 01/10] MAKEALL: Configure before setting up CROSS_COMPILE

[Sascha Hauer]

Do make defconfig before setting up CROSS_COMPILE. This will allow
us to select a ARM 32/64 bit toolchain from .config in the next step.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
 MAKEALL | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)

diff --git a/MAKEALL b/MAKEALL
index b4cf4c649b..ad95cfe040 100755
--- a/MAKEALL
+++ b/MAKEALL
@@ -98,6 +98,12 @@ do_build_target() {
 
 	tmp=$(echo "${target}" | tr - _)
 
+	MAKE="make -j${JOBS} ARCH=${arch} O=${BUILDDIR}"
+	${MAKE} ${target} 2>&1 > "${log_report}" | tee "${log_err}"
+
+	check_pipe_status
+	configure_result="$?"
+
 	cross_compile=$(eval echo '$CROSS_COMPILE_'${tmp})
 	cross_compile_set=$(eval echo '${CROSS_COMPILE_'${tmp}'+set}')
 	if [ "${cross_compile_set}" = "" ]
@@ -110,25 +116,20 @@ do_build_target() {
 		fi
 	fi
 
-	MAKE="make -j${JOBS} CROSS_COMPILE=${cross_compile} ARCH=${arch} O=${BUILDDIR}"
-	${MAKE} ${target} 2>&1 > "${log_report}" | tee "${log_err}"
-
-	check_pipe_status
-	result="$?"
-
 	printf "Configure: " | tee -a "${log_report}"
 
-	if [ "$result" = "0" ]; then
+	if [ "$configure_result" = "0" ]; then
 		printf "OK     \n" | tee -a "${log_report}"
 
+		MAKE="make -j${JOBS} CROSS_COMPILE=${cross_compile} ARCH=${arch} O=${BUILDDIR}"
 		${MAKE} -s 2>&1 >> "${log_report}" | tee -a "${log_err}"
 
 		check_pipe_status
-		result="$?"
+		compile_result="$?"
 
 		printf "Compile: " ${target} | tee -a "${log_report}"
 
-		if [ "$result" = "0" ]; then
+		if [ "$compile_result" = "0" ]; then
 			printf "OK     \n" | tee -a "${log_report}"
 			${cross_compile}size ${BUILDDIR}/barebox | tee -a "${log_report}"
 		else
-- 
2.39.2

[PATCH 02/10] MAKEALL: Select arm64 toolchain for 64bit ARM configs

[Sascha Hauer]

For both 32bit and 64bit architectures ARCH=arm is used, but we need
different toolchains to compile them. Pick the correct one based on
the CONFIG_CPU_64 symbol.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
 MAKEALL | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/MAKEALL b/MAKEALL
index ad95cfe040..dd61f89827 100755
--- a/MAKEALL
+++ b/MAKEALL
@@ -104,6 +104,13 @@ do_build_target() {
 	check_pipe_status
 	configure_result="$?"
 
+	if [ ${arch} = "arm" ]; then
+		grep -q "CONFIG_CPU_64=y" ${BUILDDIR}/.config
+		if [ $? = 0 ]; then
+			arch=arm64
+		fi
+	fi
+
 	cross_compile=$(eval echo '$CROSS_COMPILE_'${tmp})
 	cross_compile_set=$(eval echo '${CROSS_COMPILE_'${tmp}'+set}')
 	if [ "${cross_compile_set}" = "" ]
-- 
2.39.2

[PATCH 03/10] MAKEALL: order configs alphabetically

[Sascha Hauer]

Expanding wildcards returns the files in filesystem order. Sort them
alphabetically to compile the defconfigs in alphabetical order.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
 MAKEALL | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/MAKEALL b/MAKEALL
index dd61f89827..5b1fb3ddd6 100755
--- a/MAKEALL
+++ b/MAKEALL
@@ -167,7 +167,7 @@ do_build() {
 	local arch=$1
 	local regex=$2
 
-	find arch/${arch}/configs -name "${regex}_defconfig" | while read i
+	find arch/${arch}/configs -name "${regex}_defconfig" | sort | while read i
 	do
 		local target=$(basename $i)
 
-- 
2.39.2

[PATCH 04/10] MAKEALL: Do not print sizes

[Sascha Hauer]

The size of the barebox proper binary is not very useful to print,
the binary will be compressed which decreases its size and the PBL
will be added to it which then adds to the resulting size again.

Printing the sizes will only show what we all know already: Software
gets bigger over time when new features are added and more corner
cases are handled. Just drop printing of the sizes.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
 MAKEALL | 1 -
 1 file changed, 1 deletion(-)

diff --git a/MAKEALL b/MAKEALL
index 5b1fb3ddd6..c1048a21e2 100755
--- a/MAKEALL
+++ b/MAKEALL
@@ -138,7 +138,6 @@ do_build_target() {
 
 		if [ "$compile_result" = "0" ]; then
 			printf "OK     \n" | tee -a "${log_report}"
-			${cross_compile}size ${BUILDDIR}/barebox | tee -a "${log_report}"
 		else
 			printf "FAILED \n" | tee -a "${log_report}"
 			ret=1
-- 
2.39.2

[PATCH 05/10] MAKEALL: Do not trap

[Sascha Hauer]

Just print the compile statistics at the end. There's no need to print
them when MAKEALL gets interrupted, so just simplify the code by
removing trapping signals.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
 MAKEALL | 12 +++---------
 1 file changed, 3 insertions(+), 9 deletions(-)

diff --git a/MAKEALL b/MAKEALL
index c1048a21e2..4dcfa4138c 100755
--- a/MAKEALL
+++ b/MAKEALL
@@ -2,10 +2,6 @@
 #
 # SPDX-License-Identifier: GPL-2.0-only
 
-# Print statistics when we exit
-trap exit 1 2 3 15
-trap stats 0
-
 # Keep track of the number of builds and errors
 nb_errors=0
 errors_list=""
@@ -16,12 +12,9 @@ time_start=$(date +%s)
 
 filename=$(basename $0)
 
-is_print_stats=1
-
 #-----------------------------------------------------------------------
 
 usage() {
-	is_print_stats=0
 	echo "Usage: ${filename} [OPTION]..."
 	echo "Barebox MAKEALL tools."
 	echo ""
@@ -61,8 +54,6 @@ usage() {
 }
 
 stats() {
-	[ ${is_print_stats} -lt 1 ] && return
-
 	echo ""
 	echo "--------------------- SUMMARY ----------------------------"
 	echo "defconfigs compiled: ${nb_defconfigs}"
@@ -280,4 +271,7 @@ then
 else
 	do_build_target ${ARCH} $1
 fi
+
+stats
+
 exit $nb_errors
-- 
2.39.2

[PATCH 06/10] MAKEALL: fix printing number of compiled configs

[Sascha Hauer]

Do not call do_build_target in a subshell, because increasing
nb_defconfigs there will act on a local variable there instead
of increasing the variable in the parent shell.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
 MAKEALL | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/MAKEALL b/MAKEALL
index 4dcfa4138c..75f2f63a1d 100755
--- a/MAKEALL
+++ b/MAKEALL
@@ -157,8 +157,8 @@ do_build() {
 	local arch=$1
 	local regex=$2
 
-	find arch/${arch}/configs -name "${regex}_defconfig" | sort | while read i
-	do
+	configs=$(find arch/${arch}/configs -name "${regex}_defconfig" | sort)
+	for i in ${configs}; do
 		local target=$(basename $i)
 
 		do_build_target ${arch} ${target}
-- 
2.39.2

[PATCH 07/10] MAKEALL: support Kconfig fragments

[Sascha Hauer]

Add the ability to add kconfig fragments before building a defconfig.
The fragments can be added by either the -k option or by setting
the KCONFIG_ADD envrionment variable to a list of filenames.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
 MAKEALL | 26 ++++++++++++++++++--------
 1 file changed, 18 insertions(+), 8 deletions(-)

diff --git a/MAKEALL b/MAKEALL
index 75f2f63a1d..3ea0087a29 100755
--- a/MAKEALL
+++ b/MAKEALL
@@ -43,13 +43,14 @@ usage() {
 	echo "CONFIG=./MAKEALL.cfg ./MAKEALL"
 	echo ""
 	echo "you can specify via env or option"
-	echo "env       option"
-	echo "ARCH      -a      arch"
-	echo "CONFIG    -c      config"
-	echo "JOBS      -j      jobs"
-	echo "BUILDDIR  -O      build dir"
-	echo "LOGDIR    -l      log dir"
-	echo "REGEX     -e      regex"
+	echo "env         option"
+	echo "ARCH        -a      arch"
+	echo "CONFIG      -c      config"
+	echo "JOBS        -j      jobs"
+	echo "BUILDDIR    -O      build dir"
+	echo "LOGDIR      -l      log dir"
+	echo "REGEX       -e      regex"
+	echo "KCONFIG_ADD -k      kconfig fragment"
 	echo ""
 }
 
@@ -91,6 +92,12 @@ do_build_target() {
 
 	MAKE="make -j${JOBS} ARCH=${arch} O=${BUILDDIR}"
 	${MAKE} ${target} 2>&1 > "${log_report}" | tee "${log_err}"
+	for i in ${KCONFIG_ADD}; do
+		./scripts/kconfig/merge_config.sh -m -O \
+			${BUILDDIR} ${BUILDDIR}/.config $i \
+			2>&1 > "${log_report}" | tee "${log_err}"
+	done
+	${MAKE} olddefconfig 2>&1 > "${log_report}" | tee "${log_err}"
 
 	check_pipe_status
 	configure_result="$?"
@@ -182,7 +189,7 @@ do_build_all() {
 	return $build_target
 }
 
-while getopts "hc:j:O:l:a:e:" Option
+while getopts "hc:j:O:l:a:e:k:" Option
 do
 case $Option in
 	a )
@@ -203,6 +210,9 @@ case $Option in
 	e )
 		REGEX=${OPTARG}
 		;;
+	k )
+		KCONFIG_ADD="${KCONFIG_ADD} ${OPTARG}"
+		;;
 	h )
 		usage
 		exit 0
-- 
2.39.2

[PATCH 08/10] MAKEALL: rename variables

[Sascha Hauer]

nb_errors and errors_list contain the number/list of warnings, not
errors, so rename the variables accordingly.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
 MAKEALL | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/MAKEALL b/MAKEALL
index 3ea0087a29..dc0fac0f44 100755
--- a/MAKEALL
+++ b/MAKEALL
@@ -3,8 +3,8 @@
 # SPDX-License-Identifier: GPL-2.0-only
 
 # Keep track of the number of builds and errors
-nb_errors=0
-errors_list=""
+nb_warnings=0
+warnings_list=""
 nb_defconfigs=0
 ret=0
 
@@ -61,8 +61,8 @@ stats() {
 	time_stop=$(date +%s)
 	time_diff=$((${time_stop} - ${time_start}))
 	printf "compiled in %4is\n" ${time_diff}
-	if [ ${nb_errors} -gt 0 ] ; then
-		echo "defconfigs with warnings or errors: ${nb_errors} (${errors_list} )"
+	if [ ${nb_warnings} -gt 0 ] ; then
+		echo "defconfigs with warnings or errors: ${nb_warnings} (${warnings_list} )"
 	fi
 	echo "----------------------------------------------------------"
 
@@ -147,8 +147,8 @@ do_build_target() {
 	fi
 
 	if [ -s "${log_err}" ] ; then
-		nb_errors=$((nb_errors + 1))
-		errors_list="${errors_list} ${target}"
+		nb_warnings=$((nb_warnings + 1))
+		warnings_list="${warnings_list} ${target}"
 	else
 		rm "${log_err}"
 	fi
@@ -284,4 +284,4 @@ fi
 
 stats
 
-exit $nb_errors
+exit $nb_warnings
-- 
2.39.2

[PATCH 09/10] MAKEALL: separate errors and warnings

[Sascha Hauer]

Count and print errors and warnings separately to get and overview which
configs failed and which ones issued warnings.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
 MAKEALL | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/MAKEALL b/MAKEALL
index dc0fac0f44..039368b027 100755
--- a/MAKEALL
+++ b/MAKEALL
@@ -5,6 +5,8 @@
 # Keep track of the number of builds and errors
 nb_warnings=0
 warnings_list=""
+nb_errors=0
+errors_list=""
 nb_defconfigs=0
 ret=0
 
@@ -61,8 +63,11 @@ stats() {
 	time_stop=$(date +%s)
 	time_diff=$((${time_stop} - ${time_start}))
 	printf "compiled in %4is\n" ${time_diff}
+	if [ ${nb_errors} -gt 0 ] ; then
+		echo "defconfigs with errors: ${nb_errors} (${errors_list} )"
+	fi
 	if [ ${nb_warnings} -gt 0 ] ; then
-		echo "defconfigs with warnings or errors: ${nb_warnings} (${warnings_list} )"
+		echo "defconfigs with warnings: ${nb_warnings} (${warnings_list} )"
 	fi
 	echo "----------------------------------------------------------"
 
@@ -138,6 +143,8 @@ do_build_target() {
 			printf "OK     \n" | tee -a "${log_report}"
 		else
 			printf "FAILED \n" | tee -a "${log_report}"
+			nb_errors=$((nb_errors + 1))
+			errors_list="${errors_list} ${target}"
 			ret=1
 		fi
 	else
@@ -284,4 +291,4 @@ fi
 
 stats
 
-exit $nb_warnings
+exit ${nb_errors}
-- 
2.39.2

[PATCH 10/10] MAKEALL: allow multiple defconfigs as arguments

[Sascha Hauer]

Allow to pass multiple defconfigs as arguments.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
 MAKEALL | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/MAKEALL b/MAKEALL
index 039368b027..a4e268d54c 100755
--- a/MAKEALL
+++ b/MAKEALL
@@ -286,7 +286,9 @@ if [ $# -eq 0 ]
 then
 	do_build ${ARCH} "${REGEX}"
 else
-	do_build_target ${ARCH} $1
+	for i in $*; do
+		do_build_target ${ARCH} $i
+	done
 fi
 
 stats
-- 
2.39.2