* [PATCH] genenv: create a gcc-like .d file for depenencies. @ 2019-02-18 15:57 Tomaz Solc 2019-02-20 8:57 ` Sascha Hauer 0 siblings, 1 reply; 6+ messages in thread From: Tomaz Solc @ 2019-02-18 15:57 UTC (permalink / raw) To: barebox; +Cc: Tomaz Solc Commit 658af1ca mentions that environment build dependencies are not tracked and the complete environment is rebuilt on every build. However commit 105201e0 added if_changed to the make rule, so this is currently not the case. Environment is only rebuilt if barebox_default_env is missing or if CONFIG_DEFAULT_ENVIRONMENT_PATH has changed. genenv is not re-run if any individual source files are modified. So if environment source files are edited, Barebox is rebuilt with a stale barebox_default_env unless it is manually deleted first. With this commit genenv creates a .d file, similar to those created by gcc to track C header dependencies. This is then passed to Kbuild with if_changed_dep in the Makefile. This makes make re-run genenv if any source environment files are changed. However, new environment files are still not detected automatically. --- defaultenv/Makefile | 2 +- scripts/genenv | 18 +++++++++--------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/defaultenv/Makefile b/defaultenv/Makefile index f313b04e8..383b48e29 100644 --- a/defaultenv/Makefile +++ b/defaultenv/Makefile @@ -14,7 +14,7 @@ quiet_cmd_env_default = ENV $@ cmd_env_default = ($(srctree)/scripts/genenv $(srctree) $(objtree) $@ $(CONFIG_DEFAULT_ENVIRONMENT_PATH)) $(obj)/barebox_default_env: FORCE - $(call if_changed,env_default) + $(call if_changed_dep,env_default) quiet_cmd_env_h = ENVH $@ cmd_env_h = cat $< | (cd $(obj) && $(objtree)/scripts/bin2c "__aligned(4) default_environment") > $@; \ diff --git a/scripts/genenv b/scripts/genenv index 5ebe69963..756a2522b 100755 --- a/scripts/genenv +++ b/scripts/genenv @@ -30,29 +30,29 @@ abspath() { export -f abspath tempdir=$(abspath "${target}.genenv.tmp") -tmpfile="$(mktemp)" +depfile=$(dirname "${target}")/.$(basename "${target}").d mkdir -p "$tempdir" (cd $basedir +# First listed dependency is ignored by fixdep, but must be a readable file. +deps=scripts/genenv for i in $*; do if [ -d $i ]; then cp -r $i/* $tempdir + deps="$deps $(find $i -type f)" else cp -a $i $tempdir + deps="$deps $i" fi done +# This doesn't work with spaces in filenames, but Kbuild's fixdep doesn't seem +# to support such filenames in any case. +echo "${target}: $(echo ${deps})" > ${depfile} ) find $tempdir -name '.svn' -o -name '*~' -delete -$objtree/scripts/bareboxenv -s $tempdir ${tmpfile} - -diff "${tmpfile}" "${target}" >/dev/null 2>/dev/null -if [ $? != 0 ]; then - mv "${tmpfile}" "${target}" -else - rm ${tmpfile} -fi +$objtree/scripts/bareboxenv -s $tempdir ${target} rm -r $tempdir -- 2.11.0 _______________________________________________ barebox mailing list barebox@lists.infradead.org http://lists.infradead.org/mailman/listinfo/barebox ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] genenv: create a gcc-like .d file for depenencies. 2019-02-18 15:57 [PATCH] genenv: create a gcc-like .d file for depenencies Tomaz Solc @ 2019-02-20 8:57 ` Sascha Hauer 2019-02-20 11:14 ` Tomaž Šolc 0 siblings, 1 reply; 6+ messages in thread From: Sascha Hauer @ 2019-02-20 8:57 UTC (permalink / raw) To: Tomaz Solc; +Cc: barebox Hi Tomaz, On Mon, Feb 18, 2019 at 04:57:12PM +0100, Tomaz Solc wrote: > Commit 658af1ca mentions that environment build dependencies are not tracked > and the complete environment is rebuilt on every build. > > However commit 105201e0 added if_changed to the make rule, so this is currently > not the case. Environment is only rebuilt if barebox_default_env is missing or > if CONFIG_DEFAULT_ENVIRONMENT_PATH has changed. genenv is not re-run if any > individual source files are modified. So if environment source files are > edited, Barebox is rebuilt with a stale barebox_default_env unless it is > manually deleted first. > > With this commit genenv creates a .d file, similar to those created by gcc to > track C header dependencies. This is then passed to Kbuild with if_changed_dep > in the Makefile. This makes make re-run genenv if any source environment files are > changed. However, new environment files are still not detected automatically. Please try the attached patch. It should solve this issue once and for all. With this the environments are always built, but only if they have actually changed then the corresponding .S files are built. Sascha ------------------------------8<-------------------------------- From 6c1275c02d9635ccdf7c263cbb3f853a20c670b3 Mon Sep 17 00:00:00 2001 From: Sascha Hauer <s.hauer@pengutronix.de> Date: Wed, 20 Feb 2019 09:45:32 +0100 Subject: [PATCH] defaultenv: Fix dependencies The defaultenv should be rebuilt once a file in it has changed. the genenv script always generates the environment file to a temporary file. Only if it has changed to the last target file the temporary file is moved over the target file. This means we always have to call genenv, thus replace "if_changed" with "cmd". With this dependencies are correctly tracked. New or changed files result in new image builds whereas unchanged environments do not unnecessarily result in new images. Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de> --- scripts/Makefile.lib | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 7b8643bf57..561247ab34 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -325,7 +325,7 @@ quiet_cmd_env = ENV $@ cmd_env=$(srctree)/scripts/genenv $(srctree) $(objtree) $@ $< %.bbenv$(DEFAULT_COMPRESSION_SUFFIX): % FORCE - $(call if_changed,env) + $(call cmd,env) # Bzip2 # --------------------------------------------------------------------------- -- 2.20.1 -- Pengutronix e.K. | | Industrial Linux Solutions | http://www.pengutronix.de/ | Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 | _______________________________________________ barebox mailing list barebox@lists.infradead.org http://lists.infradead.org/mailman/listinfo/barebox ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] genenv: create a gcc-like .d file for depenencies. 2019-02-20 8:57 ` Sascha Hauer @ 2019-02-20 11:14 ` Tomaž Šolc 2019-02-20 11:28 ` Sascha Hauer 0 siblings, 1 reply; 6+ messages in thread From: Tomaž Šolc @ 2019-02-20 11:14 UTC (permalink / raw) To: Sascha Hauer; +Cc: barebox Hi Sascha, On 20. 02. 19 09:57, Sascha Hauer wrote: > Hi Tomaz, > > On Mon, Feb 18, 2019 at 04:57:12PM +0100, Tomaz Solc wrote: >> With this commit genenv creates a .d file, similar to those created by gcc to >> track C header dependencies. This is then passed to Kbuild with if_changed_dep >> in the Makefile. This makes make re-run genenv if any source environment files are >> changed. However, new environment files are still not detected automatically. > > Please try the attached patch. It should solve this issue once and for > all. With this the environments are always built, but only if they have > actually changed then the corresponding .S files are built. Unfortunately with your patch the final image still gets rebuilt with a stale environment unless I delete defaultenv/barebox_default_env. See transcript bellow, where I'm using your patch. I modify one environment file for Raspberry Pi. After just calling "make", the image doesn't include the change. Best regards Tomaž $ make rpi_defconfig $ make [...] $ strings images/barebox-raspberry-pi-3.img | grep global.test $ echo global.test="foo" >> arch/arm/boards/raspberry-pi/env/init/bootargs-base $ make make[1]: 'include/generated/mach-types.h' is up to date. CHK include/generated/version.h CHK include/generated/utsrelease.h UPD include/generated/utsrelease.h CREATE include/config.h CHK include/generated/compile.h CC common/version.o CC common/globalvar.o CHK include/generated/passwd.h LD common/built-in.o ENV defaultenv/defaultenv-2-base.bbenv GEN .version CHK include/generated/compile.h UPD include/generated/compile.h CC common/version.o CHK include/generated/passwd.h LD common/built-in.o LD .tmp_barebox1 KSYM .tmp_kallsyms1.S AS .tmp_kallsyms1.o LD .tmp_barebox2 KSYM .tmp_kallsyms2.S AS .tmp_kallsyms2.o LD barebox SYSMAP System.map OBJCOPY barebox.bin SHIPPED_S images/barebox.z AS images/piggy.o LD images/start_raspberry_pi1.pbl OBJCOPYB images/start_raspberry_pi1.pblb SHIPPED images/barebox-raspberry-pi-1.img LD images/start_raspberry_pi2.pbl OBJCOPYB images/start_raspberry_pi2.pblb SHIPPED images/barebox-raspberry-pi-2.img LD images/start_raspberry_pi3.pbl OBJCOPYB images/start_raspberry_pi3.pblb SHIPPED images/barebox-raspberry-pi-3.img LN images/../barebox-flash-image images built: barebox-raspberry-pi-1.img barebox-raspberry-pi-2.img barebox-raspberry-pi-3.img $ strings images/barebox-raspberry-pi-3.img | grep global.test $ rm defaultenv/barebox_default_env $ make make[1]: 'include/generated/mach-types.h' is up to date. CHK include/generated/version.h CHK include/generated/utsrelease.h CREATE include/config.h CHK include/generated/compile.h CHK include/generated/passwd.h ENV defaultenv/barebox_default_env ENVH defaultenv/barebox_default_env.h CC defaultenv/defaultenv.o ENV defaultenv/defaultenv-2-base.bbenv LD defaultenv/built-in.o GEN .version CHK include/generated/compile.h UPD include/generated/compile.h CC common/version.o CHK include/generated/passwd.h LD common/built-in.o LD .tmp_barebox1 KSYM .tmp_kallsyms1.S AS .tmp_kallsyms1.o LD .tmp_barebox2 KSYM .tmp_kallsyms2.S AS .tmp_kallsyms2.o LD barebox SYSMAP System.map OBJCOPY barebox.bin SHIPPED_S images/barebox.z AS images/piggy.o LD images/start_raspberry_pi1.pbl OBJCOPYB images/start_raspberry_pi1.pblb SHIPPED images/barebox-raspberry-pi-1.img LD images/start_raspberry_pi2.pbl OBJCOPYB images/start_raspberry_pi2.pblb SHIPPED images/barebox-raspberry-pi-2.img LD images/start_raspberry_pi3.pbl OBJCOPYB images/start_raspberry_pi3.pblb SHIPPED images/barebox-raspberry-pi-3.img LN images/../barebox-flash-image images built: barebox-raspberry-pi-1.img barebox-raspberry-pi-2.img barebox-raspberry-pi-3.img $ strings images/barebox-raspberry-pi-3.img | grep global.test global.test=foo $ _______________________________________________ barebox mailing list barebox@lists.infradead.org http://lists.infradead.org/mailman/listinfo/barebox ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] genenv: create a gcc-like .d file for depenencies. 2019-02-20 11:14 ` Tomaž Šolc @ 2019-02-20 11:28 ` Sascha Hauer 2019-02-21 8:34 ` Tomaž Šolc 0 siblings, 1 reply; 6+ messages in thread From: Sascha Hauer @ 2019-02-20 11:28 UTC (permalink / raw) To: Tomaž Šolc; +Cc: barebox On Wed, Feb 20, 2019 at 12:14:43PM +0100, Tomaž Šolc wrote: > Hi Sascha, > > On 20. 02. 19 09:57, Sascha Hauer wrote: > > Hi Tomaz, > > > > On Mon, Feb 18, 2019 at 04:57:12PM +0100, Tomaz Solc wrote: > > > With this commit genenv creates a .d file, similar to those created by gcc to > > > track C header dependencies. This is then passed to Kbuild with if_changed_dep > > > in the Makefile. This makes make re-run genenv if any source environment files are > > > changed. However, new environment files are still not detected automatically. > > > > Please try the attached patch. It should solve this issue once and for > > all. With this the environments are always built, but only if they have > > actually changed then the corresponding .S files are built. > > Unfortunately with your patch the final image still gets rebuilt with a > stale environment unless I delete defaultenv/barebox_default_env. > > See transcript bellow, where I'm using your patch. I modify one environment > file for Raspberry Pi. After just calling "make", the image doesn't include > the change. Right, I missed the environment snippet that is generated from CONFIG_DEFAULT_ENVIRONMENT_PATH which is generated at a different place. Please try this updated patch. Sascha -----------------------------------8<------------------------------- From c41a478e59a74b66a3de5c597224f02e0b39fa12 Mon Sep 17 00:00:00 2001 From: Sascha Hauer <s.hauer@pengutronix.de> Date: Wed, 20 Feb 2019 09:45:32 +0100 Subject: [PATCH] defaultenv: Fix dependencies The defaultenv should be rebuilt once a file in it has changed. the genenv script always generates the environment file to a temporary file. Only if it has changed to the last target file the temporary file is moved over the target file. This means we always have to call genenv, thus replace "if_changed" with "cmd". With this dependencies are correctly tracked. New or changed files result in new image builds whereas unchanged environments do not unnecessarily result in new images. Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de> --- defaultenv/Makefile | 2 +- scripts/Makefile.lib | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/defaultenv/Makefile b/defaultenv/Makefile index f313b04e84..03b329c4dd 100644 --- a/defaultenv/Makefile +++ b/defaultenv/Makefile @@ -14,7 +14,7 @@ quiet_cmd_env_default = ENV $@ cmd_env_default = ($(srctree)/scripts/genenv $(srctree) $(objtree) $@ $(CONFIG_DEFAULT_ENVIRONMENT_PATH)) $(obj)/barebox_default_env: FORCE - $(call if_changed,env_default) + $(call cmd,env_default) quiet_cmd_env_h = ENVH $@ cmd_env_h = cat $< | (cd $(obj) && $(objtree)/scripts/bin2c "__aligned(4) default_environment") > $@; \ diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 7b8643bf57..561247ab34 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -325,7 +325,7 @@ quiet_cmd_env = ENV $@ cmd_env=$(srctree)/scripts/genenv $(srctree) $(objtree) $@ $< %.bbenv$(DEFAULT_COMPRESSION_SUFFIX): % FORCE - $(call if_changed,env) + $(call cmd,env) # Bzip2 # --------------------------------------------------------------------------- -- 2.20.1 -- Pengutronix e.K. | | Industrial Linux Solutions | http://www.pengutronix.de/ | Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 | _______________________________________________ barebox mailing list barebox@lists.infradead.org http://lists.infradead.org/mailman/listinfo/barebox ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] genenv: create a gcc-like .d file for depenencies. 2019-02-20 11:28 ` Sascha Hauer @ 2019-02-21 8:34 ` Tomaž Šolc 2019-02-22 7:34 ` Sascha Hauer 0 siblings, 1 reply; 6+ messages in thread From: Tomaž Šolc @ 2019-02-21 8:34 UTC (permalink / raw) To: Sascha Hauer; +Cc: barebox Hi Sascha, On 20. 02. 19 12:28, Sascha Hauer wrote: > Right, I missed the environment snippet that is generated from > CONFIG_DEFAULT_ENVIRONMENT_PATH which is generated at a different place. > Please try this updated patch. Thanks, as far as I can see this patch works correctly now. I suggest adding a comment in defaultenv/Makefile that rebuilding the environment on every build is the intended behavior. It seems this was already once erroneously considered a bug (see commit 105201e0). Best regards Tomaž _______________________________________________ barebox mailing list barebox@lists.infradead.org http://lists.infradead.org/mailman/listinfo/barebox ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] genenv: create a gcc-like .d file for depenencies. 2019-02-21 8:34 ` Tomaž Šolc @ 2019-02-22 7:34 ` Sascha Hauer 0 siblings, 0 replies; 6+ messages in thread From: Sascha Hauer @ 2019-02-22 7:34 UTC (permalink / raw) To: Tomaž Šolc; +Cc: barebox On Thu, Feb 21, 2019 at 09:34:47AM +0100, Tomaž Šolc wrote: > Hi Sascha, > > On 20. 02. 19 12:28, Sascha Hauer wrote: > > Right, I missed the environment snippet that is generated from > > CONFIG_DEFAULT_ENVIRONMENT_PATH which is generated at a different place. > > Please try this updated patch. > > Thanks, as far as I can see this patch works correctly now. > > I suggest adding a comment in defaultenv/Makefile that rebuilding the > environment on every build is the intended behavior. It seems this was > already once erroneously considered a bug (see commit 105201e0). Good idea, just did that. Regards, Sascha -- Pengutronix e.K. | | Industrial Linux Solutions | http://www.pengutronix.de/ | Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 | _______________________________________________ barebox mailing list barebox@lists.infradead.org http://lists.infradead.org/mailman/listinfo/barebox ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-02-22 7:34 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-02-18 15:57 [PATCH] genenv: create a gcc-like .d file for depenencies Tomaz Solc 2019-02-20 8:57 ` Sascha Hauer 2019-02-20 11:14 ` Tomaž Šolc 2019-02-20 11:28 ` Sascha Hauer 2019-02-21 8:34 ` Tomaž Šolc 2019-02-22 7:34 ` Sascha Hauer
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox