[PATCH] defaultenv: apply external environment after board overlays again

[PATCH] defaultenv: apply external environment after board overlays again

[Florijan Plohl]

Since commit 7f1cc5dd869d ("defaultenv: add base and external defaultenv
in pure_initcall"), the environment given with
CONFIG_DEFAULT_ENVIRONMENT_PATH is added before boards add their own
overlays with defaultenv_append_directory(). As overlays are applied in
the order they were added, board overlays now overwrite the external
environment, contrary to Documentation/user/defaultenv-2.rst, and BSPs
can no longer replace a board's default environment files, e.g. its boot
scripts.

Restore the documented order by giving each overlay a priority and
keeping the list sorted by it: the base and board specific overlays are
applied first, followed by CONFIG_DEFAULT_ENVIRONMENT_PATH. Overlays
added at runtime with defaultenv_append_runtime_directory(), like the
QEMU fw_cfg environment, are still applied last, so they keep
overwriting the external environment.

No in-tree board adds its own overlay and sets
CONFIG_DEFAULT_ENVIRONMENT_PATH at the same time, so only out-of-tree
configurations are affected. Add a migration guide entry for them.

Fixes: 7f1cc5dd869d ("defaultenv: add base and external defaultenv in pure_initcall")
Reported-by: Primoz Fiser <primoz.fiser@norik.com>
Signed-off-by: Florijan Plohl <florijan.plohl@norik.com>
---
RFC: https://lore.barebox.org/barebox/20261007055953.1001705-1-florijan.plohl@norik.com/

Changes since RFC, following Ahmad's review:
- Fix the order generically in defaultenv.c instead of a per-board
  opt-in
- Keep runtime overlays like QEMU fw_cfg after the external
  environment
- Add a migration guide entry
- Drop the phytec-som-imx6 board patch

Priorities are used, as appending the external environment from
defaultenv_load() would put it after fw_cfg again, and a later initcall
would depend on link and probe order.

Tested in sandbox (external overrides board, /env unchanged without
it, runtime overlay still wins) and on a phyCORE-i.MX6 with the Yocto
BSP environment, not with a real QEMU fw_cfg.

 .../migration-guides/migration-master.rst     | 15 ++++
 Documentation/user/defaultenv-2.rst           |  5 ++
 defaultenv/defaultenv.c                       | 82 ++++++++++++++++---
 3 files changed, 89 insertions(+), 13 deletions(-)

diff --git a/Documentation/migration-guides/migration-master.rst b/Documentation/migration-guides/migration-master.rst
index 715edd5934..59b9514997 100644
--- a/Documentation/migration-guides/migration-master.rst
+++ b/Documentation/migration-guides/migration-master.rst
@@ -20,3 +20,18 @@ points at a FIT.
 Existing users, if any, will fail-secure: The command will now exit with a failure::
 
   bootm: invalid option -- s
+
+Default environment overlay order
+---------------------------------
+
+Releases since v2026.01.0 applied board specific default environment overlays
+added with ``defaultenv_append_directory()`` after the environment given
+with ``CONFIG_DEFAULT_ENVIRONMENT_PATH`` and thus overwrote its files.
+
+The documented order has been restored: ``CONFIG_DEFAULT_ENVIRONMENT_PATH``
+is applied after the board specific overlays again and can overwrite their
+files. Overlays only available at runtime, like an environment passed via
+QEMU fw_cfg, are still applied last.
+
+BSPs that started to rely on the board specific overlay taking precedence
+need to remove the conflicting files from their external environment.
diff --git a/Documentation/user/defaultenv-2.rst b/Documentation/user/defaultenv-2.rst
index 719abe2782..78adb69293 100644
--- a/Documentation/user/defaultenv-2.rst
+++ b/Documentation/user/defaultenv-2.rst
@@ -34,6 +34,11 @@ board specific overlay, or overlays, could be at any path, but usually follows
 the form given.  They must be added to a Makefile in ``bbenv-y`` and
 ``defaultenv_append_directory()`` used to add them in board init code.
 
+Overlays that are only available at runtime, like an environment passed
+via QEMU fw_cfg as ``opt/org.barebox.env``, are applied after all of the
+above and can thus also overwrite files from
+``$(CONFIG_DEFAULT_ENVIRONMENT_PATH)``.
+
 Note that not all of the above directories will necessarily be
 included in your default environment, it depends on your barebox
 configuration settings. You can see the configuration variables
diff --git a/defaultenv/defaultenv.c b/defaultenv/defaultenv.c
index 573b1f862f..9a7600a7f9 100644
--- a/defaultenv/defaultenv.c
+++ b/defaultenv/defaultenv.c
@@ -24,14 +24,63 @@
 
 static LIST_HEAD(defaultenv_list);
 
+/*
+ * defaultenv_list is kept sorted by ascending priority, overlays of the same
+ * priority stay in the order they were added. defaultenv_load() applies them
+ * in list order, so a file from a later overlay replaces the same file of an
+ * earlier one, unless ENV_FLAG_NO_OVERWRITE is passed.
+ */
+enum defaultenv_prio {
+	/* base and board specific overlays, added with defaultenv_append() */
+	DEFAULTENV_PRIO_BOARD,
+	/* the environment given with CONFIG_DEFAULT_ENVIRONMENT_PATH */
+	DEFAULTENV_PRIO_EXTERNAL,
+	/* overlays added with defaultenv_append_runtime_directory() */
+	DEFAULTENV_PRIO_RUNTIME,
+};
+
 struct defaultenv {
 	struct list_head list;
+	enum defaultenv_prio prio;
 	const char *srcdir;
 	const char *name;
 	void *buf;
 	size_t size;
 };
 
+static void defaultenv_add(struct defaultenv *df)
+{
+	struct defaultenv *pos;
+
+	/*
+	 * Insert behind all entries of the same priority. list_add_sort()
+	 * would insert in front of them and thus reverse the order in which
+	 * overlays of the same priority were added.
+	 */
+	list_for_each_entry(pos, &defaultenv_list, list) {
+		if (pos->prio > df->prio) {
+			list_add_tail(&df->list, &pos->list);
+			return;
+		}
+	}
+
+	list_add_tail(&df->list, &defaultenv_list);
+}
+
+static void __defaultenv_append(void *buf, unsigned int size, const char *name,
+				enum defaultenv_prio prio)
+{
+	struct defaultenv *df;
+
+	df = xzalloc(sizeof(*df));
+	df->buf = buf;
+	df->size = size;
+	df->name = name;
+	df->prio = prio;
+
+	defaultenv_add(df);
+}
+
 /*
  * defaultenv_append - append a envfs buffer to the default environment
  * @buf:	The buffer containing the binary environment. If it is
@@ -39,20 +88,24 @@ struct defaultenv {
  * @size:	The length of the buffer
  *
  * This adds an overlay to the default environment. New files will be created,
- * existing files will be overwritten with the overlay.
+ * existing files will be overwritten with the overlay. The environment given
+ * with CONFIG_DEFAULT_ENVIRONMENT_PATH is applied after all overlays added
+ * with this function and can thus overwrite their files.
  */
 void defaultenv_append(void *buf, unsigned int size, const char *name)
 {
-	struct defaultenv *df;
-
-	df = xzalloc(sizeof(*df));
-	df->buf = buf;
-	df->size = size;
-	df->name = name;
-
-	list_add_tail(&df->list, &defaultenv_list);
+	__defaultenv_append(buf, size, name, DEFAULTENV_PRIO_BOARD);
 }
 
+/*
+ * defaultenv_append_runtime_directory - append a directory to the default
+ *                                       environment
+ * @srcdir:	The directory whose contents should be added
+ *
+ * This adds an overlay that is copied from @srcdir when the default environment
+ * is loaded. It is applied after the environment given with
+ * CONFIG_DEFAULT_ENVIRONMENT_PATH and can thus overwrite its files.
+ */
 void defaultenv_append_runtime_directory(const char *srcdir)
 {
 	struct defaultenv *df;
@@ -60,8 +113,9 @@ void defaultenv_append_runtime_directory(const char *srcdir)
 	df = xzalloc(sizeof(*df));
 	df->srcdir = srcdir;
 	df->name = srcdir;
+	df->prio = DEFAULTENV_PRIO_RUNTIME;
 
-	list_add_tail(&df->list, &defaultenv_list);
+	defaultenv_add(df);
 }
 
 static int defaultenv_load_one(struct defaultenv *df, const char *dir,
@@ -158,10 +212,12 @@ static int defaultenv_init(void)
 
 	/*
 	 * The traditional or external environment given with
-	 * CONFIG_DEFAULT_ENVIRONMENT_PATH.
+	 * CONFIG_DEFAULT_ENVIRONMENT_PATH. It is added here already, but
+	 * still applied after the board specific overlays added later on.
 	 */
-	defaultenv_append((void *)default_environment,
-			default_environment_size, "defaultenv");
+	__defaultenv_append((void *)default_environment,
+			    default_environment_size, "defaultenv",
+			    DEFAULTENV_PRIO_EXTERNAL);
 
 	return 0;
 }
-- 
2.43.0

Re: [PATCH] defaultenv: apply external environment after board overlays again

[Ahmad Fatoum]

On 10/16/26 8:17 AM, Florijan Plohl wrote:
> Since commit 7f1cc5dd869d ("defaultenv: add base and external defaultenv
> in pure_initcall"), the environment given with
> CONFIG_DEFAULT_ENVIRONMENT_PATH is added before boards add their own
> overlays with defaultenv_append_directory(). As overlays are applied in
> the order they were added, board overlays now overwrite the external
> environment, contrary to Documentation/user/defaultenv-2.rst, and BSPs
> can no longer replace a board's default environment files, e.g. its boot
> scripts.
> 
> Restore the documented order by giving each overlay a priority and
> keeping the list sorted by it: the base and board specific overlays are
> applied first, followed by CONFIG_DEFAULT_ENVIRONMENT_PATH. Overlays
> added at runtime with defaultenv_append_runtime_directory(), like the
> QEMU fw_cfg environment, are still applied last, so they keep
> overwriting the external environment.
> 
> No in-tree board adds its own overlay and sets
> CONFIG_DEFAULT_ENVIRONMENT_PATH at the same time, so only out-of-tree
> configurations are affected. Add a migration guide entry for them.
> 
> Fixes: 7f1cc5dd869d ("defaultenv: add base and external defaultenv in pure_initcall")
> Reported-by: Primoz Fiser <primoz.fiser@norik.com>
> Signed-off-by: Florijan Plohl <florijan.plohl@norik.com>

Tested-by: Ahmad Fatoum <a.fatoum@pengutronix.de>

> Tested in sandbox (external overrides board, /env unchanged without
> it, runtime overlay still wins) and on a phyCORE-i.MX6 with the Yocto
> BSP environment, not with a real QEMU fw_cfg.

Works as expected with QEMU fw_cfg:

$ make multi_v8_defconfig
$ make
$ ./scripts/qemu_interactive.py --env 'nv/user=test' \
	--env 'init/greet=echo hi $global.user'

Observation: "hi test" output during startup that masks
an external environment if added.


> 
>  .../migration-guides/migration-master.rst     | 15 ++++
>  Documentation/user/defaultenv-2.rst           |  5 ++
>  defaultenv/defaultenv.c                       | 82 ++++++++++++++++---
>  3 files changed, 89 insertions(+), 13 deletions(-)
> 
> diff --git a/Documentation/migration-guides/migration-master.rst b/Documentation/migration-guides/migration-master.rst
> index 715edd5934..59b9514997 100644
> --- a/Documentation/migration-guides/migration-master.rst
> +++ b/Documentation/migration-guides/migration-master.rst
> @@ -20,3 +20,18 @@ points at a FIT.
>  Existing users, if any, will fail-secure: The command will now exit with a failure::
>  
>    bootm: invalid option -- s
> +
> +Default environment overlay order
> +---------------------------------
> +
> +Releases since v2026.01.0 applied board specific default environment overlays
> +added with ``defaultenv_append_directory()`` after the environment given
> +with ``CONFIG_DEFAULT_ENVIRONMENT_PATH`` and thus overwrote its files.
> +
> +The documented order has been restored: ``CONFIG_DEFAULT_ENVIRONMENT_PATH``
> +is applied after the board specific overlays again and can overwrite their
> +files. Overlays only available at runtime, like an environment passed via
> +QEMU fw_cfg, are still applied last.
> +
> +BSPs that started to rely on the board specific overlay taking precedence
> +need to remove the conflicting files from their external environment.
> diff --git a/Documentation/user/defaultenv-2.rst b/Documentation/user/defaultenv-2.rst
> index 719abe2782..78adb69293 100644
> --- a/Documentation/user/defaultenv-2.rst
> +++ b/Documentation/user/defaultenv-2.rst
> @@ -34,6 +34,11 @@ board specific overlay, or overlays, could be at any path, but usually follows
>  the form given.  They must be added to a Makefile in ``bbenv-y`` and
>  ``defaultenv_append_directory()`` used to add them in board init code.
>  
> +Overlays that are only available at runtime, like an environment passed
> +via QEMU fw_cfg as ``opt/org.barebox.env``, are applied after all of the
> +above and can thus also overwrite files from
> +``$(CONFIG_DEFAULT_ENVIRONMENT_PATH)``.
> +
>  Note that not all of the above directories will necessarily be
>  included in your default environment, it depends on your barebox
>  configuration settings. You can see the configuration variables
> diff --git a/defaultenv/defaultenv.c b/defaultenv/defaultenv.c
> index 573b1f862f..9a7600a7f9 100644
> --- a/defaultenv/defaultenv.c
> +++ b/defaultenv/defaultenv.c
> @@ -24,14 +24,63 @@
>  
>  static LIST_HEAD(defaultenv_list);
>  
> +/*
> + * defaultenv_list is kept sorted by ascending priority, overlays of the same
> + * priority stay in the order they were added. defaultenv_load() applies them
> + * in list order, so a file from a later overlay replaces the same file of an
> + * earlier one, unless ENV_FLAG_NO_OVERWRITE is passed.
> + */
> +enum defaultenv_prio {
> +	/* base and board specific overlays, added with defaultenv_append() */
> +	DEFAULTENV_PRIO_BOARD,
> +	/* the environment given with CONFIG_DEFAULT_ENVIRONMENT_PATH */
> +	DEFAULTENV_PRIO_EXTERNAL,
> +	/* overlays added with defaultenv_append_runtime_directory() */
> +	DEFAULTENV_PRIO_RUNTIME,
> +};
> +
>  struct defaultenv {
>  	struct list_head list;
> +	enum defaultenv_prio prio;
>  	const char *srcdir;
>  	const char *name;
>  	void *buf;
>  	size_t size;
>  };
>  
> +static void defaultenv_add(struct defaultenv *df)
> +{
> +	struct defaultenv *pos;
> +
> +	/*
> +	 * Insert behind all entries of the same priority. list_add_sort()
> +	 * would insert in front of them and thus reverse the order in which
> +	 * overlays of the same priority were added.
> +	 */
> +	list_for_each_entry(pos, &defaultenv_list, list) {
> +		if (pos->prio > df->prio) {
> +			list_add_tail(&df->list, &pos->list);
> +			return;
> +		}
> +	}
> +
> +	list_add_tail(&df->list, &defaultenv_list);
> +}
> +
> +static void __defaultenv_append(void *buf, unsigned int size, const char *name,
> +				enum defaultenv_prio prio)
> +{
> +	struct defaultenv *df;
> +
> +	df = xzalloc(sizeof(*df));
> +	df->buf = buf;
> +	df->size = size;
> +	df->name = name;
> +	df->prio = prio;
> +
> +	defaultenv_add(df);
> +}
> +
>  /*
>   * defaultenv_append - append a envfs buffer to the default environment
>   * @buf:	The buffer containing the binary environment. If it is
> @@ -39,20 +88,24 @@ struct defaultenv {
>   * @size:	The length of the buffer
>   *
>   * This adds an overlay to the default environment. New files will be created,
> - * existing files will be overwritten with the overlay.
> + * existing files will be overwritten with the overlay. The environment given
> + * with CONFIG_DEFAULT_ENVIRONMENT_PATH is applied after all overlays added
> + * with this function and can thus overwrite their files.
>   */
>  void defaultenv_append(void *buf, unsigned int size, const char *name)
>  {
> -	struct defaultenv *df;
> -
> -	df = xzalloc(sizeof(*df));
> -	df->buf = buf;
> -	df->size = size;
> -	df->name = name;
> -
> -	list_add_tail(&df->list, &defaultenv_list);
> +	__defaultenv_append(buf, size, name, DEFAULTENV_PRIO_BOARD);
>  }
>  
> +/*
> + * defaultenv_append_runtime_directory - append a directory to the default
> + *                                       environment
> + * @srcdir:	The directory whose contents should be added
> + *
> + * This adds an overlay that is copied from @srcdir when the default environment
> + * is loaded. It is applied after the environment given with
> + * CONFIG_DEFAULT_ENVIRONMENT_PATH and can thus overwrite its files.
> + */
>  void defaultenv_append_runtime_directory(const char *srcdir)
>  {
>  	struct defaultenv *df;
> @@ -60,8 +113,9 @@ void defaultenv_append_runtime_directory(const char *srcdir)
>  	df = xzalloc(sizeof(*df));
>  	df->srcdir = srcdir;
>  	df->name = srcdir;
> +	df->prio = DEFAULTENV_PRIO_RUNTIME;
>  
> -	list_add_tail(&df->list, &defaultenv_list);
> +	defaultenv_add(df);
>  }
>  
>  static int defaultenv_load_one(struct defaultenv *df, const char *dir,
> @@ -158,10 +212,12 @@ static int defaultenv_init(void)
>  
>  	/*
>  	 * The traditional or external environment given with
> -	 * CONFIG_DEFAULT_ENVIRONMENT_PATH.
> +	 * CONFIG_DEFAULT_ENVIRONMENT_PATH. It is added here already, but
> +	 * still applied after the board specific overlays added later on.
>  	 */
> -	defaultenv_append((void *)default_environment,
> -			default_environment_size, "defaultenv");
> +	__defaultenv_append((void *)default_environment,
> +			    default_environment_size, "defaultenv",
> +			    DEFAULTENV_PRIO_EXTERNAL);
>  
>  	return 0;
>  }

-- 
Pengutronix e.K.                  |                             |
Steuerwalder Str. 21              | http://www.pengutronix.de/  |
31137 Hildesheim, Germany         | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686  | Fax:   +49-5121-206917-5555 |

Re: [PATCH] defaultenv: apply external environment after board overlays again

[Sascha Hauer]

On Fri, 16 Oct 2026 08:17:11 +0200, Florijan Plohl wrote:
> Since commit 7f1cc5dd869d ("defaultenv: add base and external defaultenv
> in pure_initcall"), the environment given with
> CONFIG_DEFAULT_ENVIRONMENT_PATH is added before boards add their own
> overlays with defaultenv_append_directory(). As overlays are applied in
> the order they were added, board overlays now overwrite the external
> environment, contrary to Documentation/user/defaultenv-2.rst, and BSPs
> can no longer replace a board's default environment files, e.g. its boot
> scripts.
> 
> [...]

Applied, thanks!

[1/1] defaultenv: apply external environment after board overlays again
      https://git.pengutronix.de/cgit/barebox/commit/?id=6b2dee07d718 (link may not be stable)

Best regards,
-- 
Sascha Hauer <s.hauer@pengutronix.de>