[PATCH v2] ARM: i.MX8M: esdctl: split memory banks for devices with >4G

[PATCH v2] ARM: i.MX8M: esdctl: split memory banks for devices with >4G

[Marco Felsch]

At the moment the whole available memory is added to one single memory
bank "ram0". This can cause barebox chainload issues on devices with a
huge amount of memory like the i.MX8MP-EVK which has 6G of RAM if the
barebox pbl binary is to large.

The reason for this issues is that memory_bank_first_find_space()
returns the memory area with the largest amount of free space on the
first memory bank. So in case of Debix SOM-A 8G and i.MX8MP-EVK 6G this
is the area crossing the 4G boundary. This cause the barebox pbl code to
trigger a MMU exception once the early MMU gets enabled which is
configured for sizes <=4G.

Split the memory space into two memory banks: "ram0" and "ram1" to fix
this issue.

Signed-off-by: Marco Felsch <m.felsch@pengutronix.de>
---
v2:
- drop add_mem() usage and use arm_add_mem_device() directly

 arch/arm/mach-imx/esdctl.c | 23 +++++++++++++++++++----
 1 file changed, 19 insertions(+), 4 deletions(-)

diff --git a/arch/arm/mach-imx/esdctl.c b/arch/arm/mach-imx/esdctl.c
index 1798ad48e50a..7f313b2337ae 100644
--- a/arch/arm/mach-imx/esdctl.c
+++ b/arch/arm/mach-imx/esdctl.c
@@ -510,16 +510,31 @@ static resource_size_t imx8m_ddrc_sdram_size(void __iomem *ddrc, unsigned buswid
 				   reduced_adress_space, mstr);
 }
 
+static int _imx8m_ddrc_add_mem(void *mmdcbase, struct imx_esdctl_data *data,
+			       unsigned int buswidth)
+{
+	resource_size_t size = imx8m_ddrc_sdram_size(mmdcbase, buswidth);
+	resource_size_t size0, size1;
+	int ret;
+
+	size0 = min_t(resource_size_t, SZ_4G - MX8M_DDR_CSD1_BASE_ADDR, size);
+	size1 = size - size0;
+
+	ret = arm_add_mem_device("ram0", data->base0, size0);
+	if (ret || size1 == 0)
+		return ret;
+
+	return arm_add_mem_device("ram1", SZ_4G, size1);
+}
+
 static int imx8m_ddrc_add_mem(void *mmdcbase, struct imx_esdctl_data *data)
 {
-	return arm_add_mem_device("ram0", data->base0,
-			   imx8m_ddrc_sdram_size(mmdcbase, 32));
+	return _imx8m_ddrc_add_mem(mmdcbase, data, 32);
 }
 
 static int imx8mn_ddrc_add_mem(void *mmdcbase, struct imx_esdctl_data *data)
 {
-	return arm_add_mem_device("ram0", data->base0,
-			   imx8m_ddrc_sdram_size(mmdcbase, 16));
+	return _imx8m_ddrc_add_mem(mmdcbase, data, 16);
 }
 
 static resource_size_t imx7d_ddrc_sdram_size(void __iomem *ddrc)
-- 
2.39.2

Re: [PATCH v2] ARM: i.MX8M: esdctl: split memory banks for devices with >4G

[Sascha Hauer]

Hi Marco,

On Thu, Aug 31, 2023 at 04:47:24PM +0200, Marco Felsch wrote:
> At the moment the whole available memory is added to one single memory
> bank "ram0". This can cause barebox chainload issues on devices with a
> huge amount of memory like the i.MX8MP-EVK which has 6G of RAM if the
> barebox pbl binary is to large.
> 
> The reason for this issues is that memory_bank_first_find_space()
> returns the memory area with the largest amount of free space on the
> first memory bank. So in case of Debix SOM-A 8G and i.MX8MP-EVK 6G this
> is the area crossing the 4G boundary. This cause the barebox pbl code to
> trigger a MMU exception once the early MMU gets enabled which is
> configured for sizes <=4G.
> 
> Split the memory space into two memory banks: "ram0" and "ram1" to fix
> this issue.

In the long run we'll need a proper solution for this. I think this
patch is fine for now, but please add a comment why we are doing this.
Otherwise we might remove this superfluous code at some point.

Sascha

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [PATCH v2] ARM: i.MX8M: esdctl: split memory banks for devices with >4G

[Marco Felsch]

On 23-09-04, Sascha Hauer wrote:
> Hi Marco,
> 
> On Thu, Aug 31, 2023 at 04:47:24PM +0200, Marco Felsch wrote:
> > At the moment the whole available memory is added to one single memory
> > bank "ram0". This can cause barebox chainload issues on devices with a
> > huge amount of memory like the i.MX8MP-EVK which has 6G of RAM if the
> > barebox pbl binary is to large.
> > 
> > The reason for this issues is that memory_bank_first_find_space()
> > returns the memory area with the largest amount of free space on the
> > first memory bank. So in case of Debix SOM-A 8G and i.MX8MP-EVK 6G this
> > is the area crossing the 4G boundary. This cause the barebox pbl code to
> > trigger a MMU exception once the early MMU gets enabled which is
> > configured for sizes <=4G.
> > 
> > Split the memory space into two memory banks: "ram0" and "ram1" to fix
> > this issue.
> 
> In the long run we'll need a proper solution for this. I think this

Of course but I didn't wanted to step into this hole ;)

> patch is fine for now, but please add a comment why we are doing this.
> Otherwise we might remove this superfluous code at some point.

Sure, I will add a comment.

Regards,
  Marco

> 
> Sascha
> 
> -- 
> Pengutronix e.K.                           |                             |
> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
> 

Re: [PATCH v2] ARM: i.MX8M: esdctl: split memory banks for devices with >4G

[Ahmad Fatoum]

On 31.08.23 16:47, Marco Felsch wrote:
> At the moment the whole available memory is added to one single memory
> bank "ram0". This can cause barebox chainload issues on devices with a
> huge amount of memory like the i.MX8MP-EVK which has 6G of RAM if the
> barebox pbl binary is to large.
> 
> The reason for this issues is that memory_bank_first_find_space()
> returns the memory area with the largest amount of free space on the
> first memory bank. So in case of Debix SOM-A 8G and i.MX8MP-EVK 6G this
> is the area crossing the 4G boundary. This cause the barebox pbl code to
> trigger a MMU exception once the early MMU gets enabled which is
> configured for sizes <=4G.
> 
> Split the memory space into two memory banks: "ram0" and "ram1" to fix
> this issue.
> 
> Signed-off-by: Marco Felsch <m.felsch@pengutronix.de>
> ---
> v2:
> - drop add_mem() usage and use arm_add_mem_device() directly
> 
>  arch/arm/mach-imx/esdctl.c | 23 +++++++++++++++++++----
>  1 file changed, 19 insertions(+), 4 deletions(-)
> 
> diff --git a/arch/arm/mach-imx/esdctl.c b/arch/arm/mach-imx/esdctl.c
> index 1798ad48e50a..7f313b2337ae 100644
> --- a/arch/arm/mach-imx/esdctl.c
> +++ b/arch/arm/mach-imx/esdctl.c
> @@ -510,16 +510,31 @@ static resource_size_t imx8m_ddrc_sdram_size(void __iomem *ddrc, unsigned buswid
>  				   reduced_adress_space, mstr);
>  }
>  
> +static int _imx8m_ddrc_add_mem(void *mmdcbase, struct imx_esdctl_data *data,
> +			       unsigned int buswidth)
> +{
> +	resource_size_t size = imx8m_ddrc_sdram_size(mmdcbase, buswidth);
> +	resource_size_t size0, size1;
> +	int ret;
> +
> +	size0 = min_t(resource_size_t, SZ_4G - MX8M_DDR_CSD1_BASE_ADDR, size);
> +	size1 = size - size0;
> +
> +	ret = arm_add_mem_device("ram0", data->base0, size0);
> +	if (ret || size1 == 0)
> +		return ret;
> +
> +	return arm_add_mem_device("ram1", SZ_4G, size1);

This gives a warning when compiling the file for 32-bit, because resource_size_t
is 32-bit there, while SZ_4G is always an unsigned long long.

> +}
> +
>  static int imx8m_ddrc_add_mem(void *mmdcbase, struct imx_esdctl_data *data)
>  {
> -	return arm_add_mem_device("ram0", data->base0,
> -			   imx8m_ddrc_sdram_size(mmdcbase, 32));
> +	return _imx8m_ddrc_add_mem(mmdcbase, data, 32);
>  }
>  
>  static int imx8mn_ddrc_add_mem(void *mmdcbase, struct imx_esdctl_data *data)
>  {
> -	return arm_add_mem_device("ram0", data->base0,
> -			   imx8m_ddrc_sdram_size(mmdcbase, 16));
> +	return _imx8m_ddrc_add_mem(mmdcbase, data, 16);
>  }
>  
>  static resource_size_t imx7d_ddrc_sdram_size(void __iomem *ddrc)

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [PATCH v2] ARM: i.MX8M: esdctl: split memory banks for devices with >4G

[Marco Felsch]

On 23-09-04, Ahmad Fatoum wrote:
> On 31.08.23 16:47, Marco Felsch wrote:
> > At the moment the whole available memory is added to one single memory
> > bank "ram0". This can cause barebox chainload issues on devices with a
> > huge amount of memory like the i.MX8MP-EVK which has 6G of RAM if the
> > barebox pbl binary is to large.
> > 
> > The reason for this issues is that memory_bank_first_find_space()
> > returns the memory area with the largest amount of free space on the
> > first memory bank. So in case of Debix SOM-A 8G and i.MX8MP-EVK 6G this
> > is the area crossing the 4G boundary. This cause the barebox pbl code to
> > trigger a MMU exception once the early MMU gets enabled which is
> > configured for sizes <=4G.
> > 
> > Split the memory space into two memory banks: "ram0" and "ram1" to fix
> > this issue.
> > 
> > Signed-off-by: Marco Felsch <m.felsch@pengutronix.de>
> > ---
> > v2:
> > - drop add_mem() usage and use arm_add_mem_device() directly
> > 
> >  arch/arm/mach-imx/esdctl.c | 23 +++++++++++++++++++----
> >  1 file changed, 19 insertions(+), 4 deletions(-)
> > 
> > diff --git a/arch/arm/mach-imx/esdctl.c b/arch/arm/mach-imx/esdctl.c
> > index 1798ad48e50a..7f313b2337ae 100644
> > --- a/arch/arm/mach-imx/esdctl.c
> > +++ b/arch/arm/mach-imx/esdctl.c
> > @@ -510,16 +510,31 @@ static resource_size_t imx8m_ddrc_sdram_size(void __iomem *ddrc, unsigned buswid
> >  				   reduced_adress_space, mstr);
> >  }
> >  
> > +static int _imx8m_ddrc_add_mem(void *mmdcbase, struct imx_esdctl_data *data,
> > +			       unsigned int buswidth)
> > +{
> > +	resource_size_t size = imx8m_ddrc_sdram_size(mmdcbase, buswidth);
> > +	resource_size_t size0, size1;
> > +	int ret;
> > +
> > +	size0 = min_t(resource_size_t, SZ_4G - MX8M_DDR_CSD1_BASE_ADDR, size);
> > +	size1 = size - size0;
> > +
> > +	ret = arm_add_mem_device("ram0", data->base0, size0);
> > +	if (ret || size1 == 0)
> > +		return ret;
> > +
> > +	return arm_add_mem_device("ram1", SZ_4G, size1);
> 
> This gives a warning when compiling the file for 32-bit, because resource_size_t
> is 32-bit there, while SZ_4G is always an unsigned long long.

Thanks for this info. Size >= 4G on 32-bit systems is useless anyways. I
will guard this by an #ifdef CONFIG_64BIT and add a comment.

Regards,
  Marco

> 
> > +}
> > +
> >  static int imx8m_ddrc_add_mem(void *mmdcbase, struct imx_esdctl_data *data)
> >  {
> > -	return arm_add_mem_device("ram0", data->base0,
> > -			   imx8m_ddrc_sdram_size(mmdcbase, 32));
> > +	return _imx8m_ddrc_add_mem(mmdcbase, data, 32);
> >  }
> >  
> >  static int imx8mn_ddrc_add_mem(void *mmdcbase, struct imx_esdctl_data *data)
> >  {
> > -	return arm_add_mem_device("ram0", data->base0,
> > -			   imx8m_ddrc_sdram_size(mmdcbase, 16));
> > +	return _imx8m_ddrc_add_mem(mmdcbase, data, 16);
> >  }
> >  
> >  static resource_size_t imx7d_ddrc_sdram_size(void __iomem *ddrc)
> 
> -- 
> Pengutronix e.K.                           |                             |
> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
> 
>