RK3568 fail with 2023.11.0

RK3568 fail with 2023.11.0

[Alexander Shiyan]

Hello All.

The latest version of the barebox does not work on a RK3568 with 4G memory.
With a similar board with 2G memory, everything works fine.
...
NOTICE:  BL31: v2.3():v2.3-607-gbf602aff1:cl
NOTICE:  BL31: Built : 10:16:03, Jun  5 2023
INFO:    GICv3 without legacy support detected.
INFO:    ARM GICv3 driver initialized in EL3
INFO:    pmu v1 is valid 220114
INFO:    Using opteed sec cpu_context!
INFO:    boot cpu mask: 0
INFO:    BL31: Initializing runtime services
WARNING: No OPTEE provided by BL2 boot loader, Booting device without
OPTEE initialization. SMC`s destined for OPTEE will return SMC_UNK
ERROR:   Error initializing runtime service opteed_fast
INFO:    BL31: Preparing for EL3 exit to normal world
INFO:    Entry point address = 0xb00000
INFO:    SPSR = 0x3c9
>rockchip-dmc: rockchip_sdram_size(reg2=1000eaf1, reg3=30000001)
rockchip-dmc: rank 2 cs0_col 10 cs1_col 10 bk 3 cs0_row 16 cs1_row 16
bw 2 row_3_4 0
rockchip-dmc: rk3568_ram0_size() = 4026531840
uncompress.c: memory at 0x00a00000, size 0xef600000
mmu: enabling MMU, ttb @ 0xeffe0000
===STALL HERE===

Through various manipulations, I identified the faulty patch: "ARM:
mmu64: mark barebox text section executable during early init".
After its rollback everything works as it should.

Re: RK3568 fail with 2023.11.0

[Ahmad Fatoum]

On 30.11.23 15:28, Alexander Shiyan wrote:
> Hello All.
> 
> The latest version of the barebox does not work on a RK3568 with 4G memory.
> With a similar board with 2G memory, everything works fine.
> ...
> NOTICE:  BL31: v2.3():v2.3-607-gbf602aff1:cl
> NOTICE:  BL31: Built : 10:16:03, Jun  5 2023
> INFO:    GICv3 without legacy support detected.
> INFO:    ARM GICv3 driver initialized in EL3
> INFO:    pmu v1 is valid 220114
> INFO:    Using opteed sec cpu_context!
> INFO:    boot cpu mask: 0
> INFO:    BL31: Initializing runtime services
> WARNING: No OPTEE provided by BL2 boot loader, Booting device without
> OPTEE initialization. SMC`s destined for OPTEE will return SMC_UNK
> ERROR:   Error initializing runtime service opteed_fast
> INFO:    BL31: Preparing for EL3 exit to normal world
> INFO:    Entry point address = 0xb00000
> INFO:    SPSR = 0x3c9
>> rockchip-dmc: rockchip_sdram_size(reg2=1000eaf1, reg3=30000001)
> rockchip-dmc: rank 2 cs0_col 10 cs1_col 10 bk 3 cs0_row 16 cs1_row 16
> bw 2 row_3_4 0
> rockchip-dmc: rk3568_ram0_size() = 4026531840
> uncompress.c: memory at 0x00a00000, size 0xef600000
> mmu: enabling MMU, ttb @ 0xeffe0000
> ===STALL HERE===
> 
> Through various manipulations, I identified the faulty patch: "ARM:
> mmu64: mark barebox text section executable during early init".
> After its rollback everything works as it should.

Can you print the output of the iomem command after reverting my patch?

Thanks,
Ahmad

> 
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: RK3568 fail with 2023.11.0

[Alexander Shiyan]

barebox@Diasom DS-RK3568-EVB:/ iomem
0x0000000000000000 - 0xffffffffffffffff (size 0x0000000000000000) iomem
  0x000000000010f000 - 0x000000000010f0ff (size 0x0000000000000100)
10f000.sram@10f000.of
  0x0000000000a00000 - 0x00000000efffffff (size 0x00000000ef600000) ram0
    0x00000000afce0e00 - 0x00000000efce0dff (size 0x0000000040000000)
malloc space
    0x00000000efce0e00 - 0x00000000efcffff5 (size 0x000000000001f1f6) board data
    0x00000000efd00000 - 0x00000000efdad1a7 (size 0x00000000000ad1a8) barebox
    0x00000000efdad1a8 - 0x00000000efdff13f (size 0x0000000000051f98)
barebox data
    0x00000000efdff140 - 0x00000000efe0608f (size 0x0000000000006f50) bss
    0x00000000effe0000 - 0x00000000effeffff (size 0x0000000000010000) ttb
    0x00000000efff0000 - 0x00000000efff7fff (size 0x0000000000008000) stack
  0x00000000fcc00000 - 0x00000000fcffffff (size 0x0000000000400000) xHCI0
  0x00000000fd000000 - 0x00000000fd3fffff (size 0x0000000000400000) xHCI1
  0x00000000fdd40000 - 0x00000000fdd40fff (size 0x0000000000001000)
fdd40000.i2c@fdd40000.of
  0x00000000fdd60000 - 0x00000000fdd600ff (size 0x0000000000000100)
fdd60000.gpio@fdd60000.of
  0x00000000fe010000 - 0x00000000fe01ffff (size 0x0000000000010000)
fe010000.ethernet@fe010000.of
  0x00000000fe2a0000 - 0x00000000fe2affff (size 0x0000000000010000)
fe2a0000.ethernet@fe2a0000.of
  0x00000000fe2b0000 - 0x00000000fe2b3fff (size 0x0000000000004000)
fe2b0000.mmc@fe2b0000.of
  0x00000000fe310000 - 0x00000000fe31ffff (size 0x0000000000010000)
fe310000.mmc@fe310000.of
  0x00000000fe38c000 - 0x00000000fe38ffff (size 0x0000000000004000)
fe38c000.nvmem@fe38c000.of
  0x00000000fe5a0000 - 0x00000000fe5a0fff (size 0x0000000000001000)
fe5a0000.i2c@fe5a0000.of
  0x00000000fe5d0000 - 0x00000000fe5d0fff (size 0x0000000000001000)
fe5d0000.i2c@fe5d0000.of
  0x00000000fe600000 - 0x00000000fe6000ff (size 0x0000000000000100)
fe600000.watchdog@fe600000.of
  0x00000000fe660000 - 0x00000000fe6600ff (size 0x0000000000000100)
fe660000.serial@fe660000.of
  0x00000000fe670000 - 0x00000000fe6700ff (size 0x0000000000000100)
fe670000.serial@fe670000.of
  0x00000000fe6b0000 - 0x00000000fe6b00ff (size 0x0000000000000100)
fe6b0000.serial@fe6b0000.of
  0x00000000fe6c0000 - 0x00000000fe6c00ff (size 0x0000000000000100)
fe6c0000.serial@fe6c0000.of
  0x00000000fe6d0000 - 0x00000000fe6d00ff (size 0x0000000000000100)
fe6d0000.serial@fe6d0000.of
  0x00000000fe720000 - 0x00000000fe7200ff (size 0x0000000000000100)
fe720000.saradc@fe720000.of
  0x00000000fe740000 - 0x00000000fe7400ff (size 0x0000000000000100)
fe740000.gpio@fe740000.of
  0x00000000fe750000 - 0x00000000fe7500ff (size 0x0000000000000100)
fe750000.gpio@fe750000.of
  0x00000000fe760000 - 0x00000000fe7600ff (size 0x0000000000000100)
fe760000.gpio@fe760000.of
  0x00000000fe770000 - 0x00000000fe7700ff (size 0x0000000000000100)
fe770000.gpio@fe770000.of
  0x00000000fe820000 - 0x00000000fe8200ff (size 0x0000000000000100)
fe820000.phy@fe820000.of
  0x00000000fe830000 - 0x00000000fe8300ff (size 0x0000000000000100)
fe830000.phy@fe830000.of
  0x00000000fe840000 - 0x00000000fe8400ff (size 0x0000000000000100)
fe840000.phy@fe840000.of
  0x00000000fe8a0000 - 0x00000000fe8affff (size 0x0000000000010000)
fe8a0000.usb2phy@fe8a0000.of

чт, 30 нояб. 2023 г. в 17:59, Ahmad Fatoum <a.fatoum@pengutronix.de>:
>
> On 30.11.23 15:28, Alexander Shiyan wrote:
> > Hello All.
> >
> > The latest version of the barebox does not work on a RK3568 with 4G memory.
> > With a similar board with 2G memory, everything works fine.
> > ...
> > NOTICE:  BL31: v2.3():v2.3-607-gbf602aff1:cl
> > NOTICE:  BL31: Built : 10:16:03, Jun  5 2023
> > INFO:    GICv3 without legacy support detected.
> > INFO:    ARM GICv3 driver initialized in EL3
> > INFO:    pmu v1 is valid 220114
> > INFO:    Using opteed sec cpu_context!
> > INFO:    boot cpu mask: 0
> > INFO:    BL31: Initializing runtime services
> > WARNING: No OPTEE provided by BL2 boot loader, Booting device without
> > OPTEE initialization. SMC`s destined for OPTEE will return SMC_UNK
> > ERROR:   Error initializing runtime service opteed_fast
> > INFO:    BL31: Preparing for EL3 exit to normal world
> > INFO:    Entry point address = 0xb00000
> > INFO:    SPSR = 0x3c9
> >> rockchip-dmc: rockchip_sdram_size(reg2=1000eaf1, reg3=30000001)
> > rockchip-dmc: rank 2 cs0_col 10 cs1_col 10 bk 3 cs0_row 16 cs1_row 16
> > bw 2 row_3_4 0
> > rockchip-dmc: rk3568_ram0_size() = 4026531840
> > uncompress.c: memory at 0x00a00000, size 0xef600000
> > mmu: enabling MMU, ttb @ 0xeffe0000
> > ===STALL HERE===
> >
> > Through various manipulations, I identified the faulty patch: "ARM:
> > mmu64: mark barebox text section executable during early init".
> > After its rollback everything works as it should.
>
> Can you print the output of the iomem command after reverting my patch?
>
> Thanks,
> Ahmad
>
> >
> >
>
> --
> Pengutronix e.K.                           |                             |
> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
>

Re: RK3568 fail with 2023.11.0

[Alexander Shiyan]

Hello all.

Update:
In both of the problems I described for the RK3568 with 4 GB memory,
the dmc driver is probably not working correctly.

rockchip-dmc: rk3568_ram0_size() = 4026531840
Here is 0xf0000000, i.e. 4G minus 256M.
Next, the barebox writes that 0x0000000100000000 memory is detected.
In what direction should I look for the problem?

Thanks!

чт, 30 нояб. 2023 г. в 18:07, Alexander Shiyan <eagle.alexander923@gmail.com>:
>
> barebox@Diasom DS-RK3568-EVB:/ iomem
> 0x0000000000000000 - 0xffffffffffffffff (size 0x0000000000000000) iomem
>   0x000000000010f000 - 0x000000000010f0ff (size 0x0000000000000100)
> 10f000.sram@10f000.of
>   0x0000000000a00000 - 0x00000000efffffff (size 0x00000000ef600000) ram0
>     0x00000000afce0e00 - 0x00000000efce0dff (size 0x0000000040000000)
> malloc space
>     0x00000000efce0e00 - 0x00000000efcffff5 (size 0x000000000001f1f6) board data
>     0x00000000efd00000 - 0x00000000efdad1a7 (size 0x00000000000ad1a8) barebox
>     0x00000000efdad1a8 - 0x00000000efdff13f (size 0x0000000000051f98)
> barebox data
>     0x00000000efdff140 - 0x00000000efe0608f (size 0x0000000000006f50) bss
>     0x00000000effe0000 - 0x00000000effeffff (size 0x0000000000010000) ttb
>     0x00000000efff0000 - 0x00000000efff7fff (size 0x0000000000008000) stack
>   0x00000000fcc00000 - 0x00000000fcffffff (size 0x0000000000400000) xHCI0
>   0x00000000fd000000 - 0x00000000fd3fffff (size 0x0000000000400000) xHCI1
>   0x00000000fdd40000 - 0x00000000fdd40fff (size 0x0000000000001000)
> fdd40000.i2c@fdd40000.of
>   0x00000000fdd60000 - 0x00000000fdd600ff (size 0x0000000000000100)
> fdd60000.gpio@fdd60000.of
>   0x00000000fe010000 - 0x00000000fe01ffff (size 0x0000000000010000)
> fe010000.ethernet@fe010000.of
>   0x00000000fe2a0000 - 0x00000000fe2affff (size 0x0000000000010000)
> fe2a0000.ethernet@fe2a0000.of
>   0x00000000fe2b0000 - 0x00000000fe2b3fff (size 0x0000000000004000)
> fe2b0000.mmc@fe2b0000.of
>   0x00000000fe310000 - 0x00000000fe31ffff (size 0x0000000000010000)
> fe310000.mmc@fe310000.of
>   0x00000000fe38c000 - 0x00000000fe38ffff (size 0x0000000000004000)
> fe38c000.nvmem@fe38c000.of
>   0x00000000fe5a0000 - 0x00000000fe5a0fff (size 0x0000000000001000)
> fe5a0000.i2c@fe5a0000.of
>   0x00000000fe5d0000 - 0x00000000fe5d0fff (size 0x0000000000001000)
> fe5d0000.i2c@fe5d0000.of
>   0x00000000fe600000 - 0x00000000fe6000ff (size 0x0000000000000100)
> fe600000.watchdog@fe600000.of
>   0x00000000fe660000 - 0x00000000fe6600ff (size 0x0000000000000100)
> fe660000.serial@fe660000.of
>   0x00000000fe670000 - 0x00000000fe6700ff (size 0x0000000000000100)
> fe670000.serial@fe670000.of
>   0x00000000fe6b0000 - 0x00000000fe6b00ff (size 0x0000000000000100)
> fe6b0000.serial@fe6b0000.of
>   0x00000000fe6c0000 - 0x00000000fe6c00ff (size 0x0000000000000100)
> fe6c0000.serial@fe6c0000.of
>   0x00000000fe6d0000 - 0x00000000fe6d00ff (size 0x0000000000000100)
> fe6d0000.serial@fe6d0000.of
>   0x00000000fe720000 - 0x00000000fe7200ff (size 0x0000000000000100)
> fe720000.saradc@fe720000.of
>   0x00000000fe740000 - 0x00000000fe7400ff (size 0x0000000000000100)
> fe740000.gpio@fe740000.of
>   0x00000000fe750000 - 0x00000000fe7500ff (size 0x0000000000000100)
> fe750000.gpio@fe750000.of
>   0x00000000fe760000 - 0x00000000fe7600ff (size 0x0000000000000100)
> fe760000.gpio@fe760000.of
>   0x00000000fe770000 - 0x00000000fe7700ff (size 0x0000000000000100)
> fe770000.gpio@fe770000.of
>   0x00000000fe820000 - 0x00000000fe8200ff (size 0x0000000000000100)
> fe820000.phy@fe820000.of
>   0x00000000fe830000 - 0x00000000fe8300ff (size 0x0000000000000100)
> fe830000.phy@fe830000.of
>   0x00000000fe840000 - 0x00000000fe8400ff (size 0x0000000000000100)
> fe840000.phy@fe840000.of
>   0x00000000fe8a0000 - 0x00000000fe8affff (size 0x0000000000010000)
> fe8a0000.usb2phy@fe8a0000.of
>
> чт, 30 нояб. 2023 г. в 17:59, Ahmad Fatoum <a.fatoum@pengutronix.de>:
> >
> > On 30.11.23 15:28, Alexander Shiyan wrote:
> > > Hello All.
> > >
> > > The latest version of the barebox does not work on a RK3568 with 4G memory.
> > > With a similar board with 2G memory, everything works fine.
> > > ...
> > > NOTICE:  BL31: v2.3():v2.3-607-gbf602aff1:cl
> > > NOTICE:  BL31: Built : 10:16:03, Jun  5 2023
> > > INFO:    GICv3 without legacy support detected.
> > > INFO:    ARM GICv3 driver initialized in EL3
> > > INFO:    pmu v1 is valid 220114
> > > INFO:    Using opteed sec cpu_context!
> > > INFO:    boot cpu mask: 0
> > > INFO:    BL31: Initializing runtime services
> > > WARNING: No OPTEE provided by BL2 boot loader, Booting device without
> > > OPTEE initialization. SMC`s destined for OPTEE will return SMC_UNK
> > > ERROR:   Error initializing runtime service opteed_fast
> > > INFO:    BL31: Preparing for EL3 exit to normal world
> > > INFO:    Entry point address = 0xb00000
> > > INFO:    SPSR = 0x3c9
> > >> rockchip-dmc: rockchip_sdram_size(reg2=1000eaf1, reg3=30000001)
> > > rockchip-dmc: rank 2 cs0_col 10 cs1_col 10 bk 3 cs0_row 16 cs1_row 16
> > > bw 2 row_3_4 0
> > > rockchip-dmc: rk3568_ram0_size() = 4026531840
> > > uncompress.c: memory at 0x00a00000, size 0xef600000
> > > mmu: enabling MMU, ttb @ 0xeffe0000
> > > ===STALL HERE===
> > >
> > > Through various manipulations, I identified the faulty patch: "ARM:
> > > mmu64: mark barebox text section executable during early init".
> > > After its rollback everything works as it should.
> >
> > Can you print the output of the iomem command after reverting my patch?
> >
> > Thanks,
> > Ahmad
> >
> > >
> > >
> >
> > --
> > Pengutronix e.K.                           |                             |
> > Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> > 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> > Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
> >

Re: RK3568 fail with 2023.11.0

[Sascha Hauer]

On Thu, Nov 30, 2023 at 05:28:44PM +0300, Alexander Shiyan wrote:
> Hello All.
> 
> The latest version of the barebox does not work on a RK3568 with 4G memory.
> With a similar board with 2G memory, everything works fine.
> ...
> NOTICE:  BL31: v2.3():v2.3-607-gbf602aff1:cl
> NOTICE:  BL31: Built : 10:16:03, Jun  5 2023
> INFO:    GICv3 without legacy support detected.
> INFO:    ARM GICv3 driver initialized in EL3
> INFO:    pmu v1 is valid 220114
> INFO:    Using opteed sec cpu_context!
> INFO:    boot cpu mask: 0
> INFO:    BL31: Initializing runtime services
> WARNING: No OPTEE provided by BL2 boot loader, Booting device without
> OPTEE initialization. SMC`s destined for OPTEE will return SMC_UNK
> ERROR:   Error initializing runtime service opteed_fast
> INFO:    BL31: Preparing for EL3 exit to normal world
> INFO:    Entry point address = 0xb00000
> INFO:    SPSR = 0x3c9
> >rockchip-dmc: rockchip_sdram_size(reg2=1000eaf1, reg3=30000001)
> rockchip-dmc: rank 2 cs0_col 10 cs1_col 10 bk 3 cs0_row 16 cs1_row 16
> bw 2 row_3_4 0
> rockchip-dmc: rk3568_ram0_size() = 4026531840
> uncompress.c: memory at 0x00a00000, size 0xef600000
> mmu: enabling MMU, ttb @ 0xeffe0000
> ===STALL HERE===
> 
> Through various manipulations, I identified the faulty patch: "ARM:
> mmu64: mark barebox text section executable during early init".

This patch just triggers a bug elsewhere. alloc_pte() places PTEs with
offset 0x8000 between them instead of the expected 0x1000. This means we
leave the space reserved for PTEs. In case of the RK3568 with 8GiB of
memory the PTEs go into the internal register space.

I've sent a fix to the list, you're on Cc:.

Sascha

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: RK3568 fail with 2023.11.0

[Alexander Shiyan]

Hello Sascha.

So this bug can probably affect CPUs other than RK3568, right?
After this series of patches, both problems described earlier went away.
Can you apply this to the master branch please?

Thanks!

пт, 1 дек. 2023 г. в 18:14, Sascha Hauer <sha@pengutronix.de>:
>
> On Thu, Nov 30, 2023 at 05:28:44PM +0300, Alexander Shiyan wrote:
> > Hello All.
> >
> > The latest version of the barebox does not work on a RK3568 with 4G memory.
> > With a similar board with 2G memory, everything works fine.
> > ...
> > NOTICE:  BL31: v2.3():v2.3-607-gbf602aff1:cl
> > NOTICE:  BL31: Built : 10:16:03, Jun  5 2023
> > INFO:    GICv3 without legacy support detected.
> > INFO:    ARM GICv3 driver initialized in EL3
> > INFO:    pmu v1 is valid 220114
> > INFO:    Using opteed sec cpu_context!
> > INFO:    boot cpu mask: 0
> > INFO:    BL31: Initializing runtime services
> > WARNING: No OPTEE provided by BL2 boot loader, Booting device without
> > OPTEE initialization. SMC`s destined for OPTEE will return SMC_UNK
> > ERROR:   Error initializing runtime service opteed_fast
> > INFO:    BL31: Preparing for EL3 exit to normal world
> > INFO:    Entry point address = 0xb00000
> > INFO:    SPSR = 0x3c9
> > >rockchip-dmc: rockchip_sdram_size(reg2=1000eaf1, reg3=30000001)
> > rockchip-dmc: rank 2 cs0_col 10 cs1_col 10 bk 3 cs0_row 16 cs1_row 16
> > bw 2 row_3_4 0
> > rockchip-dmc: rk3568_ram0_size() = 4026531840
> > uncompress.c: memory at 0x00a00000, size 0xef600000
> > mmu: enabling MMU, ttb @ 0xeffe0000
> > ===STALL HERE===
> >
> > Through various manipulations, I identified the faulty patch: "ARM:
> > mmu64: mark barebox text section executable during early init".
>
> This patch just triggers a bug elsewhere. alloc_pte() places PTEs with
> offset 0x8000 between them instead of the expected 0x1000. This means we
> leave the space reserved for PTEs. In case of the RK3568 with 8GiB of
> memory the PTEs go into the internal register space.
>
> I've sent a fix to the list, you're on Cc:.
>
> Sascha
>
> --
> Pengutronix e.K.                           |                             |
> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: RK3568 fail with 2023.11.0

[Sascha Hauer]

Hi Alexander,

On Sun, Dec 03, 2023 at 08:41:23PM +0300, Alexander Shiyan wrote:
> Hello Sascha.
> 
> So this bug can probably affect CPUs other than RK3568, right?

Yes.

> After this series of patches, both problems described earlier went away.
> Can you apply this to the master branch please?

Did that.

Sascha

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |