RE: [PATCH v2] Porting barebox to a new SoC

[Lior Weintraub <liorw@pliops.com>]

Hello Ahmad,

I failed to reproduce this issue on virt because the addresses and peripherals on virt machine are different and it is difficult to change our code to match that.
If you think this is critical I will make extra effort to make it work.
AFAIU, this suggestion was made to debug the "conflict" issue.
Currently the workaround I am using is just to set the size of the kernel partition to match the exact size of the "Image" file.

The other issue I am facing is that Kernel seems stuck on cpu_do_idle and there is no login prompt from the kernel.
As you recall, I am running on a custom QEMU that tries to emulate our platform.
I suspect that I did something wrong with the GICv3 and Timers connectivity.
The code I used was based on examples I saw on sbsa-ref.c and virt.c.
In addition, I declared the GICv3 and timers on our device tree.

I running QEMU with "-d int" so I am also getting trace of exceptions and interrupts.
The kernel log until it is getting stuck is:
Loaded initrd unknown '/dev/ram0.rootfs'
initrd is at 0x0000000008cb0000-0x00000000094affff
Loading devicetree from '/dev/ram0.oftree'
__request_region ok: 0x094b0000:0x094b15e4 flags=0x200
commandline: <NULL>
Loaded kernel to 0x08000000, devicetree at 0x094b0000
exitcall-> nv_exit+0x0/0x30
exitcall-> devices_shutdown+0x0/0xbc
    remove-> machine
    remove-> mem1
    remove-> mem0
    remove-> pstore0
    remove-> devfs0
    remove-> ramfs0
    remove-> d000307000.serial@d000307000.of
    remove-> soc:timer.of
exitcall-> arch_shutdown+0x0/0x14
Exception return from AArch64 EL2 to AArch64 EL1 PC 0x893e098
Taking exception 11 [Hypervisor Call] on CPU 0
...from EL1 to EL2
...with ESR 0x16/0x5a000000
...with ELR 0xffffffc00801c988
...to EL2 PC 0x875d400 PSTATE 0x3c9
Exception return from AArch64 EL2 to AArch64 EL1 PC 0xffffffc00801c988
Taking exception 13 [Secure Monitor Call] on CPU 0
...from EL1 to EL3
...with ESR 0x17/0x5e000000
...with ELR 0xffffffc00802112c
...to EL3 PC 0x10005400 PSTATE 0x3cd
Exception return from AArch64 EL3 to AArch64 EL1 PC 0xffffffc00802112c
Taking exception 13 [Secure Monitor Call] on CPU 0
...from EL1 to EL3
...with ESR 0x17/0x5e000000
...with ELR 0xffffffc00802112c
...to EL3 PC 0x10005400 PSTATE 0x3cd
Exception return from AArch64 EL3 to AArch64 EL1 PC 0xffffffc00802112c
Taking exception 13 [Secure Monitor Call] on CPU 0
...from EL1 to EL3
...with ESR 0x17/0x5e000000
...with ELR 0xffffffc00802112c
...to EL3 PC 0x10005400 PSTATE 0x3cd
Exception return from AArch64 EL3 to AArch64 EL1 PC 0xffffffc00802112c
Taking exception 13 [Secure Monitor Call] on CPU 0
...from EL1 to EL3
...with ESR 0x17/0x5e000000
...with ELR 0xffffffc00802112c
...to EL3 PC 0x10005400 PSTATE 0x3cd
Exception return from AArch64 EL3 to AArch64 EL1 PC 0xffffffc00802112c
Taking exception 13 [Secure Monitor Call] on CPU 0
...from EL1 to EL3
...with ESR 0x17/0x5e000000
...with ELR 0xffffffc00897a1ac
...to EL3 PC 0x10005400 PSTATE 0x3cd
Exception return from AArch64 EL3 to AArch64 EL1 PC 0xffffffc00897a1ac
Taking exception 13 [Secure Monitor Call] on CPU 0
...from EL1 to EL3
...with ESR 0x17/0x5e000000
...with ELR 0xffffffc00897a230
...to EL3 PC 0x10005400 PSTATE 0x3cd
Exception return from AArch64 EL3 to AArch64 EL1 PC 0xffffffc00897a230
Taking exception 13 [Secure Monitor Call] on CPU 0
...from EL1 to EL3
...with ESR 0x17/0x5e000000
...with ELR 0xffffffc00802112c
...to EL3 PC 0x10005400 PSTATE 0x3cd
Exception return from AArch64 EL3 to AArch64 EL1 PC 0xffffffc00802112c
Taking exception 13 [Secure Monitor Call] on CPU 0
...from EL1 to EL3
...with ESR 0x17/0x5e000000
...with ELR 0xffffffc00802112c
...to EL3 PC 0x10005400 PSTATE 0x3cd
Exception return from AArch64 EL3 to AArch64 EL1 PC 0xffffffc00802112c
Taking exception 13 [Secure Monitor Call] on CPU 0
...from EL1 to EL3
...with ESR 0x17/0x5e000000
...with ELR 0xffffffc00802112c
...to EL3 PC 0x10005400 PSTATE 0x3cd
Exception return from AArch64 EL3 to AArch64 EL1 PC 0xffffffc00802112c
Taking exception 13 [Secure Monitor Call] on CPU 0
...from EL1 to EL3
...with ESR 0x17/0x5e000000
...with ELR 0xffffffc008021640
...to EL3 PC 0x10005400 PSTATE 0x3cd
Exception return from AArch64 EL3 to AArch64 EL1 PC 0xffffffc008021640
Pliops Kernel
Booting Linux on physical CPU 0x0000000000 [0x410fd034]
Linux version 6.4.0-rc7 (pliops@dev-liorw) (aarch64-buildroot-linux-gnu-gcc.br_real (Buildroot 2023.02.1-95-g8391404e23) 11.3.0, GNU ld (GNU Binutils) 2.38) #11 SMP Sun Jun 25 22:03:04 IDT 2023
Machine model: Pliops Spider MK-I EVK
efi: UEFI not found.
Zone ranges:
  DMA      [mem 0x0000000000000000-0x000000002fffffff]
  DMA32    empty
  Normal   empty
Movable zone start for each node
Early memory node ranges
  node   0: [mem 0x0000000000000000-0x000000002fffffff]
Initmem setup node 0 [mem 0x0000000000000000-0x000000002fffffff]
psci: probing for conduit method from DT.
psci: PSCIv1.1 detected in firmware.
psci: Using standard PSCI v0.2 function IDs
psci: MIGRATE_INFO_TYPE not supported.
psci: SMC Calling Convention v1.2
percpu: Embedded 26 pages/cpu s67752 r8192 d30552 u106496
Detected VIPT I-cache on CPU0
CPU features: detected: GIC system register CPU interface
CPU features: detected: ARM erratum 843419
CPU features: detected: ARM erratum 845719
alternatives: applying boot alternatives
Kernel command line: cpuidle.off=1
Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
Inode-cache hash table entries: 65536 (order: 7, 524288 bytes, linear)
Built 1 zonelists, mobility grouping on.  Total pages: 193536
mem auto-init: stack:off, heap alloc:off, heap free:off
Memory: 749712K/786432K available (7488K kernel code, 1300K rwdata, 1892K rodata, 1664K init, 427K bss, 36720K reserved, 0K cma-reserved)
SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
trace event string verifier disabled
rcu: Hierarchical RCU implementation.
rcu:    RCU event tracing is enabled.
rcu:    RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=1.
rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
GICv3: GIC: Using split EOI/Deactivate mode
GICv3: 64 SPIs implemented
GICv3: 0 Extended SPIs implemented
Root IRQ handler: gic_handle_irq
GICv3: GICv3 features: 16 PPIs
GICv3: CPU0: found redistributor 0 region 0:0x000000e000060000
ITS [mem 0xe000040000-0xe00005ffff]
ITS@0x000000e000040000: allocated 8192 Devices @a0000 (indirect, esz 8, psz 64K, shr 1)
ITS@0x000000e000040000: allocated 8192 Interrupt Collections @b0000 (flat, esz 8, psz 64K, shr 1)
GICv3: using LPI property table @0x00000000000c0000
GICv3: CPU0: using allocated LPI pending table @0x00000000000d0000
rcu: srcu_init: Setting srcu_struct sizes based on contention.
arch_timer: cp15 timer(s) running at 62.50MHz (phys).
clocksource: arch_sys_counter: mask: 0x1ffffffffffffff max_cycles: 0x1cd42e208c, max_idle_ns: 881590405314 ns
sched_clock: 57 bits at 63MHz, resolution 16ns, wraps every 4398046511096ns
Console: colour dummy device 80x25
printk: console [tty0] enabled
Calibrating delay loop (skipped), value calculated using timer frequency.. 125.00 BogoMIPS (lpj=250000)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 2048 (order: 2, 16384 bytes, linear)
Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes, linear)
rcu: Hierarchical SRCU implementation.
rcu:    Max phase no-delay instances is 1000.
Platform MSI: gic-its@E000040000 domain created
PCI/MSI: /soc/interrupt-controller@E000000000/gic-its@E000040000 domain created
EFI services will not be available.
smp: Bringing up secondary CPUs ...
smp: Brought up 1 node, 1 CPU
SMP: Total of 1 processors activated.
CPU features: detected: 32-bit EL0 Support
CPU features: detected: CRC32 instructions
CPU: All CPU(s) started at EL2
alternatives: applying system-wide alternatives
devtmpfs: initialized
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
futex hash table entries: 256 (order: 2, 16384 bytes, linear)
DMI not present or invalid.
NET: Registered PF_NETLINK/PF_ROUTE protocol family
DMA: preallocated 128 KiB GFP_KERNEL pool for atomic allocations
DMA: preallocated 128 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
DMA: preallocated 128 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
ASID allocator initialised with 65536 entries
Serial: AMBA PL011 UART driver
d000307000.serial: ttyAMA0 at MMIO 0xd000307000 (irq = 14, base_baud = 0) is a PL011 rev1
printk: console [ttyAMA0] enabled
iommu: Default domain type: Translated 
iommu: DMA domain TLB invalidation policy: strict mode 
SCSI subsystem initialized
pps_core: LinuxPPS API ver. 1 registered
pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
PTP clock support registered
mctp: management component transport protocol core
NET: Registered PF_MCTP protocol family
vgaarb: loaded
clocksource: Switched to clocksource arch_sys_counter
NET: Registered PF_INET protocol family
IP idents hash table entries: 16384 (order: 5, 131072 bytes, linear)
tcp_listen_portaddr_hash hash table entries: 512 (order: 1, 8192 bytes, linear)
Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
TCP established hash table entries: 8192 (order: 4, 65536 bytes, linear)
TCP bind hash table entries: 8192 (order: 6, 262144 bytes, linear)
TCP: Hash tables configured (established 8192 bind 8192)
UDP hash table entries: 512 (order: 2, 16384 bytes, linear)
UDP-Lite hash table entries: 512 (order: 2, 16384 bytes, linear)
NET: Registered PF_UNIX/PF_LOCAL protocol family
PCI: CLS 0 bytes, default 64
workingset: timestamp_bits=46 max_order=18 bucket_order=0
fuse: init (API version 7.38)
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 249)
io scheduler mq-deadline registered
io scheduler kyber registered
Unpacking initramfs...
Freeing initrd memory: 8192K
SMCCC: SOC_ID: ARCH_SOC_ID not implemented, skipping ....
hw perfevents: enabled with armv8_cortex_a53 PMU driver, 7 counters available
NET: Registered PF_INET6 protocol family
Segment Routing with IPv6
In-situ OAM (IOAM) with IPv6
sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
NET: Registered PF_PACKET protocol family
NET: Registered PF_KEY protocol family
NET: Registered PF_VSOCK protocol family
registered taskstats version 1
clk: Disabling unused clocks
Freeing unused kernel memory: 1664K

Your advise is much appreciated,
Cheers,
Lior.


> -----Original Message-----
> From: Ahmad Fatoum <a.fatoum@pengutronix.de>
> Sent: Monday, June 19, 2023 6:23 PM
> To: Lior Weintraub <liorw@pliops.com>; Ahmad Fatoum <ahmad@a3f.at>;
> barebox@lists.infradead.org
> Subject: Re: [PATCH v2] Porting barebox to a new SoC
> 
> CAUTION: External Sender
> 
> On 19.06.23 08:40, Lior Weintraub wrote:
> > Hello Ahmad,
> >
> > Thanks again for your great tips.
> > Using the addpart command seems to work and give same results as the
> pre-partitioned flash node on the DT.
> > When I deleted the whole flash node and extended the memory node to
> include all 768MB the following commands worked on barebox:
> > addpart /dev/ram0
> "512M@0(dram512),0xB30000@0x20000000(kernel),0x8000@0x20FF800
> 0(oftree),0x800000@0x21000000(rootfs),-"
> > After that, checking the devinfo gave:
> >    `-- mem0
> >       `-- 0x00000000-0x2fffffff (   768 MiB): /dev/ram0
> >       `-- 0x00000000-0x1fffffff (   512 MiB): /dev/ram0.dram512
> >       `-- 0x20000000-0x20b2ffff (  11.2 MiB): /dev/ram0.kernel
> >       `-- 0x20ff8000-0x20ffffff (    32 KiB): /dev/ram0.oftree
> >       `-- 0x21000000-0x217fffff (     8 MiB): /dev/ram0.rootfs
> >       `-- 0x21800000-0x2fffffff (   232 MiB): /dev/
> > Now running the bootm:
> > bootm -o /dev/ram0.oftree -r /dev/ram0.rootfs -a 0 /dev/ram0.kernel
> >
> > The Linux kernel starts running but has issues (still probably missing parts on
> my QEMU machine).
> >
> > Notice that I had to set the "kernel" partition to size 0xB30000 because
> otherwise it gave me the same error of "conflict":
> > __request_region ok: 0x00000000:0x00ffffff flags=0x200
> > __request_region ok: 0x00000000:0x00ff7fff flags=0x200
> > __request_region: 0x00b30000:0x00b4ffff (image) conflicts with
> 0x00000000:0x00ff7fff (image)
> > unable to request SDRAM 0x00b30000-0x00b4ffff
> > handler failed with: Out of memory
> >
> > Could it be somehow related to the way I build the kernel?
> 
> Can you reproduce this with normal qemu aarch64 virt machine?
> 
> Cheers,
> Ahmad
> 
> > Cheers,
> > Lior.
> >
> >
> >> -----Original Message-----
> >> From: Ahmad Fatoum <a.fatoum@pengutronix.de>
> >> Sent: Friday, June 16, 2023 7:21 PM
> >> To: Lior Weintraub <liorw@pliops.com>; Ahmad Fatoum <ahmad@a3f.at>;
> >> barebox@lists.infradead.org
> >> Subject: Re: [PATCH v2] Porting barebox to a new SoC
> >>
> >> CAUTION: External Sender
> >>
> >> Hello Lior,
> >>
> >> On 14.06.23 08:42, Lior Weintraub wrote:
> >>> Hi Ahmad,
> >>>
> >>> Looks like I found the issue :-)
> >>>
> >>> When I declared the flash@0 node I used about 16MB for the kernel
> >> partition even though currently my kernel image is around 11MB:
> >>> flash@0 {
> >>>     compatible = "mtd-rom";
> >>>     probe-type = "map_rom";
> >>>     reg = <0x0 0x20000000 0x0 0x10000000>; /* Offset 512MB, size
> 256MB
> >> */
> >>>     bank-width = <4>;
> >>>     device-width = <1>;
> >>>
> >>>     #address-cells = <1>;
> >>>     #size-cells = <1>;
> >>>     kernel@0 { /* 16MB (minus 32KB) */
> >>>         label = "kernel";
> >>>         //reg = <0x0 0x00FF8000>; // This line caused the error
> >>>         reg =   <0x0 0x00b30000>;
> >>>     };
> >>>     oftree@00FF8000 { /* 32KB for DTB image */
> >>>         label = "oftree";
> >>>         reg = <0x00FF8000 0x00008000>;
> >>>     };
> >>>     rootfs@1000000 { /* Offset 512MB+16MB with size 8MB */
> >>>         label = "rootfs";
> >>>         reg = <0x01000000 0x00800000>;
> >>>     };
> >>> };
> >>>
> >>> From the error message I saw that barebox tried to place the rootfs on
> >> address 0xb30000 and that conflicted with the kernel:
> >>> mtdrom0.kernel: read ofs: 0x00fe0000 count: 0x00018000
> >>> __request_region ok: 0x00000000:0x00ff7fff flags=0x200
> >>> mtdrom0.rootfs: read ofs: 0x00000000 count: 0x00000800
> >>> __request_region: 0x00b30000:0x00b4ffff (image) conflicts with
> >> 0x00000000:0x00ff7fff (image)
> >>
> >> This error message still doesn't make sense to me. No idea why this
> conflicts.
> >> I see now though that I made this more complicated that it needs be. Please
> >> scratch all that, remove the mtd-rom and revert the /memory node to
> >> encompass
> >> all memory. Then you can at runtime, just create the partitions with the
> >> addpart
> >> command (see help addpart for info). Then you can boot normally.
> >>
> >>>
> >>> So I changed the kernel@0 node and used size 0x00b30000 and now
> kernel
> >> starts running.
> >>> The kernel is producing all kinds of errors but at least it's a start :-)
> >>> The errors are now related to pieces of HW that we didn't implement on
> >> QEMU so it is understandable.
> >>>
> >>> How can we control where the rootfs is copied to?
> >>
> >> Normally, global.bootm.initrd.loadaddr, but you shouldn't need to do
> >> that. Try it with addpart and let me know.
> >>
> >> Cheers,
> >> Ahmad
> >>
> >>>
> >>> Thanks again for your kind advise and patience.
> >>> Cheers,
> >>> Lior.
> >>>
> >>>> -----Original Message-----
> >>>> From: Lior Weintraub
> >>>> Sent: Tuesday, June 13, 2023 4:28 PM
> >>>> To: Ahmad Fatoum <a.fatoum@pengutronix.de>; Ahmad Fatoum
> >>>> <ahmad@a3f.at>; barebox@lists.infradead.org
> >>>> Subject: RE: [PATCH v2] Porting barebox to a new SoC
> >>>>
> >>>> Hi Ahmad,
> >>>>
> >>>> I also thought that load address was suspicious (0x8000000) but then I
> >> saw
> >>>> the following comment on nxp_imx8mq_evk_start:
> >>>>              /*
> >>>>               * On completion the TF-A will jump to
> >>>> MX8MQ_ATF_BL33_BASE_ADDR
> >>>>               * in EL2. Copy the image there, but replace the PBL part of
> >>>>               * that image with ourselves. On a high assurance boot only
> >>>> the
> >>>>               * currently running code is validated and contains the
> >>>> checksum
> >>>>               * for the piggy data, so we need to ensure that we are
> >>>> running
> >>>>               * the same code in DRAM.
> >>>>               */
> >>>> Our code was based on this function and as a result it copies the barebox
> >>>> image into address 0x8000000 (which is our settings of
> >>>> ATF_BL33_BASE_ADDR).
> >>>> The fact that Linux image is also copied there seemed reasonable (I might
> be
> >>>> wrong here) because once loading is done and we jump to Linux there is
> no
> >>>> need for barebox code anymore.
> >>>>
> >>>> In any case, at that time when I thought it was wrong I also tried to run
> the
> >>>> bootm command with "-a 0" but I got exactly the same error:
> >>>> barebox:/ bootm -o /dev/mtdrom0.oftree -r /dev/mtdrom0.rootfs -a 0
> >>>> /dev/mtdrom0.kernel
> >>>> getopt: optindex: 1 nonopts: 0 optind: 1
> >>>> getopt: optindex: 1 nonopts: 0 optind: 3
> >>>> getopt: optindex: 1 nonopts: 0 optind: 5
> >>>> getopt: optindex: 1 nonopts: 0 optind: 7
> >>>> mtdrom0.kernel: read ofs: 0x00000000 count: 0x00001000
> >>>>
> >>>> Loading ARM aarch64 Linux image '/dev/mtdrom0.kernel'
> >>>> header 00000000: 4d 5a 40 fa 3f 1a 23 14 00 00 00 00 00 00 00 00
> >>>> MZ@.?.#.........
> >>>> header 00000010: 00 00 b3 00 00 00 00 00 0a 00 00 00 00 00 00 00
> >>>> ................
> >>>> header 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >>>> ................
> >>>> header 00000030: 00 00 00 00 00 00 00 00 41 52 4d 64 40 00 00 00
> >>>> ........ARMd@...
> >>>> header 00000040: 50 45 00 00 64 aa 02 00 00 00 00 00 00 00 00 00
> >>>> PE..d...........
> >>>> booti: Kernel to be loaded to 0+b30000
> >>>> __request_region ok: 0x00000000:0x0001ffff flags=0x200
> >>>> mtdrom0.kernel: read ofs: 0x00000000 count: 0x00020000
> >>>> __request_region ok: 0x00000000:0x0003ffff flags=0x200
> >>>> mtdrom0.kernel: read ofs: 0x00020000 count: 0x00020000
> >>>> __request_region ok: 0x00000000:0x0005ffff flags=0x200
> >>>> .
> >>>> .
> >>>> mtdrom0.kernel: read ofs: 0x00fe0000 count: 0x00018000
> >>>> __request_region ok: 0x00000000:0x00ff7fff flags=0x200
> >>>> mtdrom0.rootfs: read ofs: 0x00000000 count: 0x00000800
> >>>> __request_region: 0x00b30000:0x00b4ffff (image) conflicts with
> >>>> 0x00000000:0x00ff7fff (image)
> >>>> unable to request SDRAM 0x00b30000-0x00b4ffff
> >>>> handler failed with: Out of memory
> >>>>
> >>>>
> >>>> The iomem before:
> >>>> barebox:/ iomem
> >>>> 0x0000000000000000 - 0xffffffffffffffff (size 0x0000000000000000)
> >> iomem
> >>>>   0x0000000000000000 - 0x000000001fffffff (size
> >> 0x0000000020000000)
> >>>> ram0
> >>>>     0x00000000079fef00 - 0x0000000007dfeeff (size
> >> 0x0000000000400000)
> >>>> malloc space
> >>>>     0x0000000007dfef00 - 0x0000000007dfffd1 (size
> >> 0x00000000000010d2)
> >>>> board data
> >>>>     0x0000000007e00000 - 0x0000000007e3d95f (size
> >>>> 0x000000000003d960) barebox
> >>>>     0x0000000007e3d960 - 0x0000000007e50bb7 (size
> >>>> 0x0000000000013258) barebox data
> >>>>     0x0000000007e50bb8 - 0x0000000007e53b3f (size
> >>>> 0x0000000000002f88) bss
> >>>>     0x0000000007ff0000 - 0x0000000007ff7fff (size
> >> 0x0000000000008000)
> >>>> stack
> >>>>   0x0000000020000000 - 0x000000002fffffff (size
> >> 0x0000000010000000)
> >>>> 20000000.flash@0.of
> >>>>   0x000000d000307000 - 0x000000d000307fff (size
> >>>> 0x0000000000001000) amba
> >>>>
> >>>> After:
> >>>> barebox:/ iomem
> >>>> 0x0000000000000000 - 0xffffffffffffffff (size 0x0000000000000000)
> >> iomem
> >>>>   0x0000000000000000 - 0x000000001fffffff (size
> >> 0x0000000020000000)
> >>>> ram0
> >>>>     0x00000000079fef00 - 0x0000000007dfeeff (size
> >> 0x0000000000400000)
> >>>> malloc space
> >>>>     0x0000000007dfef00 - 0x0000000007dfffd1 (size
> >> 0x00000000000010d2)
> >>>> board data
> >>>>     0x0000000007e00000 - 0x0000000007e3d95f (size
> >>>> 0x000000000003d960) barebox
> >>>>     0x0000000007e3d960 - 0x0000000007e50bb7 (size
> >>>> 0x0000000000013258) barebox data
> >>>>     0x0000000007e50bb8 - 0x0000000007e53b3f (size
> >>>> 0x0000000000002f88) bss
> >>>>     0x0000000007ff0000 - 0x0000000007ff7fff (size
> >> 0x0000000000008000)
> >>>> stack
> >>>>   0x0000000020000000 - 0x000000002fffffff (size
> >> 0x0000000010000000)
> >>>> 20000000.flash@0.of
> >>>>   0x000000d000307000 - 0x000000d000307fff (size
> >>>> 0x0000000000001000) amba
> >>>>
> >>>> I did not set global.bootm.image.loadaddr
> >>>> Speaking of env, the last few lines on the barebox terminal are:
> >>>> initcall-> load_environment+0x0/0x4c
> >>>> loading defaultenv
> >>>> environment load /dev/env0: No such file or directory
> >>>> Maybe you have to create the partition.
> >>>> initcalls done
> >>>>
> >>>> When I run "printenv" I am getting:
> >>>> barebox:/ printenv
> >>>> locals:
> >>>> globals:
> >>>> bootsource=unknown
> >>>> bootsource_instance=unknown
> >>>> PATH=/env/bin
> >>>>
> >>>> Hope this info can help.
> >>>> Cheers,
> >>>> Lior.
> >>>>
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: Ahmad Fatoum <a.fatoum@pengutronix.de>
> >>>>> Sent: Tuesday, June 13, 2023 3:51 PM
> >>>>> To: Lior Weintraub <liorw@pliops.com>; Ahmad Fatoum
> >> <ahmad@a3f.at>;
> >>>>> barebox@lists.infradead.org
> >>>>> Subject: Re: [PATCH v2] Porting barebox to a new SoC
> >>>>>
> >>>>> CAUTION: External Sender
> >>>>>
> >>>>> On 13.06.23 14:39, Lior Weintraub wrote:
> >>>>>> Hi Ahmad,
> >>>>>>
> >>>>>> As usual, your comments are spot on :-)
> >>>>>>
> >>>>>> After fixing the DT, devinfo command shows correct flash node:
> >>>>>>    `-- 20000000.flash@0.of
> >>>>>>       `-- mtdrom0
> >>>>>>          `-- 0x00000000-0x0fffffff (   256 MiB): /dev/mtdrom0
> >>>>>>          `-- mtdrom0.kernel
> >>>>>>             `-- 0x00000000-0x00ff7fff (    16 MiB): /dev/mtdrom0.kernel
> >>>>>>          `-- mtdrom0.oftree
> >>>>>>             `-- 0x00000000-0x00007fff (    32 KiB): /dev/mtdrom0.oftree
> >>>>>>          `-- mtdrom0.rootfs
> >>>>>>             `-- 0x00000000-0x007fffff (     8 MiB): /dev/mtdrom0.rootfs
> >>>>>>
> >>>>>> As can be seen above, we chose to map this "flash" node on DRAM
> >>>> address
> >>>>> 0x20000000 (offset 512MB).
> >>>>>> The BL1 code correctly copy the 3 artifacts into those location (verified
> >> the
> >>>>> content via "md -l -s /dev/mtdrom0.kernel").
> >>>>>> Running "drvinfo" shows:
> >>>>>> mtdram
> >>>>>>         20000000.flash@0.of
> >>>>>>
> >>>>>> Now when I try to run "bootm -o /dev/mtdrom0.oftree -r
> >>>>> /dev/mtdrom0.rootfs /dev/mtdrom0.kernel" I am getting this error:
> >>>>>>
> >>>>>> barebox:/ bootm -o /dev/mtdrom0.oftree -r /dev/mtdrom0.rootfs
> >>>>> /dev/mtdrom0.kernel
> >>>>>> getopt: optindex: 1 nonopts: 0 optind: 1
> >>>>>> getopt: optindex: 1 nonopts: 0 optind: 3
> >>>>>> getopt: optindex: 1 nonopts: 0 optind: 5
> >>>>>> mtdrom0.kernel: read ofs: 0x00000000 count: 0x00001000
> >>>>>>
> >>>>>> Loading ARM aarch64 Linux image '/dev/mtdrom0.kernel'
> >>>>>> header 00000000: 4d 5a 40 fa 3f 1a 23 14 00 00 00 00 00 00 00 00
> >>>>> MZ@.?.#.........
> >>>>>> header 00000010: 00 00 b3 00 00 00 00 00 0a 00 00 00 00 00 00 00
> >>>>> ................
> >>>>>> header 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >>>>> ................
> >>>>>> header 00000030: 00 00 00 00 00 00 00 00 41 52 4d 64 40 00 00 00
> >>>>> ........ARMd@...
> >>>>>> header 00000040: 50 45 00 00 64 aa 02 00 00 00 00 00 00 00 00 00
> >>>>> PE..d...........
> >>>>>> booti: Kernel to be loaded to 8000000+b30000
> >>>>>
> >>>>> Kernel image is relocatable and ARM64 header in hexdump says that
> >>>>> load offset is zero. Yet, barebox wants to load your image to
> >>>>> 8000000, which is already reserved.
> >>>>>
> >>>>> What's the output of the iomem command before and after the failed
> >>>>> boot attempt?
> >>>>>
> >>>>>> __request_region ok: 0x08000000:0x0801ffff flags=0x200
> >>>>>> mtdrom0.kernel: read ofs: 0x00000000 count: 0x00020000
> >>>>>> __request_region ok: 0x08000000:0x0803ffff flags=0x200
> >>>>>> mtdrom0.kernel: read ofs: 0x00020000 count: 0x00020000
> >>>>>> .
> >>>>>> .
> >>>>>> mtdrom0.kernel: read ofs: 0x00fe0000 count: 0x00018000
> >>>>>> __request_region ok: 0x08000000:0x08ff7fff flags=0x200
> >>>>>> mtdrom0.rootfs: read ofs: 0x00000000 count: 0x00000800
> >>>>>> __request_region: 0x08b30000:0x08b4ffff (image) conflicts with
> >>>>> 0x08000000:0x08ff7fff (image)
> >>>>>> unable to request SDRAM 0x08b30000-0x08b4ffff
> >>>>>> handler failed with: Out of memory
> >>>>>
> >>>>> Just to be sure: You don't set global.bootm.image.loadaddr, right?
> >>>>>
> >>>>> Cheers,
> >>>>> Ahmad
> >>>>>
> >>>>>>
> >>>>>> Appreciate your advice,
> >>>>>> Cheers,
> >>>>>> Lior.
> >>>>>>
> >>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: Ahmad Fatoum <a.fatoum@pengutronix.de>
> >>>>>>> Sent: Monday, June 12, 2023 6:08 PM
> >>>>>>> To: Lior Weintraub <liorw@pliops.com>; Ahmad Fatoum
> >>>> <ahmad@a3f.at>;
> >>>>>>> barebox@lists.infradead.org
> >>>>>>> Subject: Re: [PATCH v2] Porting barebox to a new SoC
> >>>>>>>
> >>>>>>> CAUTION: External Sender
> >>>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> On 12.06.23 16:59, Lior Weintraub wrote:
> >>>>>>>> Hello Ahmad,
> >>>>>>>>
> >>>>>>>> Just for simple simulations we make the ROM size 192MB so it can
> >>>> include
> >>>>> all
> >>>>>>> needed artifacts.
> >>>>>>>
> >>>>>>> I am not convinced that this is much of a simplification over having
> >>>>>>> your Qemu machine provide cfi-flash.
> >>>>>>>
> >>>>>>>> When we get this simple system to work we will move the relevant
> >> parts
> >>>>> to
> >>>>>>> BL2.
> >>>>>>>
> >>>>>>> Ok.
> >>>>>>>
> >>>>>>>> DRAM is also simulated now as SRAM so we are not worried about
> >>>>>>> initializations.
> >>>>>>>>
> >>>>>>>> So if I understand correctly, we can decide that address 0x10000000
> >> will
> >>>>> be
> >>>>>>> reserved for "flash" and add the following node:
> >>>>>>>> flash@0 {
> >>>>>>>
> >>>>>>> flash@10000000, but that's just for informational purposes.
> >>>>>>>
> >>>>>>>>     compatible = "mtd-rom";
> >>>>>>>>     probe-type = "map_rom";
> >>>>>>>>     reg = <0x10000000 0x10000000>;
> >>>>>>>>     bank-width = <4>;
> >>>>>>>>     device-width = <1>;
> >>>>>>>>
> >>>>>>>>     #address-cells = <1>;
> >>>>>>>>     #size-cells = <1>;
> >>>>>>>>     kernel@0 {
> >>>>>>>>         label = "kernel";
> >>>>>>>>         reg = <0x0 0x01000000>;
> >>>>>>>>     };
> >>>>>>>>     rootfs@1000000 {
> >>>>>>>>         label = "rootfs";
> >>>>>>>>         reg = <0x01000000 0x00800000>;
> >>>>>>>>     };
> >>>>>>>> };
> >>>>>>>>
> >>>>>>>> When I use this node on our DT I see the following devinfo:
> >>>>>>>> barebox:/ devinfo
> >>>>>>>> `-- global
> >>>>>>>> `-- nv
> >>>>>>>> `-- platform
> >>>>>>>>    `-- machine
> >>>>>>>>    `-- psci.of
> >>>>>>>>    `-- 1000000010000000.flash@0.of
> >>>>>>>
> >>>>>>> As you can see the address is wrong. That's because you have
> >>>>>>> #address-cells = <2>;
> >>>>>>> #size-cells = <2>;
> >>>>>>>
> >>>>>>> Yet, you wrote reg as if it was <1>/<1>. Change it to
> >>>>>>>
> >>>>>>>   reg = <0x0 0x10000000 0x0 0x10000000>;
> >>>>>>>
> >>>>>>> Remember all device tree integers are 32-bit big endian.
> >>>>>>>
> >>>>>>>>    `-- soc.of
> >>>>>>>>       `-- c000000000.sram@c000000000.of
> >>>>>>>>       `-- soc:pmu.of
> >>>>>>>>       `-- soc:timer.of
> >>>>>>>>       `-- e000000000.interrupt-controller@E000000000.of
> >>>>>>>>    `-- mem0
> >>>>>>>>       `-- 0x00000000-0x0fffffff (   256 MiB): /dev/ram0
> >>>>>>>>    `-- mem1
> >>>>>>>>       `-- 0x00000000-0xffffffffffffffff (   0 Bytes): /dev/mem
> >>>>>>>> `-- amba
> >>>>>>>>    `-- d000307000.serial@d000307000.of
> >>>>>>>>       `-- cs0
> >>>>>>>>          `-- 0x00000000-0xffffffffffffffff (   0 Bytes): /dev/cs0
> >>>>>>>> `-- spi
> >>>>>>>> `-- fs
> >>>>>>>>    `-- ramfs0
> >>>>>>>>    `-- devfs0
> >>>>>>>>    `-- pstore0
> >>>>>>>>
> >>>>>>>> Not sure how to proceed from here...
> >>>>>>>
> >>>>>>> Type drvinfo command to see what drivers are bound to what
> devices.
> >>>>>>> If you see no driver bound to your flash device, then maybe you need
> >>>>>>> to enable the correct driver, that would be CONFIG_MTD_MTDRAM
> >>>> (which
> >>>>>>> handles both read-only and read-write memory mapped MTD).
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>> Ahmad
> >>>>>>>
> >>>>>>>> Cheers,
> >>>>>>>> Lior.
> >>>>>>>>
> >>>>>>>>> -----Original Message-----
> >>>>>>>>> From: Ahmad Fatoum <a.fatoum@pengutronix.de>
> >>>>>>>>> Sent: Monday, June 12, 2023 3:29 PM
> >>>>>>>>> To: Lior Weintraub <liorw@pliops.com>; Ahmad Fatoum
> >>>>> <ahmad@a3f.at>;
> >>>>>>>>> barebox@lists.infradead.org
> >>>>>>>>> Subject: Re: [PATCH v2] Porting barebox to a new SoC
> >>>>>>>>>
> >>>>>>>>> CAUTION: External Sender
> >>>>>>>>>
> >>>>>>>>> Hello Lior,
> >>>>>>>>>
> >>>>>>>>> On 12.06.23 11:27, Lior Weintraub wrote:
> >>>>>>>>>> Hi Ahmad,
> >>>>>>>>>>
> >>>>>>>>>> Regarding the rootfs and Linux run question:
> >>>>>>>>>> Our board doesn't include eMMC\SD card.
> >>>>>>>>>> We only have our NOR Flash to store the binaries and our BL1
> code
> >>>> will
> >>>>>>> make
> >>>>>>>>> sure to copy those blobs into DRAM.
> >>>>>>>>>
> >>>>>>>>> How could BL1 copy artifacts in DRAM when BL2 first needs to set
> up
> >>>>>>> DRAM?
> >>>>>>>>>
> >>>>>>>>>> The use of QEMU needs to be as simple as possible so no hidden
> >> virtio
> >>>>>>>>> drivers and such because we would like to simulate the real HW
> flow.
> >>>>>>>>>
> >>>>>>>>> cfi-flash is just memory-mapped flash, which is the next simplest
> >> thing
> >>>>>>>>> after BL1 copies stuff into (limited-to-4M) on-chip SRAM.
> >>>>>>>>>
> >>>>>>>>>> Our DTS file now includes the GIC, timer and arm,psci-1.0.
> >>>>>>>>>> We have an initial build of Linux kernel Image (using buildroot)
> and a
> >>>>>>>>> rootfs.cpio.xz.
> >>>>>>>>>> I assume we can somehow reserve a portion of the DRAM to store
> >>>>> those
> >>>>>>>>> binaries and let barebox boot this Linux.
> >>>>>>>>>
> >>>>>>>>> You can make this work, but this is not how your actual system will
> >>>>>>>>> look like and trying to make this work is harder than it needs to be.
> >>>>>>>>>
> >>>>>>>>> Just add a cfi-flash that corresponds to the Linux/rootfs flash in
> >>>>>>>>> your actual system, then boot with bootm (type help bootm for
> info
> >>>>>>>>> on how to use it). You don't need to hardcode the load addresses,
> >>>>>>>>> barebox will determine them automatically.
> >>>>>>>>>
> >>>>>>>>>> Can you please advise how to make it work?
> >>>>>>>>>
> >>>>>>>>> For completion's sake, if you have 64M of RAM that's preloaded
> with
> >>>>>>>>> boot images:
> >>>>>>>>>
> >>>>>>>>>   - Remove the 64M from the barebox /memory node. You can use
> a
> >>>>>>> different
> >>>>>>>>>     DT for kernel, but if you have memory that barebox shouldn't
> >>>> override,
> >>>>>>>>>     you need to tell it that.
> >>>>>>>>>
> >>>>>>>>>   - Add a mtd-rom node that describes these 64M that you have.
> You
> >>>> can
> >>>>>>>>>     add partitions for the region used for kernel and oftree
> >>>>>>>>>
> >>>>>>>>>   - boot with bootm -o /dev/mtdrom0.oftree -r
> /dev/mtdrom0.initrd
> >> \
> >>>>>>>>>     /dev/mtdrom0.kernel
> >>>>>>>>>
> >>>>>>>>> That's admittedly cumbersome, but necessary, so barebox knows
> >> what
> >>>>>>>>> memory
> >>>>>>>>> it may use for itself and what memory may be used for boot.
> >>>>>>>>>
> >>>>>>>>> Correct solution is to use cfi-flash or similar.
> >>>>>>>>>
> >>>>>>>>> Cheers,
> >>>>>>>>> Ahmad
> >>>>>>>>> --
> >>>>>>>>> Pengutronix e.K.                           |                             |
> >>>>>>>>> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> >>>>>>>>> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-
> 0
> >> |
> >>>>>>>>> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-
> >>>> 5555
> >>>>> |
> >>>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Pengutronix e.K.                           |                             |
> >>>>>>> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> >>>>>>> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0
> |
> >>>>>>> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-
> >> 5555
> >>>> |
> >>>>>>
> >>>>>
> >>>>> --
> >>>>> Pengutronix e.K.                           |                             |
> >>>>> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> >>>>> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> >>>>> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-
> 5555 |
> >>>
> >>
> >> --
> >> Pengutronix e.K.                           |                             |
> >> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> >> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> >> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
> >
> 
> --
> Pengutronix e.K.                           |                             |
> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |