RFC: Gitlab CI for barebox

RFC: Gitlab CI for barebox

[Antony Pavlov]

Hi!

I have made simple Gitlab CI for barebox.

Gitlab CI runner setup instruction and
the source for Debian 11 docker image can be found at:

  https://gitlab.com/frantony/barebox-gitlab-ci-runner

Sample gitlab-ci.yaml config file:

  https://gitlab.com/frantony/barebox/-/commit/b1ed597d8e67c8f76f4f98cd1c6605b936cf2471

Corresponding gitlab pipeline results:

  https://gitlab.com/frantony/barebox/-/pipelines/459422690


Some notes:

  * there are several "images size > maximum size" errors during 'build ARM'
  * fake firmware blobs for ARM are used
  * 'build doc' produces sphinx generated html docs archive artifact
  * 'build ARM/MIPS/RISC-V/X86/sandbox' produce log/ directory archive artifacts
  * no support for kvx, openrisc and powerpc

@Ahmad

I suppose we can add your qemu barebox/test/emulate.pl tests into CI.

-- 
Best regards,
  Antony Pavlov

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: RFC: Gitlab CI for barebox

[Ahmad Fatoum]

Hello Antony,

On 30.01.22 13:00, Antony Pavlov wrote:
> Hi!
> 
> I have made simple Gitlab CI for barebox.

Nice.

> Gitlab CI runner setup instruction and
> the source for Debian 11 docker image can be found at:
> 
>   https://gitlab.com/frantony/barebox-gitlab-ci-runner

I had tried something similar, but with Github actions

https://github.com/a3f/barebox/actions/runs/740206381

I didn't have the time to pursue this further. I am fine
with Gitlab as well.

> 
> Sample gitlab-ci.yaml config file:
> 
>   https://gitlab.com/frantony/barebox/-/commit/b1ed597d8e67c8f76f4f98cd1c6605b936cf2471

Here's how it looks for Github actions with emulate.pl:

https://github.com/a3f/barebox/blob/emulate.pl/.github/workflows/build-configs.yaml


> Corresponding gitlab pipeline results:
> 
>   https://gitlab.com/frantony/barebox/-/pipelines/459422690
> 
> 
> Some notes:
> 
>   * there are several "images size > maximum size" errors during 'build ARM'

Ye. IIRC Sascha disables these for his build tests..? Would be nice to get
those resolved sometime.

>   * fake firmware blobs for ARM are used

Could we just fetch them from the correct location? That way we could
just provide the prebuilt binaries for download. But for a first step,
fakes are fine.

>   * 'build doc' produces sphinx generated html docs archive artifact
>   * 'build ARM/MIPS/RISC-V/X86/sandbox' produce log/ directory archive artifacts

Nice.

>   * no support for kvx, openrisc and powerpc

My GA script supports openrisc. Perhaps Jules is interested in getting
CI working for kvx as well once we have the infrastructure in place?

> 
> @Ahmad
> 
> I suppose we can add your qemu barebox/test/emulate.pl tests into CI.

Here is the Github action config for the tests:
https://github.com/a3f/barebox/blob/emulate.pl/.github/workflows/vm-tests.yaml

Any thoughts on how/if to integrate with the mailing list?

Cheers,
Ahmad

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: RFC: Gitlab CI for barebox

[Sascha Hauer]

On Mon, Jan 31, 2022 at 11:39:45AM +0100, Ahmad Fatoum wrote:
> Hello Antony,
> 
> On 30.01.22 13:00, Antony Pavlov wrote:
> > Hi!
> > 
> > I have made simple Gitlab CI for barebox.
> 
> Nice.
> 
> > Gitlab CI runner setup instruction and
> > the source for Debian 11 docker image can be found at:
> > 
> >   https://gitlab.com/frantony/barebox-gitlab-ci-runner
> 
> I had tried something similar, but with Github actions
> 
> https://github.com/a3f/barebox/actions/runs/740206381
> 
> I didn't have the time to pursue this further. I am fine
> with Gitlab as well.
> 
> > 
> > Sample gitlab-ci.yaml config file:
> > 
> >   https://gitlab.com/frantony/barebox/-/commit/b1ed597d8e67c8f76f4f98cd1c6605b936cf2471
> 
> Here's how it looks for Github actions with emulate.pl:
> 
> https://github.com/a3f/barebox/blob/emulate.pl/.github/workflows/build-configs.yaml
> 
> 
> > Corresponding gitlab pipeline results:
> > 
> >   https://gitlab.com/frantony/barebox/-/pipelines/459422690
> > 
> > 
> > Some notes:
> > 
> >   * there are several "images size > maximum size" errors during 'build ARM'
> 
> Ye. IIRC Sascha disables these for his build tests..? Would be nice to get
> those resolved sometime.

Yes, I disable these checks for my build tests. Getting these issues
resolved only holds for a certain time. Overall bloat will break it
sooner or later anyway :(
It's fighting against windmills.

> 
> >   * fake firmware blobs for ARM are used
> 
> Could we just fetch them from the correct location? That way we could
> just provide the prebuilt binaries for download. But for a first step,
> fakes are fine.

I'm not sure we even have all binaries. I guess some of the mvebu
startup binaries would have to be extracted from the hardware and I
don't know if we are allowed to share them for download if we extracted
them.

Sascha

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: RFC: Gitlab CI for barebox

[Antony Pavlov]

On Mon, 31 Jan 2022 11:39:45 +0100
Ahmad Fatoum <a.fatoum@pengutronix.de> wrote:

Hello Ahmad!

> On 30.01.22 13:00, Antony Pavlov wrote:
> > Hi!
> > 
> > I have made simple Gitlab CI for barebox.
> 
> Nice.
> 
> > Gitlab CI runner setup instruction and
> > the source for Debian 11 docker image can be found at:
> > 
> >   https://gitlab.com/frantony/barebox-gitlab-ci-runner
> 
> I had tried something similar, but with Github actions
> 
> https://github.com/a3f/barebox/actions/runs/740206381
> 
> I didn't have the time to pursue this further. I am fine
> with Gitlab as well.
> 
> > 
> > Sample gitlab-ci.yaml config file:
> > 
> >   https://gitlab.com/frantony/barebox/-/commit/b1ed597d8e67c8f76f4f98cd1c6605b936cf2471
> 
> Here's how it looks for Github actions with emulate.pl:
> 
> https://github.com/a3f/barebox/blob/emulate.pl/.github/workflows/build-configs.yaml

I have just update gitlab runner, please see https://gitlab.com/frantony/barebox-gitlab-ci-runner/-/commit/f90c3932f1e90f900c4d4304c6a54898a5ee0e1e

As a result runner can build barebox for openrisc architecture and run emulate.pl, see
https://gitlab.com/frantony/barebox/-/pipelines/464224894

There is a problem with emulate.pl for MIPS (see https://gitlab.com/frantony/barebox/-/jobs/2058215656).

log/mips/console_main artifact contains the fragment:

---- begin ----
Hit any to stop autoboot:    3
barebox@qemu malta:/ echo "YGIS""XTJCSB"
YGISXTJCSB
barebox@qemu malta:/ echo -o /cmd 'cat /env/data/config'; echo "EOVL""SAWCFS"; sh /cmd; echo "EOVL""SAWCFS" $?;
EOVLSAWCFS
could not open /env/data/config: No such file or directory
EOVLSAWCFS 1
barebox@qemu malta:/ echo -o /cmd selftest; echo "BOCZ""OJHGFV"; sh /cmd; echo "BOCZ""OJHGFV" $?;
BOCZOJHGFV
printf: all 116 tests passed
progress_notifier: all 12 tests passed
BOCZOJHGFV 0
barebox@qemu malta:/
---- end ----

there is a problem: "could not open /env/data/config: No such file or directory"

How we can pass error to gitlab runner? Now runner reports that there is no error.

> > 
> > @Ahmad
> > 
> > I suppose we can add your qemu barebox/test/emulate.pl tests into CI.
> 
> Here is the Github action config for the tests:
> https://github.com/a3f/barebox/blob/emulate.pl/.github/workflows/vm-tests.yaml
> 
> Any thoughts on how/if to integrate with the mailing list?

I suppose gitlab itself can send e-mails on CI pipeline fail.


-- 
Best regards,
  Antony Pavlov

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: RFC: Gitlab CI for barebox

[Antony Pavlov]

On Sat, 5 Feb 2022 17:12:33 +0300
Antony Pavlov <antonynpavlov@gmail.com> wrote:

> On Mon, 31 Jan 2022 11:39:45 +0100
> Ahmad Fatoum <a.fatoum@pengutronix.de> wrote:
> 
> Hello Ahmad!
> 
> > On 30.01.22 13:00, Antony Pavlov wrote:
> > > Hi!
> > > 
> > > I have made simple Gitlab CI for barebox.
> > 
> > Nice.
> > 
> > > Gitlab CI runner setup instruction and
> > > the source for Debian 11 docker image can be found at:
> > > 
> > >   https://gitlab.com/frantony/barebox-gitlab-ci-runner
> > 
> > I had tried something similar, but with Github actions
> > 
> > https://github.com/a3f/barebox/actions/runs/740206381
> > 
> > I didn't have the time to pursue this further. I am fine
> > with Gitlab as well.
> > 
> > > 
> > > Sample gitlab-ci.yaml config file:
> > > 
> > >   https://gitlab.com/frantony/barebox/-/commit/b1ed597d8e67c8f76f4f98cd1c6605b936cf2471
> > 
> > Here's how it looks for Github actions with emulate.pl:
> > 
> > https://github.com/a3f/barebox/blob/emulate.pl/.github/workflows/build-configs.yaml
> 
> I have just update gitlab runner, please see https://gitlab.com/frantony/barebox-gitlab-ci-runner/-/commit/f90c3932f1e90f900c4d4304c6a54898a5ee0e1e
> 
> As a result runner can build barebox for openrisc architecture and run emulate.pl, see
> https://gitlab.com/frantony/barebox/-/pipelines/464224894
> 
> There is a problem with emulate.pl for MIPS (see https://gitlab.com/frantony/barebox/-/jobs/2058215656).
> 
> log/mips/console_main artifact contains the fragment:
> 
> ---- begin ----
> Hit any to stop autoboot:    3
> barebox@qemu malta:/ echo "YGIS""XTJCSB"
> YGISXTJCSB
> barebox@qemu malta:/ echo -o /cmd 'cat /env/data/config'; echo "EOVL""SAWCFS"; sh /cmd; echo "EOVL""SAWCFS" $?;
> EOVLSAWCFS
> could not open /env/data/config: No such file or directory
> EOVLSAWCFS 1
> barebox@qemu malta:/ echo -o /cmd selftest; echo "BOCZ""OJHGFV"; sh /cmd; echo "BOCZ""OJHGFV" $?;
> BOCZOJHGFV
> printf: all 116 tests passed
> progress_notifier: all 12 tests passed
> BOCZOJHGFV 0
> barebox@qemu malta:/
> ---- end ----
> 
> there is a problem: "could not open /env/data/config: No such file or directory"

The same problem with ARM architecture.
It looks like CONFIG_DEFAULT_ENVIRONMENT_GENERIC_NEW_IKCONFIG=y is set only in RISC-V virt{32/64}_defconfig.

-- 
Best regards,
  Antony Pavlov

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: RFC: Gitlab CI for barebox

[Ahmad Fatoum]

Hello Antony,

On 08.02.22 17:16, Antony Pavlov wrote:
> On Sat, 5 Feb 2022 17:12:33 +0300
> Antony Pavlov <antonynpavlov@gmail.com> wrote:
> 
>> On Mon, 31 Jan 2022 11:39:45 +0100
>> Ahmad Fatoum <a.fatoum@pengutronix.de> wrote:
>>
>> Hello Ahmad!
>>
>>> On 30.01.22 13:00, Antony Pavlov wrote:
>>>> Hi!
>>>>
>>>> I have made simple Gitlab CI for barebox.
>>>
>>> Nice.
>>>
>>>> Gitlab CI runner setup instruction and
>>>> the source for Debian 11 docker image can be found at:
>>>>
>>>>   https://gitlab.com/frantony/barebox-gitlab-ci-runner
>>>
>>> I had tried something similar, but with Github actions
>>>
>>> https://github.com/a3f/barebox/actions/runs/740206381
>>>
>>> I didn't have the time to pursue this further. I am fine
>>> with Gitlab as well.
>>>
>>>>
>>>> Sample gitlab-ci.yaml config file:
>>>>
>>>>   https://gitlab.com/frantony/barebox/-/commit/b1ed597d8e67c8f76f4f98cd1c6605b936cf2471
>>>
>>> Here's how it looks for Github actions with emulate.pl:
>>>
>>> https://github.com/a3f/barebox/blob/emulate.pl/.github/workflows/build-configs.yaml
>>
>> I have just update gitlab runner, please see https://gitlab.com/frantony/barebox-gitlab-ci-runner/-/commit/f90c3932f1e90f900c4d4304c6a54898a5ee0e1e
>>
>> As a result runner can build barebox for openrisc architecture and run emulate.pl, see
>> https://gitlab.com/frantony/barebox/-/pipelines/464224894
>>
>> There is a problem with emulate.pl for MIPS (see https://gitlab.com/frantony/barebox/-/jobs/2058215656).
>>
>> log/mips/console_main artifact contains the fragment:
>>
>> ---- begin ----
>> Hit any to stop autoboot:    3
>> barebox@qemu malta:/ echo "YGIS""XTJCSB"
>> YGISXTJCSB
>> barebox@qemu malta:/ echo -o /cmd 'cat /env/data/config'; echo "EOVL""SAWCFS"; sh /cmd; echo "EOVL""SAWCFS" $?;
>> EOVLSAWCFS
>> could not open /env/data/config: No such file or directory
>> EOVLSAWCFS 1
>> barebox@qemu malta:/ echo -o /cmd selftest; echo "BOCZ""OJHGFV"; sh /cmd; echo "BOCZ""OJHGFV" $?;
>> BOCZOJHGFV
>> printf: all 116 tests passed
>> progress_notifier: all 12 tests passed
>> BOCZOJHGFV 0
>> barebox@qemu malta:/
>> ---- end ----
>>
>> there is a problem: "could not open /env/data/config: No such file or directory"
> 
> The same problem with ARM architecture.
> It looks like CONFIG_DEFAULT_ENVIRONMENT_GENERIC_NEW_IKCONFIG=y is set only in RISC-V virt{32/64}_defconfig.

There is a fallback to get .config out of the build directory instead.
Alternatively, you can call emulate.pl with --kconfig-add CONFIG_DEFAULT_ENVIRONMENT_GENERIC_NEW_IKCONFIG=y

Cheers,
Ahmad

> 


-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: RFC: Gitlab CI for barebox

[Ahmad Fatoum]

On 05.02.22 15:12, Antony Pavlov wrote:
> On Mon, 31 Jan 2022 11:39:45 +0100
> Ahmad Fatoum <a.fatoum@pengutronix.de> wrote:
> 
> Hello Ahmad!
> 
>> On 30.01.22 13:00, Antony Pavlov wrote:
>>> Hi!
>>>
>>> I have made simple Gitlab CI for barebox.
>>
>> Nice.
>>
>>> Gitlab CI runner setup instruction and
>>> the source for Debian 11 docker image can be found at:
>>>
>>>   https://gitlab.com/frantony/barebox-gitlab-ci-runner
>>
>> I had tried something similar, but with Github actions
>>
>> https://github.com/a3f/barebox/actions/runs/740206381
>>
>> I didn't have the time to pursue this further. I am fine
>> with Gitlab as well.
>>
>>>
>>> Sample gitlab-ci.yaml config file:
>>>
>>>   https://gitlab.com/frantony/barebox/-/commit/b1ed597d8e67c8f76f4f98cd1c6605b936cf2471
>>
>> Here's how it looks for Github actions with emulate.pl:
>>
>> https://github.com/a3f/barebox/blob/emulate.pl/.github/workflows/build-configs.yaml
> 
> I have just update gitlab runner, please see https://gitlab.com/frantony/barebox-gitlab-ci-runner/-/commit/f90c3932f1e90f900c4d4304c6a54898a5ee0e1e
> 
> As a result runner can build barebox for openrisc architecture and run emulate.pl, see
> https://gitlab.com/frantony/barebox/-/pipelines/464224894
> 
> There is a problem with emulate.pl for MIPS (see https://gitlab.com/frantony/barebox/-/jobs/2058215656).
> 
> log/mips/console_main artifact contains the fragment:
> 
> ---- begin ----
> Hit any to stop autoboot:    3
> barebox@qemu malta:/ echo "YGIS""XTJCSB"
> YGISXTJCSB
> barebox@qemu malta:/ echo -o /cmd 'cat /env/data/config'; echo "EOVL""SAWCFS"; sh /cmd; echo "EOVL""SAWCFS" $?;
> EOVLSAWCFS
> could not open /env/data/config: No such file or directory
> EOVLSAWCFS 1
> barebox@qemu malta:/ echo -o /cmd selftest; echo "BOCZ""OJHGFV"; sh /cmd; echo "BOCZ""OJHGFV" $?;
> BOCZOJHGFV
> printf: all 116 tests passed
> progress_notifier: all 12 tests passed
> BOCZOJHGFV 0
> barebox@qemu malta:/
> ---- end ----
> 
> there is a problem: "could not open /env/data/config: No such file or directory"
> 
> How we can pass error to gitlab runner? Now runner reports that there is no error.

That's ok. It's preferable to have the config built into barebox, but
the fallback for getting the .config out of the build directory seems
to work alright.

> 
>>>
>>> @Ahmad
>>>
>>> I suppose we can add your qemu barebox/test/emulate.pl tests into CI.
>>
>> Here is the Github action config for the tests:
>> https://github.com/a3f/barebox/blob/emulate.pl/.github/workflows/vm-tests.yaml
>>
>> Any thoughts on how/if to integrate with the mailing list?
> 
> I suppose gitlab itself can send e-mails on CI pipeline fail.
> 
> 


-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox