mail archive of the barebox mailing list
 help / color / mirror / Atom feed
* [PATCH 00/12] poller: run pollers as proper coroutines (green threads)
@ 2021-02-15 10:36 Ahmad Fatoum
  2021-02-15 10:36 ` [PATCH 01/12] common: add coroutine support Ahmad Fatoum
                   ` (11 more replies)
  0 siblings, 12 replies; 14+ messages in thread
From: Ahmad Fatoum @ 2021-02-15 10:36 UTC (permalink / raw)
  To: barebox

barebox pollers were so far very limited coroutines that only yield at
the end of the function. This series leverages setjmp/longjmp to allow
pollers to yield at any point of their execution. The motivation behind
this is that it finally becomes feasible to port threaded code to barebox with
minimal modification: If your thread has a delay, just stick in a
poller_yield() and give other threads as well as the main thread a chance
to run before resuming execution again.

How that can look like is showing in "usbgadget: ums: run gadget loop in a
background coroutine if possible", which turns a blocking loop very
naturally into a thread.

The implementation is basically: setjmp() to store the original place of
execution, longjmp() to jump back on a context switch, initjmp() to start
a new poller the very first time. The reason why we need an extra initjmp()
beyond standard C setjmp and longjmp is because those two only save a
stack pointer, so coroutines can clobber each other's stack.
With initjmp, the new coroutine starts directly executing on a disjoint stack.
initjmp, like setjmp and longjmp should be implemented in assembly, but it
should be fairly straight forward.  e.g. the ARM implementation for initjmp is
a mere 4 instructions long.

For now, I only implemented this for ARM and sandbox. If this is accepted and
becomes available for other architectures as well, it'll likely result in
changes to other parts of barebox:

  - Workqueues will become obsolete. This series already has pollers
    doing file system access yielding until they can be run in command context

  - Slices will become obsolete. They are equivalent to a mutex_try_lock
    and we can implement blocking mutexes now

  - Functions calling poller_call can be changed to call poller_reschedule()
    to become available from calling from any context

  - Backtraces should indicate which coroutine is currently running

The execution tax on context switching is that for an idle i.MX8MM system
30% less pollers can be executed in a given time frame. The extra accounting
overhead for a yielding poller is in the range of a 100 byte or so, which
is negligible compared to the stack newly allocated on poller creation, which
is CONFIG_STACK_SIZE bytes long, usually 32K. Pollers probably don't need that
much stack space, but for now, we just use the same stack size everywhere.

This is series is based on:
 - <20210215102442.28731-1-a.fatoum@pengutronix.de> to avoid a conflict
 - <20210215102740.30418-1-a.fatoum@pengutronix.de> which adds blocking
   usb mass storage, which is made non-blocing here

Note: Despite the name green threads (or fibers), this is wholly cooperative.
There is no preemption and thus no new extra precautions that need to be taken
when writing pollers. On the contrary, if we make this available everywhere,
you can basically do anything in a poller.

Looking forward to feedback,
Ahmad Fatoum (12):
  common: add coroutine support
  poller: run pollers as proper coroutines if architecture supports it
  ARM: asm: setjmp: annotate setjmp/longjmp for GCC
  ARM: asm: setjmp: implement coroutine dependency initjmp()
  sandbox: asm: implement setjmp/longjmp/initjmp
  poller: command: add new coroutine check
  slice: have assert_command_context() yield until true if possible
  poller: implement basic Linux-like completion API
  include: add kthread wrappers for pollers
  usbgadget: ums: run gadget loop in a background coroutine if possible
  usbgadget: refactor usbgadget_register to accept array
  usbgadget: multi: wire mass storage gadget into composite gadget

 Documentation/user/usb.rst          |   2 +
 arch/arm/Kconfig                    |   1 +
 arch/arm/include/asm/setjmp.h       |   6 +-
 arch/arm/lib32/setjmp.S             |   8 ++
 arch/arm/lib64/setjmp.S             |   9 ++
 arch/sandbox/Kconfig                |   1 +
 arch/sandbox/Makefile               |   5 +-
 arch/sandbox/include/asm/setjmp.h   |  17 +++
 arch/sandbox/os/Makefile            |   5 +-
 arch/sandbox/os/setjmp.c            | 180 ++++++++++++++++++++++++++++
 commands/Kconfig                    |  10 ++
 commands/Makefile                   |   2 +-
 commands/usbgadget.c                |  29 +++--
 common/Kconfig                      |  17 +++
 common/Makefile                     |   1 +
 common/coroutine.c                  | 178 +++++++++++++++++++++++++++
 common/poller.c                     | 111 ++++++++++++++---
 common/usbgadget.c                  |  55 +++++++--
 drivers/usb/gadget/Kconfig          |  12 +-
 drivers/usb/gadget/f_mass_storage.c |  14 +++
 drivers/usb/gadget/multi.c          |  36 ++++++
 include/coroutine.h                 |  43 +++++++
 include/linux/completion.h          |  61 ++++++++++
 include/linux/kthread.h             |  72 +++++++++++
 include/poller.h                    |  16 ++-
 include/slice.h                     |  14 ++-
 include/usb/gadget-multi.h          |  19 ++-
 27 files changed, 869 insertions(+), 55 deletions(-)
 create mode 100644 arch/sandbox/include/asm/setjmp.h
 create mode 100644 arch/sandbox/os/setjmp.c
 create mode 100644 common/coroutine.c
 create mode 100644 include/coroutine.h
 create mode 100644 include/linux/completion.h
 create mode 100644 include/linux/kthread.h

-- 
2.29.2


_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2021-02-15 12:31 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-15 10:36 [PATCH 00/12] poller: run pollers as proper coroutines (green threads) Ahmad Fatoum
2021-02-15 10:36 ` [PATCH 01/12] common: add coroutine support Ahmad Fatoum
2021-02-15 10:36 ` [PATCH 02/12] poller: run pollers as proper coroutines if architecture supports it Ahmad Fatoum
2021-02-15 10:36 ` [PATCH 03/12] ARM: asm: setjmp: annotate setjmp/longjmp for GCC Ahmad Fatoum
2021-02-15 10:36 ` [PATCH 04/12] ARM: asm: setjmp: implement coroutine dependency initjmp() Ahmad Fatoum
2021-02-15 10:36 ` [PATCH 05/12] sandbox: asm: implement setjmp/longjmp/initjmp Ahmad Fatoum
2021-02-15 10:36 ` [PATCH 06/12] poller: command: add new coroutine check Ahmad Fatoum
2021-02-15 10:37 ` [PATCH 07/12] slice: have assert_command_context() yield until true if possible Ahmad Fatoum
2021-02-15 10:37 ` [PATCH 08/12] poller: implement basic Linux-like completion API Ahmad Fatoum
2021-02-15 10:37 ` [PATCH 09/12] include: add kthread wrappers for pollers Ahmad Fatoum
2021-02-15 12:31   ` Sascha Hauer
2021-02-15 10:37 ` [PATCH 10/12] usbgadget: ums: run gadget loop in a background coroutine if possible Ahmad Fatoum
2021-02-15 10:37 ` [PATCH 11/12] usbgadget: refactor usbgadget_register to accept array Ahmad Fatoum
2021-02-15 10:37 ` [PATCH 12/12] usbgadget: multi: wire mass storage gadget into composite gadget Ahmad Fatoum

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox