[PATCH master] sandbox: libc_malloc: multiply, not add, args in calloc overflow check

[PATCH master] sandbox: libc_malloc: multiply, not add, args in calloc overflow check

[Ahmad Fatoum]

libc_malloc is built without access to barebox headers, so instead of
size_mul it's using GCC builtins directly.

The check against BAREBOX_MALLOC_MAX_SIZE should've been done using the
product as indicated by the variable name, but erroneously
__builtin_add_overflow was used instead of __builtin_mul_overflow.

Fixes: 8839004152f7 ("sandbox: libc_malloc: fail allocations exceeding MALLOC_MAX_SIZE")
Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 arch/sandbox/os/libc_malloc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/sandbox/os/libc_malloc.c b/arch/sandbox/os/libc_malloc.c
index ac97fc37eee5..dd0f8670ff71 100644
--- a/arch/sandbox/os/libc_malloc.c
+++ b/arch/sandbox/os/libc_malloc.c
@@ -67,7 +67,7 @@ void *barebox_calloc(size_t n, size_t elem_size)
 	size_t product;
 	void *mem = NULL;
 
-	if (!__builtin_add_overflow(n, elem_size, &product) &&
+	if (!__builtin_mul_overflow(n, elem_size, &product) &&
 	    product <= BAREBOX_MALLOC_MAX_SIZE)
 		mem = calloc(n, elem_size);
 	if (!mem)
-- 
2.39.5

Re: [PATCH master] sandbox: libc_malloc: multiply, not add, args in calloc overflow check

[Sascha Hauer]

On Mon, 03 Mar 2025 14:28:00 +0100, Ahmad Fatoum wrote:
> libc_malloc is built without access to barebox headers, so instead of
> size_mul it's using GCC builtins directly.
> 
> The check against BAREBOX_MALLOC_MAX_SIZE should've been done using the
> product as indicated by the variable name, but erroneously
> __builtin_add_overflow was used instead of __builtin_mul_overflow.
> 
> [...]

Applied, thanks!

[1/1] sandbox: libc_malloc: multiply, not add, args in calloc overflow check
      https://git.pengutronix.de/cgit/barebox/commit/?id=75296e2aa356 (link may not be stable)

Best regards,
-- 
Sascha Hauer <s.hauer@pengutronix.de>