* [PATCH 0/3] pbl: add lz4 compression support
@ 2013-07-12 9:14 Jean-Christophe PLAGNIOL-VILLARD
2013-07-12 9:22 ` [PATCH 1/3] decompressor: Add LZ4 decompressor module Jean-Christophe PLAGNIOL-VILLARD
0 siblings, 1 reply; 7+ messages in thread
From: Jean-Christophe PLAGNIOL-VILLARD @ 2013-07-12 9:14 UTC (permalink / raw)
To: barebox
Hi,
The following patch serie add the lz4 support to the pbl
Jean-Christophe PLAGNIOL-VILLARD (1):
pbl: add lz4 support
Kyungsik Lee (2):
decompressor: Add LZ4 decompressor module
lib: Add support for LZ4-compressed kernel
arch/arm/pbl/Makefile | 3 +-
arch/arm/pbl/piggy.lz4.S | 6 ++++
arch/mips/pbl/Makefile | 3 +-
arch/mips/pbl/piggy.lz4.S | 6 ++++
common/filetype.c | 4 +++
images/Makefile | 1 +
include/filetype.h | 1 +
include/linux/decompress/unlz4.h | 10 ++++++
include/linux/lz4.h | 87 ++++++++++++++++++++++++++++++++++++++++++++++++
lib/Kconfig | 4 +++
lib/Makefile | 2 ++
lib/decompress_unlz4.c | 188 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
lib/lz4/Makefile | 1 +
lib/lz4/lz4_decompress.c | 327 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
lib/lz4/lz4defs.h | 156 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
lib/uncompress.c | 6 ++++
pbl/Kconfig | 3 ++
pbl/decomp.c | 4 +++
scripts/Makefile.lib | 5 +++
19 files changed, 815 insertions(+), 2 deletions(-)
create mode 100644 arch/arm/pbl/piggy.lz4.S
create mode 100644 arch/mips/pbl/piggy.lz4.S
create mode 100644 include/linux/decompress/unlz4.h
create mode 100644 include/linux/lz4.h
create mode 100644 lib/decompress_unlz4.c
create mode 100644 lib/lz4/Makefile
create mode 100644 lib/lz4/lz4_decompress.c
create mode 100644 lib/lz4/lz4defs.h
Best Regards,
J.
_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 1/3] decompressor: Add LZ4 decompressor module
2013-07-12 9:14 [PATCH 0/3] pbl: add lz4 compression support Jean-Christophe PLAGNIOL-VILLARD
@ 2013-07-12 9:22 ` Jean-Christophe PLAGNIOL-VILLARD
2013-07-12 9:22 ` [PATCH 2/3] lib: Add support for LZ4-compressed kernel Jean-Christophe PLAGNIOL-VILLARD
2013-07-12 9:22 ` [PATCH 3/3] pbl: add lz4 support Jean-Christophe PLAGNIOL-VILLARD
0 siblings, 2 replies; 7+ messages in thread
From: Jean-Christophe PLAGNIOL-VILLARD @ 2013-07-12 9:22 UTC (permalink / raw)
To: barebox; +Cc: Kyungsik Lee
From: Kyungsik Lee <kyungsik.lee@lge.com>
This patch adds support for LZ4 decompression in the Linux Kernel.
LZ4 Decompression APIs for kernel are based on LZ4 implementation
by Yann Collet.
LZ4 homepage : http://fastcompression.blogspot.com/p/lz4.html
LZ4 source repository : http://code.google.com/p/lz4/
Signed-off-by: Kyungsik Lee <kyungsik.lee@lge.com>
Signed-off-by: Jean-Christophe PLAGNIOL-VILLARD <plagnioj@jcrosoft.com>
---
include/linux/lz4.h | 87 +++++++++++++
lib/lz4/lz4_decompress.c | 326 +++++++++++++++++++++++++++++++++++++++++++++++
lib/lz4/lz4defs.h | 156 +++++++++++++++++++++++
3 files changed, 569 insertions(+)
create mode 100644 include/linux/lz4.h
create mode 100644 lib/lz4/lz4_decompress.c
create mode 100644 lib/lz4/lz4defs.h
diff --git a/include/linux/lz4.h b/include/linux/lz4.h
new file mode 100644
index 0000000..d21c13f
--- /dev/null
+++ b/include/linux/lz4.h
@@ -0,0 +1,87 @@
+#ifndef __LZ4_H__
+#define __LZ4_H__
+/*
+ * LZ4 Kernel Interface
+ *
+ * Copyright (C) 2013, LG Electronics, Kyungsik Lee <kyungsik.lee@lge.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+#define LZ4_MEM_COMPRESS (4096 * sizeof(unsigned char *))
+#define LZ4HC_MEM_COMPRESS (65538 * sizeof(unsigned char *))
+
+/*
+ * lz4_compressbound()
+ * Provides the maximum size that LZ4 may output in a "worst case" scenario
+ * (input data not compressible)
+ */
+static inline size_t lz4_compressbound(size_t isize)
+{
+ return isize + (isize / 255) + 16;
+}
+
+/*
+ * lz4_compress()
+ * src : source address of the original data
+ * src_len : size of the original data
+ * dst : output buffer address of the compressed data
+ * This requires 'dst' of size LZ4_COMPRESSBOUND.
+ * dst_len : is the output size, which is returned after compress done
+ * workmem : address of the working memory.
+ * This requires 'workmem' of size LZ4_MEM_COMPRESS.
+ * return : Success if return 0
+ * Error if return (< 0)
+ * note : Destination buffer and workmem must be already allocated with
+ * the defined size.
+ */
+int lz4_compress(const unsigned char *src, size_t src_len,
+ unsigned char *dst, size_t *dst_len, void *wrkmem);
+
+ /*
+ * lz4hc_compress()
+ * src : source address of the original data
+ * src_len : size of the original data
+ * dst : output buffer address of the compressed data
+ * This requires 'dst' of size LZ4_COMPRESSBOUND.
+ * dst_len : is the output size, which is returned after compress done
+ * workmem : address of the working memory.
+ * This requires 'workmem' of size LZ4HC_MEM_COMPRESS.
+ * return : Success if return 0
+ * Error if return (< 0)
+ * note : Destination buffer and workmem must be already allocated with
+ * the defined size.
+ */
+int lz4hc_compress(const unsigned char *src, size_t src_len,
+ unsigned char *dst, size_t *dst_len, void *wrkmem);
+
+/*
+ * lz4_decompress()
+ * src : source address of the compressed data
+ * src_len : is the input size, whcih is returned after decompress done
+ * dest : output buffer address of the decompressed data
+ * actual_dest_len: is the size of uncompressed data, supposing it's known
+ * return : Success if return 0
+ * Error if return (< 0)
+ * note : Destination buffer must be already allocated.
+ * slightly faster than lz4_decompress_unknownoutputsize()
+ */
+int lz4_decompress(const char *src, size_t *src_len, char *dest,
+ size_t actual_dest_len);
+
+/*
+ * lz4_decompress_unknownoutputsize()
+ * src : source address of the compressed data
+ * src_len : is the input size, therefore the compressed size
+ * dest : output buffer address of the decompressed data
+ * dest_len: is the max size of the destination buffer, which is
+ * returned with actual size of decompressed data after
+ * decompress done
+ * return : Success if return 0
+ * Error if return (< 0)
+ * note : Destination buffer must be already allocated.
+ */
+int lz4_decompress_unknownoutputsize(const char *src, size_t src_len,
+ char *dest, size_t *dest_len);
+#endif
diff --git a/lib/lz4/lz4_decompress.c b/lib/lz4/lz4_decompress.c
new file mode 100644
index 0000000..d3414ea
--- /dev/null
+++ b/lib/lz4/lz4_decompress.c
@@ -0,0 +1,326 @@
+/*
+ * LZ4 Decompressor for Linux kernel
+ *
+ * Copyright (C) 2013, LG Electronics, Kyungsik Lee <kyungsik.lee@lge.com>
+ *
+ * Based on LZ4 implementation by Yann Collet.
+ *
+ * LZ4 - Fast LZ compression algorithm
+ * Copyright (C) 2011-2012, Yann Collet.
+ * BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are
+ * met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following disclaimer
+ * in the documentation and/or other materials provided with the
+ * distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * You can contact the author at :
+ * - LZ4 homepage : http://fastcompression.blogspot.com/p/lz4.html
+ * - LZ4 source repository : http://code.google.com/p/lz4/
+ */
+
+#ifndef STATIC
+#include <linux/module.h>
+#include <linux/kernel.h>
+#endif
+#include <linux/lz4.h>
+
+#include <asm/unaligned.h>
+
+#include "lz4defs.h"
+
+static int lz4_uncompress(const char *source, char *dest, int osize)
+{
+ const BYTE *ip = (const BYTE *) source;
+ const BYTE *ref;
+ BYTE *op = (BYTE *) dest;
+ BYTE * const oend = op + osize;
+ BYTE *cpy;
+ unsigned token;
+ size_t length;
+ size_t dec32table[] = {0, 3, 2, 3, 0, 0, 0, 0};
+#if LZ4_ARCH64
+ size_t dec64table[] = {0, 0, 0, -1, 0, 1, 2, 3};
+#endif
+
+ while (1) {
+
+ /* get runlength */
+ token = *ip++;
+ length = (token >> ML_BITS);
+ if (length == RUN_MASK) {
+ size_t len;
+
+ len = *ip++;
+ for (; len == 255; length += 255)
+ len = *ip++;
+ length += len;
+ }
+
+ /* copy literals */
+ cpy = op + length;
+ if (unlikely(cpy > oend - COPYLENGTH)) {
+ /*
+ * Error: not enough place for another match
+ * (min 4) + 5 literals
+ */
+ if (cpy != oend)
+ goto _output_error;
+
+ memcpy(op, ip, length);
+ ip += length;
+ break; /* EOF */
+ }
+ LZ4_WILDCOPY(ip, op, cpy);
+ ip -= (op - cpy);
+ op = cpy;
+
+ /* get offset */
+ LZ4_READ_LITTLEENDIAN_16(ref, cpy, ip);
+ ip += 2;
+
+ /* Error: offset create reference outside destination buffer */
+ if (unlikely(ref < (BYTE *const) dest))
+ goto _output_error;
+
+ /* get matchlength */
+ length = token & ML_MASK;
+ if (length == ML_MASK) {
+ for (; *ip == 255; length += 255)
+ ip++;
+ length += *ip++;
+ }
+
+ /* copy repeated sequence */
+ if (unlikely((op - ref) < STEPSIZE)) {
+#if LZ4_ARCH64
+ size_t dec64 = dec64table[op - ref];
+#else
+ const int dec64 = 0;
+#endif
+ op[0] = ref[0];
+ op[1] = ref[1];
+ op[2] = ref[2];
+ op[3] = ref[3];
+ op += 4;
+ ref += 4;
+ ref -= dec32table[op-ref];
+ PUT4(ref, op);
+ op += STEPSIZE - 4;
+ ref -= dec64;
+ } else {
+ LZ4_COPYSTEP(ref, op);
+ }
+ cpy = op + length - (STEPSIZE - 4);
+ if (cpy > (oend - COPYLENGTH)) {
+
+ /* Error: request to write beyond destination buffer */
+ if (cpy > oend)
+ goto _output_error;
+ LZ4_SECURECOPY(ref, op, (oend - COPYLENGTH));
+ while (op < cpy)
+ *op++ = *ref++;
+ op = cpy;
+ /*
+ * Check EOF (should never happen, since last 5 bytes
+ * are supposed to be literals)
+ */
+ if (op == oend)
+ goto _output_error;
+ continue;
+ }
+ LZ4_SECURECOPY(ref, op, cpy);
+ op = cpy; /* correction */
+ }
+ /* end of decoding */
+ return (int) (((char *)ip) - source);
+
+ /* write overflow error detected */
+_output_error:
+ return (int) (-(((char *)ip) - source));
+}
+
+static int lz4_uncompress_unknownoutputsize(const char *source, char *dest,
+ int isize, size_t maxoutputsize)
+{
+ const BYTE *ip = (const BYTE *) source;
+ const BYTE *const iend = ip + isize;
+ const BYTE *ref;
+
+
+ BYTE *op = (BYTE *) dest;
+ BYTE * const oend = op + maxoutputsize;
+ BYTE *cpy;
+
+ size_t dec32table[] = {0, 3, 2, 3, 0, 0, 0, 0};
+#if LZ4_ARCH64
+ size_t dec64table[] = {0, 0, 0, -1, 0, 1, 2, 3};
+#endif
+
+ /* Main Loop */
+ while (ip < iend) {
+
+ unsigned token;
+ size_t length;
+
+ /* get runlength */
+ token = *ip++;
+ length = (token >> ML_BITS);
+ if (length == RUN_MASK) {
+ int s = 255;
+ while ((ip < iend) && (s == 255)) {
+ s = *ip++;
+ length += s;
+ }
+ }
+ /* copy literals */
+ cpy = op + length;
+ if ((cpy > oend - COPYLENGTH) ||
+ (ip + length > iend - COPYLENGTH)) {
+
+ if (cpy > oend)
+ goto _output_error;/* writes beyond buffer */
+
+ if (ip + length != iend)
+ goto _output_error;/*
+ * Error: LZ4 format requires
+ * to consume all input
+ * at this stage
+ */
+ memcpy(op, ip, length);
+ op += length;
+ break;/* Necessarily EOF, due to parsing restrictions */
+ }
+ LZ4_WILDCOPY(ip, op, cpy);
+ ip -= (op - cpy);
+ op = cpy;
+
+ /* get offset */
+ LZ4_READ_LITTLEENDIAN_16(ref, cpy, ip);
+ ip += 2;
+ if (ref < (BYTE * const) dest)
+ goto _output_error;
+ /*
+ * Error : offset creates reference
+ * outside of destination buffer
+ */
+
+ /* get matchlength */
+ length = (token & ML_MASK);
+ if (length == ML_MASK) {
+ while (ip < iend) {
+ int s = *ip++;
+ length += s;
+ if (s == 255)
+ continue;
+ break;
+ }
+ }
+
+ /* copy repeated sequence */
+ if (unlikely((op - ref) < STEPSIZE)) {
+#if LZ4_ARCH64
+ size_t dec64 = dec64table[op - ref];
+#else
+ const int dec64 = 0;
+#endif
+ op[0] = ref[0];
+ op[1] = ref[1];
+ op[2] = ref[2];
+ op[3] = ref[3];
+ op += 4;
+ ref += 4;
+ ref -= dec32table[op - ref];
+ PUT4(ref, op);
+ op += STEPSIZE - 4;
+ ref -= dec64;
+ } else {
+ LZ4_COPYSTEP(ref, op);
+ }
+ cpy = op + length - (STEPSIZE-4);
+ if (cpy > oend - COPYLENGTH) {
+ if (cpy > oend)
+ goto _output_error; /* write outside of buf */
+
+ LZ4_SECURECOPY(ref, op, (oend - COPYLENGTH));
+ while (op < cpy)
+ *op++ = *ref++;
+ op = cpy;
+ /*
+ * Check EOF (should never happen, since last 5 bytes
+ * are supposed to be literals)
+ */
+ if (op == oend)
+ goto _output_error;
+ continue;
+ }
+ LZ4_SECURECOPY(ref, op, cpy);
+ op = cpy; /* correction */
+ }
+ /* end of decoding */
+ return (int) (((char *) op) - dest);
+
+ /* write overflow error detected */
+_output_error:
+ return (int) (-(((char *) ip) - source));
+}
+
+int lz4_decompress(const char *src, size_t *src_len, char *dest,
+ size_t actual_dest_len)
+{
+ int ret = -1;
+ int input_len = 0;
+
+ input_len = lz4_uncompress(src, dest, actual_dest_len);
+ if (input_len < 0)
+ goto exit_0;
+ *src_len = input_len;
+
+ return 0;
+exit_0:
+ return ret;
+}
+#ifndef STATIC
+EXPORT_SYMBOL_GPL(lz4_decompress);
+#endif
+
+int lz4_decompress_unknownoutputsize(const char *src, size_t src_len,
+ char *dest, size_t *dest_len)
+{
+ int ret = -1;
+ int out_len = 0;
+
+ out_len = lz4_uncompress_unknownoutputsize(src, dest, src_len,
+ *dest_len);
+ if (out_len < 0)
+ goto exit_0;
+ *dest_len = out_len;
+
+ return 0;
+exit_0:
+ return ret;
+}
+#ifndef STATIC
+EXPORT_SYMBOL_GPL(lz4_decompress_unknownoutputsize);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("LZ4 Decompressor");
+#endif
diff --git a/lib/lz4/lz4defs.h b/lib/lz4/lz4defs.h
new file mode 100644
index 0000000..abcecdc
--- /dev/null
+++ b/lib/lz4/lz4defs.h
@@ -0,0 +1,156 @@
+/*
+ * lz4defs.h -- architecture specific defines
+ *
+ * Copyright (C) 2013, LG Electronics, Kyungsik Lee <kyungsik.lee@lge.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+/*
+ * Detects 64 bits mode
+ */
+#if (defined(__x86_64__) || defined(__x86_64) || defined(__amd64__) \
+ || defined(__ppc64__) || defined(__LP64__))
+#define LZ4_ARCH64 1
+#else
+#define LZ4_ARCH64 0
+#endif
+
+/*
+ * Architecture-specific macros
+ */
+#define BYTE u8
+typedef struct _U16_S { u16 v; } U16_S;
+typedef struct _U32_S { u32 v; } U32_S;
+typedef struct _U64_S { u64 v; } U64_S;
+#if defined(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS) \
+ || defined(CONFIG_ARM) && __LINUX_ARM_ARCH__ >= 6 \
+ && defined(ARM_EFFICIENT_UNALIGNED_ACCESS)
+
+#define A16(x) (((U16_S *)(x))->v)
+#define A32(x) (((U32_S *)(x))->v)
+#define A64(x) (((U64_S *)(x))->v)
+
+#define PUT4(s, d) (A32(d) = A32(s))
+#define PUT8(s, d) (A64(d) = A64(s))
+#define LZ4_WRITE_LITTLEENDIAN_16(p, v) \
+ do { \
+ A16(p) = v; \
+ p += 2; \
+ } while (0)
+#else /* CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS */
+
+#define A64(x) get_unaligned((u64 *)&(((U16_S *)(x))->v))
+#define A32(x) get_unaligned((u32 *)&(((U16_S *)(x))->v))
+#define A16(x) get_unaligned((u16 *)&(((U16_S *)(x))->v))
+
+#define PUT4(s, d) \
+ put_unaligned(get_unaligned((const u32 *) s), (u32 *) d)
+#define PUT8(s, d) \
+ put_unaligned(get_unaligned((const u64 *) s), (u64 *) d)
+
+#define LZ4_WRITE_LITTLEENDIAN_16(p, v) \
+ do { \
+ put_unaligned(v, (u16 *)(p)); \
+ p += 2; \
+ } while (0)
+#endif
+
+#define COPYLENGTH 8
+#define ML_BITS 4
+#define ML_MASK ((1U << ML_BITS) - 1)
+#define RUN_BITS (8 - ML_BITS)
+#define RUN_MASK ((1U << RUN_BITS) - 1)
+#define MEMORY_USAGE 14
+#define MINMATCH 4
+#define SKIPSTRENGTH 6
+#define LASTLITERALS 5
+#define MFLIMIT (COPYLENGTH + MINMATCH)
+#define MINLENGTH (MFLIMIT + 1)
+#define MAXD_LOG 16
+#define MAXD (1 << MAXD_LOG)
+#define MAXD_MASK (u32)(MAXD - 1)
+#define MAX_DISTANCE (MAXD - 1)
+#define HASH_LOG (MAXD_LOG - 1)
+#define HASHTABLESIZE (1 << HASH_LOG)
+#define MAX_NB_ATTEMPTS 256
+#define OPTIMAL_ML (int)((ML_MASK-1)+MINMATCH)
+#define LZ4_64KLIMIT ((1<<16) + (MFLIMIT - 1))
+#define HASHLOG64K ((MEMORY_USAGE - 2) + 1)
+#define HASH64KTABLESIZE (1U << HASHLOG64K)
+#define LZ4_HASH_VALUE(p) (((A32(p)) * 2654435761U) >> \
+ ((MINMATCH * 8) - (MEMORY_USAGE-2)))
+#define LZ4_HASH64K_VALUE(p) (((A32(p)) * 2654435761U) >> \
+ ((MINMATCH * 8) - HASHLOG64K))
+#define HASH_VALUE(p) (((A32(p)) * 2654435761U) >> \
+ ((MINMATCH * 8) - HASH_LOG))
+
+#if LZ4_ARCH64/* 64-bit */
+#define STEPSIZE 8
+
+#define LZ4_COPYSTEP(s, d) \
+ do { \
+ PUT8(s, d); \
+ d += 8; \
+ s += 8; \
+ } while (0)
+
+#define LZ4_COPYPACKET(s, d) LZ4_COPYSTEP(s, d)
+
+#define LZ4_SECURECOPY(s, d, e) \
+ do { \
+ if (d < e) { \
+ LZ4_WILDCOPY(s, d, e); \
+ } \
+ } while (0)
+#define HTYPE u32
+
+#ifdef __BIG_ENDIAN
+#define LZ4_NBCOMMONBYTES(val) (__builtin_clzll(val) >> 3)
+#else
+#define LZ4_NBCOMMONBYTES(val) (__builtin_ctzll(val) >> 3)
+#endif
+
+#else /* 32-bit */
+#define STEPSIZE 4
+
+#define LZ4_COPYSTEP(s, d) \
+ do { \
+ PUT4(s, d); \
+ d += 4; \
+ s += 4; \
+ } while (0)
+
+#define LZ4_COPYPACKET(s, d) \
+ do { \
+ LZ4_COPYSTEP(s, d); \
+ LZ4_COPYSTEP(s, d); \
+ } while (0)
+
+#define LZ4_SECURECOPY LZ4_WILDCOPY
+#define HTYPE const u8*
+
+#ifdef __BIG_ENDIAN
+#define LZ4_NBCOMMONBYTES(val) (__builtin_clz(val) >> 3)
+#else
+#define LZ4_NBCOMMONBYTES(val) (__builtin_ctz(val) >> 3)
+#endif
+
+#endif
+
+#define LZ4_READ_LITTLEENDIAN_16(d, s, p) \
+ (d = s - get_unaligned_le16(p))
+
+#define LZ4_WILDCOPY(s, d, e) \
+ do { \
+ LZ4_COPYPACKET(s, d); \
+ } while (d < e)
+
+#define LZ4_BLINDCOPY(s, d, l) \
+ do { \
+ u8 *e = (d) + l; \
+ LZ4_WILDCOPY(s, d, e); \
+ d = e; \
+ } while (0)
--
1.8.3.2
_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 2/3] lib: Add support for LZ4-compressed kernel
2013-07-12 9:22 ` [PATCH 1/3] decompressor: Add LZ4 decompressor module Jean-Christophe PLAGNIOL-VILLARD
@ 2013-07-12 9:22 ` Jean-Christophe PLAGNIOL-VILLARD
2013-07-12 9:22 ` [PATCH 3/3] pbl: add lz4 support Jean-Christophe PLAGNIOL-VILLARD
1 sibling, 0 replies; 7+ messages in thread
From: Jean-Christophe PLAGNIOL-VILLARD @ 2013-07-12 9:22 UTC (permalink / raw)
To: barebox; +Cc: Kyungsik Lee
From: Kyungsik Lee <kyungsik.lee@lge.com>
This patch adds support for extracting LZ4-compressed kernel images,
as well as LZ4-compressed ramdisk images in the kernel boot process.
This depends on the patch below
decompressor: Add LZ4 decompressor module
Signed-off-by: Kyungsik Lee <kyungsik.lee@lge.com>
Signed-off-by: Jean-Christophe PLAGNIOL-VILLARD <plagnioj@jcrosoft.com>
---
common/filetype.c | 4 +
include/filetype.h | 1 +
include/linux/decompress/unlz4.h | 10 +++
lib/Kconfig | 4 +
lib/Makefile | 2 +
lib/decompress_unlz4.c | 188 +++++++++++++++++++++++++++++++++++++++
lib/lz4/Makefile | 1 +
lib/lz4/lz4_decompress.c | 3 +-
lib/uncompress.c | 6 ++
scripts/Makefile.lib | 5 ++
10 files changed, 223 insertions(+), 1 deletion(-)
create mode 100644 include/linux/decompress/unlz4.h
create mode 100644 lib/decompress_unlz4.c
create mode 100644 lib/lz4/Makefile
diff --git a/common/filetype.c b/common/filetype.c
index 1ff3dd2..13c7994 100644
--- a/common/filetype.c
+++ b/common/filetype.c
@@ -33,6 +33,7 @@ static const struct filetype_str filetype_str[] = {
[filetype_unknown] = { "unknown", "unkown" },
[filetype_arm_zimage] = { "arm Linux zImage", "arm-zimage" },
[filetype_lzo_compressed] = { "lzo compressed", "lzo" },
+ [filetype_lz4_compressed] = { "lz4 compressed", "lz4" },
[filetype_arm_barebox] = { "arm barebox image", "arm-barebox" },
[filetype_uimage] = { "U-Boot uImage", "u-boot" },
[filetype_ubi] = { "UBI image", "ubi" },
@@ -195,6 +196,9 @@ enum filetype file_detect_type(const void *_buf, size_t bufsize)
if (buf8[0] == 0x89 && buf8[1] == 0x4c && buf8[2] == 0x5a &&
buf8[3] == 0x4f)
return filetype_lzo_compressed;
+ if (buf8[0] == 0x02 && buf8[1] == 0x21 && buf8[2] == 0x4c &&
+ buf8[3] == 0x18)
+ return filetype_lz4_compressed;
if (buf[0] == be32_to_cpu(0x27051956))
return filetype_uimage;
if (buf[0] == 0x23494255)
diff --git a/include/filetype.h b/include/filetype.h
index c73c64a..1b90805 100644
--- a/include/filetype.h
+++ b/include/filetype.h
@@ -10,6 +10,7 @@ enum filetype {
filetype_unknown,
filetype_arm_zimage,
filetype_lzo_compressed,
+ filetype_lz4_compressed,
filetype_arm_barebox,
filetype_uimage,
filetype_ubi,
diff --git a/include/linux/decompress/unlz4.h b/include/linux/decompress/unlz4.h
new file mode 100644
index 0000000..7aaafc2
--- /dev/null
+++ b/include/linux/decompress/unlz4.h
@@ -0,0 +1,10 @@
+#ifndef DECOMPRESS_UNLZ4_H
+#define DECOMPRESS_UNLZ4_H
+
+int decompress_unlz4(unsigned char *inbuf, int len,
+ int(*fill)(void*, unsigned int),
+ int(*flush)(void*, unsigned int),
+ unsigned char *output,
+ int *pos,
+ void(*error)(char *x));
+#endif
diff --git a/lib/Kconfig b/lib/Kconfig
index 646fdb7..b5ac22a 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -14,6 +14,10 @@ config BZLIB
bool "include bzip2 uncompression support"
select UNCOMPRESS
+config LZ4_DECOMPRESS
+ bool "include lz4 uncompression support"
+ select UNCOMPRESS
+
config GENERIC_FIND_NEXT_BIT
def_bool n
diff --git a/lib/Makefile b/lib/Makefile
index 7c42537..61f1e63 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -28,8 +28,10 @@ obj-y += notifier.o
obj-y += copy_file.o
obj-y += random.o
obj-y += lzo/
+obj-$(CONFIG_LZ4_DECOMPRESS) += lz4/
obj-y += show_progress.o
obj-$(CONFIG_LZO_DECOMPRESS) += decompress_unlzo.o
+obj-$(CONFIG_LZ4_DECOMPRESS) += decompress_unlz4.o
obj-$(CONFIG_PROCESS_ESCAPE_SEQUENCE) += process_escape_sequence.o
obj-$(CONFIG_UNCOMPRESS) += uncompress.o
obj-$(CONFIG_BCH) += bch.o
diff --git a/lib/decompress_unlz4.c b/lib/decompress_unlz4.c
new file mode 100644
index 0000000..894cc37
--- /dev/null
+++ b/lib/decompress_unlz4.c
@@ -0,0 +1,188 @@
+/*
+ * Wrapper for decompressing LZ4-compressed kernel, initramfs, and initrd
+ *
+ * Copyright (C) 2013, LG Electronics, Kyungsik Lee <kyungsik.lee@lge.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifdef STATIC
+#define PREBOOT
+#include <linux/decompress/mm.h>
+#include "lz4/lz4_decompress.c"
+#else
+#include <linux/decompress/unlz4.h>
+#include <malloc.h>
+#endif
+#include <linux/types.h>
+#include <linux/lz4.h>
+#include <linux/decompress/mm.h>
+#include <linux/compiler.h>
+
+#include <asm/unaligned.h>
+
+/*
+ * Note: Uncompressed chunk size is used in the compressor side
+ * (userspace side for compression).
+ * It is hardcoded because there is not proper way to extract it
+ * from the binary stream which is generated by the preliminary
+ * version of LZ4 tool so far.
+ */
+#define LZ4_DEFAULT_UNCOMPRESSED_CHUNK_SIZE (8 << 20)
+#define ARCHIVE_MAGICNUMBER 0x184C2102
+
+static inline int unlz4(u8 *input, int in_len,
+ int (*fill) (void *, unsigned int),
+ int (*flush) (void *, unsigned int),
+ u8 *output, int *posp,
+ void (*error) (char *x))
+{
+ int ret = -1;
+ size_t chunksize = 0;
+ size_t uncomp_chunksize = LZ4_DEFAULT_UNCOMPRESSED_CHUNK_SIZE;
+ u8 *inp;
+ u8 *inp_start;
+ u8 *outp;
+ int size = in_len;
+#ifdef PREBOOT
+ size_t out_len = get_unaligned_le32(input + in_len);
+#endif
+ size_t dest_len;
+
+
+ if (output) {
+ outp = output;
+ } else if (!flush) {
+ error("NULL output pointer and no flush function provided");
+ goto exit_0;
+ } else {
+ outp = malloc(uncomp_chunksize);
+ if (!outp) {
+ error("Could not allocate output buffer");
+ goto exit_0;
+ }
+ }
+
+ if (input && fill) {
+ error("Both input pointer and fill function provided,");
+ goto exit_1;
+ } else if (input) {
+ inp = input;
+ } else if (!fill) {
+ error("NULL input pointer and missing fill function");
+ goto exit_1;
+ } else {
+ inp = malloc(lz4_compressbound(uncomp_chunksize));
+ if (!inp) {
+ error("Could not allocate input buffer");
+ goto exit_1;
+ }
+ }
+ inp_start = inp;
+
+ if (posp)
+ *posp = 0;
+
+ if (fill)
+ fill(inp, 4);
+
+ chunksize = get_unaligned_le32(inp);
+ if (chunksize == ARCHIVE_MAGICNUMBER) {
+ inp += 4;
+ size -= 4;
+ } else {
+ error("invalid header");
+ goto exit_2;
+ }
+
+ if (posp)
+ *posp += 4;
+
+ for (;;) {
+
+ if (fill)
+ fill(inp, 4);
+
+ chunksize = get_unaligned_le32(inp);
+ if (chunksize == ARCHIVE_MAGICNUMBER) {
+ inp += 4;
+ size -= 4;
+ if (posp)
+ *posp += 4;
+ continue;
+ }
+ inp += 4;
+ size -= 4;
+
+ if (posp)
+ *posp += 4;
+
+ if (fill) {
+ if (chunksize > lz4_compressbound(uncomp_chunksize)) {
+ error("chunk length is longer than allocated");
+ goto exit_2;
+ }
+ fill(inp, chunksize);
+ }
+#ifdef PREBOOT
+ if (out_len >= uncomp_chunksize) {
+ dest_len = uncomp_chunksize;
+ out_len -= dest_len;
+ } else
+ dest_len = out_len;
+ ret = lz4_decompress(inp, &chunksize, outp, dest_len);
+#else
+ dest_len = uncomp_chunksize;
+ ret = lz4_decompress_unknownoutputsize(inp, chunksize, outp,
+ &dest_len);
+#endif
+ if (ret < 0) {
+ error("Decoding failed");
+ goto exit_2;
+ }
+
+ if (flush && flush(outp, dest_len) != dest_len)
+ goto exit_2;
+ if (output)
+ outp += dest_len;
+ if (posp)
+ *posp += chunksize;
+
+ size -= chunksize;
+
+ if (size == 0)
+ break;
+ else if (size < 0) {
+ error("data corrupted");
+ goto exit_2;
+ }
+
+ inp += chunksize;
+ if (fill)
+ inp = inp_start;
+ }
+
+ ret = 0;
+exit_2:
+ if (!input)
+ free(inp_start);
+exit_1:
+ if (!output)
+ free(outp);
+exit_0:
+ return ret;
+}
+
+int decompress_unlz4(unsigned char *buf, int in_len,
+ int(*fill)(void*, unsigned int),
+ int(*flush)(void*, unsigned int),
+ unsigned char *output,
+ int *posp,
+ void(*error)(char *x)
+ )
+{
+ return unlz4(buf, in_len - 4, fill, flush, output, posp, error);
+}
+#define decompress decompress_unlz4
diff --git a/lib/lz4/Makefile b/lib/lz4/Makefile
new file mode 100644
index 0000000..7f548c6
--- /dev/null
+++ b/lib/lz4/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_LZ4_DECOMPRESS) += lz4_decompress.o
diff --git a/lib/lz4/lz4_decompress.c b/lib/lz4/lz4_decompress.c
index d3414ea..8e64ce6 100644
--- a/lib/lz4/lz4_decompress.c
+++ b/lib/lz4/lz4_decompress.c
@@ -38,10 +38,11 @@
*/
#ifndef STATIC
-#include <linux/module.h>
+#include <module.h>
#include <linux/kernel.h>
#endif
#include <linux/lz4.h>
+#include <linux/string.h>
#include <asm/unaligned.h>
diff --git a/lib/uncompress.c b/lib/uncompress.c
index e0a69df..a4225aa 100644
--- a/lib/uncompress.c
+++ b/lib/uncompress.c
@@ -21,6 +21,7 @@
#include <bunzip2.h>
#include <gunzip.h>
#include <lzo.h>
+#include <linux/decompress/unlz4.h>
#include <errno.h>
#include <filetype.h>
#include <malloc.h>
@@ -112,6 +113,11 @@ int uncompress(unsigned char *inbuf, int len,
compfn = decompress_unlzo;
break;
#endif
+#ifdef CONFIG_LZ4_DECOMPRESS
+ case filetype_lz4_compressed:
+ compfn = decompress_unlz4;
+ break;
+#endif
default:
err = asprintf("cannot handle filetype %s", file_type_to_string(ft));
error_fn(err);
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 307412f..722b410 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -289,6 +289,11 @@ cmd_xzmisc = (cat $(filter-out FORCE,$^) | \
xz --check=crc32 --lzma2=dict=1MiB) > $@ || \
(rm -f $@ ; false)
+quiet_cmd_lz4 = LZ4 $@
+cmd_lz4 = (cat $(filter-out FORCE,$^) | \
+ lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
+ (rm -f $@ ; false)
+
quiet_cmd_disasm = DISASM $@
cmd_disasm = $(OBJDUMP) -d $< > $@
--
1.8.3.2
_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 3/3] pbl: add lz4 support
2013-07-12 9:22 ` [PATCH 1/3] decompressor: Add LZ4 decompressor module Jean-Christophe PLAGNIOL-VILLARD
2013-07-12 9:22 ` [PATCH 2/3] lib: Add support for LZ4-compressed kernel Jean-Christophe PLAGNIOL-VILLARD
@ 2013-07-12 9:22 ` Jean-Christophe PLAGNIOL-VILLARD
2013-07-15 9:00 ` Sascha Hauer
1 sibling, 1 reply; 7+ messages in thread
From: Jean-Christophe PLAGNIOL-VILLARD @ 2013-07-12 9:22 UTC (permalink / raw)
To: barebox
Signed-off-by: Jean-Christophe PLAGNIOL-VILLARD <plagnioj@jcrosoft.com>
---
arch/arm/pbl/Makefile | 3 ++-
arch/arm/pbl/piggy.lz4.S | 6 ++++++
arch/mips/pbl/Makefile | 3 ++-
arch/mips/pbl/piggy.lz4.S | 6 ++++++
images/Makefile | 1 +
pbl/Kconfig | 3 +++
pbl/decomp.c | 4 ++++
7 files changed, 24 insertions(+), 2 deletions(-)
create mode 100644 arch/arm/pbl/piggy.lz4.S
create mode 100644 arch/mips/pbl/piggy.lz4.S
diff --git a/arch/arm/pbl/Makefile b/arch/arm/pbl/Makefile
index 8923a70..dd3e946 100644
--- a/arch/arm/pbl/Makefile
+++ b/arch/arm/pbl/Makefile
@@ -1,6 +1,7 @@
suffix_$(CONFIG_IMAGE_COMPRESSION_GZIP) = gzip
suffix_$(CONFIG_IMAGE_COMPRESSION_LZO) = lzo
+suffix_$(CONFIG_IMAGE_COMPRESSION_LZ4) = lz4
suffix_$(CONFIG_IMAGE_COMPRESSION_NONE) = shipped
OBJCOPYFLAGS_zbarebox.bin = -O binary
@@ -10,7 +11,7 @@ targets := zbarebox.lds zbarebox zbarebox.bin zbarebox.S \
$(piggy_o) piggy.$(suffix_y)
# Make sure files are removed during clean
-extra-y += piggy.gzip piggy.lzo piggy.lzma piggy.xzkern piggy.shipped zbarebox.map
+extra-y += piggy.gzip piggy.lz4 piggy.lzo piggy.lzma piggy.xzkern piggy.shipped zbarebox.map
$(obj)/zbarebox.bin: $(obj)/zbarebox FORCE
$(call if_changed,objcopy)
diff --git a/arch/arm/pbl/piggy.lz4.S b/arch/arm/pbl/piggy.lz4.S
new file mode 100644
index 0000000..fa9b246
--- /dev/null
+++ b/arch/arm/pbl/piggy.lz4.S
@@ -0,0 +1,6 @@
+ .section .piggydata,#alloc
+ .globl input_data
+input_data:
+ .incbin "arch/arm/pbl/piggy.lz4"
+ .globl input_data_end
+input_data_end:
diff --git a/arch/mips/pbl/Makefile b/arch/mips/pbl/Makefile
index 6eeee73..fea1f24 100644
--- a/arch/mips/pbl/Makefile
+++ b/arch/mips/pbl/Makefile
@@ -1,6 +1,7 @@
suffix_$(CONFIG_IMAGE_COMPRESSION_GZIP) = gzip
suffix_$(CONFIG_IMAGE_COMPRESSION_LZO) = lzo
+suffix_$(CONFIG_IMAGE_COMPRESSION_LZ4) = lz4
suffix_$(CONFIG_IMAGE_COMPRESSION_NONE) = shipped
OBJCOPYFLAGS_zbarebox.bin = -O binary
@@ -10,7 +11,7 @@ targets := zbarebox.lds zbarebox zbarebox.bin zbarebox.S \
$(piggy_o) piggy.$(suffix_y)
# Make sure files are removed during clean
-extra-y += piggy.gzip piggy.lzo piggy.lzma piggy.xzkern piggy.shipped zbarebox.map
+extra-y += piggy.gzip piggy.lz4 piggy.lzo piggy.lzma piggy.xzkern piggy.shipped zbarebox.map
$(obj)/zbarebox.bin: $(obj)/zbarebox FORCE
$(call if_changed,objcopy)
diff --git a/arch/mips/pbl/piggy.lz4.S b/arch/mips/pbl/piggy.lz4.S
new file mode 100644
index 0000000..6f1af1f
--- /dev/null
+++ b/arch/mips/pbl/piggy.lz4.S
@@ -0,0 +1,6 @@
+#include <asm/asm.h>
+
+ .section .data
+EXPORT(input_data)
+ .incbin "arch/mips/pbl/piggy.lzo"
+EXPORT(input_data_end)
diff --git a/images/Makefile b/images/Makefile
index 925a987..65c533a 100644
--- a/images/Makefile
+++ b/images/Makefile
@@ -81,6 +81,7 @@ $(obj)/%.s: $(obj)/% FORCE
suffix_$(CONFIG_IMAGE_COMPRESSION_GZIP) = gzip
suffix_$(CONFIG_IMAGE_COMPRESSION_LZO) = lzo
+suffix_$(CONFIG_IMAGE_COMPRESSION_LZ4) = lz4
suffix_$(CONFIG_IMAGE_COMPRESSION_NONE) = shipped
# barebox.z - compressed barebox binary
diff --git a/pbl/Kconfig b/pbl/Kconfig
index a37c976..836060e 100644
--- a/pbl/Kconfig
+++ b/pbl/Kconfig
@@ -54,6 +54,9 @@ if IMAGE_COMPRESSION
choice
prompt "Compression"
+config IMAGE_COMPRESSION_LZ4
+ bool "lz4"
+
config IMAGE_COMPRESSION_LZO
bool "lzo"
diff --git a/pbl/decomp.c b/pbl/decomp.c
index aa6a31e..ca0df64 100644
--- a/pbl/decomp.c
+++ b/pbl/decomp.c
@@ -10,6 +10,10 @@
#define STATIC static
+#ifdef CONFIG_IMAGE_COMPRESSION_LZ4
+#include "../../../lib/decompress_unlz4.c"
+#endif
+
#ifdef CONFIG_IMAGE_COMPRESSION_LZO
#include "../../../lib/decompress_unlzo.c"
#endif
--
1.8.3.2
_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 3/3] pbl: add lz4 support
2013-07-12 9:22 ` [PATCH 3/3] pbl: add lz4 support Jean-Christophe PLAGNIOL-VILLARD
@ 2013-07-15 9:00 ` Sascha Hauer
2013-07-15 10:08 ` Jean-Christophe PLAGNIOL-VILLARD
0 siblings, 1 reply; 7+ messages in thread
From: Sascha Hauer @ 2013-07-15 9:00 UTC (permalink / raw)
To: Jean-Christophe PLAGNIOL-VILLARD; +Cc: barebox
On Fri, Jul 12, 2013 at 11:22:39AM +0200, Jean-Christophe PLAGNIOL-VILLARD wrote:
> @@ -54,6 +54,9 @@ if IMAGE_COMPRESSION
> choice
> prompt "Compression"
>
> +config IMAGE_COMPRESSION_LZ4
> + bool "lz4"
> +
> config IMAGE_COMPRESSION_LZO
> bool "lzo"
Can we add this in a way that it doesn't get the default for all configs
currently using lzo? Either put it at the end of the choice or add a
'default IMAGE_COMPRESSION_LZO'?
Sascha
--
Pengutronix e.K. | |
Industrial Linux Solutions | http://www.pengutronix.de/ |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 |
Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 3/3] pbl: add lz4 support
2013-07-15 9:00 ` Sascha Hauer
@ 2013-07-15 10:08 ` Jean-Christophe PLAGNIOL-VILLARD
0 siblings, 0 replies; 7+ messages in thread
From: Jean-Christophe PLAGNIOL-VILLARD @ 2013-07-15 10:08 UTC (permalink / raw)
To: Sascha Hauer; +Cc: barebox
On 11:00 Mon 15 Jul , Sascha Hauer wrote:
> On Fri, Jul 12, 2013 at 11:22:39AM +0200, Jean-Christophe PLAGNIOL-VILLARD wrote:
> > @@ -54,6 +54,9 @@ if IMAGE_COMPRESSION
> > choice
> > prompt "Compression"
> >
> > +config IMAGE_COMPRESSION_LZ4
> > + bool "lz4"
> > +
> > config IMAGE_COMPRESSION_LZO
> > bool "lzo"
>
> Can we add this in a way that it doesn't get the default for all configs
> currently using lzo? Either put it at the end of the choice or add a
> 'default IMAGE_COMPRESSION_LZO'?
yeah this default will do the tric
I plan to update the LZO too as we can get 10 to 20% decompression speed up
Best Regards,
J.
>
> Sascha
>
> --
> Pengutronix e.K. | |
> Industrial Linux Solutions | http://www.pengutronix.de/ |
> Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 |
> Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 2/3] lib: Add support for LZ4-compressed kernel
2013-07-15 10:20 ` [PATCH 1/3] decompressor: Add LZ4 decompressor module Jean-Christophe PLAGNIOL-VILLARD
@ 2013-07-15 10:20 ` Jean-Christophe PLAGNIOL-VILLARD
0 siblings, 0 replies; 7+ messages in thread
From: Jean-Christophe PLAGNIOL-VILLARD @ 2013-07-15 10:20 UTC (permalink / raw)
To: barebox; +Cc: Kyungsik Lee
From: Kyungsik Lee <kyungsik.lee@lge.com>
This patch adds support for extracting LZ4-compressed kernel images,
as well as LZ4-compressed ramdisk images in the kernel boot process.
This depends on the patch below
decompressor: Add LZ4 decompressor module
Signed-off-by: Kyungsik Lee <kyungsik.lee@lge.com>
Signed-off-by: Jean-Christophe PLAGNIOL-VILLARD <plagnioj@jcrosoft.com>
---
common/filetype.c | 4 +
include/filetype.h | 1 +
include/linux/decompress/unlz4.h | 10 +++
lib/Kconfig | 4 +
lib/Makefile | 2 +
lib/decompress_unlz4.c | 188 +++++++++++++++++++++++++++++++++++++++
lib/lz4/Makefile | 1 +
lib/lz4/lz4_decompress.c | 3 +-
lib/uncompress.c | 6 ++
scripts/Makefile.lib | 5 ++
10 files changed, 223 insertions(+), 1 deletion(-)
create mode 100644 include/linux/decompress/unlz4.h
create mode 100644 lib/decompress_unlz4.c
create mode 100644 lib/lz4/Makefile
diff --git a/common/filetype.c b/common/filetype.c
index 1ff3dd2..13c7994 100644
--- a/common/filetype.c
+++ b/common/filetype.c
@@ -33,6 +33,7 @@ static const struct filetype_str filetype_str[] = {
[filetype_unknown] = { "unknown", "unkown" },
[filetype_arm_zimage] = { "arm Linux zImage", "arm-zimage" },
[filetype_lzo_compressed] = { "lzo compressed", "lzo" },
+ [filetype_lz4_compressed] = { "lz4 compressed", "lz4" },
[filetype_arm_barebox] = { "arm barebox image", "arm-barebox" },
[filetype_uimage] = { "U-Boot uImage", "u-boot" },
[filetype_ubi] = { "UBI image", "ubi" },
@@ -195,6 +196,9 @@ enum filetype file_detect_type(const void *_buf, size_t bufsize)
if (buf8[0] == 0x89 && buf8[1] == 0x4c && buf8[2] == 0x5a &&
buf8[3] == 0x4f)
return filetype_lzo_compressed;
+ if (buf8[0] == 0x02 && buf8[1] == 0x21 && buf8[2] == 0x4c &&
+ buf8[3] == 0x18)
+ return filetype_lz4_compressed;
if (buf[0] == be32_to_cpu(0x27051956))
return filetype_uimage;
if (buf[0] == 0x23494255)
diff --git a/include/filetype.h b/include/filetype.h
index c73c64a..1b90805 100644
--- a/include/filetype.h
+++ b/include/filetype.h
@@ -10,6 +10,7 @@ enum filetype {
filetype_unknown,
filetype_arm_zimage,
filetype_lzo_compressed,
+ filetype_lz4_compressed,
filetype_arm_barebox,
filetype_uimage,
filetype_ubi,
diff --git a/include/linux/decompress/unlz4.h b/include/linux/decompress/unlz4.h
new file mode 100644
index 0000000..7aaafc2
--- /dev/null
+++ b/include/linux/decompress/unlz4.h
@@ -0,0 +1,10 @@
+#ifndef DECOMPRESS_UNLZ4_H
+#define DECOMPRESS_UNLZ4_H
+
+int decompress_unlz4(unsigned char *inbuf, int len,
+ int(*fill)(void*, unsigned int),
+ int(*flush)(void*, unsigned int),
+ unsigned char *output,
+ int *pos,
+ void(*error)(char *x));
+#endif
diff --git a/lib/Kconfig b/lib/Kconfig
index 646fdb7..b5ac22a 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -14,6 +14,10 @@ config BZLIB
bool "include bzip2 uncompression support"
select UNCOMPRESS
+config LZ4_DECOMPRESS
+ bool "include lz4 uncompression support"
+ select UNCOMPRESS
+
config GENERIC_FIND_NEXT_BIT
def_bool n
diff --git a/lib/Makefile b/lib/Makefile
index 7c42537..61f1e63 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -28,8 +28,10 @@ obj-y += notifier.o
obj-y += copy_file.o
obj-y += random.o
obj-y += lzo/
+obj-$(CONFIG_LZ4_DECOMPRESS) += lz4/
obj-y += show_progress.o
obj-$(CONFIG_LZO_DECOMPRESS) += decompress_unlzo.o
+obj-$(CONFIG_LZ4_DECOMPRESS) += decompress_unlz4.o
obj-$(CONFIG_PROCESS_ESCAPE_SEQUENCE) += process_escape_sequence.o
obj-$(CONFIG_UNCOMPRESS) += uncompress.o
obj-$(CONFIG_BCH) += bch.o
diff --git a/lib/decompress_unlz4.c b/lib/decompress_unlz4.c
new file mode 100644
index 0000000..894cc37
--- /dev/null
+++ b/lib/decompress_unlz4.c
@@ -0,0 +1,188 @@
+/*
+ * Wrapper for decompressing LZ4-compressed kernel, initramfs, and initrd
+ *
+ * Copyright (C) 2013, LG Electronics, Kyungsik Lee <kyungsik.lee@lge.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifdef STATIC
+#define PREBOOT
+#include <linux/decompress/mm.h>
+#include "lz4/lz4_decompress.c"
+#else
+#include <linux/decompress/unlz4.h>
+#include <malloc.h>
+#endif
+#include <linux/types.h>
+#include <linux/lz4.h>
+#include <linux/decompress/mm.h>
+#include <linux/compiler.h>
+
+#include <asm/unaligned.h>
+
+/*
+ * Note: Uncompressed chunk size is used in the compressor side
+ * (userspace side for compression).
+ * It is hardcoded because there is not proper way to extract it
+ * from the binary stream which is generated by the preliminary
+ * version of LZ4 tool so far.
+ */
+#define LZ4_DEFAULT_UNCOMPRESSED_CHUNK_SIZE (8 << 20)
+#define ARCHIVE_MAGICNUMBER 0x184C2102
+
+static inline int unlz4(u8 *input, int in_len,
+ int (*fill) (void *, unsigned int),
+ int (*flush) (void *, unsigned int),
+ u8 *output, int *posp,
+ void (*error) (char *x))
+{
+ int ret = -1;
+ size_t chunksize = 0;
+ size_t uncomp_chunksize = LZ4_DEFAULT_UNCOMPRESSED_CHUNK_SIZE;
+ u8 *inp;
+ u8 *inp_start;
+ u8 *outp;
+ int size = in_len;
+#ifdef PREBOOT
+ size_t out_len = get_unaligned_le32(input + in_len);
+#endif
+ size_t dest_len;
+
+
+ if (output) {
+ outp = output;
+ } else if (!flush) {
+ error("NULL output pointer and no flush function provided");
+ goto exit_0;
+ } else {
+ outp = malloc(uncomp_chunksize);
+ if (!outp) {
+ error("Could not allocate output buffer");
+ goto exit_0;
+ }
+ }
+
+ if (input && fill) {
+ error("Both input pointer and fill function provided,");
+ goto exit_1;
+ } else if (input) {
+ inp = input;
+ } else if (!fill) {
+ error("NULL input pointer and missing fill function");
+ goto exit_1;
+ } else {
+ inp = malloc(lz4_compressbound(uncomp_chunksize));
+ if (!inp) {
+ error("Could not allocate input buffer");
+ goto exit_1;
+ }
+ }
+ inp_start = inp;
+
+ if (posp)
+ *posp = 0;
+
+ if (fill)
+ fill(inp, 4);
+
+ chunksize = get_unaligned_le32(inp);
+ if (chunksize == ARCHIVE_MAGICNUMBER) {
+ inp += 4;
+ size -= 4;
+ } else {
+ error("invalid header");
+ goto exit_2;
+ }
+
+ if (posp)
+ *posp += 4;
+
+ for (;;) {
+
+ if (fill)
+ fill(inp, 4);
+
+ chunksize = get_unaligned_le32(inp);
+ if (chunksize == ARCHIVE_MAGICNUMBER) {
+ inp += 4;
+ size -= 4;
+ if (posp)
+ *posp += 4;
+ continue;
+ }
+ inp += 4;
+ size -= 4;
+
+ if (posp)
+ *posp += 4;
+
+ if (fill) {
+ if (chunksize > lz4_compressbound(uncomp_chunksize)) {
+ error("chunk length is longer than allocated");
+ goto exit_2;
+ }
+ fill(inp, chunksize);
+ }
+#ifdef PREBOOT
+ if (out_len >= uncomp_chunksize) {
+ dest_len = uncomp_chunksize;
+ out_len -= dest_len;
+ } else
+ dest_len = out_len;
+ ret = lz4_decompress(inp, &chunksize, outp, dest_len);
+#else
+ dest_len = uncomp_chunksize;
+ ret = lz4_decompress_unknownoutputsize(inp, chunksize, outp,
+ &dest_len);
+#endif
+ if (ret < 0) {
+ error("Decoding failed");
+ goto exit_2;
+ }
+
+ if (flush && flush(outp, dest_len) != dest_len)
+ goto exit_2;
+ if (output)
+ outp += dest_len;
+ if (posp)
+ *posp += chunksize;
+
+ size -= chunksize;
+
+ if (size == 0)
+ break;
+ else if (size < 0) {
+ error("data corrupted");
+ goto exit_2;
+ }
+
+ inp += chunksize;
+ if (fill)
+ inp = inp_start;
+ }
+
+ ret = 0;
+exit_2:
+ if (!input)
+ free(inp_start);
+exit_1:
+ if (!output)
+ free(outp);
+exit_0:
+ return ret;
+}
+
+int decompress_unlz4(unsigned char *buf, int in_len,
+ int(*fill)(void*, unsigned int),
+ int(*flush)(void*, unsigned int),
+ unsigned char *output,
+ int *posp,
+ void(*error)(char *x)
+ )
+{
+ return unlz4(buf, in_len - 4, fill, flush, output, posp, error);
+}
+#define decompress decompress_unlz4
diff --git a/lib/lz4/Makefile b/lib/lz4/Makefile
new file mode 100644
index 0000000..7f548c6
--- /dev/null
+++ b/lib/lz4/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_LZ4_DECOMPRESS) += lz4_decompress.o
diff --git a/lib/lz4/lz4_decompress.c b/lib/lz4/lz4_decompress.c
index d3414ea..8e64ce6 100644
--- a/lib/lz4/lz4_decompress.c
+++ b/lib/lz4/lz4_decompress.c
@@ -38,10 +38,11 @@
*/
#ifndef STATIC
-#include <linux/module.h>
+#include <module.h>
#include <linux/kernel.h>
#endif
#include <linux/lz4.h>
+#include <linux/string.h>
#include <asm/unaligned.h>
diff --git a/lib/uncompress.c b/lib/uncompress.c
index e0a69df..a4225aa 100644
--- a/lib/uncompress.c
+++ b/lib/uncompress.c
@@ -21,6 +21,7 @@
#include <bunzip2.h>
#include <gunzip.h>
#include <lzo.h>
+#include <linux/decompress/unlz4.h>
#include <errno.h>
#include <filetype.h>
#include <malloc.h>
@@ -112,6 +113,11 @@ int uncompress(unsigned char *inbuf, int len,
compfn = decompress_unlzo;
break;
#endif
+#ifdef CONFIG_LZ4_DECOMPRESS
+ case filetype_lz4_compressed:
+ compfn = decompress_unlz4;
+ break;
+#endif
default:
err = asprintf("cannot handle filetype %s", file_type_to_string(ft));
error_fn(err);
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 307412f..722b410 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -289,6 +289,11 @@ cmd_xzmisc = (cat $(filter-out FORCE,$^) | \
xz --check=crc32 --lzma2=dict=1MiB) > $@ || \
(rm -f $@ ; false)
+quiet_cmd_lz4 = LZ4 $@
+cmd_lz4 = (cat $(filter-out FORCE,$^) | \
+ lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
+ (rm -f $@ ; false)
+
quiet_cmd_disasm = DISASM $@
cmd_disasm = $(OBJDUMP) -d $< > $@
--
1.8.3.2
_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2013-07-15 10:21 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-07-12 9:14 [PATCH 0/3] pbl: add lz4 compression support Jean-Christophe PLAGNIOL-VILLARD
2013-07-12 9:22 ` [PATCH 1/3] decompressor: Add LZ4 decompressor module Jean-Christophe PLAGNIOL-VILLARD
2013-07-12 9:22 ` [PATCH 2/3] lib: Add support for LZ4-compressed kernel Jean-Christophe PLAGNIOL-VILLARD
2013-07-12 9:22 ` [PATCH 3/3] pbl: add lz4 support Jean-Christophe PLAGNIOL-VILLARD
2013-07-15 9:00 ` Sascha Hauer
2013-07-15 10:08 ` Jean-Christophe PLAGNIOL-VILLARD
2013-07-15 10:09 [PATCH 0/3 v2] pbl: add lz4 compression support Jean-Christophe PLAGNIOL-VILLARD
2013-07-15 10:20 ` [PATCH 1/3] decompressor: Add LZ4 decompressor module Jean-Christophe PLAGNIOL-VILLARD
2013-07-15 10:20 ` [PATCH 2/3] lib: Add support for LZ4-compressed kernel Jean-Christophe PLAGNIOL-VILLARD
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox