Skip to contentSkip to conversation

package/libmdbx: new package (library/database).

PATCH Superseded Leonid Yuriev 3 participants 15 messages Oct 2, 2020 – Nov 27, 2020

v2 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v2 cover package/libmdbx: new package (library/database).

This patch adds libmdbx v0.9.1 and below is a brief overview of libmdbx.

Please merge.

Regards,
Leonid.

--

libmdbx is an extremely fast, compact, powerful, embedded, transactional key-value database, with permissive license. libmdbx has a specific set of properties and capabilities, focused on creating unique lightweight solutions.

Historically, libmdbx (MDBX) is a deeply revised and extended descendant of the legendary LMDB (Lightning Memory-Mapped Database). libmdbx inherits all benefits from LMDB, but resolves some issues and adds a set of improvements.

According to developers, for now libmdbx surpasses the LMDB in terms of reliability, features and performance.

The most important differences MDBX from LMDB:
==============================================

1) More attention is paid to the quality of the code, to an "unbreakability" of the API, to testing and automatic checks (i.e. sanitizers, etc). So there:
- more control during operation;
- more checking parameters, internal audit of database structures;
- no warnings from compiler;
- no issues from ASAN, UBSAN, Valgrind, Coverity;
- etc.

2) Keys could be more than 2 times longer than LMDB.

3) Up to 20% faster than??LMDB??in??CRUD??benchmarks.

4) Automatic on-the-fly database size adjustment,
both increment and reduction.

5) Automatic continuous zero-overhead database compactification.

6) The same database format for 32- and 64-bit builds.

7) LIFO policy for Garbage Collection recycling (this can significantly increase write performance due write-back disk cache up to several times in a best case scenario).

8) Range query estimation.

9) Utility for checking the integrity of the database structure with some recovery capabilities.

For more info please refer:
- https://github.com/erthink/libmdbx for source code and README.
- https://erthink.github.io/libmdbx for API description.

Leonid Yuriev (1):
package/libmdbx: new package (library/database).

 DEVELOPERS                   |  3 ++
 package/Config.in            |  1 +
 package/libmdbx/Config.in    | 35 ++++++++++++++++++++++
 package/libmdbx/libmdbx.hash |  2 ++
 package/libmdbx/libmdbx.mk   | 57 ++++++++++++++++++++++++++++++++++++
 5 files changed, 98 insertions(+)
 create mode 100644 package/libmdbx/Config.in
 create mode 100644 package/libmdbx/libmdbx.hash
 create mode 100644 package/libmdbx/libmdbx.mk

--

MDBX is a Btree-based database management library modeled loosely on the BerkeleyDB API, but much simplified. The entire database (aka "environment") is exposed in a memory map, and all data fetches return data directly from the mapped memory, so no malloc's or memcpy's occur during data fetches. As such, the library is extremely simple because it requires no page caching layer of its own, and it is extremely high performance and memory-efficient. It is also fully transactional with full ACID semantics, and when the memory map is read-only, the database integrity cannot be corrupted by stray pointer writes from application code.

The library is fully thread-aware and supports concurrent read/write access from multiple processes and threads. Data pages use a copy-on-write strategy so no active data pages are ever overwritten, which also provides resistance to corruption and eliminates the need of any special recovery procedures after a system crash. Writes are fully serialized; only one write transaction may be active at a time, which guarantees that writers can never deadlock. The database structure is multi-versioned so readers run with no locks; writers cannot block readers, and readers don't block writers.

Unlike other well-known database mechanisms which use either write-ahead transaction logs or append-only data writes, MDBX requires no maintenance during operation. Both write-ahead loggers and append-only databases require periodic checkpointing and/or compaction of their log or database files otherwise they grow without bound. MDBX tracks retired/freed pages within the database and re-uses them for new write operations, so the database size does not grow without bound in normal use.

The memory map can be used as a read-only or read-write map. It is read-only by default as this provides total immunity to corruption. Using read-write mode offers much higher write performance, but adds the possibility for stray application writes thru pointers to silently corrupt the database.

Features
========

- Key-value data model, keys are always sorted.

- Fully ACID-compliant, through to MVCC and CoW.

- Multiple key-value sub-databases within a single datafile.

- Range lookups, including range query estimation.

- Efficient support for short fixed length keys, including native 32/64-bit integers.

- Ultra-efficient support for multimaps. Multi-values sorted, searchable and iterable. Keys stored without duplication.

- Data is memory-mapped and accessible directly/zero-copy. Traversal of database records is extremely-fast.

- Transactions for readers and writers, ones do not block others.

- Writes are strongly serialized. No transaction conflicts nor deadlocks.

- Readers are non-blocking, notwithstanding snapshot isolation.

- Nested write transactions.

- Reads scale linearly across CPUs.

- Continuous zero-overhead database compactification.

- Automatic on-the-fly database size adjustment.

- Customizable database page size.

- Olog(N) cost of lookup, insert, update, and delete operations by virtue of B+ tree characteristics.

- Online hot backup.

- Append operation for efficient bulk insertion of pre-sorted data.

- No WAL nor any transaction journal. No crash recovery needed. No maintenance is required.

- No internal cache and/or memory management, all done by basic OS services.

Limitations
===========

- Page size: a power of 2, maximum 65536 bytes, default 4096 bytes.

- Key size: minimum 0, maximum ????? pagesize (1300 bytes for default 4K pagesize, 21780 bytes for 64K pagesize).

- Value size: minimum 0, maximum 2146435072 (0x7FF00000) bytes for maps, ????? pagesize for multimaps (1348 bytes default 4K pagesize, 21828 bytes for 64K pagesize).

- Write transaction size: up to 4194301 (0x3FFFFD) pages (16 GiB for default 4K pagesize, 256 GiB for 64K pagesize).

- Database size: up to 2147483648 pages (8 TiB for default 4K pagesize, 128 TiB for 64K pagesize).

- Maximum sub-databases: 32765.

Gotchas
=======

- There cannot be more than one writer at a time, i.e. no more than one write transaction at a time.

- libmdbx is based on B+ tree, so access to database pages is mostly random. Thus SSDs provide a significant performance boost over spinning disks for large databases.

- libmdbx uses shadow paging instead of WAL. Thus syncing data to disk might be a bottleneck for write intensive workload.

- libmdbx uses copy-on-write for snapshot isolation during updates, but read transactions prevents recycling an old retired/freed pages, since it read ones. Thus altering of data during a parallel long-lived read operation will increase the process work set, may exhaust entire free database space, the database can grow quickly, and result in performance degradation. Try to avoid long running read transactions.

- libmdbx is extraordinarily fast and provides minimal overhead for data access, so you should reconsider using brute force techniques and double check your code. On the one hand, in the case of libmdbx, a simple linear search may be more profitable than complex indexes. On the other hand, if you make something suboptimally, you can notice detrimentally only on sufficiently large data.

#1 lore raw

PATCH v2 1/1 package/libmdbx: new package (library/database).

This patch adds libmdbx v0.9.1:
- libmdbx is one of the fastest compact embeddable key-value ACID database.
- libmdbx has a specific set of properties and capabilities, focused on creating unique lightweight solutions.
- libmdbx surpasses the legendary LMDB (Lightning Memory-Mapped Database) in terms of reliability, features and performance.

https://github.com/erthink/libmdbx

---
Changes v1 -> v2:
- libmdbx version v0.8.2 -> v0.9.1 (released 2020-09-30)

---
 DEVELOPERS                   |  3 ++
 package/Config.in            |  1 +
 package/libmdbx/Config.in    | 35 ++++++++++++++++++++++
 package/libmdbx/libmdbx.hash |  2 ++
 package/libmdbx/libmdbx.mk   | 57 ++++++++++++++++++++++++++++++++++++
 5 files changed, 98 insertions(+)
 create mode 100644 package/libmdbx/Config.in
 create mode 100644 package/libmdbx/libmdbx.hash
 create mode 100644 package/libmdbx/libmdbx.mk
DEVELOPERS +3 -0
index 9352d891d3..301372fc40 100644
@@ -1544,6 +1544,9 @@ N:	Leon Anavi <[email protected]>
 F:	board/olimex/a10_olinuxino
 F:	configs/olimex_a10_olinuxino_lime_defconfig
 
+N:	Leonid Yuriev <[email protected]>
+F:	package/libmdbx
+
 N:	Lionel Flandrin <[email protected]>
 F:	package/python-babel/
 F:	package/python-daemonize/
package/Config.in +1 -0
index 51583d07d6..f87e0388a0 100644
@@ -1370,6 +1370,7 @@ menu "Database"
 	source "package/kompexsqlite/Config.in"
 	source "package/leveldb/Config.in"
 	source "package/libgit2/Config.in"
+	source "package/libmdbx/Config.in"
 	source "package/libodb/Config.in"
 	source "package/libodb-boost/Config.in"
 	source "package/libodb-mysql/Config.in"
package/libmdbx/Config.in new file +35 -0
new file mode 100644
index 0000000000..7d2d321095
@@ -0,0 +1,35 @@
+config BR2_PACKAGE_LIBMDBX
+	bool "libmdbx"
+	depends on BR2_USE_MMU
+	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8
+	depends on BR2_TOOLCHAIN_HAS_SYNC_4
+	help
+	  One of the fastest compact key-value ACID database
+	  without WAL. libmdbx has a specific set of properties
+	  and capabilities, focused on creating unique lightweight
+	  solutions.
+
+	  libmdbx surpasses the legendary legendary LMDB in terms of
+	  reliability, features and performance.
+
+	  https://github.com/erthink/libmdbx
+
+if BR2_PACKAGE_LIBMDBX
+
+config BR2_PACKAGE_LIBMDBX_TOOLS
+	bool "install tools"
+	help
+	  Install libmdbx tools for checking, dump, restore
+	  and show statistics of databases.
+
+# C++ API is preliminary for now
+#config BR2_PACKAGE_LIBMDBX_CXX
+#	bool "C++ API"
+#	select BR2_INSTALL_LIBSTDCPP
+#	help
+#	  Enable modern C++ API for libmdbx.
+
+endif
+
+comment "libmdbx needs a toolchain w/ threads"
+	depends on !BR2_TOOLCHAIN_HAS_THREADS
package/libmdbx/libmdbx.hash new file +2 -0
new file mode 100644
index 0000000000..f48cd81a7d
@@ -0,0 +1,2 @@
+sha256  310fe25c858a9515fc8c8d7d1f24a67c9496f84a91e0a0e41ea9975b1371e569  LICENSE
+sha256  c7fb24381eb4d92f2e2edc17e577cb721269683c816c6cca307c58f6f346e786  libmdbx-amalgamated-0.9.1.tar.gz
package/libmdbx/libmdbx.mk new file +57 -0

57 lines not on this page · raw message

Signature · 1 line
2.28.0
#2 lore raw

PATCH v2 1/1 package/libmdbx: new package (library/database).

Hi Leonid,

Thank you for this contribution.

Below are some comments.

Am Fr., 2. Okt. 2020 um 13:22 Uhr schrieb Leonid Yuriev <[email protected]>:

Quoted 80 lines

This patch adds libmdbx v0.9.1:
- libmdbx is one of the fastest compact embeddable key-value ACID database.
- libmdbx has a specific set of properties and capabilities, focused on creating unique lightweight solutions.
- libmdbx surpasses the legendary LMDB (Lightning Memory-Mapped Database) in terms of reliability, features and performance.

https://github.com/erthink/libmdbx

---
Changes v1 -> v2:
- libmdbx version v0.8.2 -> v0.9.1 (released 2020-09-30)

---
 DEVELOPERS                   |  3 ++
 package/Config.in            |  1 +
 package/libmdbx/Config.in    | 35 ++++++++++++++++++++++
 package/libmdbx/libmdbx.hash |  2 ++
 package/libmdbx/libmdbx.mk   | 57 ++++++++++++++++++++++++++++++++++++
 5 files changed, 98 insertions(+)
 create mode 100644 package/libmdbx/Config.in
 create mode 100644 package/libmdbx/libmdbx.hash
 create mode 100644 package/libmdbx/libmdbx.mk
DEVELOPERS +3 -0
index 9352d891d3..301372fc40 100644
@@ -1544,6 +1544,9 @@ N:        Leon Anavi <[email protected]>
 F:     board/olimex/a10_olinuxino
 F:     configs/olimex_a10_olinuxino_lime_defconfig
 
+N:     Leonid Yuriev <[email protected]>
+F:     package/libmdbx
+
 N:     Lionel Flandrin <[email protected]>
 F:     package/python-babel/
 F:     package/python-daemonize/
package/Config.in +1 -0
index 51583d07d6..f87e0388a0 100644
@@ -1370,6 +1370,7 @@ menu "Database"
        source "package/kompexsqlite/Config.in"
        source "package/leveldb/Config.in"
        source "package/libgit2/Config.in"
+       source "package/libmdbx/Config.in"
        source "package/libodb/Config.in"
        source "package/libodb-boost/Config.in"
        source "package/libodb-mysql/Config.in"
package/libmdbx/Config.in new file +20 -0
new file mode 100644
index 0000000000..7d2d321095
@@ -0,0 +1,35 @@
+config BR2_PACKAGE_LIBMDBX
+       bool "libmdbx"
+       depends on BR2_USE_MMU
+       depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8
+       depends on BR2_TOOLCHAIN_HAS_SYNC_4
+       help
+         One of the fastest compact key-value ACID database
+         without WAL. libmdbx has a specific set of properties
+         and capabilities, focused on creating unique lightweight
+         solutions.
+
+         libmdbx surpasses the legendary legendary LMDB in terms of
+         reliability, features and performance.
+
+         https://github.com/erthink/libmdbx
+
+if BR2_PACKAGE_LIBMDBX
+
+config BR2_PACKAGE_LIBMDBX_TOOLS
+       bool "install tools"

package/libmdbx/Config.in line 30

Quoted 10 lines
package/libmdbx/Config.in new file +10 -0
+       help
+         Install libmdbx tools for checking, dump, restore
+         and show statistics of databases.
+
+# C++ API is preliminary for now
+#config BR2_PACKAGE_LIBMDBX_CXX
+#      bool "C++ API"
+#      select BR2_INSTALL_LIBSTDCPP
+#      help
+#        Enable modern C++ API for libmdbx.

You should not keep this outcommented stuff here.

Quoted 19 lines
package/libmdbx/Config.in new file +5 -0
+
+endif
+
+comment "libmdbx needs a toolchain w/ threads"
+       depends on !BR2_TOOLCHAIN_HAS_THREADS
package/libmdbx/libmdbx.hash new file +2 -0
new file mode 100644
index 0000000000..f48cd81a7d
@@ -0,0 +1,2 @@
+sha256  310fe25c858a9515fc8c8d7d1f24a67c9496f84a91e0a0e41ea9975b1371e569  LICENSE
+sha256  c7fb24381eb4d92f2e2edc17e577cb721269683c816c6cca307c58f6f346e786  libmdbx-amalgamated-0.9.1.tar.gz
package/libmdbx/libmdbx.mk new file +0 -0
new file mode 100644
index 0000000000..894b7f0838

package/libmdbx/libmdbx.mk line 8

Quoted 8 lines
package/libmdbx/libmdbx.mk new file +8 -0
@@ -0,0 +1,57 @@
+################################################################################
+#
+# libmdbx
+#
+# One of the fastest embeddable key-value ACID database without WAL. libmdbx is
+# superior to legendary LMDB in terms of reliability, features and performance.
+#
+################################################################################

The header of the makefile should only hold the name of the package without the description.

package/libmdbx/libmdbx.mk line 20

Quoted 12 lines
package/libmdbx/libmdbx.mk new file +12 -0
+
+LIBMDBX_VERSION = 0.9.1
+LIBMDBX_SOURCE = libmdbx-amalgamated-$(LIBMDBX_VERSION).tar.gz
+LIBMDBX_SITE = https://github.com/erthink/libmdbx/releases/download/v$(LIBMDBX_VERSION)
+LIBMDBX_SUPPORTS_IN_SOURCE_BUILD = NO
+LIBMDBX_LICENSE = OLDAP-2.8
+LIBMDBX_LICENSE_FILES = LICENSE
+LIBMDBX_REDISTRIBUTE = YES
+LIBMDBX_STRIP_COMPONENTS = 0
+LIBMDBX_INSTALL_STAGING = YES
+
+LIBMDBX_CONF_OPTS = -DMDBX_INSTALL_MANPAGES:BOOL=OFF -DBUILD_FOR_NATIVE_CPU:BOOL=OFF

You should omit the type stuff for the cmake config, like ":BOOL".

package/libmdbx/libmdbx.mk line 27

Quoted 7 lines
package/libmdbx/libmdbx.mk new file +7 -0
+
+# C++ API is preliminary for now
+#ifeq ($(BR2_PACKAGE_LIBMDBX_CXX),y)
+#LIBMDBX_CONF_OPTS += -DMDBX_BUILD_CXX:BOOL=ON
+#else
+#LIBMDBX_CONF_OPTS += -DMDBX_BUILD_CXX:BOOL=OFF
+#endif

Same here .. no out commented stuff.

package/libmdbx/libmdbx.mk line 33

Quoted 6 lines
package/libmdbx/libmdbx.mk new file +6 -0
+
+ifeq ($(BR2_STATIC_LIBS),y)
+LIBMDBX_CONF_OPTS += -DMDBX_INSTALL_STATIC:BOOL=ON
+else
+LIBMDBX_CONF_OPTS += -DMDBX_INSTALL_STATIC:BOOL=OFF
+endif

This can be:

LIBMDBX_CONF_OPTS += \
    -DMDBX_INSTALL_STATIC=$(if $(BR2_STATIC_LIBS),ON,OFF)

The same pattern also for DMDBX_BUILD_TOOLS

LIBMDBX_CONF_OPTS += \
    -DMDBX_BUILD_TOOLS=$(if $(BR2_PACKAGE_LIBMDBX_TOOLS),ON,OFF)

package/libmdbx/libmdbx.mk line 37

Quoted 4 lines
package/libmdbx/libmdbx.mk new file +4 -0
+
+ifeq ($(BR2_SHARED_LIBS),y)
+LIBMDBX_CONF_OPTS += -DMDBX_BUILD_SHARED_LIBRARY:BOOL=ON
+LIBMDBX_INSTALL_TARGET = YES

'YES' is the default value for <pkg>_INSGTALL_TARGET. so you do not need to set it.

Quoted 10 lines
package/libmdbx/libmdbx.mk new file +10 -0
+ifeq ($(BR2_PACKAGE_LIBMDBX_TOOLS),y)
+LIBMDBX_CONF_OPTS += -DMDBX_BUILD_TOOLS:BOOL=ON -DMDBX_LINK_TOOLS_NONSTATIC:BOOL=ON
+else
+LIBMDBX_CONF_OPTS += -DMDBX_BUILD_TOOLS:BOOL=OFF
+endif
+
+else
+
+LIBMDBX_CONF_OPTS += -DMDBX_BUILD_SHARED_LIBRARY:BOOL=OFF
+ifeq ($(BR2_PACKAGE_LIBMDBX_TOOLS),y)

package/libmdbx/libmdbx.mk line 57

Quoted 10 lines
package/libmdbx/libmdbx.mk new file +10 -0
+LIBMDBX_CONF_OPTS += -DMDBX_BUILD_TOOLS:BOOL=ON -DMDBX_LINK_TOOLS_NONSTATIC:BOOL=OFF
+LIBMDBX_INSTALL_TARGET = YES
+else
+LIBMDBX_CONF_OPTS += -DMDBX_BUILD_TOOLS:BOOL=OFF
+LIBMDBX_INSTALL_TARGET = NO
+endif
+
+endif
+
+$(eval $(cmake-package))

By the way .. there are two tools that help validating new package:

utils/tes-pkg -p libmdbx
utils/check-package package/libmdbx/*

Please check and fix the findings of these tests.

Thank you

Signature · 1 line
Heiko
#3 lore raw

v3 cover package/libmdbx: new package (library/database)

This patch adds libmdbx v0.9.1 and below is a brief overview of libmdbx.

Changes v1 -> v2:
- libmdbx version v0.8.2 -> v0.9.1 (released 2020-09-30)

Changes v2 -> v3:
- removed outcommented stuff (suggested by Heiko Thiery).
- cleaned up and simplified the makefile (suggested by Heiko Thiery).
- added patch for C++ header installation.
- added patch with fix minor copy&paste typo.
- added patch with pthread workaround for buggy toolchain/cmake/buildroot.
- added patch for pthread_yield().
- passed utils/check-package package/libmdbx/* (suggested by Heiko Thiery).
- passed utils/test-pkg -a -p libmdbx (suggested by Heiko Thiery).

Please merge.

Regards,
Leonid.

--

libmdbx is an extremely fast, compact, powerful, embedded, transactional key-value database, with permissive license. libmdbx has a specific set of properties and capabilities, focused on creating unique lightweight solutions.

Historically, libmdbx (MDBX) is a deeply revised and extended descendant of the legendary LMDB (Lightning Memory-Mapped Database). libmdbx inherits all benefits from LMDB, but resolves some issues and adds a set of improvements.

According to developers, for now libmdbx surpasses the LMDB in terms of reliability, features and performance.

The most important differences MDBX from LMDB:
==============================================

1. More attention is paid to the quality of the code, to an "unbreakability" of the API, to testing and automatic checks (i.e. sanitizers, etc). So there:
- more control during operation;
- more checking parameters, internal audit of database structures;
- no warnings from compiler;
- no issues from ASAN, UBSAN, Valgrind, Coverity;
- etc.

2. Keys could be more than 2 times longer than LMDB.

3. Up to 20% faster than LMDB in CRUD benchmarks.

4. Automatic on-the-fly database size adjustment,
both increment and reduction.

5. Automatic continuous zero-overhead database compactification.

6. The same database format for 32- and 64-bit builds.

7. LIFO policy for Garbage Collection recycling (this can significantly increase write performance due write-back disk cache up to several times in a best case scenario).

8. Range query estimation.

9. Utility for checking the integrity of the database structure with some recovery capabilities.

For more info please refer:
- https://github.com/erthink/libmdbx for source code and README.
- https://erthink.github.io/libmdbx for API description.

--

MDBX is a Btree-based database management library modeled loosely on the BerkeleyDB API, but much simplified. The entire database (aka "environment") is exposed in a memory map, and all data fetches return data directly from the mapped memory, so no malloc's or memcpy's occur during data fetches. As such, the library is extremely simple because it requires no page caching layer of its own, and it is extremely high performance and memory-efficient. It is also fully transactional with full ACID semantics, and when the memory map is read-only, the database integrity cannot be corrupted by stray pointer writes from application code.

The library is fully thread-aware and supports concurrent read/write access from multiple processes and threads. Data pages use a copy-on-write strategy so no active data pages are ever overwritten, which also provides resistance to corruption and eliminates the need of any special recovery procedures after a system crash. Writes are fully serialized; only one write transaction may be active at a time, which guarantees that writers can never deadlock. The database structure is multi-versioned so readers run with no locks; writers cannot block readers, and readers don't block writers.

Unlike other well-known database mechanisms which use either write-ahead transaction logs or append-only data writes, MDBX requires no maintenance during operation. Both write-ahead loggers and append-only databases require periodic checkpointing and/or compaction of their log or database files otherwise they grow without bound. MDBX tracks retired/freed pages within the database and re-uses them for new write operations, so the database size does not grow without bound in normal use.

The memory map can be used as a read-only or read-write map. It is read-only by default as this provides total immunity to corruption. Using read-write mode offers much higher write performance, but adds the possibility for stray application writes thru pointers to silently corrupt the database.

Features
========

- Key-value data model, keys are always sorted.

- Fully ACID-compliant, through to MVCC and CoW.

- Multiple key-value sub-databases within a single datafile.

- Range lookups, including range query estimation.

- Efficient support for short fixed length keys, including native 32/64-bit integers.

- Ultra-efficient support for multimaps. Multi-values sorted, searchable and iterable. Keys stored without duplication.

- Data is memory-mapped and accessible directly/zero-copy. Traversal of database records is extremely-fast.

- Transactions for readers and writers, ones do not block others.

- Writes are strongly serialized. No transaction conflicts nor deadlocks.

- Readers are non-blocking, notwithstanding snapshot isolation.

- Nested write transactions.

- Reads scale linearly across CPUs.

- Continuous zero-overhead database compactification.

- Automatic on-the-fly database size adjustment.

- Customizable database page size.

- Olog(N) cost of lookup, insert, update, and delete operations by virtue of B+ tree characteristics.

- Online hot backup.

- Append operation for efficient bulk insertion of pre-sorted data.

- No WAL nor any transaction journal. No crash recovery needed. No maintenance is required.

- No internal cache and/or memory management, all done by basic OS services.

Limitations
===========

- Page size: a power of 2, maximum 65536 bytes, default 4096 bytes.

- Key size: minimum 0, maximum ?? pagesize (1300 bytes for default 4K pagesize, 21780 bytes for 64K pagesize).

- Value size: minimum 0, maximum 2146435072 (0x7FF00000) bytes for maps, ?? pagesize for multimaps (1348 bytes default 4K pagesize, 21828 bytes for 64K pagesize).

- Write transaction size: up to 4194301 (0x3FFFFD) pages (16 GiB for default 4K pagesize, 256 GiB for 64K pagesize).

- Database size: up to 2147483648 pages (8 TiB for default 4K pagesize, 128 TiB for 64K pagesize).

- Maximum sub-databases: 32765.

Gotchas
=======

- There cannot be more than one writer at a time, i.e. no more than one write transaction at a time.

- libmdbx is based on B+ tree, so access to database pages is mostly random. Thus SSDs provide a significant performance boost over spinning disks for large databases.

- libmdbx uses shadow paging instead of WAL. Thus syncing data to disk might be a bottleneck for write intensive workload.

- libmdbx uses copy-on-write for snapshot isolation during updates, but read transactions prevents recycling an old retired/freed pages, since it read ones. Thus altering of data during a parallel long-lived read operation will increase the process work set, may exhaust entire free database space, the database can grow quickly, and result in performance degradation. Try to avoid long running read transactions.

- libmdbx is extraordinarily fast and provides minimal overhead for data access, so you should reconsider using brute force techniques and double check your code. On the one hand, in the case of libmdbx, a simple linear search may be more profitable than complex indexes. On the other hand, if you make something suboptimally, you can notice detrimentally only on sufficiently large data.

--
Leonid Yuriev (1):
package/libmdbx: new package (library/database).

 DEVELOPERS                                    |  3 +
 package/Config.in                             |  1 +
 .../0001-mdbx-fix-minor-copy-paste-typo.patch | 27 ++++++++
 ...e-fix-missing-installation-of-mdbx.h.patch | 54 +++++++++++++++
 .../0003-mdbx-fix-obsolete-__noreturn.patch   | 27 ++++++++
 ...ield-instruction-on-ARM-if-unsupport.patch | 28 ++++++++
 ...fix-minor-false-positive-GCC-warning.patch | 27 ++++++++
 ...ad-workaround-for-buggy-toolchain-cm.patch | 65 +++++++++++++++++++
 ...mdbx-fix-pthread_yield-for-non-GLIBC.patch | 42 ++++++++++++
 package/libmdbx/Config.in                     | 35 ++++++++++
 package/libmdbx/libmdbx.hash                  |  2 +
 package/libmdbx/libmdbx.mk                    | 23 +++++++
 12 files changed, 334 insertions(+)
 create mode 100644 package/libmdbx/0001-mdbx-fix-minor-copy-paste-typo.patch
 create mode 100644 package/libmdbx/0002-mdbx-cmake-fix-missing-installation-of-mdbx.h.patch
 create mode 100644 package/libmdbx/0003-mdbx-fix-obsolete-__noreturn.patch
 create mode 100644 package/libmdbx/0004-mdbx-don-t-use-yield-instruction-on-ARM-if-unsupport.patch
 create mode 100644 package/libmdbx/0005-mdbx-load-fix-minor-false-positive-GCC-warning.patch
 create mode 100644 package/libmdbx/0006-mdbx-cmake-pthread-workaround-for-buggy-toolchain-cm.patch
 create mode 100644 package/libmdbx/0007-mdbx-fix-pthread_yield-for-non-GLIBC.patch
 create mode 100644 package/libmdbx/Config.in
 create mode 100644 package/libmdbx/libmdbx.hash
 create mode 100644 package/libmdbx/libmdbx.mk
Signature · 1 line
2.28.0
#4 lore raw

PATCH v3 1/1 package/libmdbx: new package (library/database).

In reply to Heiko Thiery

This patch adds libmdbx v0.9.1:
- libmdbx is one of the fastest compact embeddable key-value ACID database.
- libmdbx has a specific set of properties and capabilities, focused on creating unique lightweight solutions.
- libmdbx surpasses the legendary LMDB (Lightning Memory-Mapped Database) in terms of reliability, features and performance.

https://github.com/erthink/libmdbx

---
Changes v1 -> v2:
- libmdbx version v0.8.2 -> v0.9.1 (released 2020-09-30)

Changes v2 -> v3:
- removed outcommented stuff (suggested by Heiko Thiery).
- cleaned up and simplified the makefile (suggested by Heiko Thiery).
- added patch for C++ header installation.
- added patch with fix minor copy&paste typo.
- added patch with pthread workaround for buggy toolchain/cmake/buildroot.
- added patch for pthread_yield().
- passed utils/check-package package/libmdbx/* (suggested by Heiko Thiery).
- passed utils/test-pkg -a -p libmdbx (suggested by Heiko Thiery).

---
 DEVELOPERS                                    |  3 +
 package/Config.in                             |  1 +
 .../0001-mdbx-fix-minor-copy-paste-typo.patch | 27 ++++++++
 ...e-fix-missing-installation-of-mdbx.h.patch | 54 +++++++++++++++
 .../0003-mdbx-fix-obsolete-__noreturn.patch   | 27 ++++++++
 ...ield-instruction-on-ARM-if-unsupport.patch | 28 ++++++++
 ...fix-minor-false-positive-GCC-warning.patch | 27 ++++++++
 ...ad-workaround-for-buggy-toolchain-cm.patch | 65 +++++++++++++++++++
 ...mdbx-fix-pthread_yield-for-non-GLIBC.patch | 42 ++++++++++++
 package/libmdbx/Config.in                     | 35 ++++++++++
 package/libmdbx/libmdbx.hash                  |  2 +
 package/libmdbx/libmdbx.mk                    | 23 +++++++
 12 files changed, 334 insertions(+)
 create mode 100644 package/libmdbx/0001-mdbx-fix-minor-copy-paste-typo.patch
 create mode 100644 package/libmdbx/0002-mdbx-cmake-fix-missing-installation-of-mdbx.h.patch
 create mode 100644 package/libmdbx/0003-mdbx-fix-obsolete-__noreturn.patch
 create mode 100644 package/libmdbx/0004-mdbx-don-t-use-yield-instruction-on-ARM-if-unsupport.patch
 create mode 100644 package/libmdbx/0005-mdbx-load-fix-minor-false-positive-GCC-warning.patch
 create mode 100644 package/libmdbx/0006-mdbx-cmake-pthread-workaround-for-buggy-toolchain-cm.patch
 create mode 100644 package/libmdbx/0007-mdbx-fix-pthread_yield-for-non-GLIBC.patch
 create mode 100644 package/libmdbx/Config.in
 create mode 100644 package/libmdbx/libmdbx.hash
 create mode 100644 package/libmdbx/libmdbx.mk
DEVELOPERS +3 -0
index 95942cbeee..7ac362fbe1 100644
@@ -1545,6 +1545,9 @@ N:	Leon Anavi <[email protected]>
 F:	board/olimex/a10_olinuxino
 F:	configs/olimex_a10_olinuxino_lime_defconfig
 
+N:	Leonid Yuriev <[email protected]>
+F:	package/libmdbx
+
 N:	Lionel Flandrin <[email protected]>
 F:	package/python-babel/
 F:	package/python-daemonize/
package/Config.in +1 -0
index 51583d07d6..f87e0388a0 100644
@@ -1370,6 +1370,7 @@ menu "Database"
 	source "package/kompexsqlite/Config.in"
 	source "package/leveldb/Config.in"
 	source "package/libgit2/Config.in"
+	source "package/libmdbx/Config.in"
 	source "package/libodb/Config.in"
 	source "package/libodb-boost/Config.in"
 	source "package/libodb-mysql/Config.in"
package/libmdbx/0001-mdbx-fix-minor-copy-paste-typo.patch new file +27 -0
new file mode 100644
index 0000000000..d9ff3c5e71
@@ -0,0 +1,27 @@
+From 5807e2eda046bff35946972289236bbef5c0c597 Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Thu, 1 Oct 2020 02:27:29 +0300
+Subject: [PATCH] mdbx++: fix minor copy&paste typo.
+
+Change-Id: I0af3e7ffbbd1231069a60f9f48880df3df2141d7
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ mdbx.h++ | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/mdbx.h++ b/mdbx.h++
+index 095ff9abb..2050f4b92 100644
+--- a/mdbx.h++
++++ b/mdbx.h++
+@@ -538,7 +538,7 @@ struct LIBMDBX_API_TYPE slice : public ::MDBX_val {
+   byte *from_base58(byte *dest, size_t dest_size,
+                     bool ignore_spaces = false) const;
+ 
+-  /// vReturns the buffer size in bytes needed for conversion
++  /// \brief Returns the buffer size in bytes needed for conversion
+   /// [Base58](https://en.wikipedia.org/wiki/Base58) dump to data.
+   MDBX_CXX11_CONSTEXPR size_t from_base58_bytes() const noexcept {
+     return length() / 11 * 8 + length() % 11 * 32 / 43;
+-- 
+2.28.0
+
package/libmdbx/0002-mdbx-cmake-fix-missing-installation-of-mdbx.h.patch new file +54 -0

54 lines not on this page · raw message

package/libmdbx/0003-mdbx-fix-obsolete-__noreturn.patch new file +27 -0
new file mode 100644
index 0000000000..cb04334220
@@ -0,0 +1,27 @@
+From 280ed17ea2dff464c9688fcc92079f36d8b74621 Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Sat, 3 Oct 2020 11:35:26 +0300
+Subject: [PATCH] mdbx: fix obsolete `__noreturn`.
+
+Change-Id: Ic78843d6f16de2a409c16ceecc7acb2ed8aa3e68
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ mdbx.c | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/mdbx.c b/mdbx.c
+index 04ea5140d..9212640fe 100644
+--- a/mdbx.c
++++ b/mdbx.c
+@@ -141,7 +141,7 @@ __extern_C void __assert(const char *, const char *, unsigned int, const char *)
+ #else
+     __nothrow
+ #endif /* __THROW */
+-    __noreturn;
++    MDBX_NORETURN;
+ #define __assert_fail(assertion, file, line, function)                         \
+   __assert(assertion, file, line, function)
+ 
+-- 
+2.28.0
+
package/libmdbx/0004-mdbx-don-t-use-yield-instruction-on-ARM-if-unsupport.patch new file +28 -0
new file mode 100644
index 0000000000..23f91d859b
@@ -0,0 +1,28 @@
+From 70b615e8d4d10cda2d961a815dd15eb87a1f3925 Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Sun, 4 Oct 2020 14:54:11 +0300
+Subject: [PATCH] mdbx: don't use `yield` instruction on ARM if unsupported.
+
+Change-Id: I0b01d783fe4336b089f4b051fb61c203b5879aa5
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ mdbx.c | 3 ++-
+ 1 file changed, 2 insertions(+), 1 deletion(-)
+
+diff --git a/mdbx.c b/mdbx.c
+index aba445424..a9ba1ee85 100644
+--- a/mdbx.c
++++ b/mdbx.c
+@@ -763,7 +763,8 @@ static __always_inline void atomic_yield(void) {
+ #else
+   __asm__ __volatile__("hint @pause");
+ #endif
+-#elif defined(__arm__) || defined(__aarch64__)
++#elif defined(__aarch64__) || (defined(__ARM_ARCH) && __ARM_ARCH > 6) ||       \
++    defined(__ARM_ARCH_6K__)
+ #ifdef __CC_ARM
+   __yield();
+ #else
+-- 
+2.28.0
+
package/libmdbx/0005-mdbx-load-fix-minor-false-positive-GCC-warning.patch new file +27 -0
new file mode 100644
index 0000000000..29d69e3faa
@@ -0,0 +1,27 @@
+From 8d4e7994c0c5c0cf69326139717e324f90bff65b Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Mon, 5 Oct 2020 15:09:10 +0300
+Subject: [PATCH] mdbx-load: fix minor false-positive GCC warning.
+
+Change-Id: Ie75c793712d050e8d3da76a4d0a8df9b81dc5275
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ mdbx_load.c | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/mdbx_load.c b/mdbx_load.c
+index e5e72fc8f..54e179553 100644
+--- a/mdbx_load.c
++++ b/mdbx_load.c
+@@ -318,7 +318,7 @@ static int readhdr(void) {
+     }
+ 
+     for (int i = 0; dbflags[i].bit; i++) {
+-      bool value;
++      bool value = false;
+       if (valbool(dbuf.iov_base, dbflags[i].name, &value)) {
+         if (value)
+           dbi_flags |= dbflags[i].bit;
+-- 
+2.28.0
+
package/libmdbx/0006-mdbx-cmake-pthread-workaround-for-buggy-toolchain-cm.patch new file +65 -0

65 lines not on this page · raw message

package/libmdbx/0007-mdbx-fix-pthread_yield-for-non-GLIBC.patch new file +42 -0

42 lines not on this page · raw message

package/libmdbx/Config.in new file +35 -0
new file mode 100644
index 0000000000..3d21ca4cb1
@@ -0,0 +1,35 @@
+config BR2_PACKAGE_LIBMDBX
+	bool "libmdbx"
+	depends on BR2_USE_MMU
+	depends on BR2_TOOLCHAIN_HAS_THREADS
+	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8
+	depends on BR2_TOOLCHAIN_HAS_SYNC_4
+	help
+	  One of the fastest compact key-value ACID database
+	  without WAL. libmdbx has a specific set of properties
+	  and capabilities, focused on creating unique lightweight
+	  solutions.
+
+	  libmdbx surpasses the legendary legendary LMDB in terms of
+	  reliability, features and performance.
+
+	  https://github.com/erthink/libmdbx
+
+if BR2_PACKAGE_LIBMDBX
+
+config BR2_PACKAGE_LIBMDBX_TOOLS
+	bool "install tools"
+	help
+	  Install libmdbx tools for checking, dump, restore
+	  and show statistics of databases.
+
+config BR2_PACKAGE_LIBMDBX_CXX
+	bool "C++ API"
+	select BR2_INSTALL_LIBSTDCPP
+	help
+	  Enable modern C++ API for libmdbx.
+
+endif
+
+comment "libmdbx needs a toolchain w/ threads"
+	depends on !BR2_TOOLCHAIN_HAS_THREADS
package/libmdbx/libmdbx.hash new file +2 -0
new file mode 100644
index 0000000000..f48cd81a7d
@@ -0,0 +1,2 @@
+sha256  310fe25c858a9515fc8c8d7d1f24a67c9496f84a91e0a0e41ea9975b1371e569  LICENSE
+sha256  c7fb24381eb4d92f2e2edc17e577cb721269683c816c6cca307c58f6f346e786  libmdbx-amalgamated-0.9.1.tar.gz
package/libmdbx/libmdbx.mk new file +23 -0
new file mode 100644
index 0000000000..06aa0fc051
@@ -0,0 +1,23 @@
+################################################################################
+#
+# libmdbx
+#
+################################################################################
+
+LIBMDBX_VERSION = 0.9.1
+LIBMDBX_SOURCE = libmdbx-amalgamated-$(LIBMDBX_VERSION).tar.gz
+LIBMDBX_SITE = https://github.com/erthink/libmdbx/releases/download/v$(LIBMDBX_VERSION)
+LIBMDBX_SUPPORTS_IN_SOURCE_BUILD = NO
+LIBMDBX_LICENSE = OLDAP-2.8
+LIBMDBX_LICENSE_FILES = LICENSE
+LIBMDBX_REDISTRIBUTE = YES
+LIBMDBX_STRIP_COMPONENTS = 0
+LIBMDBX_INSTALL_STAGING = YES
+
+LIBMDBX_CONF_OPTS = -DMDBX_INSTALL_MANPAGES=OFF -DBUILD_FOR_NATIVE_CPU=OFF \
+	-DMDBX_INSTALL_STATIC=$(if $(BR2_STATIC_LIBS),ON,OFF) \
+	-DMDBX_BUILD_SHARED_LIBRARY=$(if $(BR2_SHARED_LIBS),ON,OFF) \
+	-DMDBX_BUILD_CXX=$(if $(BR2_PACKAGE_LIBMDBX_CXX),ON,OFF) \
+	-DMDBX_LINK_TOOLS_NONSTATIC=$(if $(BR2_SHARED_LIBS),ON,OFF)
+
+$(eval $(cmake-package))
Signature · 1 line
2.28.0
#5 lore raw

PATCH v3 1/1 package/libmdbx: new package (library/database).

Hi Leonid,

thank you for the new version. Some comments below.

Am Mo., 5. Okt. 2020 um 22:16 Uhr schrieb Leonid Yuriev <[email protected]>:

Quoted 38 lines

This patch adds libmdbx v0.9.1:
- libmdbx is one of the fastest compact embeddable key-value ACID database.
- libmdbx has a specific set of properties and capabilities, focused on creating unique lightweight solutions.
- libmdbx surpasses the legendary LMDB (Lightning Memory-Mapped Database) in terms of reliability, features and performance.

https://github.com/erthink/libmdbx

---
Changes v1 -> v2:
- libmdbx version v0.8.2 -> v0.9.1 (released 2020-09-30)

Changes v2 -> v3:
- removed outcommented stuff (suggested by Heiko Thiery).
- cleaned up and simplified the makefile (suggested by Heiko Thiery).
- added patch for C++ header installation.
- added patch with fix minor copy&paste typo.
- added patch with pthread workaround for buggy toolchain/cmake/buildroot.
- added patch for pthread_yield().
- passed utils/check-package package/libmdbx/* (suggested by Heiko Thiery).
- passed utils/test-pkg -a -p libmdbx (suggested by Heiko Thiery).

---
 DEVELOPERS                                    |  3 +
 package/Config.in                             |  1 +
 .../0001-mdbx-fix-minor-copy-paste-typo.patch | 27 ++++++++
 ...e-fix-missing-installation-of-mdbx.h.patch | 54 +++++++++++++++
 .../0003-mdbx-fix-obsolete-__noreturn.patch   | 27 ++++++++
 ...ield-instruction-on-ARM-if-unsupport.patch | 28 ++++++++
 ...fix-minor-false-positive-GCC-warning.patch | 27 ++++++++
 ...ad-workaround-for-buggy-toolchain-cm.patch | 65 +++++++++++++++++++
 ...mdbx-fix-pthread_yield-for-non-GLIBC.patch | 42 ++++++++++++
 package/libmdbx/Config.in                     | 35 ++++++++++
 package/libmdbx/libmdbx.hash                  |  2 +
 package/libmdbx/libmdbx.mk                    | 23 +++++++
Quoted 8 lines
 12 files changed, 334 insertions(+)
 create mode 100644 package/libmdbx/0001-mdbx-fix-minor-copy-paste-typo.patch
 create mode 100644 package/libmdbx/0002-mdbx-cmake-fix-missing-installation-of-mdbx.h.patch
 create mode 100644 package/libmdbx/0003-mdbx-fix-obsolete-__noreturn.patch
 create mode 100644 package/libmdbx/0004-mdbx-don-t-use-yield-instruction-on-ARM-if-unsupport.patch
 create mode 100644 package/libmdbx/0005-mdbx-load-fix-minor-false-positive-GCC-warning.patch
 create mode 100644 package/libmdbx/0006-mdbx-cmake-pthread-workaround-for-buggy-toolchain-cm.patch
 create mode 100644 package/libmdbx/0007-mdbx-fix-pthread_yield-for-non-GLIBC.patch

Is it required to add all these patches? They seem all to be in the upstream repo. So you could also set the version to the last commit id that contains all these changes.

Quoted 362 lines
 create mode 100644 package/libmdbx/Config.in
 create mode 100644 package/libmdbx/libmdbx.hash
 create mode 100644 package/libmdbx/libmdbx.mk
DEVELOPERS +3 -0
index 95942cbeee..7ac362fbe1 100644
@@ -1545,6 +1545,9 @@ N:        Leon Anavi <[email protected]>
 F:     board/olimex/a10_olinuxino
 F:     configs/olimex_a10_olinuxino_lime_defconfig
 
+N:     Leonid Yuriev <[email protected]>
+F:     package/libmdbx
+
 N:     Lionel Flandrin <[email protected]>
 F:     package/python-babel/
 F:     package/python-daemonize/
package/Config.in +1 -0
index 51583d07d6..f87e0388a0 100644
@@ -1370,6 +1370,7 @@ menu "Database"
        source "package/kompexsqlite/Config.in"
        source "package/leveldb/Config.in"
        source "package/libgit2/Config.in"
+       source "package/libmdbx/Config.in"
        source "package/libodb/Config.in"
        source "package/libodb-boost/Config.in"
        source "package/libodb-mysql/Config.in"
package/libmdbx/0001-mdbx-fix-minor-copy-paste-typo.patch new file +27 -0
new file mode 100644
index 0000000000..d9ff3c5e71
@@ -0,0 +1,27 @@
+From 5807e2eda046bff35946972289236bbef5c0c597 Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Thu, 1 Oct 2020 02:27:29 +0300
+Subject: [PATCH] mdbx++: fix minor copy&paste typo.
+
+Change-Id: I0af3e7ffbbd1231069a60f9f48880df3df2141d7
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ mdbx.h++ | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/mdbx.h++ b/mdbx.h++
+index 095ff9abb..2050f4b92 100644
+--- a/mdbx.h++
++++ b/mdbx.h++
+@@ -538,7 +538,7 @@ struct LIBMDBX_API_TYPE slice : public ::MDBX_val {
+   byte *from_base58(byte *dest, size_t dest_size,
+                     bool ignore_spaces = false) const;
+
+-  /// vReturns the buffer size in bytes needed for conversion
++  /// \brief Returns the buffer size in bytes needed for conversion
+   /// [Base58](https://en.wikipedia.org/wiki/Base58) dump to data.
+   MDBX_CXX11_CONSTEXPR size_t from_base58_bytes() const noexcept {
+     return length() / 11 * 8 + length() % 11 * 32 / 43;
+--
+2.28.0
+
package/libmdbx/0002-mdbx-cmake-fix-missing-installation-of-mdbx.h.patch new file +54 -0
new file mode 100644
index 0000000000..df49342ccb
@@ -0,0 +1,54 @@
+From dc2cd19d566e84451ee47cf04277f97bc22771c5 Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Fri, 2 Oct 2020 00:05:02 +0300
+Subject: [PATCH] mdbx-cmake: fix missing installation of `mdbx.h++`
+
+Change-Id: I41975e4eeff6583a266b273b9a4f8486982ede90
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ CMakeLists.txt | 8 +++++---
+ 1 file changed, 5 insertions(+), 3 deletions(-)
+
+diff --git a/CMakeLists.txt b/CMakeLists.txt
+index 28db425c5..95f116521 100644
+--- a/CMakeLists.txt
++++ b/CMakeLists.txt
+@@ -485,7 +485,8 @@ fetch_version(MDBX "${CMAKE_CURRENT_SOURCE_DIR}" FALSE)
+ message(STATUS "libmdbx version is ${MDBX_VERSION}")
+
+ # sources list
+-set(LIBMDBX_SOURCES mdbx.h "${CMAKE_CURRENT_BINARY_DIR}/config.h" )
++set(LIBMDBX_PUBLIC_HEADERS mdbx.h)
++set(LIBMDBX_SOURCES mdbx.h "${CMAKE_CURRENT_BINARY_DIR}/config.h")
+ if(MDBX_AMALGAMATED_SOURCE)
+   list(APPEND LIBMDBX_SOURCES mdbx.c)
+ else()
+@@ -511,6 +512,7 @@ else()
+ endif(MDBX_AMALGAMATED_SOURCE)
+ if(MDBX_BUILD_CXX)
+   message(STATUS "Use C${MDBX_C_STANDARD} and C++${MDBX_CXX_STANDARD} for libmdbx")
++  list(APPEND LIBMDBX_PUBLIC_HEADERS mdbx.h++)
+   list(APPEND LIBMDBX_SOURCES "${MDBX_SOURCE_DIR}/mdbx.c++" mdbx.h++)
+ else()
+   message(STATUS "Use C${MDBX_C_STANDARD} for libmdbx but C++ portion is disabled")
+@@ -561,7 +563,7 @@ if(MDBX_INSTALL_STATIC)
+ else()
+   add_library(mdbx-static STATIC EXCLUDE_FROM_ALL ${LIBMDBX_SOURCES})
+ endif()
+-set_target_properties(mdbx-static PROPERTIES PUBLIC_HEADER mdbx.h)
++set_target_properties(mdbx-static PROPERTIES PUBLIC_HEADER "${LIBMDBX_PUBLIC_HEADERS}")
+ target_compile_definitions(mdbx-static PRIVATE MDBX_BUILD_SHARED_LIBRARY=0)
+ target_setup_options(mdbx-static)
+ libmdbx_setup_libs(mdbx-static INTERFACE)
+@@ -576,7 +578,7 @@ endif()
+ # build shared library
+ if(MDBX_BUILD_SHARED_LIBRARY)
+   add_library(mdbx SHARED ${LIBMDBX_SOURCES})
+-  set_target_properties(mdbx PROPERTIES PUBLIC_HEADER mdbx.h)
++  set_target_properties(mdbx PROPERTIES PUBLIC_HEADER "${LIBMDBX_PUBLIC_HEADERS}")
+   target_compile_definitions(mdbx PRIVATE LIBMDBX_EXPORTS MDBX_BUILD_SHARED_LIBRARY=1 INTERFACE LIBMDBX_IMPORTS)
+   target_setup_options(mdbx)
+   libmdbx_setup_libs(mdbx PRIVATE)
+--
+2.28.0
+
package/libmdbx/0003-mdbx-fix-obsolete-__noreturn.patch new file +27 -0
new file mode 100644
index 0000000000..cb04334220
@@ -0,0 +1,27 @@
+From 280ed17ea2dff464c9688fcc92079f36d8b74621 Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Sat, 3 Oct 2020 11:35:26 +0300
+Subject: [PATCH] mdbx: fix obsolete `__noreturn`.
+
+Change-Id: Ic78843d6f16de2a409c16ceecc7acb2ed8aa3e68
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ mdbx.c | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/mdbx.c b/mdbx.c
+index 04ea5140d..9212640fe 100644
+--- a/mdbx.c
++++ b/mdbx.c
+@@ -141,7 +141,7 @@ __extern_C void __assert(const char *, const char *, unsigned int, const char *)
+ #else
+     __nothrow
+ #endif /* __THROW */
+-    __noreturn;
++    MDBX_NORETURN;
+ #define __assert_fail(assertion, file, line, function)                         \
+   __assert(assertion, file, line, function)
+
+--
+2.28.0
+
package/libmdbx/0004-mdbx-don-t-use-yield-instruction-on-ARM-if-unsupport.patch new file +28 -0
new file mode 100644
index 0000000000..23f91d859b
@@ -0,0 +1,28 @@
+From 70b615e8d4d10cda2d961a815dd15eb87a1f3925 Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Sun, 4 Oct 2020 14:54:11 +0300
+Subject: [PATCH] mdbx: don't use `yield` instruction on ARM if unsupported.
+
+Change-Id: I0b01d783fe4336b089f4b051fb61c203b5879aa5
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ mdbx.c | 3 ++-
+ 1 file changed, 2 insertions(+), 1 deletion(-)
+
+diff --git a/mdbx.c b/mdbx.c
+index aba445424..a9ba1ee85 100644
+--- a/mdbx.c
++++ b/mdbx.c
+@@ -763,7 +763,8 @@ static __always_inline void atomic_yield(void) {
+ #else
+   __asm__ __volatile__("hint @pause");
+ #endif
+-#elif defined(__arm__) || defined(__aarch64__)
++#elif defined(__aarch64__) || (defined(__ARM_ARCH) && __ARM_ARCH > 6) ||       \
++    defined(__ARM_ARCH_6K__)
+ #ifdef __CC_ARM
+   __yield();
+ #else
+--
+2.28.0
+
package/libmdbx/0005-mdbx-load-fix-minor-false-positive-GCC-warning.patch new file +27 -0
new file mode 100644
index 0000000000..29d69e3faa
@@ -0,0 +1,27 @@
+From 8d4e7994c0c5c0cf69326139717e324f90bff65b Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Mon, 5 Oct 2020 15:09:10 +0300
+Subject: [PATCH] mdbx-load: fix minor false-positive GCC warning.
+
+Change-Id: Ie75c793712d050e8d3da76a4d0a8df9b81dc5275
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ mdbx_load.c | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/mdbx_load.c b/mdbx_load.c
+index e5e72fc8f..54e179553 100644
+--- a/mdbx_load.c
++++ b/mdbx_load.c
+@@ -318,7 +318,7 @@ static int readhdr(void) {
+     }
+
+     for (int i = 0; dbflags[i].bit; i++) {
+-      bool value;
++      bool value = false;
+       if (valbool(dbuf.iov_base, dbflags[i].name, &value)) {
+         if (value)
+           dbi_flags |= dbflags[i].bit;
+--
+2.28.0
+
package/libmdbx/0006-mdbx-cmake-pthread-workaround-for-buggy-toolchain-cm.patch new file +65 -0
new file mode 100644
index 0000000000..37525cf0d5
@@ -0,0 +1,65 @@
+From 787eaaa373073e17f3a53658b085c255bc2c8ff8 Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Mon, 5 Oct 2020 19:12:20 +0300
+Subject: [PATCH] mdbx-cmake: pthread workaround for buggy
+ toolchain/cmake/buildroot.
+
+Change-Id: I0d95e783abbd10a63cd1595a9de50593e814a967
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ CMakeLists.txt | 21 ++++++++++++++++++---
+ 1 file changed, 18 insertions(+), 3 deletions(-)
+
+diff --git a/CMakeLists.txt b/CMakeLists.txt
+index 95f116521..20a50a453 100644
+--- a/CMakeLists.txt
++++ b/CMakeLists.txt
+@@ -173,10 +173,27 @@ if(NOT CMAKE_BUILD_TYPE)
+ endif()
+ string(TOUPPER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE_UPPERCASE)
+
++set(CMAKE_THREAD_PREFER_PTHREAD TRUE)
++set(THREADS_PREFER_PTHREAD_FLAG TRUE)
++find_package(Threads REQUIRED)
++
+ include(cmake/utils.cmake)
+ include(cmake/compiler.cmake)
+ include(cmake/profile.cmake)
+
++# Workaround for `-pthread` toolchain/cmake bug
++if(NOT APPLE AND NOT MSVC
++    AND CMAKE_USE_PTHREADS_INIT AND NOT CMAKE_THREAD_LIBS_INIT
++    AND (CMAKE_COMPILER_IS_GNUCC OR CMAKE_COMPILER_IS_CLANG))
++  check_compiler_flag("-pthread" CC_HAS_PTHREAD)
++  if(CC_HAS_PTHREAD AND NOT CMAKE_EXE_LINKER_FLAGS MATCHES "-pthread")
++    message(STATUS "Force add -pthread for linker flags to avoid troubles")
++    set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -pthread")
++    set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -pthread")
++    set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} -pthread")
++  endif()
++endif()
++
+ CHECK_FUNCTION_EXISTS(pow NOT_NEED_LIBM)
+ if(NOT_NEED_LIBM)
+   set(LIB_MATH "")
+@@ -190,8 +207,6 @@ else()
+   endif()
+ endif()
+
+-find_package(Threads REQUIRED)
+-
+ if(SUBPROJECT)
+   if(NOT DEFINED BUILD_SHARED_LIBS)
+     option(BUILD_SHARED_LIBS "Build shared libraries (DLLs)" OFF)
+@@ -541,7 +556,7 @@ macro(target_setup_options TARGET)
+ endmacro()
+
+ macro(libmdbx_setup_libs TARGET MODE)
+-  target_link_libraries(${TARGET} ${MODE} ${CMAKE_THREAD_LIBS_INIT})
++  target_link_libraries(${TARGET} ${MODE} Threads::Threads)
+   if(${CMAKE_SYSTEM_NAME} STREQUAL "Windows")
+     target_link_libraries(${TARGET} ${MODE} ntdll.lib)
+     if(MDBX_NTDLL_EXTRA_IMPLIB AND MDBX_AVOID_CRT)
+--
+2.28.0
+
package/libmdbx/0007-mdbx-fix-pthread_yield-for-non-GLIBC.patch new file +42 -0
new file mode 100644
index 0000000000..5d7e83d4c7
@@ -0,0 +1,42 @@
+From 16d43332a9d28ba27aa3d127e8f6b77a87b2f5eb Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Mon, 5 Oct 2020 21:08:09 +0300
+Subject: [PATCH] mdbx: fix `pthread_yield()` for non-GLIBC.
+
+Change-Id: I080e37a42b62e524896dea8747e9f23e2fcd584f
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ mdbx.c | 4 +++-
+ mdbx.c | 1 +
+ 2 files changed, 4 insertions(+), 1 deletion(-)
+
+diff --git a/mdbx.c b/mdbx.c
+index a9ba1ee85..ace6f2756 100644
+--- a/mdbx.c
++++ b/mdbx.c
+@@ -777,7 +777,9 @@ static __always_inline void atomic_yield(void) {
+     defined(__mips64__) || defined(_M_MRX000) || defined(_MIPS_) ||            \
+     defined(__MWERKS__) || defined(__sgi)
+   __asm__ __volatile__(".word 0x00000140");
+-#else
++#elif defined(__linux__) || defined(__gnu_linux__) || defined(_UNIX03_SOURCE)
++  sched_yield();
++#elif (defined(_GNU_SOURCE) &&  __GLIBC_PREREQ(2, 1)) || defined(_OPEN_THREADS)
+   pthread_yield();
+ #endif
+ }
+diff --git a/mdbx.c b/mdbx.c
+index 3509942dc..3af23ce0b 100644
+--- a/mdbx.c
++++ b/mdbx.c
+@@ -119,6 +119,7 @@
+
+ #if defined(__linux__) || defined(__gnu_linux__)
+ #include <linux/sysctl.h>
++#include <sched.h>
+ #include <sys/sendfile.h>
+ #include <sys/statfs.h>
+ #endif /* Linux */
+--
+2.28.0
+
package/libmdbx/Config.in new file +14 -0
new file mode 100644
index 0000000000..3d21ca4cb1
@@ -0,0 +1,35 @@
+config BR2_PACKAGE_LIBMDBX
+       bool "libmdbx"
+       depends on BR2_USE_MMU
+       depends on BR2_TOOLCHAIN_HAS_THREADS
+       depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8
+       depends on BR2_TOOLCHAIN_HAS_SYNC_4
+       help
+         One of the fastest compact key-value ACID database
+         without WAL. libmdbx has a specific set of properties
+         and capabilities, focused on creating unique lightweight
+         solutions.
+
+         libmdbx surpasses the legendary legendary LMDB in terms of
+         reliability, features and performance.

package/libmdbx/Config.in line 24

Quoted 10 lines
package/libmdbx/Config.in new file +10 -0
+
+         https://github.com/erthink/libmdbx
+
+if BR2_PACKAGE_LIBMDBX
+
+config BR2_PACKAGE_LIBMDBX_TOOLS
+       bool "install tools"
+       help
+         Install libmdbx tools for checking, dump, restore
+         and show statistics of databases.

This config option is nowhere used? Do you miss to add it in the makefile or is it still a leftover?

package/libmdbx/Config.in line 31

Quoted 7 lines
package/libmdbx/Config.in new file +7 -0
+
+config BR2_PACKAGIBMDBX_CXX
+       bool "C++ API"
+       select BR2_INSTALL_LIBSTDCPP
+       help
+         Enable modern C++ API for libmdbx.
+

When enabling CXX support you could add a comment that a toolchain with cpp support is required and not enable it by selecting this option.

config BR2_PACKAGIBMDBX_CXX
bool "C++ API"
help
Enable modern C++ API for libmdbx.

comment "libmdbx c++ support needs a toolchain w/ C++"
depends on !BR2_INSTALL_LIBSTDCPP

See also: https://git.buildroot.net/buildroot/tree/package/rrdtool/Config.in#n31

package/libmdbx/Config.in line 35

Quoted 4 lines
package/libmdbx/Config.in new file +4 -0
+endif
+
+comment "libmdbx needs a toolchain w/ threads"
+       depends on !BR2_TOOLCHAIN_HAS_THREADS

Here you the comment should look something like:
comment "libmdbx needs a toolchain w/ threads, gcc >= 4.8"
depends on BR2_USE_MMU
depends on !BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 || !BR2_TOOLCHAIN_HAS_THREADS

Quoted 27 lines
package/libmdbx/libmdbx.hash new file +2 -0
new file mode 100644
index 0000000000..f48cd81a7d
@@ -0,0 +1,2 @@
+sha256  310fe25c858a9515fc8c8d7d1f24a67c9496f84a91e0a0e41ea9975b1371e569  LICENSE
+sha256  c7fb24381eb4d92f2e2edc17e577cb721269683c816c6cca307c58f6f346e786  libmdbx-amalgamated-0.9.1.tar.gz
package/libmdbx/libmdbx.mk new file +13 -0
new file mode 100644
index 0000000000..06aa0fc051
@@ -0,0 +1,23 @@
+################################################################################
+#
+# libmdbx
+#
+################################################################################
+
+LIBMDBX_VERSION = 0.9.1
+LIBMDBX_SOURCE = libmdbx-amalgamated-$(LIBMDBX_VERSION).tar.gz
+LIBMDBX_SITE = https://github.com/erthink/libmdbx/releases/download/v$(LIBMDBX_VERSION)
+LIBMDBX_SUPPORTS_IN_SOURCE_BUILD = NO
+LIBMDBX_LICENSE = OLDAP-2.8
+LIBMDBX_LICENSE_FILES = LICENSE
+LIBMDBX_REDISTRIBUTE = YES

package/libmdbx/libmdbx.mk line 23

Quoted 10 lines
package/libmdbx/libmdbx.mk new file +10 -0
+LIBMDBX_STRIP_COMPONENTS = 0
+LIBMDBX_INSTALL_STAGING = YES
+
+LIBMDBX_CONF_OPTS = -DMDBX_INSTALL_MANPAGES=OFF -DBUILD_FOR_NATIVE_CPU=OFF \
+       -DMDBX_INSTALL_STATIC=$(if $(BR2_STATIC_LIBS),ON,OFF) \
+       -DMDBX_BUILD_SHARED_LIBRARY=$(if $(BR2_SHARED_LIBS),ON,OFF) \
+       -DMDBX_BUILD_CXX=$(if $(BR2_PACKAGE_LIBMDBX_CXX),ON,OFF) \
+       -DMDBX_LINK_TOOLS_NONSTATIC=$(if $(BR2_SHARED_LIBS),ON,OFF)
+
+$(eval $(cmake-package))

Thank you

Signature · 1 line
Heiko
#6 lore raw

PATCH v3 1/1 package/libmdbx: new package (library/database).

Hi, Heiko.

Thanks for comments.
I will send an updated (hope final) version of the patch soon. And here are the answers to some of your questions/comments below.

Quoted 4 lines · Heiko Thiery

Hi Leonid,

thank you for the new version. Some comments below.

[...]

Quoted 11 lines
Quoted 7 lines
 create mode 100644 package/libmdbx/0001-mdbx-fix-minor-copy-paste-typo.patch
 create mode 100644 package/libmdbx/0002-mdbx-cmake-fix-missing-installation-of-mdbx.h.patch
 create mode 100644 package/libmdbx/0003-mdbx-fix-obsolete-__noreturn.patch
 create mode 100644 package/libmdbx/0004-mdbx-don-t-use-yield-instruction-on-ARM-if-unsupport.patch
 create mode 100644 package/libmdbx/0005-mdbx-load-fix-minor-false-positive-GCC-warning.patch
 create mode 100644 package/libmdbx/0006-mdbx-cmake-pthread-workaround-for-buggy-toolchain-cm.patch
 create mode 100644 package/libmdbx/0007-mdbx-fix-pthread_yield-for-non-GLIBC.patch

Is it required to add all these patches? They seem all to be in the upstream repo. So you could also set the version to the last commit id that contains all these changes.

Yes, these patches are required for amalgamated source code which is available only for a releases (i.e. github's tags).

Quoted 5 lines

package/libmdbx/Config.in line 21

Quoted 2 lines
package/libmdbx/Config.in new file +2 -0
+config BR2_PACKAGE_LIBMDBX_TOOLS
+       bool "install tools"

This config option is nowhere used? Do you miss to add it in the makefile or is it still a leftover?

This is my mistake.
Now fixed in the makefile.

Quoted 7 lines

package/libmdbx/Config.in line 28

Quoted 3 lines
package/libmdbx/Config.in new file +3 -0
+config BR2_PACKAGE_LIBMDBX_CXX
+       bool "C++ API"
+       select BR2_INSTALL_LIBSTDCPP

When enabling CXX support you could add a comment that a toolchain with cpp support is required and not enable it by selecting this option.

[..]

Quoted 2 lines

comment "libmdbx c++ support needs a toolchain w/ C++"
depends on !BR2_INSTALL_LIBSTDCPP

[..]

Quoted 7 lines

package/libmdbx/Config.in line 35

Quoted 2 lines
package/libmdbx/Config.in new file +2 -0
+comment "libmdbx needs a toolchain w/ threads"
+       depends on !BR2_TOOLCHAIN_HAS_THREADS

Here you the comment should look something like:
comment "libmdbx needs a toolchain w/ threads, gcc >= 4.8"
depends on BR2_USE_MMU
depends on !BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 || !BR2_TOOLCHAIN_HAS_THREADS

Done.
I also clarified these dependencies.

Regards,
Leonid.

#7 lore raw

v4 cover package/libmdbx: new package (library/database)

This patch adds libmdbx v0.9.1 and below is a brief overview of libmdbx.

Changes v1 -> v2:
- libmdbx version v0.8.2 -> v0.9.1 (released 2020-09-30)

Changes v2 -> v3:
- removed outcommented stuff (suggested by Heiko Thiery).
- cleaned up and simplified the makefile (suggested by Heiko Thiery).
- added patch for C++ header installation.
- added patch with fix minor copy&paste typo.
- added patch with pthread workaround for buggy toolchain/cmake/buildroot.
- added patch for pthread_yield().
- passed utils/check-package package/libmdbx/* (suggested by Heiko Thiery).
- passed utils/test-pkg -a -p libmdbx (suggested by Heiko Thiery).

Changes v3 -> v4:
- fix passing BR2_PACKAGE_LIBMDBX_TOOLS option to cmake.
- fix using depend on instead of select,
and add comment for C++ toolchain (suggested by Heiko Thiery).
- fix minor help typo.

Please merge.

Regards,
Leonid.

--

libmdbx is an extremely fast, compact, powerful, embedded, transactional key-value database, with permissive license. libmdbx has a specific set of properties and capabilities, focused on creating unique lightweight solutions.

Historically, libmdbx (MDBX) is a deeply revised and extended descendant of the legendary LMDB (Lightning Memory-Mapped Database). libmdbx inherits all benefits from LMDB, but resolves some issues and adds a set of improvements.

According to developers, for now libmdbx surpasses the LMDB in terms of reliability, features and performance.

The most important differences MDBX from LMDB:
==============================================

1. More attention is paid to the quality of the code, to an "unbreakability" of the API, to testing and automatic checks (i.e. sanitizers, etc). So there:
- more control during operation;
- more checking parameters, internal audit of database structures;
- no warnings from compiler;
- no issues from ASAN, UBSAN, Valgrind, Coverity;
- etc.

2. Keys could be more than 2 times longer than LMDB.

3. Up to 20% faster than LMDB in CRUD benchmarks.

4. Automatic on-the-fly database size adjustment,
both increment and reduction.

5. Automatic continuous zero-overhead database compactification.

6. The same database format for 32- and 64-bit builds.

7. LIFO policy for Garbage Collection recycling (this can significantly increase write performance due write-back disk cache up to several times in a best case scenario).

8. Range query estimation.

9. Utility for checking the integrity of the database structure with some recovery capabilities.

For more info please refer:
- https://github.com/erthink/libmdbx for source code and README.
- https://erthink.github.io/libmdbx for API description.

--

MDBX is a Btree-based database management library modeled loosely on the BerkeleyDB API, but much simplified. The entire database (aka "environment") is exposed in a memory map, and all data fetches return data directly from the mapped memory, so no malloc's or memcpy's occur during data fetches. As such, the library is extremely simple because it requires no page caching layer of its own, and it is extremely high performance and memory-efficient. It is also fully transactional with full ACID semantics, and when the memory map is read-only, the database integrity cannot be corrupted by stray pointer writes from application code.

The library is fully thread-aware and supports concurrent read/write access from multiple processes and threads. Data pages use a copy-on-write strategy so no active data pages are ever overwritten, which also provides resistance to corruption and eliminates the need of any special recovery procedures after a system crash. Writes are fully serialized; only one write transaction may be active at a time, which guarantees that writers can never deadlock. The database structure is multi-versioned so readers run with no locks; writers cannot block readers, and readers don't block writers.

Unlike other well-known database mechanisms which use either write-ahead transaction logs or append-only data writes, MDBX requires no maintenance during operation. Both write-ahead loggers and append-only databases require periodic checkpointing and/or compaction of their log or database files otherwise they grow without bound. MDBX tracks retired/freed pages within the database and re-uses them for new write operations, so the database size does not grow without bound in normal use.

The memory map can be used as a read-only or read-write map. It is read-only by default as this provides total immunity to corruption. Using read-write mode offers much higher write performance, but adds the possibility for stray application writes thru pointers to silently corrupt the database.

Features
========

- Key-value data model, keys are always sorted.

- Fully ACID-compliant, through to MVCC and CoW.

- Multiple key-value sub-databases within a single datafile.

- Range lookups, including range query estimation.

- Efficient support for short fixed length keys, including native 32/64-bit integers.

- Ultra-efficient support for multimaps. Multi-values sorted, searchable and iterable. Keys stored without duplication.

- Data is memory-mapped and accessible directly/zero-copy. Traversal of database records is extremely-fast.

- Transactions for readers and writers, ones do not block others.

- Writes are strongly serialized. No transaction conflicts nor deadlocks.

- Readers are non-blocking, notwithstanding snapshot isolation.

- Nested write transactions.

- Reads scale linearly across CPUs.

- Continuous zero-overhead database compactification.

- Automatic on-the-fly database size adjustment.

- Customizable database page size.

- Olog(N) cost of lookup, insert, update, and delete operations by virtue of B+ tree characteristics.

- Online hot backup.

- Append operation for efficient bulk insertion of pre-sorted data.

- No WAL nor any transaction journal. No crash recovery needed. No maintenance is required.

- No internal cache and/or memory management, all done by basic OS services.

Limitations
===========

- Page size: a power of 2, maximum 65536 bytes, default 4096 bytes.

- Key size: minimum 0, maximum ?? pagesize (1300 bytes for default 4K pagesize, 21780 bytes for 64K pagesize).

- Value size: minimum 0, maximum 2146435072 (0x7FF00000) bytes for maps, ?? pagesize for multimaps (1348 bytes default 4K pagesize, 21828 bytes for 64K pagesize).

- Write transaction size: up to 4194301 (0x3FFFFD) pages (16 GiB for default 4K pagesize, 256 GiB for 64K pagesize).

- Database size: up to 2147483648 pages (8 TiB for default 4K pagesize, 128 TiB for 64K pagesize).

- Maximum sub-databases: 32765.

Gotchas
=======

- There cannot be more than one writer at a time, i.e. no more than one write transaction at a time.

- libmdbx is based on B+ tree, so access to database pages is mostly random. Thus SSDs provide a significant performance boost over spinning disks for large databases.

- libmdbx uses shadow paging instead of WAL. Thus syncing data to disk might be a bottleneck for write intensive workload.

- libmdbx uses copy-on-write for snapshot isolation during updates, but read transactions prevents recycling an old retired/freed pages, since it read ones. Thus altering of data during a parallel long-lived read operation will increase the process work set, may exhaust entire free database space, the database can grow quickly, and result in performance degradation. Try to avoid long running read transactions.

- libmdbx is extraordinarily fast and provides minimal overhead for data access, so you should reconsider using brute force techniques and double check your code. On the one hand, in the case of libmdbx, a simple linear search may be more profitable than complex indexes. On the other hand, if you make something suboptimally, you can notice detrimentally only on sufficiently large data.

--
Leonid Yuriev (1):
package/libmdbx: new package (library/database).

 DEVELOPERS                                    |  3 +
 package/Config.in                             |  1 +
 .../0001-mdbx-fix-minor-copy-paste-typo.patch | 27 ++++++++
 ...e-fix-missing-installation-of-mdbx.h.patch | 54 +++++++++++++++
 .../0003-mdbx-fix-obsolete-__noreturn.patch   | 27 ++++++++
 ...ield-instruction-on-ARM-if-unsupport.patch | 28 ++++++++
 ...fix-minor-false-positive-GCC-warning.patch | 27 ++++++++
 ...ad-workaround-for-buggy-toolchain-cm.patch | 65 +++++++++++++++++++
 ...mdbx-fix-pthread_yield-for-non-GLIBC.patch | 42 ++++++++++++
 package/libmdbx/Config.in                     | 42 ++++++++++++
 package/libmdbx/libmdbx.hash                  |  2 +
 package/libmdbx/libmdbx.mk                    | 24 +++++++
 12 files changed, 342 insertions(+)
 create mode 100644 package/libmdbx/0001-mdbx-fix-minor-copy-paste-typo.patch
 create mode 100644 package/libmdbx/0002-mdbx-cmake-fix-missing-installation-of-mdbx.h.patch
 create mode 100644 package/libmdbx/0003-mdbx-fix-obsolete-__noreturn.patch
 create mode 100644 package/libmdbx/0004-mdbx-don-t-use-yield-instruction-on-ARM-if-unsupport.patch
 create mode 100644 package/libmdbx/0005-mdbx-load-fix-minor-false-positive-GCC-warning.patch
 create mode 100644 package/libmdbx/0006-mdbx-cmake-pthread-workaround-for-buggy-toolchain-cm.patch
 create mode 100644 package/libmdbx/0007-mdbx-fix-pthread_yield-for-non-GLIBC.patch
 create mode 100644 package/libmdbx/Config.in
 create mode 100644 package/libmdbx/libmdbx.hash
 create mode 100644 package/libmdbx/libmdbx.mk
Signature · 1 line
2.28.0
#8 lore raw

PATCH v4 1/1 package/libmdbx: new package (library/database).

In reply to Леонид Юрьев

This patch adds libmdbx v0.9.1:
- libmdbx is one of the fastest compact embeddable key-value ACID database.
- libmdbx has a specific set of properties and capabilities, focused on creating unique lightweight solutions.
- libmdbx surpasses the legendary LMDB (Lightning Memory-Mapped Database) in terms of reliability, features and performance.

https://github.com/erthink/libmdbx

---
Changes v1 -> v2:
- libmdbx version v0.8.2 -> v0.9.1 (released 2020-09-30)

Changes v2 -> v3:
- removed outcommented stuff (suggested by Heiko Thiery).
- cleaned up and simplified the makefile (suggested by Heiko Thiery).
- added patch for C++ header installation.
- added patch with fix minor copy&paste typo.
- added patch with pthread workaround for buggy toolchain/cmake/buildroot.
- added patch for pthread_yield().
- passed utils/check-package package/libmdbx/* (suggested by Heiko Thiery)
- passed utils/test-pkg -a -p libmdbx,
except w/o threads & w/o MMU (suggested by Heiko Thiery).

Changes v3 -> v4:
- fix passing BR2_PACKAGE_LIBMDBX_TOOLS option to cmake.
- fix using depend on instead of select,
and add comment for C++ toolchain (suggested by Heiko Thiery).
- fix minor help typo.

---
 DEVELOPERS                                    |  3 +
 package/Config.in                             |  1 +
 .../0001-mdbx-fix-minor-copy-paste-typo.patch | 27 ++++++++
 ...e-fix-missing-installation-of-mdbx.h.patch | 54 +++++++++++++++
 .../0003-mdbx-fix-obsolete-__noreturn.patch   | 27 ++++++++
 ...ield-instruction-on-ARM-if-unsupport.patch | 28 ++++++++
 ...fix-minor-false-positive-GCC-warning.patch | 27 ++++++++
 ...ad-workaround-for-buggy-toolchain-cm.patch | 65 +++++++++++++++++++
 ...mdbx-fix-pthread_yield-for-non-GLIBC.patch | 42 ++++++++++++
 package/libmdbx/Config.in                     | 42 ++++++++++++
 package/libmdbx/libmdbx.hash                  |  2 +
 package/libmdbx/libmdbx.mk                    | 24 +++++++
 12 files changed, 342 insertions(+)
 create mode 100644 package/libmdbx/0001-mdbx-fix-minor-copy-paste-typo.patch
 create mode 100644 package/libmdbx/0002-mdbx-cmake-fix-missing-installation-of-mdbx.h.patch
 create mode 100644 package/libmdbx/0003-mdbx-fix-obsolete-__noreturn.patch
 create mode 100644 package/libmdbx/0004-mdbx-don-t-use-yield-instruction-on-ARM-if-unsupport.patch
 create mode 100644 package/libmdbx/0005-mdbx-load-fix-minor-false-positive-GCC-warning.patch
 create mode 100644 package/libmdbx/0006-mdbx-cmake-pthread-workaround-for-buggy-toolchain-cm.patch
 create mode 100644 package/libmdbx/0007-mdbx-fix-pthread_yield-for-non-GLIBC.patch
 create mode 100644 package/libmdbx/Config.in
 create mode 100644 package/libmdbx/libmdbx.hash
 create mode 100644 package/libmdbx/libmdbx.mk
DEVELOPERS +3 -0
index f8655fa054..924e70e30c 100644
@@ -1551,6 +1551,9 @@ N:	Leon Anavi <[email protected]>
 F:	board/olimex/a10_olinuxino
 F:	configs/olimex_a10_olinuxino_lime_defconfig
 
+N:	Leonid Yuriev <[email protected]>
+F:	package/libmdbx
+
 N:	Lionel Flandrin <[email protected]>
 F:	package/python-babel/
 F:	package/python-daemonize/
package/Config.in +1 -0
index 51583d07d6..f87e0388a0 100644
@@ -1370,6 +1370,7 @@ menu "Database"
 	source "package/kompexsqlite/Config.in"
 	source "package/leveldb/Config.in"
 	source "package/libgit2/Config.in"
+	source "package/libmdbx/Config.in"
 	source "package/libodb/Config.in"
 	source "package/libodb-boost/Config.in"
 	source "package/libodb-mysql/Config.in"
package/libmdbx/0001-mdbx-fix-minor-copy-paste-typo.patch new file +27 -0
new file mode 100644
index 0000000000..d9ff3c5e71
@@ -0,0 +1,27 @@
+From 5807e2eda046bff35946972289236bbef5c0c597 Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Thu, 1 Oct 2020 02:27:29 +0300
+Subject: [PATCH] mdbx++: fix minor copy&paste typo.
+
+Change-Id: I0af3e7ffbbd1231069a60f9f48880df3df2141d7
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ mdbx.h++ | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/mdbx.h++ b/mdbx.h++
+index 095ff9abb..2050f4b92 100644
+--- a/mdbx.h++
++++ b/mdbx.h++
+@@ -538,7 +538,7 @@ struct LIBMDBX_API_TYPE slice : public ::MDBX_val {
+   byte *from_base58(byte *dest, size_t dest_size,
+                     bool ignore_spaces = false) const;
+ 
+-  /// vReturns the buffer size in bytes needed for conversion
++  /// \brief Returns the buffer size in bytes needed for conversion
+   /// [Base58](https://en.wikipedia.org/wiki/Base58) dump to data.
+   MDBX_CXX11_CONSTEXPR size_t from_base58_bytes() const noexcept {
+     return length() / 11 * 8 + length() % 11 * 32 / 43;
+-- 
+2.28.0
+
package/libmdbx/0002-mdbx-cmake-fix-missing-installation-of-mdbx.h.patch new file +54 -0

54 lines not on this page · raw message

package/libmdbx/0003-mdbx-fix-obsolete-__noreturn.patch new file +27 -0
new file mode 100644
index 0000000000..cb04334220
@@ -0,0 +1,27 @@
+From 280ed17ea2dff464c9688fcc92079f36d8b74621 Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Sat, 3 Oct 2020 11:35:26 +0300
+Subject: [PATCH] mdbx: fix obsolete `__noreturn`.
+
+Change-Id: Ic78843d6f16de2a409c16ceecc7acb2ed8aa3e68
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ mdbx.c | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/mdbx.c b/mdbx.c
+index 04ea5140d..9212640fe 100644
+--- a/mdbx.c
++++ b/mdbx.c
+@@ -141,7 +141,7 @@ __extern_C void __assert(const char *, const char *, unsigned int, const char *)
+ #else
+     __nothrow
+ #endif /* __THROW */
+-    __noreturn;
++    MDBX_NORETURN;
+ #define __assert_fail(assertion, file, line, function)                         \
+   __assert(assertion, file, line, function)
+ 
+-- 
+2.28.0
+
package/libmdbx/0004-mdbx-don-t-use-yield-instruction-on-ARM-if-unsupport.patch new file +28 -0
new file mode 100644
index 0000000000..23f91d859b
@@ -0,0 +1,28 @@
+From 70b615e8d4d10cda2d961a815dd15eb87a1f3925 Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Sun, 4 Oct 2020 14:54:11 +0300
+Subject: [PATCH] mdbx: don't use `yield` instruction on ARM if unsupported.
+
+Change-Id: I0b01d783fe4336b089f4b051fb61c203b5879aa5
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ mdbx.c | 3 ++-
+ 1 file changed, 2 insertions(+), 1 deletion(-)
+
+diff --git a/mdbx.c b/mdbx.c
+index aba445424..a9ba1ee85 100644
+--- a/mdbx.c
++++ b/mdbx.c
+@@ -763,7 +763,8 @@ static __always_inline void atomic_yield(void) {
+ #else
+   __asm__ __volatile__("hint @pause");
+ #endif
+-#elif defined(__arm__) || defined(__aarch64__)
++#elif defined(__aarch64__) || (defined(__ARM_ARCH) && __ARM_ARCH > 6) ||       \
++    defined(__ARM_ARCH_6K__)
+ #ifdef __CC_ARM
+   __yield();
+ #else
+-- 
+2.28.0
+
package/libmdbx/0005-mdbx-load-fix-minor-false-positive-GCC-warning.patch new file +27 -0
new file mode 100644
index 0000000000..29d69e3faa
@@ -0,0 +1,27 @@
+From 8d4e7994c0c5c0cf69326139717e324f90bff65b Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Mon, 5 Oct 2020 15:09:10 +0300
+Subject: [PATCH] mdbx-load: fix minor false-positive GCC warning.
+
+Change-Id: Ie75c793712d050e8d3da76a4d0a8df9b81dc5275
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ mdbx_load.c | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/mdbx_load.c b/mdbx_load.c
+index e5e72fc8f..54e179553 100644
+--- a/mdbx_load.c
++++ b/mdbx_load.c
+@@ -318,7 +318,7 @@ static int readhdr(void) {
+     }
+ 
+     for (int i = 0; dbflags[i].bit; i++) {
+-      bool value;
++      bool value = false;
+       if (valbool(dbuf.iov_base, dbflags[i].name, &value)) {
+         if (value)
+           dbi_flags |= dbflags[i].bit;
+-- 
+2.28.0
+
package/libmdbx/0006-mdbx-cmake-pthread-workaround-for-buggy-toolchain-cm.patch new file +65 -0

65 lines not on this page · raw message

package/libmdbx/0007-mdbx-fix-pthread_yield-for-non-GLIBC.patch new file +42 -0

42 lines not on this page · raw message

package/libmdbx/Config.in new file +42 -0

42 lines not on this page · raw message

package/libmdbx/libmdbx.hash new file +2 -0
new file mode 100644
index 0000000000..f48cd81a7d
@@ -0,0 +1,2 @@
+sha256  310fe25c858a9515fc8c8d7d1f24a67c9496f84a91e0a0e41ea9975b1371e569  LICENSE
+sha256  c7fb24381eb4d92f2e2edc17e577cb721269683c816c6cca307c58f6f346e786  libmdbx-amalgamated-0.9.1.tar.gz
package/libmdbx/libmdbx.mk new file +24 -0
new file mode 100644
index 0000000000..be0c2d7b06
@@ -0,0 +1,24 @@
+################################################################################
+#
+# libmdbx
+#
+################################################################################
+
+LIBMDBX_VERSION = 0.9.1
+LIBMDBX_SOURCE = libmdbx-amalgamated-$(LIBMDBX_VERSION).tar.gz
+LIBMDBX_SITE = https://github.com/erthink/libmdbx/releases/download/v$(LIBMDBX_VERSION)
+LIBMDBX_SUPPORTS_IN_SOURCE_BUILD = NO
+LIBMDBX_LICENSE = OLDAP-2.8
+LIBMDBX_LICENSE_FILES = LICENSE
+LIBMDBX_REDISTRIBUTE = YES
+LIBMDBX_STRIP_COMPONENTS = 0
+LIBMDBX_INSTALL_STAGING = YES
+
+LIBMDBX_CONF_OPTS = -DMDBX_INSTALL_MANPAGES=OFF -DBUILD_FOR_NATIVE_CPU=OFF \
+	-DMDBX_INSTALL_STATIC=$(if $(BR2_STATIC_LIBS),ON,OFF) \
+	-DMDBX_BUILD_SHARED_LIBRARY=$(if $(BR2_SHARED_LIBS),ON,OFF) \
+	-DMDBX_BUILD_CXX=$(if $(BR2_PACKAGE_LIBMDBX_CXX),ON,OFF) \
+	-DMDBX_BUILD_TOOLS=$(if $(BR2_PACKAGE_LIBMDBX_TOOLS),ON,OFF) \
+	-DMDBX_LINK_TOOLS_NONSTATIC=$(if $(BR2_SHARED_LIBS),ON,OFF)
+
+$(eval $(cmake-package))
Signature · 1 line
2.28.0
#9 lore raw

PATCH v4 1/1 package/libmdbx: new package (library/database).

Hi,

Am Di., 6. Okt. 2020 um 21:27 Uhr schrieb Leonid Yuriev <[email protected]>:

Quoted 25 lines

This patch adds libmdbx v0.9.1:
- libmdbx is one of the fastest compact embeddable key-value ACID database.
- libmdbx has a specific set of properties and capabilities, focused on creating unique lightweight solutions.
- libmdbx surpasses the legendary LMDB (Lightning Memory-Mapped Database) in terms of reliability, features and performance.

https://github.com/erthink/libmdbx

---
Changes v1 -> v2:
- libmdbx version v0.8.2 -> v0.9.1 (released 2020-09-30)

Changes v2 -> v3:
- removed outcommented stuff (suggested by Heiko Thiery).
- cleaned up and simplified the makefile (suggested by Heiko Thiery).
- added patch for C++ header installation.
- added patch with fix minor copy&paste typo.
- added patch with pthread workaround for buggy toolchain/cmake/buildroot.
- added patch for pthread_yield().
- passed utils/check-package package/libmdbx/* (suggested by Heiko Thiery)
- passed utils/test-pkg -a -p libmdbx,
except w/o threads & w/o MMU (suggested by Heiko Thiery).

Quoted 7 lines

Changes v3 -> v4:
- fix passing BR2_PACKAGE_LIBMDBX_TOOLS option to cmake.
- fix using depend on instead of select,
and add comment for C++ toolchain (suggested by Heiko Thiery).
- fix minor help typo.

Quoted 456 lines
---
 DEVELOPERS                                    |  3 +
 package/Config.in                             |  1 +
 .../0001-mdbx-fix-minor-copy-paste-typo.patch | 27 ++++++++
 ...e-fix-missing-installation-of-mdbx.h.patch | 54 +++++++++++++++
 .../0003-mdbx-fix-obsolete-__noreturn.patch   | 27 ++++++++
 ...ield-instruction-on-ARM-if-unsupport.patch | 28 ++++++++
 ...fix-minor-false-positive-GCC-warning.patch | 27 ++++++++
 ...ad-workaround-for-buggy-toolchain-cm.patch | 65 +++++++++++++++++++
 ...mdbx-fix-pthread_yield-for-non-GLIBC.patch | 42 ++++++++++++
 package/libmdbx/Config.in                     | 42 ++++++++++++
 package/libmdbx/libmdbx.hash                  |  2 +
 package/libmdbx/libmdbx.mk                    | 24 +++++++
 12 files changed, 342 insertions(+)
 create mode 100644 package/libmdbx/0001-mdbx-fix-minor-copy-paste-typo.patch
 create mode 100644 package/libmdbx/0002-mdbx-cmake-fix-missing-installation-of-mdbx.h.patch
 create mode 100644 package/libmdbx/0003-mdbx-fix-obsolete-__noreturn.patch
 create mode 100644 package/libmdbx/0004-mdbx-don-t-use-yield-instruction-on-ARM-if-unsupport.patch
 create mode 100644 package/libmdbx/0005-mdbx-load-fix-minor-false-positive-GCC-warning.patch
 create mode 100644 package/libmdbx/0006-mdbx-cmake-pthread-workaround-for-buggy-toolchain-cm.patch
 create mode 100644 package/libmdbx/0007-mdbx-fix-pthread_yield-for-non-GLIBC.patch
 create mode 100644 package/libmdbx/Config.in
 create mode 100644 package/libmdbx/libmdbx.hash
 create mode 100644 package/libmdbx/libmdbx.mk
DEVELOPERS +3 -0
index f8655fa054..924e70e30c 100644
@@ -1551,6 +1551,9 @@ N:        Leon Anavi <[email protected]>
 F:     board/olimex/a10_olinuxino
 F:     configs/olimex_a10_olinuxino_lime_defconfig
 
+N:     Leonid Yuriev <[email protected]>
+F:     package/libmdbx
+
 N:     Lionel Flandrin <[email protected]>
 F:     package/python-babel/
 F:     package/python-daemonize/
package/Config.in +1 -0
index 51583d07d6..f87e0388a0 100644
@@ -1370,6 +1370,7 @@ menu "Database"
        source "package/kompexsqlite/Config.in"
        source "package/leveldb/Config.in"
        source "package/libgit2/Config.in"
+       source "package/libmdbx/Config.in"
        source "package/libodb/Config.in"
        source "package/libodb-boost/Config.in"
        source "package/libodb-mysql/Config.in"
package/libmdbx/0001-mdbx-fix-minor-copy-paste-typo.patch new file +27 -0
new file mode 100644
index 0000000000..d9ff3c5e71
@@ -0,0 +1,27 @@
+From 5807e2eda046bff35946972289236bbef5c0c597 Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Thu, 1 Oct 2020 02:27:29 +0300
+Subject: [PATCH] mdbx++: fix minor copy&paste typo.
+
+Change-Id: I0af3e7ffbbd1231069a60f9f48880df3df2141d7
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ mdbx.h++ | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/mdbx.h++ b/mdbx.h++
+index 095ff9abb..2050f4b92 100644
+--- a/mdbx.h++
++++ b/mdbx.h++
+@@ -538,7 +538,7 @@ struct LIBMDBX_API_TYPE slice : public ::MDBX_val {
+   byte *from_base58(byte *dest, size_t dest_size,
+                     bool ignore_spaces = false) const;
+
+-  /// vReturns the buffer size in bytes needed for conversion
++  /// \brief Returns the buffer size in bytes needed for conversion
+   /// [Base58](https://en.wikipedia.org/wiki/Base58) dump to data.
+   MDBX_CXX11_CONSTEXPR size_t from_base58_bytes() const noexcept {
+     return length() / 11 * 8 + length() % 11 * 32 / 43;
+--
+2.28.0
+
package/libmdbx/0002-mdbx-cmake-fix-missing-installation-of-mdbx.h.patch new file +54 -0
new file mode 100644
index 0000000000..df49342ccb
@@ -0,0 +1,54 @@
+From dc2cd19d566e84451ee47cf04277f97bc22771c5 Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Fri, 2 Oct 2020 00:05:02 +0300
+Subject: [PATCH] mdbx-cmake: fix missing installation of `mdbx.h++`
+
+Change-Id: I41975e4eeff6583a266b273b9a4f8486982ede90
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ CMakeLists.txt | 8 +++++---
+ 1 file changed, 5 insertions(+), 3 deletions(-)
+
+diff --git a/CMakeLists.txt b/CMakeLists.txt
+index 28db425c5..95f116521 100644
+--- a/CMakeLists.txt
++++ b/CMakeLists.txt
+@@ -485,7 +485,8 @@ fetch_version(MDBX "${CMAKE_CURRENT_SOURCE_DIR}" FALSE)
+ message(STATUS "libmdbx version is ${MDBX_VERSION}")
+
+ # sources list
+-set(LIBMDBX_SOURCES mdbx.h "${CMAKE_CURRENT_BINARY_DIR}/config.h" )
++set(LIBMDBX_PUBLIC_HEADERS mdbx.h)
++set(LIBMDBX_SOURCES mdbx.h "${CMAKE_CURRENT_BINARY_DIR}/config.h")
+ if(MDBX_AMALGAMATED_SOURCE)
+   list(APPEND LIBMDBX_SOURCES mdbx.c)
+ else()
+@@ -511,6 +512,7 @@ else()
+ endif(MDBX_AMALGAMATED_SOURCE)
+ if(MDBX_BUILD_CXX)
+   message(STATUS "Use C${MDBX_C_STANDARD} and C++${MDBX_CXX_STANDARD} for libmdbx")
++  list(APPEND LIBMDBX_PUBLIC_HEADERS mdbx.h++)
+   list(APPEND LIBMDBX_SOURCES "${MDBX_SOURCE_DIR}/mdbx.c++" mdbx.h++)
+ else()
+   message(STATUS "Use C${MDBX_C_STANDARD} for libmdbx but C++ portion is disabled")
+@@ -561,7 +563,7 @@ if(MDBX_INSTALL_STATIC)
+ else()
+   add_library(mdbx-static STATIC EXCLUDE_FROM_ALL ${LIBMDBX_SOURCES})
+ endif()
+-set_target_properties(mdbx-static PROPERTIES PUBLIC_HEADER mdbx.h)
++set_target_properties(mdbx-static PROPERTIES PUBLIC_HEADER "${LIBMDBX_PUBLIC_HEADERS}")
+ target_compile_definitions(mdbx-static PRIVATE MDBX_BUILD_SHARED_LIBRARY=0)
+ target_setup_options(mdbx-static)
+ libmdbx_setup_libs(mdbx-static INTERFACE)
+@@ -576,7 +578,7 @@ endif()
+ # build shared library
+ if(MDBX_BUILD_SHARED_LIBRARY)
+   add_library(mdbx SHARED ${LIBMDBX_SOURCES})
+-  set_target_properties(mdbx PROPERTIES PUBLIC_HEADER mdbx.h)
++  set_target_properties(mdbx PROPERTIES PUBLIC_HEADER "${LIBMDBX_PUBLIC_HEADERS}")
+   target_compile_definitions(mdbx PRIVATE LIBMDBX_EXPORTS MDBX_BUILD_SHARED_LIBRARY=1 INTERFACE LIBMDBX_IMPORTS)
+   target_setup_options(mdbx)
+   libmdbx_setup_libs(mdbx PRIVATE)
+--
+2.28.0
+
package/libmdbx/0003-mdbx-fix-obsolete-__noreturn.patch new file +27 -0
new file mode 100644
index 0000000000..cb04334220
@@ -0,0 +1,27 @@
+From 280ed17ea2dff464c9688fcc92079f36d8b74621 Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Sat, 3 Oct 2020 11:35:26 +0300
+Subject: [PATCH] mdbx: fix obsolete `__noreturn`.
+
+Change-Id: Ic78843d6f16de2a409c16ceecc7acb2ed8aa3e68
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ mdbx.c | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/mdbx.c b/mdbx.c
+index 04ea5140d..9212640fe 100644
+--- a/mdbx.c
++++ b/mdbx.c
+@@ -141,7 +141,7 @@ __extern_C void __assert(const char *, const char *, unsigned int, const char *)
+ #else
+     __nothrow
+ #endif /* __THROW */
+-    __noreturn;
++    MDBX_NORETURN;
+ #define __assert_fail(assertion, file, line, function)                         \
+   __assert(assertion, file, line, function)
+
+--
+2.28.0
+
package/libmdbx/0004-mdbx-don-t-use-yield-instruction-on-ARM-if-unsupport.patch new file +28 -0
new file mode 100644
index 0000000000..23f91d859b
@@ -0,0 +1,28 @@
+From 70b615e8d4d10cda2d961a815dd15eb87a1f3925 Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Sun, 4 Oct 2020 14:54:11 +0300
+Subject: [PATCH] mdbx: don't use `yield` instruction on ARM if unsupported.
+
+Change-Id: I0b01d783fe4336b089f4b051fb61c203b5879aa5
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ mdbx.c | 3 ++-
+ 1 file changed, 2 insertions(+), 1 deletion(-)
+
+diff --git a/mdbx.c b/mdbx.c
+index aba445424..a9ba1ee85 100644
+--- a/mdbx.c
++++ b/mdbx.c
+@@ -763,7 +763,8 @@ static __always_inline void atomic_yield(void) {
+ #else
+   __asm__ __volatile__("hint @pause");
+ #endif
+-#elif defined(__arm__) || defined(__aarch64__)
++#elif defined(__aarch64__) || (defined(__ARM_ARCH) && __ARM_ARCH > 6) ||       \
++    defined(__ARM_ARCH_6K__)
+ #ifdef __CC_ARM
+   __yield();
+ #else
+--
+2.28.0
+
package/libmdbx/0005-mdbx-load-fix-minor-false-positive-GCC-warning.patch new file +27 -0
new file mode 100644
index 0000000000..29d69e3faa
@@ -0,0 +1,27 @@
+From 8d4e7994c0c5c0cf69326139717e324f90bff65b Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Mon, 5 Oct 2020 15:09:10 +0300
+Subject: [PATCH] mdbx-load: fix minor false-positive GCC warning.
+
+Change-Id: Ie75c793712d050e8d3da76a4d0a8df9b81dc5275
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ mdbx_load.c | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/mdbx_load.c b/mdbx_load.c
+index e5e72fc8f..54e179553 100644
+--- a/mdbx_load.c
++++ b/mdbx_load.c
+@@ -318,7 +318,7 @@ static int readhdr(void) {
+     }
+
+     for (int i = 0; dbflags[i].bit; i++) {
+-      bool value;
++      bool value = false;
+       if (valbool(dbuf.iov_base, dbflags[i].name, &value)) {
+         if (value)
+           dbi_flags |= dbflags[i].bit;
+--
+2.28.0
+
package/libmdbx/0006-mdbx-cmake-pthread-workaround-for-buggy-toolchain-cm.patch new file +65 -0
new file mode 100644
index 0000000000..37525cf0d5
@@ -0,0 +1,65 @@
+From 787eaaa373073e17f3a53658b085c255bc2c8ff8 Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Mon, 5 Oct 2020 19:12:20 +0300
+Subject: [PATCH] mdbx-cmake: pthread workaround for buggy
+ toolchain/cmake/buildroot.
+
+Change-Id: I0d95e783abbd10a63cd1595a9de50593e814a967
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ CMakeLists.txt | 21 ++++++++++++++++++---
+ 1 file changed, 18 insertions(+), 3 deletions(-)
+
+diff --git a/CMakeLists.txt b/CMakeLists.txt
+index 95f116521..20a50a453 100644
+--- a/CMakeLists.txt
++++ b/CMakeLists.txt
+@@ -173,10 +173,27 @@ if(NOT CMAKE_BUILD_TYPE)
+ endif()
+ string(TOUPPER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE_UPPERCASE)
+
++set(CMAKE_THREAD_PREFER_PTHREAD TRUE)
++set(THREADS_PREFER_PTHREAD_FLAG TRUE)
++find_package(Threads REQUIRED)
++
+ include(cmake/utils.cmake)
+ include(cmake/compiler.cmake)
+ include(cmake/profile.cmake)
+
++# Workaround for `-pthread` toolchain/cmake bug
++if(NOT APPLE AND NOT MSVC
++    AND CMAKE_USE_PTHREADS_INIT AND NOT CMAKE_THREAD_LIBS_INIT
++    AND (CMAKE_COMPILER_IS_GNUCC OR CMAKE_COMPILER_IS_CLANG))
++  check_compiler_flag("-pthread" CC_HAS_PTHREAD)
++  if(CC_HAS_PTHREAD AND NOT CMAKE_EXE_LINKER_FLAGS MATCHES "-pthread")
++    message(STATUS "Force add -pthread for linker flags to avoid troubles")
++    set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -pthread")
++    set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -pthread")
++    set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} -pthread")
++  endif()
++endif()
++
+ CHECK_FUNCTION_EXISTS(pow NOT_NEED_LIBM)
+ if(NOT_NEED_LIBM)
+   set(LIB_MATH "")
+@@ -190,8 +207,6 @@ else()
+   endif()
+ endif()
+
+-find_package(Threads REQUIRED)
+-
+ if(SUBPROJECT)
+   if(NOT DEFINED BUILD_SHARED_LIBS)
+     option(BUILD_SHARED_LIBS "Build shared libraries (DLLs)" OFF)
+@@ -541,7 +556,7 @@ macro(target_setup_options TARGET)
+ endmacro()
+
+ macro(libmdbx_setup_libs TARGET MODE)
+-  target_link_libraries(${TARGET} ${MODE} ${CMAKE_THREAD_LIBS_INIT})
++  target_link_libraries(${TARGET} ${MODE} Threads::Threads)
+   if(${CMAKE_SYSTEM_NAME} STREQUAL "Windows")
+     target_link_libraries(${TARGET} ${MODE} ntdll.lib)
+     if(MDBX_NTDLL_EXTRA_IMPLIB AND MDBX_AVOID_CRT)
+--
+2.28.0
+
package/libmdbx/0007-mdbx-fix-pthread_yield-for-non-GLIBC.patch new file +42 -0
new file mode 100644
index 0000000000..5d7e83d4c7
@@ -0,0 +1,42 @@
+From 16d43332a9d28ba27aa3d127e8f6b77a87b2f5eb Mon Sep 17 00:00:00 2001
+From: Leonid Yuriev <[email protected]>
+Date: Mon, 5 Oct 2020 21:08:09 +0300
+Subject: [PATCH] mdbx: fix `pthread_yield()` for non-GLIBC.
+
+Change-Id: I080e37a42b62e524896dea8747e9f23e2fcd584f
+Signed-off-by: Leonid Yuriev <[email protected]>
+---
+ mdbx.c | 4 +++-
+ mdbx.c | 1 +
+ 2 files changed, 4 insertions(+), 1 deletion(-)
+
+diff --git a/mdbx.c b/mdbx.c
+index a9ba1ee85..ace6f2756 100644
+--- a/mdbx.c
++++ b/mdbx.c
+@@ -777,7 +777,9 @@ static __always_inline void atomic_yield(void) {
+     defined(__mips64__) || defined(_M_MRX000) || defined(_MIPS_) ||            \
+     defined(__MWERKS__) || defined(__sgi)
+   __asm__ __volatile__(".word 0x00000140");
+-#else
++#elif defined(__linux__) || defined(__gnu_linux__) || defined(_UNIX03_SOURCE)
++  sched_yield();
++#elif (defined(_GNU_SOURCE) &&  __GLIBC_PREREQ(2, 1)) || defined(_OPEN_THREADS)
+   pthread_yield();
+ #endif
+ }
+diff --git a/mdbx.c b/mdbx.c
+index 3509942dc..3af23ce0b 100644
+--- a/mdbx.c
++++ b/mdbx.c
+@@ -119,6 +119,7 @@
+
+ #if defined(__linux__) || defined(__gnu_linux__)
+ #include <linux/sysctl.h>
++#include <sched.h>
+ #include <sys/sendfile.h>
+ #include <sys/statfs.h>
+ #endif /* Linux */
+--
+2.28.0
+
package/libmdbx/Config.in new file +42 -0
new file mode 100644
index 0000000000..dc36f12348
@@ -0,0 +1,42 @@
+config BR2_PACKAGE_LIBMDBX
+       bool "libmdbx"
+       depends on BR2_USE_MMU
+       depends on BR2_TOOLCHAIN_HAS_THREADS
+       depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_4
+       depends on BR2_TOOLCHAIN_HAS_SYNC_4
+       help
+         One of the fastest compact key-value ACID database
+         without WAL. libmdbx has a specific set of properties
+         and capabilities, focused on creating unique lightweight
+         solutions.
+
+         libmdbx surpasses the legendary LMDB in terms of
+         reliability, features and performance.
+
+         https://github.com/erthink/libmdbx
+
+if BR2_PACKAGE_LIBMDBX
+
+config BR2_PACKAGE_LIBMDBX_TOOLS
+       bool "install tools"
+       help
+         Install libmdbx tools for checking, dump, restore
+         and show statistics of databases.
+
+config BR2_PACKAGE_LIBMDBX_CXX
+       bool "C++ API"
+       depends on BR2_INSTALL_LIBSTDCPP
+       depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8
+       help
+         Enable modern C++11/14/17/20 API for libmdbx.
+
+comment "libmdbx C++ support needs a toolchain w/ C++11, gcc >= 4.8"
+       depends on !BR2_INSTALL_LIBSTDCPP || \
+               !BR2_TOOLCHAIN_GCC_AT_LEAST_4_8
+
+endif
+
+comment "libmdbx needs a toolchain w/ threads, gcc >= 4.4"
+       depends on BR2_USE_MMU
+       depends on !BR2_TOOLCHAIN_HAS_THREADS || \
+               !BR2_TOOLCHAIN_GCC_AT_LEAST_4_4
package/libmdbx/libmdbx.hash new file +2 -0
new file mode 100644
index 0000000000..f48cd81a7d
@@ -0,0 +1,2 @@
+sha256  310fe25c858a9515fc8c8d7d1f24a67c9496f84a91e0a0e41ea9975b1371e569  LICENSE
+sha256  c7fb24381eb4d92f2e2edc17e577cb721269683c816c6cca307c58f6f346e786  libmdbx-amalgamated-0.9.1.tar.gz
package/libmdbx/libmdbx.mk new file +24 -1
new file mode 100644
index 0000000000..be0c2d7b06
@@ -0,0 +1,24 @@
+################################################################################
+#
+# libmdbx
+#
+################################################################################
+
+LIBMDBX_VERSION = 0.9.1
+LIBMDBX_SOURCE = libmdbx-amalgamated-$(LIBMDBX_VERSION).tar.gz
+LIBMDBX_SITE = https://github.com/erthink/libmdbx/releases/download/v$(LIBMDBX_VERSION)
+LIBMDBX_SUPPORTS_IN_SOURCE_BUILD = NO
+LIBMDBX_LICENSE = OLDAP-2.8
+LIBMDBX_LICENSE_FILES = LICENSE
+LIBMDBX_REDISTRIBUTE = YES
+LIBMDBX_STRIP_COMPONENTS = 0
+LIBMDBX_INSTALL_STAGING = YES
+
+LIBMDBX_CONF_OPTS = -DMDBX_INSTALL_MANPAGES=OFF -DBUILD_FOR_NATIVE_CPU=OFF \
+       -DMDBX_INSTALL_STATIC=$(if $(BR2_STATIC_LIBS),ON,OFF) \
+       -DMDBX_BUILD_SHARED_LIBRARY=$(if $(BR2_SHARED_LIBS),ON,OFF) \
+       -DMDBX_BUILD_CXX=$(if $(BR2_PACKAGE_LIBMDBX_CXX),ON,OFF) \
+       -DMDBX_BUILD_TOOLS=$(if $(BR2_PACKAGE_LIBMDBX_TOOLS),ON,OFF) \
+       -DMDBX_LINK_TOOLS_NONSTATIC=$(if $(BR2_SHARED_LIBS),ON,OFF)
+
+$(eval $(cmake-package))
--

2.28.0

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

#10 lore raw

PATCH v4 1/1 package/libmdbx: new package (library/database).

In reply to Leonid Yuriev

Hello Leonid,

Thanks for this contribution and new iteration!

Quoted 9 lines · On Tue, 6 Oct 2020 22:27:09 +0300 Leonid Yuriev <[email protected]> wrote:

This patch adds libmdbx v0.9.1:
- libmdbx is one of the fastest compact embeddable key-value ACID database.
- libmdbx has a specific set of properties and capabilities, focused on creating unique lightweight solutions.
- libmdbx surpasses the legendary LMDB (Lightning Memory-Mapped Database) in terms of reliability, features and performance.

https://github.com/erthink/libmdbx

Your Signed-off-by line should be here... and not after the changelog.

Quoted 24 lines

---
Changes v1 -> v2:
- libmdbx version v0.8.2 -> v0.9.1 (released 2020-09-30)

Changes v2 -> v3:
- removed outcommented stuff (suggested by Heiko Thiery).
- cleaned up and simplified the makefile (suggested by Heiko Thiery).
- added patch for C++ header installation.
- added patch with fix minor copy&paste typo.
- added patch with pthread workaround for buggy toolchain/cmake/buildroot.
- added patch for pthread_yield().
- passed utils/check-package package/libmdbx/* (suggested by Heiko Thiery)
- passed utils/test-pkg -a -p libmdbx,
except w/o threads & w/o MMU (suggested by Heiko Thiery).

Changes v3 -> v4:
- fix passing BR2_PACKAGE_LIBMDBX_TOOLS option to cmake.
- fix using depend on instead of select,
and add comment for C++ toolchain (suggested by Heiko Thiery).
- fix minor help typo.

DEVELOPERS N: Leon Anavi <[email protected]> · line 1555

Quoted 2 lines
DEVELOPERS +2 -0
+N:	Leonid Yuriev <[email protected]>
+F:	package/libmdbx

Final / missing.

Quoted 22 lines
package/libmdbx/Config.in new file +16 -0
new file mode 100644
index 0000000000..dc36f12348
@@ -0,0 +1,42 @@
+config BR2_PACKAGE_LIBMDBX
+	bool "libmdbx"
+	depends on BR2_USE_MMU
+	depends on BR2_TOOLCHAIN_HAS_THREADS
+	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_4
+	depends on BR2_TOOLCHAIN_HAS_SYNC_4
+	help
+	  One of the fastest compact key-value ACID database
+	  without WAL. libmdbx has a specific set of properties
+	  and capabilities, focused on creating unique lightweight
+	  solutions.
+
+	  libmdbx surpasses the legendary LMDB in terms of
+	  reliability, features and performance.
+
+	  https://github.com/erthink/libmdbx

package/libmdbx/Config.in line 26

Quoted 10 lines
package/libmdbx/Config.in new file +10 -0
+
+if BR2_PACKAGE_LIBMDBX
+
+config BR2_PACKAGE_LIBMDBX_TOOLS
+	bool "install tools"
+	help
+	  Install libmdbx tools for checking, dump, restore
+	  and show statistics of databases.
+
+config BR2_PACKAGE_LIBMDBX_CXX

I don't think this option is needed, just enable C++ support in the .mk file when the relevant conditions are met.

Quoted 4 lines
package/libmdbx/Config.in new file +4 -0
+	bool "C++ API"
+	depends on BR2_INSTALL_LIBSTDCPP
+	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8
+	help

package/libmdbx/Config.in line 40

Quoted 10 lines
package/libmdbx/Config.in new file +10 -0
+	  Enable modern C++11/14/17/20 API for libmdbx.
+
+comment "libmdbx C++ support needs a toolchain w/ C++11, gcc >= 4.8"
+	depends on !BR2_INSTALL_LIBSTDCPP || \
+		!BR2_TOOLCHAIN_GCC_AT_LEAST_4_8
+
+endif
+
+comment "libmdbx needs a toolchain w/ threads, gcc >= 4.4"
+	depends on BR2_USE_MMU

depends on BR2_TOOLCHAIN_HAS_SYNC_4

should be added here as well.

package/libmdbx/libmdbx.hash

Quoted 8 lines
package/libmdbx/Config.in new file +2 -0
+	depends on !BR2_TOOLCHAIN_HAS_THREADS || \
+		!BR2_TOOLCHAIN_GCC_AT_LEAST_4_4
package/libmdbx/libmdbx.hash new file +0 -0
new file mode 100644
index 0000000000..f48cd81a7d
@@ -0,0 +1,2 @@

# Locally calculated

package/libmdbx/libmdbx.hash line 2

Quoted 2 lines
package/libmdbx/libmdbx.hash new file +2 -0
+sha256  310fe25c858a9515fc8c8d7d1f24a67c9496f84a91e0a0e41ea9975b1371e569  LICENSE
+sha256  c7fb24381eb4d92f2e2edc17e577cb721269683c816c6cca307c58f6f346e786  libmdbx-amalgamated-0.9.1.tar.gz

Please put the tarball hash before the license hash.

Quoted 9 lines
package/libmdbx/libmdbx.mk new file +3 -0
new file mode 100644
index 0000000000..be0c2d7b06
@@ -0,0 +1,24 @@
+################################################################################
+#
+# libmdbx

package/libmdbx/libmdbx.mk line 13

Quoted 10 lines
package/libmdbx/libmdbx.mk new file +10 -0
+#
+################################################################################
+
+LIBMDBX_VERSION = 0.9.1
+LIBMDBX_SOURCE = libmdbx-amalgamated-$(LIBMDBX_VERSION).tar.gz
+LIBMDBX_SITE = https://github.com/erthink/libmdbx/releases/download/v$(LIBMDBX_VERSION)
+LIBMDBX_SUPPORTS_IN_SOURCE_BUILD = NO
+LIBMDBX_LICENSE = OLDAP-2.8
+LIBMDBX_LICENSE_FILES = LICENSE
+LIBMDBX_REDISTRIBUTE = YES

Not needed, this is the default.

package/libmdbx/libmdbx.mk line 14

Quoted 1 line
package/libmdbx/libmdbx.mk new file +1 -0
+LIBMDBX_STRIP_COMPONENTS = 0

Quite odd, why do you generate your tarballs like this ?

package/libmdbx/libmdbx.mk line 19

Quoted 5 lines
package/libmdbx/libmdbx.mk new file +5 -0
+LIBMDBX_INSTALL_STAGING = YES
+
+LIBMDBX_CONF_OPTS = -DMDBX_INSTALL_MANPAGES=OFF -DBUILD_FOR_NATIVE_CPU=OFF \
+	-DMDBX_INSTALL_STATIC=$(if $(BR2_STATIC_LIBS),ON,OFF) \
+	-DMDBX_BUILD_SHARED_LIBRARY=$(if $(BR2_SHARED_LIBS),ON,OFF) \

This is not good as it doesn't account for the BR2_SHARED_STATIC_LIBS situation. So it should be:

ifeq ($(BR2_STATIC_LIBS)$(BR2_SHARED_STATIC_LIBS),y) LIBMDBX_CONF_OPTS += -DMDBX_INSTALL_STATIC=ON else
LIBMDBX_CONF_OPTS += -DMDBX_INSTALL_STATIC=OFF endif

ifeq ($(BR2_SHARED_LIBS)$(BR2_SHARED_STATIC_LIBS),y) LIBMDBX_CONF_OPTS += -DMDBX_BUILD_SHARED_LIBRARY=ON else
LIBMDBX_CONF_OPTS += -DMDBX_BUILD_SHARED_LIBRARY=OFF endif

package/libmdbx/libmdbx.mk line 22

Quoted 3 lines
package/libmdbx/libmdbx.mk new file +3 -0
+	-DMDBX_BUILD_CXX=$(if $(BR2_PACKAGE_LIBMDBX_CXX),ON,OFF) \
+	-DMDBX_BUILD_TOOLS=$(if $(BR2_PACKAGE_LIBMDBX_TOOLS),ON,OFF) \
+	-DMDBX_LINK_TOOLS_NONSTATIC=$(if $(BR2_SHARED_LIBS),ON,OFF)

Ditto here.

However, the bigger problem is that it doesn't build for me. I used the following defconfig:

BR2_arm=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-full-2020.02.tar.bz2" BR2_TOOLCHAIN_EXTERNAL_GCC_5=y
BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_14=y
BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
BR2_TOOLCHAIN_EXTERNAL_CXX=y
BR2_INIT_NONE=y
BR2_SYSTEM_BIN_SH_NONE=y
# BR2_PACKAGE_BUSYBOX is not set
BR2_PACKAGE_LIBMDBX=y
BR2_PACKAGE_LIBMDBX_TOOLS=y
BR2_PACKAGE_LIBMDBX_CXX=y
# BR2_TARGET_ROOTFS_TAR is not set

And it fails like this:

In file included from /home/thomas/projets/buildroot/output/build/libmdbx-0.9.1/mdbx.c++:105:0:
/home/thomas/projets/buildroot/output/build/libmdbx-0.9.1/mdbx.h++:223:10: error: ?exception_ptr? in namespace ?std? does not name a type

   ::std::exception_ptr captured_;
          ^
/home/thomas/projets/buildroot/output/build/libmdbx-0.9.1/mdbx.h++: In member function ?bool mdbx::exception_thunk::is_clean() const?:
/home/thomas/projets/buildroot/output/build/libmdbx-0.9.1/mdbx.h++:2730:66: error: ?captured_? was not declared in this scope
 inline bool exception_thunk::is_clean() const noexcept { return !captured_; }
                                                                  ^

/home/thomas/projets/buildroot/output/build/libmdbx-0.9.1/mdbx.h++: In member function ?void mdbx::exception_thunk::capture()?:
/home/thomas/projets/buildroot/output/build/libmdbx-0.9.1/mdbx.h++:2734:3: error: ?captured_? was not declared in this scope
captured_ = ::std::current_exception();
^
/home/thomas/projets/buildroot/output/build/libmdbx-0.9.1/mdbx.h++:2734:15: error: ?current_exception? is not a member of ?std?

   captured_ = ::std::current_exception();
               ^
/home/thomas/projets/buildroot/output/build/libmdbx-0.9.1/mdbx.h++: In member function ?void mdbx::exception_thunk::rethrow_captured() const?:
/home/thomas/projets/buildroot/output/build/libmdbx-0.9.1/mdbx.h++:2738:7: error: ?captured_? was not declared in this scope
   if (captured_)
       ^
/home/thomas/projets/buildroot/output/build/libmdbx-0.9.1/mdbx.h++:2739:25: error: ?rethrow_exception? is not a member of ?std?
     MDBX_CXX20_UNLIKELY ::std::rethrow_exception(captured_);
                         ^

make[3]: *** [CMakeFiles/mdbx.dir/build.make:96: CMakeFiles/mdbx.dir/mdbx.c++.o] Error 1

Some older versions of gcc had an issue with exception_ptr, see BR2_TOOLCHAIN_HAS_GCC_BUG_64735. You might need to depend on !BR2_TOOLCHAIN_HAS_GCC_BUG_64735. See other packages that have this dependency in Buildroot.

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#11 lore raw

PATCH v4 1/1 package/libmdbx: new package (library/database).

Hi, Thomas.

Thank you for the review, I will definitely take your comments and suggestions into account when preparing the next version of my patch. The next version/iteration will be based on the next release of libmdbx library, i.e. some internal patches will be dropped.

I am currently working on the next release of the library, which had to be postponed due to a couple of bugs.
This was the reason for the delay in my response, sorry.

Regards,
Leonid.

Quoted 235 lines · Thomas Petazzoni

Hello Leonid,

Thanks for this contribution and new iteration!

Quoted 9 lines · On Tue, 6 Oct 2020 22:27:09 +0300 Leonid Yuriev <[email protected]> wrote:

This patch adds libmdbx v0.9.1:
- libmdbx is one of the fastest compact embeddable key-value ACID database.
- libmdbx has a specific set of properties and capabilities, focused on creating unique lightweight solutions.
- libmdbx surpasses the legendary LMDB (Lightning Memory-Mapped Database) in terms of reliability, features and performance.

https://github.com/erthink/libmdbx

Your Signed-off-by line should be here... and not after the changelog.

Quoted 24 lines

---
Changes v1 -> v2:
- libmdbx version v0.8.2 -> v0.9.1 (released 2020-09-30)

Changes v2 -> v3:
- removed outcommented stuff (suggested by Heiko Thiery).
- cleaned up and simplified the makefile (suggested by Heiko Thiery).
- added patch for C++ header installation.
- added patch with fix minor copy&paste typo.
- added patch with pthread workaround for buggy toolchain/cmake/buildroot.
- added patch for pthread_yield().
- passed utils/check-package package/libmdbx/* (suggested by Heiko Thiery)
- passed utils/test-pkg -a -p libmdbx,
except w/o threads & w/o MMU (suggested by Heiko Thiery).

Changes v3 -> v4:
- fix passing BR2_PACKAGE_LIBMDBX_TOOLS option to cmake.
- fix using depend on instead of select,
and add comment for C++ toolchain (suggested by Heiko Thiery).
- fix minor help typo.

DEVELOPERS N: Leon Anavi <[email protected]> · line 1555

Quoted 2 lines
DEVELOPERS +2 -0
+N:   Leonid Yuriev <[email protected]>
+F:   package/libmdbx

Final / missing.

Quoted 22 lines
package/libmdbx/Config.in new file +16 -0
new file mode 100644
index 0000000000..dc36f12348
@@ -0,0 +1,42 @@
+config BR2_PACKAGE_LIBMDBX
+     bool "libmdbx"
+     depends on BR2_USE_MMU
+     depends on BR2_TOOLCHAIN_HAS_THREADS
+     depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_4
+     depends on BR2_TOOLCHAIN_HAS_SYNC_4
+     help
+       One of the fastest compact key-value ACID database
+       without WAL. libmdbx has a specific set of properties
+       and capabilities, focused on creating unique lightweight
+       solutions.
+
+       libmdbx surpasses the legendary LMDB in terms of
+       reliability, features and performance.
+
+       https://github.com/erthink/libmdbx

package/libmdbx/Config.in line 26

Quoted 10 lines
package/libmdbx/Config.in new file +10 -0
+
+if BR2_PACKAGE_LIBMDBX
+
+config BR2_PACKAGE_LIBMDBX_TOOLS
+     bool "install tools"
+     help
+       Install libmdbx tools for checking, dump, restore
+       and show statistics of databases.
+
+config BR2_PACKAGE_LIBMDBX_CXX

I don't think this option is needed, just enable C++ support in the .mk file when the relevant conditions are met.

Quoted 4 lines
package/libmdbx/Config.in new file +4 -0
+     bool "C++ API"
+     depends on BR2_INSTALL_LIBSTDCPP
+     depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8
+     help

package/libmdbx/Config.in line 40

Quoted 10 lines
package/libmdbx/Config.in new file +10 -0
+       Enable modern C++11/14/17/20 API for libmdbx.
+
+comment "libmdbx C++ support needs a toolchain w/ C++11, gcc >= 4.8"
+     depends on !BR2_INSTALL_LIBSTDCPP || \
+             !BR2_TOOLCHAIN_GCC_AT_LEAST_4_8
+
+endif
+
+comment "libmdbx needs a toolchain w/ threads, gcc >= 4.4"
+     depends on BR2_USE_MMU

depends on BR2_TOOLCHAIN_HAS_SYNC_4

should be added here as well.

package/libmdbx/libmdbx.hash

Quoted 8 lines
package/libmdbx/Config.in new file +2 -0
+     depends on !BR2_TOOLCHAIN_HAS_THREADS || \
+             !BR2_TOOLCHAIN_GCC_AT_LEAST_4_4
package/libmdbx/libmdbx.hash new file +0 -0
new file mode 100644
index 0000000000..f48cd81a7d
@@ -0,0 +1,2 @@

# Locally calculated

package/libmdbx/libmdbx.hash line 2

Quoted 2 lines
package/libmdbx/libmdbx.hash new file +2 -0
+sha256  310fe25c858a9515fc8c8d7d1f24a67c9496f84a91e0a0e41ea9975b1371e569  LICENSE
+sha256  c7fb24381eb4d92f2e2edc17e577cb721269683c816c6cca307c58f6f346e786  libmdbx-amalgamated-0.9.1.tar.gz

Please put the tarball hash before the license hash.

Quoted 9 lines
package/libmdbx/libmdbx.mk new file +3 -0
new file mode 100644
index 0000000000..be0c2d7b06
@@ -0,0 +1,24 @@
+################################################################################
+#
+# libmdbx

package/libmdbx/libmdbx.mk line 13

Quoted 10 lines
package/libmdbx/libmdbx.mk new file +10 -0
+#
+################################################################################
+
+LIBMDBX_VERSION = 0.9.1
+LIBMDBX_SOURCE = libmdbx-amalgamated-$(LIBMDBX_VERSION).tar.gz
+LIBMDBX_SITE = https://github.com/erthink/libmdbx/releases/download/v$(LIBMDBX_VERSION)
+LIBMDBX_SUPPORTS_IN_SOURCE_BUILD = NO
+LIBMDBX_LICENSE = OLDAP-2.8
+LIBMDBX_LICENSE_FILES = LICENSE
+LIBMDBX_REDISTRIBUTE = YES

Not needed, this is the default.

package/libmdbx/libmdbx.mk line 14

Quoted 1 line
package/libmdbx/libmdbx.mk new file +1 -0
+LIBMDBX_STRIP_COMPONENTS = 0

Quite odd, why do you generate your tarballs like this ?

package/libmdbx/libmdbx.mk line 19

Quoted 5 lines
package/libmdbx/libmdbx.mk new file +5 -0
+LIBMDBX_INSTALL_STAGING = YES
+
+LIBMDBX_CONF_OPTS = -DMDBX_INSTALL_MANPAGES=OFF -DBUILD_FOR_NATIVE_CPU=OFF \
+     -DMDBX_INSTALL_STATIC=$(if $(BR2_STATIC_LIBS),ON,OFF) \
+     -DMDBX_BUILD_SHARED_LIBRARY=$(if $(BR2_SHARED_LIBS),ON,OFF) \

This is not good as it doesn't account for the BR2_SHARED_STATIC_LIBS situation. So it should be:

ifeq ($(BR2_STATIC_LIBS)$(BR2_SHARED_STATIC_LIBS),y) LIBMDBX_CONF_OPTS += -DMDBX_INSTALL_STATIC=ON else
LIBMDBX_CONF_OPTS += -DMDBX_INSTALL_STATIC=OFF endif

ifeq ($(BR2_SHARED_LIBS)$(BR2_SHARED_STATIC_LIBS),y) LIBMDBX_CONF_OPTS += -DMDBX_BUILD_SHARED_LIBRARY=ON else
LIBMDBX_CONF_OPTS += -DMDBX_BUILD_SHARED_LIBRARY=OFF endif

package/libmdbx/libmdbx.mk line 22

Quoted 3 lines
package/libmdbx/libmdbx.mk new file +3 -0
+     -DMDBX_BUILD_CXX=$(if $(BR2_PACKAGE_LIBMDBX_CXX),ON,OFF) \
+     -DMDBX_BUILD_TOOLS=$(if $(BR2_PACKAGE_LIBMDBX_TOOLS),ON,OFF) \
+     -DMDBX_LINK_TOOLS_NONSTATIC=$(if $(BR2_SHARED_LIBS),ON,OFF)

Ditto here.

However, the bigger problem is that it doesn't build for me. I used the following defconfig:

BR2_arm=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-full-2020.02.tar.bz2" BR2_TOOLCHAIN_EXTERNAL_GCC_5=y
BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_14=y
BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
BR2_TOOLCHAIN_EXTERNAL_CXX=y
BR2_INIT_NONE=y
BR2_SYSTEM_BIN_SH_NONE=y
# BR2_PACKAGE_BUSYBOX is not set
BR2_PACKAGE_LIBMDBX=y
BR2_PACKAGE_LIBMDBX_TOOLS=y
BR2_PACKAGE_LIBMDBX_CXX=y
# BR2_TARGET_ROOTFS_TAR is not set

And it fails like this:

In file included from /home/thomas/projets/buildroot/output/build/libmdbx-0.9.1/mdbx.c++:105:0:
/home/thomas/projets/buildroot/output/build/libmdbx-0.9.1/mdbx.h++:223:10: error: ?exception_ptr? in namespace ?std? does not name a type

   ::std::exception_ptr captured_;
          ^
/home/thomas/projets/buildroot/output/build/libmdbx-0.9.1/mdbx.h++: In member function ?bool mdbx::exception_thunk::is_clean() const?:
/home/thomas/projets/buildroot/output/build/libmdbx-0.9.1/mdbx.h++:2730:66: error: ?captured_? was not declared in this scope
 inline bool exception_thunk::is_clean() const noexcept { return !captured_; }
                                                                  ^
/home/thomas/projets/buildroot/output/build/libmdbx-0.9.1/mdbx.h++: In member function ?void mdbx::exception_thunk::capture()?:
/home/thomas/projets/buildroot/output/build/libmdbx-0.9.1/mdbx.h++:2734:3: error: ?captured_? was not declared in this scope
   captured_ = ::std::current_exception();
   ^
/home/thomas/projets/buildroot/output/build/libmdbx-0.9.1/mdbx.h++:2734:15: error: ?current_exception? is not a member of ?std?
   captured_ = ::std::current_exception();
               ^
/home/thomas/projets/buildroot/output/build/libmdbx-0.9.1/mdbx.h++: In member function ?void mdbx::exception_thunk::rethrow_captured() const?:
/home/thomas/projets/buildroot/output/build/libmdbx-0.9.1/mdbx.h++:2738:7: error: ?captured_? was not declared in this scope
   if (captured_)
       ^
/home/thomas/projets/buildroot/output/build/libmdbx-0.9.1/mdbx.h++:2739:25: error: ?rethrow_exception? is not a member of ?std?
     MDBX_CXX20_UNLIKELY ::std::rethrow_exception(captured_);
                         ^

make[3]: *** [CMakeFiles/mdbx.dir/build.make:96: CMakeFiles/mdbx.dir/mdbx.c++.o] Error 1

Some older versions of gcc had an issue with exception_ptr, see BR2_TOOLCHAIN_HAS_GCC_BUG_64735. You might need to depend on !BR2_TOOLCHAIN_HAS_GCC_BUG_64735. See other packages that have this dependency in Buildroot.

Thanks!

Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

#12 lore raw

PATCH v2 1/1 package/libmdbx: new package (library/database).

In reply to Heiko Thiery

Hi, Heiko.

Thanks for your advice.
A few minutes ago, I sent the third version of the patch, where all issues are eliminated.

Thank you again.
Regards,
Leonid.

Quoted 7 lines · Heiko Thiery

Hi Leonid,

Thank you for this contribution.

Below are some comments.

[...]

Quoted 11 lines

By the way .. there are two tools that help validating new package:

utils/tes-pkg -p libmdbx
utils/check-package package/libmdbx/*

Please check and fix the findings of these tests.

Thank you
--
Heiko

#13 lore raw

v5 cover cover letter for package/libmdbx: new package (library/database)

This patch adds libmdbx v0.9.2 and below is a brief overview of libmdbx. Please merge.

Regards,
Leonid.

--

libmdbx is an extremely fast, compact, powerful, embedded, transactional key-value database, with permissive license. libmdbx has a specific set of properties and capabilities, focused on creating unique lightweight solutions.

Historically, libmdbx (MDBX) is a deeply revised and extended descendant of the legendary LMDB (Lightning Memory-Mapped Database). libmdbx inherits all benefits from LMDB, but resolves some issues and adds a set of improvements.

According to developers, for now libmdbx surpasses the LMDB in terms of reliability, features and performance.

The most important differences MDBX from LMDB:
==============================================

1. More attention is paid to the quality of the code, to an "unbreakability" of the API, to testing and automatic checks (i.e. sanitizers, etc). So there:
- more control during operation;
- more checking parameters, internal audit of database structures;
- no warnings from compiler;
- no issues from ASAN, UBSAN, Valgrind, Coverity;
- etc.

2. Keys could be more than 2 times longer than LMDB.

3. Up to 20% faster than LMDB in CRUD benchmarks.

4. Automatic on-the-fly database size adjustment,
both increment and reduction.

5. Automatic continuous zero-overhead database compactification.

6. The same database format for 32- and 64-bit builds.

7. LIFO policy for Garbage Collection recycling (this can significantly increase write performance due write-back disk cache up to several times in a best case scenario).

8. Range query estimation.

9. Utility for checking the integrity of the database structure with some recovery capabilities.

For more info please refer:
- https://github.com/erthink/libmdbx for source code and README.
- https://erthink.github.io/libmdbx for API description.

--

MDBX is a Btree-based database management library modeled loosely on the BerkeleyDB API, but much simplified. The entire database (aka "environment") is exposed in a memory map, and all data fetches return data directly from the mapped memory, so no malloc's or memcpy's occur during data fetches. As such, the library is extremely simple because it requires no page caching layer of its own, and it is extremely high performance and memory-efficient. It is also fully transactional with full ACID semantics, and when the memory map is read-only, the database integrity cannot be corrupted by stray pointer writes from application code.

The library is fully thread-aware and supports concurrent read/write access from multiple processes and threads. Data pages use a copy-on-write strategy so no active data pages are ever overwritten, which also provides resistance to corruption and eliminates the need of any special recovery procedures after a system crash. Writes are fully serialized; only one write transaction may be active at a time, which guarantees that writers can never deadlock. The database structure is multi-versioned so readers run with no locks; writers cannot block readers, and readers don't block writers.

Unlike other well-known database mechanisms which use either write-ahead transaction logs or append-only data writes, MDBX requires no maintenance during operation. Both write-ahead loggers and append-only databases require periodic checkpointing and/or compaction of their log or database files otherwise they grow without bound. MDBX tracks retired/freed pages within the database and re-uses them for new write operations, so the database size does not grow without bound in normal use.

The memory map can be used as a read-only or read-write map. It is read-only by default as this provides total immunity to corruption. Using read-write mode offers much higher write performance, but adds the possibility for stray application writes thru pointers to silently corrupt the database.

Features
========

- Key-value data model, keys are always sorted.

- Fully ACID-compliant, through to MVCC and CoW.

- Multiple key-value sub-databases within a single datafile.

- Range lookups, including range query estimation.

- Efficient support for short fixed length keys, including native 32/64-bit integers.

- Ultra-efficient support for multimaps. Multi-values sorted, searchable and iterable. Keys stored without duplication.

- Data is memory-mapped and accessible directly/zero-copy. Traversal of database records is extremely-fast.

- Transactions for readers and writers, ones do not block others.

- Writes are strongly serialized. No transaction conflicts nor deadlocks.

- Readers are non-blocking, notwithstanding snapshot isolation.

- Nested write transactions.

- Reads scale linearly across CPUs.

- Continuous zero-overhead database compactification.

- Automatic on-the-fly database size adjustment.

- Customizable database page size.

- Olog(N) cost of lookup, insert, update, and delete operations by virtue of B+ tree characteristics.

- Online hot backup.

- Append operation for efficient bulk insertion of pre-sorted data.

- No WAL nor any transaction journal. No crash recovery needed. No maintenance is required.

- No internal cache and/or memory management, all done by basic OS services.

Limitations
===========

- Page size: a power of 2, maximum 65536 bytes, default 4096 bytes.

- Key size: minimum 0, maximum ?? pagesize (1300 bytes for default 4K pagesize, 21780 bytes for 64K pagesize).

- Value size: minimum 0, maximum 2146435072 (0x7FF00000) bytes for maps, ?? pagesize for multimaps (1348 bytes default 4K pagesize, 21828 bytes for 64K pagesize).

- Write transaction size: up to 4194301 (0x3FFFFD) pages (16 GiB for default 4K pagesize, 256 GiB for 64K pagesize).

- Database size: up to 2147483648 pages (8 TiB for default 4K pagesize, 128 TiB for 64K pagesize).

- Maximum sub-databases: 32765.

Gotchas
=======

- There cannot be more than one writer at a time, i.e. no more than one write transaction at a time.

- libmdbx is based on B+ tree, so access to database pages is mostly random. Thus SSDs provide a significant performance boost over spinning disks for large databases.

- libmdbx uses shadow paging instead of WAL. Thus syncing data to disk might be a bottleneck for write intensive workload.

- libmdbx uses copy-on-write for snapshot isolation during updates, but read transactions prevents recycling an old retired/freed pages, since it read ones. Thus altering of data during a parallel long-lived read operation will increase the process work set, may exhaust entire free database space, the database can grow quickly, and result in performance degradation. Try to avoid long running read transactions.

- libmdbx is extraordinarily fast and provides minimal overhead for data access, so you should reconsider using brute force techniques and double check your code. On the one hand, in the case of libmdbx, a simple linear search may be more profitable than complex indexes. On the other hand, if you make something suboptimally, you can notice detrimentally only on sufficiently large data.

--
Leonid Yuriev (1):
package/libmdbx: new package (library/database).

 DEVELOPERS                   |  3 +++
 package/Config.in            |  1 +
 package/libmdbx/Config.in    | 45 ++++++++++++++++++++++++++++++++++++
 package/libmdbx/libmdbx.hash |  5 ++++
 package/libmdbx/libmdbx.mk   | 33 ++++++++++++++++++++++++++
 5 files changed, 87 insertions(+)
 create mode 100644 package/libmdbx/Config.in
 create mode 100644 package/libmdbx/libmdbx.hash
 create mode 100644 package/libmdbx/libmdbx.mk
Signature · 1 line
2.29.2
#14 lore raw

PATCH v5 1/1 package/libmdbx: new package (library/database).

This patch adds libmdbx v0.9.2:
- libmdbx is one of the fastest compact embeddable key-value ACID database.
- libmdbx has a specific set of properties and capabilities, focused on creating unique lightweight solutions.
- libmdbx surpasses the legendary LMDB (Lightning Memory-Mapped Database) in terms of reliability, features and performance.
- https://github.com/erthink/libmdbx

---
Changes v1 -> v2:
- libmdbx version v0.8.2 -> v0.9.1 (released 2020-09-30)

Changes v2 -> v3:
- removed outcommented stuff (suggested by Heiko Thiery).
- cleaned up and simplified the makefile (suggested by Heiko Thiery).
- added patch for C++ header installation.
- added patch with fix minor copy&paste typo.
- added patch with pthread workaround for buggy toolchain/cmake/buildroot.
- added patch for pthread_yield().
- passed utils/check-package package/libmdbx/* (suggested by Heiko Thiery)
- passed utils/test-pkg -a -p libmdbx,
except w/o threads & w/o MMU (suggested by Heiko Thiery).

Changes v3 -> v4:
- fix passing BR2_PACKAGE_LIBMDBX_TOOLS option to cmake.
- fix using depend on instead of select,
and add comment for C++ toolchain (suggested by Heiko Thiery).
- fix minor help typo.

Changes v4 -> v5:
- libmdbx version v0.9.1 -> v0.9.2 (released 2020-11-27)
- dropped all patch since not needed.
- cosmetic changes (suggested by Thomas Petazzoni <[email protected]>).
- added dependence of BR2_TOOLCHAIN_HAS_SYNC_4 (suggested by Thomas Petazzoni <[email protected]>).
- added dependence of !BR2_TOOLCHAIN_HAS_GCC_BUG_64735 (suggested by Thomas Petazzoni <[email protected]>).
- take in account the BR2_SHARED_STATIC_LIBS (suggested by Thomas Petazzoni <[email protected]>).

---
 DEVELOPERS                   |  3 +++
 package/Config.in            |  1 +
 package/libmdbx/Config.in    | 45 ++++++++++++++++++++++++++++++++++++
 package/libmdbx/libmdbx.hash |  5 ++++
 package/libmdbx/libmdbx.mk   | 33 ++++++++++++++++++++++++++
 5 files changed, 87 insertions(+)
 create mode 100644 package/libmdbx/Config.in
 create mode 100644 package/libmdbx/libmdbx.hash
 create mode 100644 package/libmdbx/libmdbx.mk
DEVELOPERS +3 -0
index 9ab1e125f4..758ff6a2d5 100644
@@ -1482,6 +1482,9 @@ N:	Leon Anavi <[email protected]>
 F:	board/olimex/a10_olinuxino
 F:	configs/olimex_a10_olinuxino_lime_defconfig
 
+N:	Leonid Yuriev <[email protected]>
+F:	package/libmdbx/
+
 N:	Lionel Flandrin <[email protected]>
 F:	package/python-babel/
 F:	package/python-daemonize/
package/Config.in +1 -0
index 016a99ed1a..a6f95bfaa9 100644
@@ -1372,6 +1372,7 @@ menu "Database"
 	source "package/kompexsqlite/Config.in"
 	source "package/leveldb/Config.in"
 	source "package/libgit2/Config.in"
+	source "package/libmdbx/Config.in"
 	source "package/libodb/Config.in"
 	source "package/libodb-boost/Config.in"
 	source "package/libodb-mysql/Config.in"
package/libmdbx/Config.in new file +45 -0

45 lines not on this page · raw message

package/libmdbx/libmdbx.hash new file +5 -0
new file mode 100644
index 0000000000..0d3501f1d9
@@ -0,0 +1,5 @@
+# Hashes from: https://github.com/erthink/libmdbx/releases/
+sha256  c35cc53d66d74ebfc86e39441ba26276541ac7892bf91dba1e70c83665a02767  libmdbx-amalgamated-0.9.2.tar.gz
+
+# Locally calculated
+sha256  310fe25c858a9515fc8c8d7d1f24a67c9496f84a91e0a0e41ea9975b1371e569  LICENSE
package/libmdbx/libmdbx.mk new file +33 -0
new file mode 100644
index 0000000000..f3720130ec
@@ -0,0 +1,33 @@
+################################################################################
+#
+# libmdbx
+#
+################################################################################
+
+LIBMDBX_VERSION = 0.9.2
+LIBMDBX_SOURCE = libmdbx-amalgamated-$(LIBMDBX_VERSION).tar.gz
+LIBMDBX_SITE = https://github.com/erthink/libmdbx/releases/download/v$(LIBMDBX_VERSION)
+LIBMDBX_SUPPORTS_IN_SOURCE_BUILD = NO
+LIBMDBX_LICENSE = OLDAP-2.8
+LIBMDBX_LICENSE_FILES = LICENSE
+LIBMDBX_REDISTRIBUTE = YES
+LIBMDBX_STRIP_COMPONENTS = 0
+LIBMDBX_INSTALL_STAGING = YES
+
+LIBMDBX_CONF_OPTS = -DMDBX_INSTALL_MANPAGES=OFF -DBUILD_FOR_NATIVE_CPU=OFF \
+	-DMDBX_BUILD_CXX=$(if $(BR2_PACKAGE_LIBMDBX_CXX),ON,OFF) \
+	-DMDBX_BUILD_TOOLS=$(if $(BR2_PACKAGE_LIBMDBX_TOOLS),ON,OFF)
+
+ifeq ($(BR2_STATIC_LIBS)$(BR2_SHARED_STATIC_LIBS),y)
+LIBMDBX_CONF_OPTS += -DMDBX_INSTALL_STATIC=ON
+else
+LIBMDBX_CONF_OPTS += -DMDBX_INSTALL_STATIC=OFF
+endif
+
+ifeq ($(BR2_SHARED_LIBS)$(BR2_SHARED_STATIC_LIBS),y)
+LIBMDBX_CONF_OPTS += -DMDBX_BUILD_SHARED_LIBRARY=ON -DMDBX_LINK_TOOLS_NONSTATIC=ON
+else
+LIBMDBX_CONF_OPTS += -DMDBX_BUILD_SHARED_LIBRARY=OFF -DMDBX_LINK_TOOLS_NONSTATIC=OFF
+endif
+
+$(eval $(cmake-package))
Signature · 1 line
2.29.2