RFC Current state of MARS
Get series with b4
b4 am [email protected]Versions newest first
- v1 RFC Latest31 patches 0 repliesNew
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/31 | mars: add new module lamportdrivers/ include/ | +87 | 0 replies | |
| 2/31 | mars: add new module brick_saydrivers/ include/ | +1012 | 0 replies | |
| 3/31 | mars: add new module brick_memdrivers/ include/ | +1299 | 0 replies | |
| 4/31 | mars: add new module brick_checkinginclude/ | +104 | 0 replies | |
| 5/31 | mars: add new module metainclude/ | +106 | 0 replies | |
| 6/31 | mars: add new module brickdrivers/ include/ | +1370 | 0 replies | |
| 7/31 | mars: add new module lib_pairing_heapinclude/ | +110 | 0 replies | |
| 8/31 | mars: add new module lib_queueinclude/ | +166 | 0 replies | |
| 9/31 | mars: add new module lib_rankdrivers/ include/ | +222 | 0 replies | |
| 10/31 | mars: add new module lib_limiterdrivers/ include/ | +178 | 0 replies | |
| 11/31 | mars: add new module lib_timingdrivers/ include/ | +252 | 0 replies | |
| 12/31 | mars: add new module vfs_compatinclude/ | +45 | 0 replies | |
| 13/31 | mars: add new module xiodrivers/ include/ | +474 | 0 replies | |
| 14/31 | mars: add new module xio_netdrivers/ include/ | +2001 | 0 replies | |
| 15/31 | mars: add new module lib_mapfreedrivers/ include/ | +464 | 0 replies | |
| 16/31 | mars: add new module lib_logdrivers/ include/ | +834 | 0 replies | |
| 17/31 | mars: add new module xio_biodrivers/ include/ | +930 | 0 replies | |
| 18/31 | mars: add new module xio_siodrivers/ include/ | +639 | 0 replies | |
| 19/31 | mars: add new module xio_clientdrivers/ include/ | +1160 | 0 replies | |
| 20/31 | mars: add new module xio_ifdrivers/ include/ | +1069 | 0 replies | |
| 21/31 | mars: add new module xio_copydrivers/ include/ | +1120 | 0 replies | |
| 22/31 | mars: add new module xio_trans_loggerdrivers/ include/ | +3572 | 0 replies | |
| 23/31 | mars: add new module xio_serverdrivers/ include/ | +577 | 0 replies | |
| 24/31 | mars: add new module light_strategydrivers/ include/ | +2368 | 0 replies | |
| 25/31 | mars: add new module light_netdrivers/ | +109 | 0 replies | |
| 26/31 | mars: add new module light_server_strategydrivers/ | +403 | 0 replies | |
| 27/31 | mars: add new module mars_procdrivers/ include/ | +403 | 0 replies | |
| 28/31 | mars: add new module mars_lightdrivers/ | +5880 | 0 replies | |
| 29/31 | mars: add new module Makefiledrivers/ | +61 | 0 replies | |
| 30/31 | mars: add new module Kconfigdrivers/ | +266 | 0 replies | |
| 31/31 | mars: activate builddrivers/ drivers/ | +3 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.
Cover letter [RFC 00/31] Current state of MARS
MARS Light is an asynchronous replication system for block storage over long distances [1,2,3]. It is the base for HA over long distances (more than 50km) and over network bottlenecks, e.g. high / varying packet loss rates.
The out-of-tree (OOT) version of MARS (branch light0.1.y) is in production at 1&1 Internet SE, currently at more than 1600 servers of various types, more than 6 petabytes of total storage, and has collected more than 6 millions of operating hours with less than 10 customer-visible incidents caused by MARS (most of them in the early beta phase).
Quoted 1 line
From my last LKML posting in 2014, I got the following major TODOs
from you:
1) no kernel prepatch anymore, no additional EXPORT_SYMBOL()s anywhere in the rest of the kernel.
=> accomplished in the enclosed patchset (however not yet fully bug-free). See also branches WIP-PORTABLE, WIP-BASE and WIP-PROPOSED-UPSTREAM at [1] for some out-of-tree (OOT) versions of MARS.
=> this means there will be zero impact from MARS to the rest of the kernel when MARS is not used. Additionally I placed it into drivers/staging/ (configurabale via migration script ./rework-mars-for-upstream.pl ).
2) replace the symlink tree by something else.
=> not yet done, due to lack of time. Sorry. I am planning to work on it in the forthcoming year, and hope to get enough time for it.
I think the attached patchset is not yet ready for submission, but please help me by giving early feedback. I would be very glad if some experienced upstream hacker would mentor me and help me at various points.
Here are some of my thoughts about further development:
In future, I would like to store the old symlink information (list of key => value pairs) in status files instead, one instance for each resource, and one for global configuration data (notice that the information _needs_ to be persistent for coping with node failures, and that dynamic storage is needed anyway for huge masses of transaction logfiles).
This should enhance scalability in the distributed system. Here is a use case: almost our full 1&1 webhosting machine park has been migrated from DRBD to MARS, retaining the conventional pair structure. Thus more than 800 DRBD clusters have been turned into MARS clusters.
In future, the whole 1&1 machine park could collapse into a _single_ big cluster consisting of thousands of machines. This has some advantages, not limited to greater flexibility with dynamic join-resource / leave-resource operations throughout that big cluster. For example, userspace can more easily detect failures cluster-wide and datacenter-wide, because the Lamport Clock algorithm used by MARS [2,3] is already a kind of "heartbeat", even remembering the timestamps of last successful information exchange. Thus it should be relatively easy to implement quorum algorithms etc on top of it in userspace.
Note that I am currently paid by 1&1 to ensure a seamless upgrade path for servers to any new data format, without downtime (other than temporarily switching the primary roles).
If you don't want the old symlink format in the upstream kernel, I would first implement the new format OOT (and strip out the old code for submission). Otherwise, I would agree to do this intree, hopefully with help and advise from some interested kernel hackers.
Cheers and a happy new year,
Thomas
[1] https://github.com/schoebel/mars
[2] https://github.com/schoebel/mars/blob/master/docu/MARS_Froscon2015.pdf
[3] https://github.com/schoebel/mars/blob/master/docu/mars-manual.pdf
Thomas Schoebel-Theuer (31):
mars: add new module lamport
mars: add new module brick_say
mars: add new module brick_mem
mars: add new module brick_checking
mars: add new module meta
mars: add new module brick
mars: add new module lib_pairing_heap
mars: add new module lib_queue
mars: add new module lib_rank
mars: add new module lib_limiter
mars: add new module lib_timing
mars: add new module vfs_compat
mars: add new module xio
mars: add new module xio_net
mars: add new module lib_mapfree
mars: add new module lib_log
mars: add new module xio_bio
mars: add new module xio_sio
mars: add new module xio_client
mars: add new module xio_if
mars: add new module xio_copy
mars: add new module xio_trans_logger
mars: add new module xio_server
mars: add new module light_strategy
mars: add new module light_net
mars: add new module light_server_strategy
mars: add new module mars_proc
mars: add new module mars_light
mars: add new module Makefile
mars: add new module Kconfig
mars: activate build
drivers/staging/Kconfig | 2 + drivers/staging/Makefile | 1 + drivers/staging/mars/Kconfig | 266 + drivers/staging/mars/Makefile | 61 + drivers/staging/mars/brick.c | 728 +++ drivers/staging/mars/brick_mem.c | 1081 ++++ drivers/staging/mars/brick_say.c | 916 +++ drivers/staging/mars/lamport.c | 61 + drivers/staging/mars/lib/lib_limiter.c | 129 + drivers/staging/mars/lib/lib_rank.c | 87 + drivers/staging/mars/lib/lib_timing.c | 71 + drivers/staging/mars/mars_light/light_net.c | 109 + .../mars/mars_light/light_server_strategy.c | 403 ++ drivers/staging/mars/mars_light/light_strategy.c | 2132 +++++++ drivers/staging/mars/mars_light/mars_light.c | 5880 ++++++++++++++++++++ drivers/staging/mars/mars_light/mars_proc.c | 369 ++ drivers/staging/mars/xio_bricks/lib_log.c | 505 ++ drivers/staging/mars/xio_bricks/lib_mapfree.c | 380 ++ drivers/staging/mars/xio_bricks/xio.c | 161 + drivers/staging/mars/xio_bricks/xio_bio.c | 845 +++ drivers/staging/mars/xio_bricks/xio_client.c | 1055 ++++ drivers/staging/mars/xio_bricks/xio_copy.c | 1005 ++++ drivers/staging/mars/xio_bricks/xio_if.c | 961 ++++ drivers/staging/mars/xio_bricks/xio_net.c | 1830 ++++++ drivers/staging/mars/xio_bricks/xio_server.c | 486 ++ drivers/staging/mars/xio_bricks/xio_sio.c | 571 ++ drivers/staging/mars/xio_bricks/xio_trans_logger.c | 3309 +++++++++++ include/linux/brick/brick.h | 642 +++ include/linux/brick/brick_checking.h | 104 + include/linux/brick/brick_mem.h | 218 + include/linux/brick/brick_say.h | 96 + include/linux/brick/lamport.h | 26 + include/linux/brick/lib_limiter.h | 49 + include/linux/brick/lib_pairing_heap.h | 110 + include/linux/brick/lib_queue.h | 166 + include/linux/brick/lib_rank.h | 135 + include/linux/brick/lib_timing.h | 181 + include/linux/brick/meta.h | 106 + include/linux/brick/vfs_compat.h | 45 + include/linux/mars_light/light_strategy.h | 236 + include/linux/mars_light/mars_proc.h | 34 + include/linux/xio/lib_log.h | 329 ++ include/linux/xio/lib_mapfree.h | 84 + include/linux/xio/xio.h | 313 ++ include/linux/xio/xio_bio.h | 85 + include/linux/xio/xio_client.h | 105 + include/linux/xio/xio_copy.h | 115 + include/linux/xio/xio_if.h | 108 + include/linux/xio/xio_net.h | 171 + include/linux/xio/xio_server.h | 91 + include/linux/xio/xio_sio.h | 68 + include/linux/xio/xio_trans_logger.h | 263 + 52 files changed, 27284 insertions(+) create mode 100644 drivers/staging/mars/Kconfig create mode 100644 drivers/staging/mars/Makefile create mode 100644 drivers/staging/mars/brick.c create mode 100644 drivers/staging/mars/brick_mem.c create mode 100644 drivers/staging/mars/brick_say.c create mode 100644 drivers/staging/mars/lamport.c create mode 100644 drivers/staging/mars/lib/lib_limiter.c create mode 100644 drivers/staging/mars/lib/lib_rank.c create mode 100644 drivers/staging/mars/lib/lib_timing.c create mode 100644 drivers/staging/mars/mars_light/light_net.c create mode 100644 drivers/staging/mars/mars_light/light_server_strategy.c create mode 100644 drivers/staging/mars/mars_light/light_strategy.c create mode 100644 drivers/staging/mars/mars_light/mars_light.c create mode 100644 drivers/staging/mars/mars_light/mars_proc.c create mode 100644 drivers/staging/mars/xio_bricks/lib_log.c create mode 100644 drivers/staging/mars/xio_bricks/lib_mapfree.c create mode 100644 drivers/staging/mars/xio_bricks/xio.c create mode 100644 drivers/staging/mars/xio_bricks/xio_bio.c create mode 100644 drivers/staging/mars/xio_bricks/xio_client.c create mode 100644 drivers/staging/mars/xio_bricks/xio_copy.c create mode 100644 drivers/staging/mars/xio_bricks/xio_if.c create mode 100644 drivers/staging/mars/xio_bricks/xio_net.c create mode 100644 drivers/staging/mars/xio_bricks/xio_server.c create mode 100644 drivers/staging/mars/xio_bricks/xio_sio.c create mode 100644 drivers/staging/mars/xio_bricks/xio_trans_logger.c create mode 100644 include/linux/brick/brick.h create mode 100644 include/linux/brick/brick_checking.h create mode 100644 include/linux/brick/brick_mem.h create mode 100644 include/linux/brick/brick_say.h create mode 100644 include/linux/brick/lamport.h create mode 100644 include/linux/brick/lib_limiter.h create mode 100644 include/linux/brick/lib_pairing_heap.h create mode 100644 include/linux/brick/lib_queue.h create mode 100644 include/linux/brick/lib_rank.h create mode 100644 include/linux/brick/lib_timing.h create mode 100644 include/linux/brick/meta.h create mode 100644 include/linux/brick/vfs_compat.h create mode 100644 include/linux/mars_light/light_strategy.h create mode 100644 include/linux/mars_light/mars_proc.h create mode 100644 include/linux/xio/lib_log.h create mode 100644 include/linux/xio/lib_mapfree.h create mode 100644 include/linux/xio/xio.h create mode 100644 include/linux/xio/xio_bio.h create mode 100644 include/linux/xio/xio_client.h create mode 100644 include/linux/xio/xio_copy.h create mode 100644 include/linux/xio/xio_if.h create mode 100644 include/linux/xio/xio_net.h create mode 100644 include/linux/xio/xio_server.h create mode 100644 include/linux/xio/xio_sio.h create mode 100644 include/linux/xio/xio_trans_logger.h
Signature · 1 line
2.6.4