Skip to contentSkip to conversation

switching between protocols A and C

Drbd-dev Cristian Zamfir 2 participants 6 messages Apr 11, 2007 DRBD driver

lore
Share & tools
lore
#0 lore raw

Hi,

Would it be possible to switch from protocol A to C and vice versa without disconnecting (for instance detach, switch protocol and re-attach)?

This would be useful when alternating between two primaries (only protocol C can be used) and a primary/secondary configuration where protocol A may increase write throughput on the primary node.

Right now this behaviour does not seem to be supported but I am interested if you consider it could be implemented easily.

Thank you.

Cristian

maintainer
#1 lore raw
Quoted 9 lines · Cristian Zamfir

Hi,

Would it be possible to switch from protocol A to C and vice versa without disconnecting (for instance detach, switch protocol and re-attach)?

This would be useful when alternating between two primaries (only protocol C can be used) and a primary/secondary configuration where protocol A may increase write throughput on the primary node.

protocol A does _NOT_ increase throughput at all.
it _MAY_ reduce latency somewhat sometimes.

Quoted 2 lines

Right now this behaviour does not seem to be supported but I am interested if you consider it could be implemented easily.

no. and it does not make much sense for me, either.

Signature · 5 lines
: Lars Ellenberg                            Tel +43-1-8178292-0  :
: LINBIT Information Technologies GmbH      Fax +43-1-8178292-82 :
: Vivenotgasse 48, A-1120 Vienna/Europe    http://www.linbit.com :
__
please use the "List-Reply" function of your email client.
#2 lore raw
Quoted 9 lines · Lars Ellenberg
Quoted 5 lines · Cristian Zamfir

Hi,

Would it be possible to switch from protocol A to C and vice versa without disconnecting (for instance detach, switch protocol and re-attach)?

This would be useful when alternating between two primaries (only protocol C can be used) and a primary/secondary configuration where protocol A may increase write throughput on the primary node.

protocol A does _NOT_ increase throughput at all.
it _MAY_ reduce latency somewhat sometimes.

I am testing with the bonnie++ benchmark over a 100Mbps link and I get a substantial increase for output in the case of protocol A at the expense of higher CPU ratio. From what I understand, it would make sense to get better throughput from the asynchronous protocol over low quality links. Or am I doing something wrong?

drbd connected protocol C

Version 1.03 ------Sequential Output------ --Sequential Input- --Random-

                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- 
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
raoul          464M 12764  52 13525   4  8401   0 18724  39 27174   0 198.4   0
                    ------Sequential Create------ --------Random 
Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- 
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP 
/sec %CP
                 16 27245  94 +++++ +++ 27305  99 26368  98 +++++ +++ 

22775 95 raoul,464M,12764,52,13525,4,8401,0,18724,39,27174,0,198.4,0,16,27245,94,+++++,+++,27305,99,26368,98,+++++,+++,22775,95

drbd connected protocol A

Version 1.03 ------Sequential Output------ --Sequential Input- --Random-

                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- 
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
raoul          464M 27292  90 33632  10 11140   0 19785  46 28524   0 218.0   0
                    ------Sequential Create------ --------Random 
Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- 
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP 
/sec %CP
                 16 27136  97 +++++ +++ 26985 100 26775  98 +++++ +++ 

23972 100 raoul,464M,27292,90,33632,10,11140,0,19785,46,28524,0,218.0,0,16,27136,97,+++++,+++,26985,100,26775,98,+++++,+++,23972,100

Quoted 5 lines
Quoted 2 lines

Right now this behaviour does not seem to be supported but I am interested if you consider it could be implemented easily.

no. and it does not make much sense for me, either.

Cristian

maintainer
#3 lore raw
Quoted 12 lines · Cristian Zamfir
Quoted 11 lines · Lars Ellenberg
Quoted 9 lines · Cristian Zamfir

Hi,

Would it be possible to switch from protocol A to C and vice versa without disconnecting (for instance detach, switch protocol and re-attach)?

This would be useful when alternating between two primaries (only protocol C can be used) and a primary/secondary configuration where protocol A may increase write throughput on the primary node.

Quoted 8 lines
Quoted 2 lines

protocol A does _NOT_ increase throughput at all.
it _MAY_ reduce latency somewhat sometimes.

I am testing with the bonnie++ benchmark over a 100Mbps link and I get a substantial increase for output in the case of protocol A at the expense of higher CPU ratio. From what I understand, it would make sense to get better throughput from the asynchronous protocol over low quality links. Or am I doing something wrong?

and your exact bonnie++ command line is:

Signature · 5 lines
: Lars Ellenberg                            Tel +43-1-8178292-0  :
: LINBIT Information Technologies GmbH      Fax +43-1-8178292-82 :
: Vivenotgasse 48, A-1120 Vienna/Europe    http://www.linbit.com :
__
please use the "List-Reply" function of your email client.
#4 lore raw
Quoted 5 lines · Lars Ellenberg
Quoted 4 lines · Cristian Zamfir
Quoted 3 lines · Lars Ellenberg
Quoted 2 lines · Cristian Zamfir

Hi,

Quoted 9 lines
Quoted 6 lines
Quoted 5 lines
Quoted 3 lines

Would it be possible to switch from protocol A to C and vice versa without disconnecting (for instance detach, switch protocol and re-attach)?

This would be useful when alternating between two primaries (only protocol C can be used) and a primary/secondary configuration where protocol A may increase write throughput on the primary node.

protocol A does _NOT_ increase throughput at all.
it _MAY_ reduce latency somewhat sometimes.

I am testing with the bonnie++ benchmark over a 100Mbps link and I get a substantial increase for output in the case of protocol A at the expense of higher CPU ratio. From what I understand, it would make sense to get better throughput from the asynchronous protocol over low quality links. Or am I doing something wrong?

and your exact bonnie++ command line is:

Bonnie++ is run with the default parameters:
bonnie++ -u zamf -d bonnie/

I have results from ./tiobench, with shows a 2.4x increase in performance in favour of protocol A:

/usr/bin/tiotest -t 8 -f 60 -r 500 -b 4096 -d . -T-T

Protocol A
Sequential Writes
File Blk Num Avg
Maximum Lat% Lat% CPU
Identifier Size Size Thr Rate (CPU%) Latency
Latency >2s >10s Eff
---------------------------- ------ ----- --- ------ ------ --------- ----------- -------- -------- -----
2.6.18-xenU 480 4096 1 24.27 11.73% 0.126

    724.87   0.00000  0.00000   207

2.6.18-xenU 480 4096 2 20.32 10.07% 0.312
1002.72 0.00000 0.00000 202
2.6.18-xenU 480 4096 4 16.57 7.733% 0.648
4505.20 0.00570 0.00000 214
2.6.18-xenU 480 4096 8 15.96 7.445% 1.107
6598.77 0.02279 0.00000 214

Protocol C

Sequential Writes
File Blk Num Avg
Maximum Lat% Lat% CPU
Identifier Size Size Thr Rate (CPU%) Latency
Latency >2s >10s Eff
---------------------------- ------ ----- --- ------ ------ --------- ----------- -------- -------- -----
2.6.18-xenU 480 4096 1 10.46 5.032% 0.301
1232.91 0.00000 0.00000 208
2.6.18-xenU 480 4096 2 10.74 5.032% 0.581
1981.85 0.00000 0.00000 213
2.6.18-xenU 480 4096 4 10.55 5.009% 0.849
3782.22 0.01302 0.00000 211
2.6.18-xenU 480 4096 8 10.32 4.923% 1.841
8250.74 0.03581 0.00000 210

maintainer
#5 lore raw
Quoted 23 lines · Cristian Zamfir
Quoted 14 lines · Lars Ellenberg
Quoted 13 lines · Cristian Zamfir
Quoted 12 lines · Lars Ellenberg
Quoted 9 lines · Cristian Zamfir

Hi,

Would it be possible to switch from protocol A to C and vice versa without disconnecting (for instance detach, switch protocol and re-attach)?

This would be useful when alternating between two primaries (only protocol C can be used) and a primary/secondary configuration where protocol A may increase write throughput on the primary node.

protocol A does _NOT_ increase throughput at all.
it _MAY_ reduce latency somewhat sometimes.

Quoted 8 lines
Quoted 5 lines

I am testing with the bonnie++ benchmark over a 100Mbps link and I get a substantial increase for output in the case of protocol A at the expense of higher CPU ratio. From what I understand, it would make sense to get better throughput from the asynchronous protocol over low quality links. Or am I doing something wrong?

and your exact bonnie++ command line is:

Quoted 7 lines

Bonnie++ is run with the default parameters:
bonnie++ -u zamf -d bonnie/

I have results from ./tiobench, with shows a 2.4x increase in performance in favour of protocol A:

/usr/bin/tiotest -t 8 -f 60 -r 500 -b 4096 -d . -T-T

sorry, I don't have the time to interpret your benchmark results in detail. it is likely that these benchmarks are just not testing what I call "throuput".
which is sustained sequential block writes (streaming), as can be measured with e.g. the "dm" tool from the benchmark sub directory in the drbd tarball.

even so, regardless of benchmark results, the answer stands:

either you allow two primaries,
and are force to use protocol C.

or you do not allow two primaries,
and are free to chose the protocol.

protocol changes involve a disconnect/reconnect on both nodes. I do not see any reason to change this.

Signature · 5 lines
: Lars Ellenberg                            Tel +43-1-8178292-0  :
: LINBIT Information Technologies GmbH      Fax +43-1-8178292-82 :
: Vivenotgasse 48, A-1120 Vienna/Europe    http://www.linbit.com :
__
please use the "List-Reply" function of your email client.