Skip to contentSkip to conversation

vhost-switch example: huge memory need and CRC off-loading issue

DISCUSSION Jan Kiszka 3 participants 5 messages Aug 7, 2015 – Aug 10, 2015

lore
Share & tools
lore

Jan Kiszka

author
#0 lore raw

Hi again,

two findings in the vhost-switch example code that can cause grey hair for starters:

- MAX_QUEUES of 512 causes pretty high memory need for the application (something between 1 and 2G) - is that really needed? I'm now running with 32, and I'm able to get away with 256M. Can we tune this default?

- hw_strip_crc is set to 0, but either the igb driver or the ET2 quad port adapter I'm using is ignoring this. It does strip the CRC, so does software, and I'm losing 4 bytes on each unpadded packet. Known issue?

Thanks,
Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux
#1 lore raw
Quoted 11 lines

-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Jan Kiszka
Sent: Saturday, August 8, 2015 1:31 AM
To: [email protected]
Subject: [dpdk-dev] vhost-switch example: huge memory need and CRC off- loading issue

Hi again,

Quoted 7 lines

two findings in the vhost-switch example code that can cause grey hair for starters:

- MAX_QUEUES of 512 causes pretty high memory need for the application (something between 1 and 2G) - is that really needed? I'm now running with 32, and I'm able to get away with 256M. Can we tune this default?

Don't think we need change default just because your platform is 32, Well, my platform is 128, other platform may have other value, :-)

Quoted 3 lines

- hw_strip_crc is set to 0, but either the igb driver or the ET2 quad port adapter I'm using is ignoring this. It does strip the CRC, so

Igb and ET2 should NOT ignore it.

Quoted 9 lines

does software, and I'm losing 4 bytes on each unpadded packet. Known issue?

Thanks,
Jan

--
Siemens AG, Corporate Technology, CT RTC ITP SES-DE Corporate Competence Center Embedded Linux

author
#2 lore raw
Quoted 14 lines · Changchun
Quoted 14 lines

-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Jan Kiszka
Sent: Saturday, August 8, 2015 1:31 AM
To: [email protected]
Subject: [dpdk-dev] vhost-switch example: huge memory need and CRC off- loading issue

Hi again,

two findings in the vhost-switch example code that can cause grey hair for starters:

Quoted 7 lines
Quoted 4 lines

- MAX_QUEUES of 512 causes pretty high memory need for the application (something between 1 and 2G) - is that really needed? I'm now running with 32, and I'm able to get away with 256M. Can we tune this default?

Don't think we need change default just because your platform is 32, Well, my platform is 128, other platform may have other value, :-)

Then let's make it configurable or explore the actual device needs before allocating the buffer. The impact on memory consumption is way too big to hard-code this, specifically as this is per physical port IIUC.

Quoted 6 lines
Quoted 4 lines

- hw_strip_crc is set to 0, but either the igb driver or the ET2 quad port adapter I'm using is ignoring this. It does strip the CRC, so

Igb and ET2 should NOT ignore it.

Well, they do not listen to us. What can I do to debug this?

According to eth_igb_rx_init, hw_strip_crc affects E1000_RCTL_SECRC in the receiver control registers, and debugging confirms this (the bit is unset in all E1000_RCTL accesses). But there is still no CRC at the end of received packets. Possibly, there is some other knob that controls CRC stripping?

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux
#3 lore raw

2015-08-08 09:17, Jan Kiszka:

Quoted 7 lines · Changchun
Quoted 6 lines

From: dev [mailto:[email protected]] On Behalf Of Jan Kiszka

Quoted 4 lines

- MAX_QUEUES of 512 causes pretty high memory need for the application (something between 1 and 2G) - is that really needed? I'm now running with 32, and I'm able to get away with 256M. Can we tune this default?

Quoted 6 lines
Quoted 2 lines

Don't think we need change default just because your platform is 32, Well, my platform is 128, other platform may have other value, :-)

Then let's make it configurable or explore the actual device needs before allocating the buffer. The impact on memory consumption is way too big to hard-code this, specifically as this is per physical port IIUC.

You can add a run-time option and/or add a comment in the documentation. It is just an example, so it must be simple to understand.

#4 lore raw
Quoted 22 lines

-----Original Message-----
From: Thomas Monjalon [mailto:[email protected]]
Sent: Sunday, August 9, 2015 4:56 PM
To: Jan Kiszka
Cc: [email protected]; Ouyang, Changchun
Subject: Re: [dpdk-dev] vhost-switch example: huge memory need and CRC off-loading issue

2015-08-08 09:17, Jan Kiszka:

Quoted 3 lines · Changchun
Quoted 2 lines

From: dev [mailto:[email protected]] On Behalf Of Jan Kiszka

Quoted 1 line

- MAX_QUEUES of 512 causes pretty high memory need for the

application

Quoted 7 lines
Quoted 6 lines
Quoted 3 lines

(something between 1 and 2G) - is that really needed? I'm now running with 32, and I'm able to get away with 256M. Can we tune this default?

Don't think we need change default just because your platform is 32, Well, my platform is 128, other platform may have other value, :-)

Quoted 6 lines
Quoted 3 lines

Then let's make it configurable or explore the actual device needs before allocating the buffer. The impact on memory consumption is way too big to hard-code this, specifically as this is per physical port IIUC.

You can add a run-time option and/or add a comment in the documentation. It is just an example, so it must be simple to understand.

Agree, that is better than changing the default macro value.