Skip to contentSkip to conversation

ARM, AF_PACKET: caching problems on Marvell Kirkwood

PATCH In review Phil Sutter 7 participants 19 messages Apr 8, 2011 – Sep 6, 2011 1 message not in the archive

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Dear lists,

I am experiencing severe caching issues using the TX_RING feature of AF_PACKET on a Kirkwood-based system (i.e., OpenRD). This may likely be a bug of the CPU/SoC itself, at least it reacts a bit picky when using the preload data instruction (pld) in rather useless cases (but that's a different story).

There is simple testing code at the end of this email, effectively just preparing a packet in the TX_RING and triggering it's delivery once per second. The experienced symptom is that sporadically nothing goes out in one iteration, and two packets in the following one.

It looks like the kernel doesn't get the changed value of tp_status in time, although userspace sees the correct value. Note that moving the sleep(1) from the end of the loop to just before calling sendto() fixes the problem.

Another (more useful) workaround is to call flush_cache_all() at the beginning of packet_sendmsg() in net/packet/af_packet.c. I was not able to fix this with some more specific flushing at that place. Anyway, the call to flush_dcache_page() from __packet_get_status() in the same source file is meant to do the trick I guess. But somehow doesn't.

Feedback regardles of which kind is highly appreciated, of course!

Greetings, Phil

------------------[start of packet_mmap_test.c]-------------------- #include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <linux/if_ether.h>
#include <linux/if_packet.h>
#include <net/if.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <sys/socket.h>
#include <sys/types.h>

#define PERROR_EXIT(rc, mesg) { \
	perror(mesg); \
	return rc; \
}

int main(void)

{
	uint32_t size;
	struct sockaddr_ll sa;
	struct ifreq ifr;
	int index;
	int tmp;
	int fd;
	struct tpacket_req packet_req;
	struct tpacket2_hdr * ps_header_start, *ps_header;

	if ((fd = socket(AF_PACKET, SOCK_DGRAM, htons(ETH_P_ALL))) < 0)
		PERROR_EXIT(EXIT_FAILURE, "socket");

	/* retrieve eth0's interface index number */
	strncpy (ifr.ifr_name, "eth0", sizeof(ifr.ifr_name));
	if (ioctl(fd, SIOCGIFINDEX, &ifr) < 0)
		PERROR_EXIT(EXIT_FAILURE, "ioctl(SIOCGIFINDEX)");

	/* set sockaddr info */
	memset(&sa, 0, sizeof(sa));
	sa.sll_family = AF_PACKET;
	sa.sll_protocol = ETH_P_ALL;
	sa.sll_ifindex = ifr.ifr_ifindex;

	/* bind port */
	if (bind(fd, (struct sockaddr *)&sa, sizeof(sa)) < 0)
		PERROR_EXIT(EXIT_FAILURE, "bind()");

	tmp = TPACKET_V2;
	if (setsockopt(fd, SOL_PACKET, PACKET_VERSION, &tmp, sizeof(tmp)) < 0)
		PERROR_EXIT(EXIT_FAILURE, "setsockopt(PACKET_VERSION)");

	/* set packet loss option */
	tmp = 1;
	if (setsockopt(fd, SOL_PACKET, PACKET_LOSS, &tmp, sizeof(tmp)) < 0)
		PERROR_EXIT(EXIT_FAILURE, "setsockopt(PACKET_LOSS)");

	/* prepare Tx ring request */
	packet_req.tp_block_size = 1024 * 8;
	packet_req.tp_frame_size = 1024 * 8;
	packet_req.tp_block_nr = 1024;
	packet_req.tp_frame_nr = 1024;

	/* send TX ring request */
	if (setsockopt(fd, SOL_PACKET, PACKET_TX_RING,
	               &packet_req, sizeof(packet_req)) < 0)
		PERROR_EXIT(EXIT_FAILURE, "setsockopt: PACKET_TX_RING");

	/* calculate memory to mmap in the kernel */
	size = packet_req.tp_block_size * packet_req.tp_block_nr;

	/* mmap Tx ring buffers memory */
	ps_header_start = mmap(0, size,
			PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
	if (ps_header_start < 0)
		PERROR_EXIT(EXIT_FAILURE, "mmap");

	/* fill peer sockaddr for SOCK_DGRAM */
	sa.sll_family = AF_PACKET;
	sa.sll_protocol = htons(ETH_P_IP);
	sa.sll_ifindex = ifr.ifr_ifindex;
	sa.sll_halen = ETH_ALEN;
	memset(&sa.sll_addr, 0xff, ETH_ALEN);

	ps_header = ps_header_start;
	while (1) {
		int sendlen, j;

		char *data = (void*)ps_header + TPACKET_HDRLEN
		              - sizeof(struct sockaddr_ll);

		switch((volatile uint32_t)ps_header->tp_status)
		{
		case TP_STATUS_AVAILABLE:
			memset(data, 0x23, 150);
			break;

		case TP_STATUS_WRONG_FORMAT:
			printf("An error has occured during transfer\n");
			exit(EXIT_FAILURE);
			break;

		default:
			printf("Buffer is not available, aborting\n");
			exit(1);
			break;
		}
		ps_header->tp_len = 150;
		ps_header->tp_status = TP_STATUS_SEND_REQUEST;

		sendlen = sendto(fd, NULL, 0, 0,
				(struct sockaddr *)&sa, sizeof(sa));
		if (sendlen < 0)
			perror("sendto");
		else if (sendlen == 0)
			printf("sendto(): nothing sent!\n");
		else
			printf("sendto(): sent %d bytes out\n", sendlen);

#define ST_IS(x) ((volatile uint32_t)ps_header->tp_status == x)
printf("tp_status after sending: %s\n",
ST_IS(TP_STATUS_AVAILABLE) ? "AVAILABLE" :
ST_IS(TP_STATUS_SEND_REQUEST) ? "SEND_REQUEST" :
ST_IS(TP_STATUS_WRONG_FORMAT) ? "WRONG_FORMAT" :
"unknown");
#undef ST_IS

		ps_header = (void *)ps_header + packet_req.tp_frame_size;
		if (ps_header >= ps_header_start + size)
			ps_header = ps_header_start;

		sleep(1);
	}
	return 0;
}

--------------------[end of packet_mmap_test.c]--------------------

#1 lore raw

Hi,

Hasn't anyone experienced this bug but me? Can anyone reproduce the described behaviour on his Kirkwood-based (or even generic ARM) machine? I am still not sure if this is a problem of just my CPU or common amongst Kirkwood/VIPT/ARM machines.

My workaround looks like this:

| diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c
| index b5362e9..0672f50 100644
| --- a/net/packet/af_packet.c
| +++ b/net/packet/af_packet.c
| @@ -1298,10 +1298,13 @@ static int packet_sendmsg(struct kiocb *iocb, struct socket *sock,
|  {
|         struct sock *sk = sock->sk;
|         struct packet_sock *po = pkt_sk(sk);
| -       if (po->tx_ring.pg_vec)
| -               return tpacket_snd(po, msg);
| -       else
| -               return packet_snd(sock, msg, len);
| +       int rc;
| +
| +       flush_cache_all();
| +       rc = po->tx_ring.pg_vec ? tpacket_snd(po, msg) :
| +                       packet_snd(sock, msg, len);
| +       flush_cache_all();
| +       return rc;
|  }
|  
|  /*

Greetings, Phil

(Full-quoting here because I've added the TX ring author and the Kirkwood maintainers to Cc.)

Quoted 168 lines · Phil Sutter

Dear lists,

I am experiencing severe caching issues using the TX_RING feature of AF_PACKET on a Kirkwood-based system (i.e., OpenRD). This may likely be a bug of the CPU/SoC itself, at least it reacts a bit picky when using the preload data instruction (pld) in rather useless cases (but that's a different story).

There is simple testing code at the end of this email, effectively just preparing a packet in the TX_RING and triggering it's delivery once per second. The experienced symptom is that sporadically nothing goes out in one iteration, and two packets in the following one.

It looks like the kernel doesn't get the changed value of tp_status in time, although userspace sees the correct value. Note that moving the sleep(1) from the end of the loop to just before calling sendto() fixes the problem.

Another (more useful) workaround is to call flush_cache_all() at the beginning of packet_sendmsg() in net/packet/af_packet.c. I was not able to fix this with some more specific flushing at that place. Anyway, the call to flush_dcache_page() from __packet_get_status() in the same source file is meant to do the trick I guess. But somehow doesn't.

Feedback regardles of which kind is highly appreciated, of course!

Greetings, Phil

------------------[start of packet_mmap_test.c]-------------------- #include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <linux/if_ether.h>
#include <linux/if_packet.h>
#include <net/if.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <sys/socket.h>
#include <sys/types.h>

#define PERROR_EXIT(rc, mesg) { \
	perror(mesg); \
	return rc; \
}

int main(void)

{
	uint32_t size;
	struct sockaddr_ll sa;
	struct ifreq ifr;
	int index;
	int tmp;
	int fd;
	struct tpacket_req packet_req;
	struct tpacket2_hdr * ps_header_start, *ps_header;

	if ((fd = socket(AF_PACKET, SOCK_DGRAM, htons(ETH_P_ALL))) < 0)
		PERROR_EXIT(EXIT_FAILURE, "socket");

	/* retrieve eth0's interface index number */
	strncpy (ifr.ifr_name, "eth0", sizeof(ifr.ifr_name));
	if (ioctl(fd, SIOCGIFINDEX, &ifr) < 0)
		PERROR_EXIT(EXIT_FAILURE, "ioctl(SIOCGIFINDEX)");

	/* set sockaddr info */
	memset(&sa, 0, sizeof(sa));
	sa.sll_family = AF_PACKET;
	sa.sll_protocol = ETH_P_ALL;
	sa.sll_ifindex = ifr.ifr_ifindex;

	/* bind port */
	if (bind(fd, (struct sockaddr *)&sa, sizeof(sa)) < 0)
		PERROR_EXIT(EXIT_FAILURE, "bind()");

	tmp = TPACKET_V2;
	if (setsockopt(fd, SOL_PACKET, PACKET_VERSION, &tmp, sizeof(tmp)) < 0)
		PERROR_EXIT(EXIT_FAILURE, "setsockopt(PACKET_VERSION)");

	/* set packet loss option */
	tmp = 1;
	if (setsockopt(fd, SOL_PACKET, PACKET_LOSS, &tmp, sizeof(tmp)) < 0)
		PERROR_EXIT(EXIT_FAILURE, "setsockopt(PACKET_LOSS)");

	/* prepare Tx ring request */
	packet_req.tp_block_size = 1024 * 8;
	packet_req.tp_frame_size = 1024 * 8;
	packet_req.tp_block_nr = 1024;
	packet_req.tp_frame_nr = 1024;

	/* send TX ring request */
	if (setsockopt(fd, SOL_PACKET, PACKET_TX_RING,
	               &packet_req, sizeof(packet_req)) < 0)
		PERROR_EXIT(EXIT_FAILURE, "setsockopt: PACKET_TX_RING");

	/* calculate memory to mmap in the kernel */
	size = packet_req.tp_block_size * packet_req.tp_block_nr;

	/* mmap Tx ring buffers memory */
	ps_header_start = mmap(0, size,
			PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
	if (ps_header_start < 0)
		PERROR_EXIT(EXIT_FAILURE, "mmap");

	/* fill peer sockaddr for SOCK_DGRAM */
	sa.sll_family = AF_PACKET;
	sa.sll_protocol = htons(ETH_P_IP);
	sa.sll_ifindex = ifr.ifr_ifindex;
	sa.sll_halen = ETH_ALEN;
	memset(&sa.sll_addr, 0xff, ETH_ALEN);

	ps_header = ps_header_start;
	while (1) {
		int sendlen, j;

		char *data = (void*)ps_header + TPACKET_HDRLEN
		              - sizeof(struct sockaddr_ll);

		switch((volatile uint32_t)ps_header->tp_status)
		{
		case TP_STATUS_AVAILABLE:
			memset(data, 0x23, 150);
			break;

		case TP_STATUS_WRONG_FORMAT:
			printf("An error has occured during transfer\n");
			exit(EXIT_FAILURE);
			break;

		default:
			printf("Buffer is not available, aborting\n");
			exit(1);
			break;
		}
		ps_header->tp_len = 150;
		ps_header->tp_status = TP_STATUS_SEND_REQUEST;

		sendlen = sendto(fd, NULL, 0, 0,
				(struct sockaddr *)&sa, sizeof(sa));
		if (sendlen < 0)
			perror("sendto");
		else if (sendlen == 0)
			printf("sendto(): nothing sent!\n");
		else
			printf("sendto(): sent %d bytes out\n", sendlen);

#define ST_IS(x) ((volatile uint32_t)ps_header->tp_status == x)
printf("tp_status after sending: %s\n",
ST_IS(TP_STATUS_AVAILABLE) ? "AVAILABLE" :
ST_IS(TP_STATUS_SEND_REQUEST) ? "SEND_REQUEST" :
ST_IS(TP_STATUS_WRONG_FORMAT) ? "WRONG_FORMAT" :
"unknown");
#undef ST_IS

		ps_header = (void *)ps_header + packet_req.tp_frame_size;
		if (ps_header >= ps_header_start + size)
			ps_header = ps_header_start;

		sleep(1);
	}
	return 0;
}

--------------------[end of packet_mmap_test.c]--------------------

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#2 lore raw

Le jeudi 05 mai 2011 ? 16:11 +0200, Phil Sutter a ?crit :

Quoted 31 lines

Hi,

Hasn't anyone experienced this bug but me? Can anyone reproduce the described behaviour on his Kirkwood-based (or even generic ARM) machine? I am still not sure if this is a problem of just my CPU or common amongst Kirkwood/VIPT/ARM machines.

My workaround looks like this:

| diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c
| index b5362e9..0672f50 100644
| --- a/net/packet/af_packet.c
| +++ b/net/packet/af_packet.c
| @@ -1298,10 +1298,13 @@ static int packet_sendmsg(struct kiocb *iocb, struct socket *sock,
|  {
|         struct sock *sk = sock->sk;
|         struct packet_sock *po = pkt_sk(sk);
| -       if (po->tx_ring.pg_vec)
| -               return tpacket_snd(po, msg);
| -       else
| -               return packet_snd(sock, msg, len);
| +       int rc;
| +
| +       flush_cache_all();
| +       rc = po->tx_ring.pg_vec ? tpacket_snd(po, msg) :
| +                       packet_snd(sock, msg, len);
| +       flush_cache_all();
| +       return rc;
|  }
|  
|  /*
Quoted 169 lines

Greetings, Phil

(Full-quoting here because I've added the TX ring author and the Kirkwood maintainers to Cc.)

Quoted 163 lines · Phil Sutter

Dear lists,

I am experiencing severe caching issues using the TX_RING feature of AF_PACKET on a Kirkwood-based system (i.e., OpenRD). This may likely be a bug of the CPU/SoC itself, at least it reacts a bit picky when using the preload data instruction (pld) in rather useless cases (but that's a different story).

There is simple testing code at the end of this email, effectively just preparing a packet in the TX_RING and triggering it's delivery once per second. The experienced symptom is that sporadically nothing goes out in one iteration, and two packets in the following one.

It looks like the kernel doesn't get the changed value of tp_status in time, although userspace sees the correct value. Note that moving the sleep(1) from the end of the loop to just before calling sendto() fixes the problem.

Another (more useful) workaround is to call flush_cache_all() at the beginning of packet_sendmsg() in net/packet/af_packet.c. I was not able to fix this with some more specific flushing at that place. Anyway, the call to flush_dcache_page() from __packet_get_status() in the same source file is meant to do the trick I guess. But somehow doesn't.

Feedback regardles of which kind is highly appreciated, of course!

Greetings, Phil

------------------[start of packet_mmap_test.c]-------------------- #include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <linux/if_ether.h>
#include <linux/if_packet.h>
#include <net/if.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <sys/socket.h>
#include <sys/types.h>

#define PERROR_EXIT(rc, mesg) { \
	perror(mesg); \
	return rc; \
}

int main(void)

{
	uint32_t size;
	struct sockaddr_ll sa;
	struct ifreq ifr;
	int index;
	int tmp;
	int fd;
	struct tpacket_req packet_req;
	struct tpacket2_hdr * ps_header_start, *ps_header;

	if ((fd = socket(AF_PACKET, SOCK_DGRAM, htons(ETH_P_ALL))) < 0)
		PERROR_EXIT(EXIT_FAILURE, "socket");

	/* retrieve eth0's interface index number */
	strncpy (ifr.ifr_name, "eth0", sizeof(ifr.ifr_name));
	if (ioctl(fd, SIOCGIFINDEX, &ifr) < 0)
		PERROR_EXIT(EXIT_FAILURE, "ioctl(SIOCGIFINDEX)");

	/* set sockaddr info */
	memset(&sa, 0, sizeof(sa));
	sa.sll_family = AF_PACKET;
	sa.sll_protocol = ETH_P_ALL;
	sa.sll_ifindex = ifr.ifr_ifindex;

	/* bind port */
	if (bind(fd, (struct sockaddr *)&sa, sizeof(sa)) < 0)
		PERROR_EXIT(EXIT_FAILURE, "bind()");

	tmp = TPACKET_V2;
	if (setsockopt(fd, SOL_PACKET, PACKET_VERSION, &tmp, sizeof(tmp)) < 0)
		PERROR_EXIT(EXIT_FAILURE, "setsockopt(PACKET_VERSION)");

	/* set packet loss option */
	tmp = 1;
	if (setsockopt(fd, SOL_PACKET, PACKET_LOSS, &tmp, sizeof(tmp)) < 0)
		PERROR_EXIT(EXIT_FAILURE, "setsockopt(PACKET_LOSS)");

	/* prepare Tx ring request */
	packet_req.tp_block_size = 1024 * 8;
	packet_req.tp_frame_size = 1024 * 8;
	packet_req.tp_block_nr = 1024;
	packet_req.tp_frame_nr = 1024;

	/* send TX ring request */
	if (setsockopt(fd, SOL_PACKET, PACKET_TX_RING,
	               &packet_req, sizeof(packet_req)) < 0)
		PERROR_EXIT(EXIT_FAILURE, "setsockopt: PACKET_TX_RING");

	/* calculate memory to mmap in the kernel */
	size = packet_req.tp_block_size * packet_req.tp_block_nr;

	/* mmap Tx ring buffers memory */
	ps_header_start = mmap(0, size,
			PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
	if (ps_header_start < 0)
		PERROR_EXIT(EXIT_FAILURE, "mmap");

	/* fill peer sockaddr for SOCK_DGRAM */
	sa.sll_family = AF_PACKET;
	sa.sll_protocol = htons(ETH_P_IP);
	sa.sll_ifindex = ifr.ifr_ifindex;
	sa.sll_halen = ETH_ALEN;
	memset(&sa.sll_addr, 0xff, ETH_ALEN);

	ps_header = ps_header_start;
	while (1) {
		int sendlen, j;

		char *data = (void*)ps_header + TPACKET_HDRLEN
		              - sizeof(struct sockaddr_ll);

		switch((volatile uint32_t)ps_header->tp_status)
		{
		case TP_STATUS_AVAILABLE:
			memset(data, 0x23, 150);
			break;

		case TP_STATUS_WRONG_FORMAT:
			printf("An error has occured during transfer\n");
			exit(EXIT_FAILURE);
			break;

		default:
			printf("Buffer is not available, aborting\n");
			exit(1);
			break;
		}
		ps_header->tp_len = 150;
		ps_header->tp_status = TP_STATUS_SEND_REQUEST;

		sendlen = sendto(fd, NULL, 0, 0,
				(struct sockaddr *)&sa, sizeof(sa));
		if (sendlen < 0)
			perror("sendto");
		else if (sendlen == 0)
			printf("sendto(): nothing sent!\n");
		else
			printf("sendto(): sent %d bytes out\n", sendlen);

#define ST_IS(x) ((volatile uint32_t)ps_header->tp_status == x)
printf("tp_status after sending: %s\n",
ST_IS(TP_STATUS_AVAILABLE) ? "AVAILABLE" :
ST_IS(TP_STATUS_SEND_REQUEST) ? "SEND_REQUEST" :
ST_IS(TP_STATUS_WRONG_FORMAT) ? "WRONG_FORMAT" :
"unknown");
#undef ST_IS

		ps_header = (void *)ps_header + packet_req.tp_frame_size;
		if (ps_header >= ps_header_start + size)
			ps_header = ps_header_start;

		sleep(1);
	}
	return 0;
}

--------------------[end of packet_mmap_test.c]--------------------

Hi Phil

I assume you use latest linux-2.6 or net-next-2.6 ?

Could you try to force vmalloc() use ?

net/packet/af_packet.c +1 -1
index b5362e9..0b5a89c 100644
@@ -2383,7 +2383,7 @@ static inline char *alloc_one_pg_vec_page(unsigned long order)
 	gfp_t gfp_flags = GFP_KERNEL | __GFP_COMP |
 			  __GFP_ZERO | __GFP_NOWARN | __GFP_NORETRY;
 
-	buffer = (char *) __get_free_pages(gfp_flags, order);
+	buffer = NULL;
 
 	if (buffer)
 		return buffer;
#4 lore raw
In reply to Phil Sutter
Quoted 6 lines · Phil Sutter

Hi,

Hasn't anyone experienced this bug but me? Can anyone reproduce the described behaviour on his Kirkwood-based (or even generic ARM) machine? I am still not sure if this is a problem of just my CPU or common amongst Kirkwood/VIPT/ARM machines.

Hi Phil

I can reproduce it on a Kirkwood:

[    0.000000] CPU: Feroceon 88FR131 [56251311] revision 1 (ARMv5TE), cr=00053977

Andrew

#3 lore raw
In reply to Eric Dumazet

Hi,

Quoted 1 line · Eric Dumazet

I assume you use latest linux-2.6 or net-next-2.6 ?

Well, initially we noticed the problem on 2.6.34.7, but I verified it against both 2.6.37 and linux-2.6 from three days ago.

Quoted 7 lines

Could you try to force vmalloc() use ?

net/packet/af_packet.c +0 -0
index b5362e9..0b5a89c 100644

net/packet/af_packet.c static inline char *alloc_one_pg_vec_page(unsigned long order) · line 2389

Quoted 8 lines
net/packet/af_packet.c +1 -1
@@ -2383,7 +2383,7 @@ static inline char *alloc_one_pg_vec_page(unsigned long order)
 	gfp_t gfp_flags = GFP_KERNEL | __GFP_COMP |
 			  __GFP_ZERO | __GFP_NOWARN | __GFP_NORETRY;
 
-	buffer = (char *) __get_free_pages(gfp_flags, order);
+	buffer = NULL;
 
 	if (buffer)
 		return buffer;

Thanks for the hint. I tried that, but the problem persists.

Greetings, Phil

#5 lore raw
In reply to Andrew Lunn

Hi,

Quoted 3 lines · Andrew Lunn

I can reproduce it on a Kirkwood:

[    0.000000] CPU: Feroceon 88FR131 [56251311] revision 1 (ARMv5TE), cr=00053977

Thanks for the information. Seems like we have the same CPU:

| [    0.000000] CPU: Feroceon 88FR131 [56251311] revision 1 (ARMv5TE), cr=00053177
| [    0.000000] CPU: VIVT data cache, VIVT instruction cache

and it's actually VIVT, not VIPT as I wrote in an earlier mail.

Greetings, Phil

#6 lore raw

Hi,

Quoted 11 lines · Phil Sutter
Quoted 3 lines · Andrew Lunn

I can reproduce it on a Kirkwood:

[    0.000000] CPU: Feroceon 88FR131 [56251311] revision 1 (ARMv5TE), cr=00053977

Thanks for the information. Seems like we have the same CPU:

| [    0.000000] CPU: Feroceon 88FR131 [56251311] revision 1 (ARMv5TE), cr=00053177
| [    0.000000] CPU: VIVT data cache, VIVT instruction cache

and it's actually VIVT, not VIPT as I wrote in an earlier mail.

Interesting news, a friend of mine can reproduce the problem on a Foxboard (FOXG20):

| root at localhost:/root # dmesg |head -4
| [    0.000000] Linux version 2.6.37 (wbx at chrom) (gcc version 4.5.2 (GCC) ) #1 Sat May 7 19:37:30 CEST 2011
| [    0.000000] CPU: ARM926EJ-S [41069265] revision 5 (ARMv5TEJ), cr=00053177
| [    0.000000] CPU: VIVT data cache, VIVT instruction cache
| [    0.000000] Machine: Acme Systems FOXG20
| root at localhost:/root # uname -a
| Linux localhost 2.6.37 #1 Sat May 7 19:37:30 CEST 2011 armv5tejl GNU/Linux
| root at localhost:/root # /mmap
| sendto(): nothing sent!
| tp_status after sending: SEND_REQUEST
| sendto(): nothing sent!
| tp_status after sending: SEND_REQUEST
| sendto(): sent 450 bytes out
| tp_status after sending: AVAILABLE
| sendto(): sent 150 bytes out
| tp_status after sending: AVAILABLE
| sendto(): nothing sent!
| tp_status after sending: SEND_REQUEST
| sendto(): nothing sent!
| tp_status after sending: SEND_REQUEST
| sendto(): sent 300 bytes out
| tp_status after sending: SEND_REQUEST
| sendto(): nothing sent!
| tp_status after sending: SEND_REQUEST
| sendto(): sent 300 bytes out
| tp_status after sending: SEND_REQUEST
| sendto(): sent 150 bytes out
| tp_status after sending: SEND_REQUEST
| sendto(): sent 150 bytes out
| tp_status after sending: SEND_REQUEST
| sendto(): sent 300 bytes out
| tp_status after sending: AVAILABLE
| sendto(): nothing sent!
| tp_status after sending: SEND_REQUEST
| sendto(): sent 150 bytes out
| tp_status after sending: SEND_REQUEST

Greetings, Phil

#7 lore raw
In reply to Phil Sutter

Hi,

Quoted 11 lines · Phil Sutter
Quoted 3 lines · Andrew Lunn

I can reproduce it on a Kirkwood:

[    0.000000] CPU: Feroceon 88FR131 [56251311] revision 1 (ARMv5TE), cr=00053977

Thanks for the information. Seems like we have the same CPU:

| [    0.000000] CPU: Feroceon 88FR131 [56251311] revision 1 (ARMv5TE), cr=00053177
| [    0.000000] CPU: VIVT data cache, VIVT instruction cache

and it's actually VIVT, not VIPT as I wrote in an earlier mail.

I have been looking at flush_dcache_page(), which is supposed to do the trick when caches need to be flushed/dropped for the mmapped memory. Interestingly, there seems to be no page mapping - so page_mapping(page) returns NULL and therefore __flush_dcache_aliases(mapping, page) is never being called.

Could that be the culprit here? I guess there should always be a mapping present for mmapped pages, right?

Greetings, Phil

#8 lore raw

PATCH 1/1 af_packet: flush complete kernel cache in packet_sendmsg

In reply to Phil Sutter

This flushes the cache before and after accessing the mmapped packet buffer. It seems like the call to flush_dcache_page from inside __packet_get_status is not enough on Kirkwood (or ARM in general).
---
I know this is far from an optimal solution, but it's in fact the only working one I found. And it shouldn't interfere with unaffected target systems. So anyone relying on a working TX_RING on Kirkwood may refer to this patch. Any ARM/cache/Marvell/Kirkwood experts out there feel free to improve this.

---
 net/packet/af_packet.c |   24 ++++++++++++++++++++----
 1 files changed, 20 insertions(+), 4 deletions(-)
net/packet/af_packet.c +20 -4
index 243946d..d7b5c2e 100644
@@ -87,6 +87,14 @@
 #include <net/inet_common.h>
 #endif
 
+/* whether we need additional cacheflushing between user- and kernel-space */
+#ifdef CONFIG_ARCH_KIRKWOOD
+#  define ENABLE_CACHEPROB_WORKAROUND
+#  define kw_extra_cache_flush()	flush_cache_all()
+#else
+#  define kw_extra_cache_flush()	/* nothing */
+#endif
+
 /*
    Assumptions:
    - if device has no dev->hard_header routine, it adds and removes ll header
@@ -1239,10 +1247,13 @@ static int packet_sendmsg(struct kiocb *iocb, struct socket *sock,
 {
 	struct sock *sk = sock->sk;
 	struct packet_sock *po = pkt_sk(sk);
-	if (po->tx_ring.pg_vec)
-		return tpacket_snd(po, msg);
-	else
-		return packet_snd(sock, msg, len);
+	int rc;
+
+	kw_extra_cache_flush();
+	rc = po->tx_ring.pg_vec ? tpacket_snd(po, msg) :
+			packet_snd(sock, msg, len);
+	kw_extra_cache_flush();
+	return rc;
 }
 
 /*
@@ -2622,6 +2633,11 @@ static int __init packet_init(void)
 	sock_register(&packet_family_ops);
 	register_pernet_subsys(&packet_net_ops);
 	register_netdevice_notifier(&packet_netdev_notifier);
+
+#ifdef ENABLE_CACHEPROB_WORKAROUND
+	printk(KERN_INFO "af_packet: cache coherency workaround for kirkwood is active!\n");
+#endif
+
 out:
 	return rc;
 }
Signature · 1 line
1.7.3.4
#9 lore raw

Re: [PATCH] af_packet: flush complete kernel cache in packet_sendmsg

Quoted 6 lines · Phil Sutter

This flushes the cache before and after accessing the mmapped packet buffer. It seems like the call to flush_dcache_page from inside __packet_get_status is not enough on Kirkwood (or ARM in general).
---
I know this is far from an optimal solution, but it's in fact the only working one I found.

[...]

This is ridiculous. If flush_dcache_page() isn't doing everything it should, you need to fix that.

Ben.

Signature · 3 lines
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.
#10 lore raw

Re: [PATCH] af_packet: flush complete kernel cache in packet_sendmsg

Quoted 11 lines · Ben Hutchings
Quoted 6 lines · Phil Sutter

This flushes the cache before and after accessing the mmapped packet buffer. It seems like the call to flush_dcache_page from inside __packet_get_status is not enough on Kirkwood (or ARM in general).
---
I know this is far from an optimal solution, but it's in fact the only working one I found.

[...]

This is ridiculous. If flush_dcache_page() isn't doing everything it should, you need to fix that.

You're absolutely correct. But in fact this problem goes way too deep for me to find it's cause. And since my time is finite, I doubt this will change in the near future. So I asked for help, a pointer in whatever direction or anything I could try to help further analyzing - without any response (unless I missed it, in which case I apologize).

Please don't get me wrong. I have no intend in this patch becoming mainline, just want to give others with the same problem a starting point.

Greetings, Phil

Signature · 15 lines
Viprinet GmbH
Mainzer Str. 43
55411 Bingen am Rhein
Germany

Zentrale:     +49-6721-49030-0
Durchwahl:    +49-6721-49030-134
Fax:          +49-6721-49030-209

phil.sutter at viprinet.com
http://www.viprinet.com

Sitz der Gesellschaft: Bingen am Rhein
Handelsregister: Amtsgericht Mainz HRB40380
Gesch?ftsf?hrer: Simon Kissel
#16 lore raw

Re: FW: [PATCH] af_packet: flush complete kernel cache in packet_sendmsg

In reply to a message not in the archive
Quoted 7 lines

This flushes the cache before and after accessing the mmapped packet buffer. It seems like the call to flush_dcache_page from inside __packet_get_status is not enough on Kirkwood (or ARM in general).

Quoted 6 lines

+ ? ? ? kw_extra_cache_flush();
+ ? ? ? rc = po->tx_ring.pg_vec ? tpacket_snd(po, msg) :
+ ? ? ? ? ? ? ? ? ? ? ? packet_snd(sock, msg, len);
+ ? ? ? kw_extra_cache_flush();
+ ? ? ? return rc;
?}

If a workaround is needed for mmap, then why not change tpacket_snd?

Also, is this workaround actually working for all the cases? Because packet_get_status is not being touched in your patch.

Also, I don't see any changes for the Rx-path. Is that working ok?

Chetan Loke

#17 lore raw

Re: FW: [PATCH] af_packet: flush complete kernel cache in packet_sendmsg

Quoted 7 lines · chetan loke
Quoted 4 lines

This flushes the cache before and after accessing the mmapped packet buffer. It seems like the call to flush_dcache_page from inside __packet_get_status is not enough on Kirkwood (or ARM in general).

Quoted 8 lines
Quoted 6 lines

+ ? ? ? kw_extra_cache_flush();
+ ? ? ? rc = po->tx_ring.pg_vec ? tpacket_snd(po, msg) :
+ ? ? ? ? ? ? ? ? ? ? ? packet_snd(sock, msg, len);
+ ? ? ? kw_extra_cache_flush();
+ ? ? ? return rc;
?}

If a workaround is needed for mmap, then why not change tpacket_snd?

I did not verify that packet_snd() is not affected. OTOH, adding it there was quite "intuitive".

Quoted 4 lines

Also, is this workaround actually working for all the cases? Because packet_get_status is not being touched in your patch.

Also, I don't see any changes for the Rx-path. Is that working ok?

So far we haven't noticed problems in that direction. I just tried some explicit test: having tcpdump print local timestamps (not the pcap-ones) on every received packet, activating icmp_echo_ignore_all and pinging the host on a dedicated line. I expected to sometimes see a second difference between the two timestamps, as like with sending from time to time a packet should get "lost" in the cache, and then occur to userspace after the next one arrived. Maybe my test is broken, or RX is indeed unaffected.

Greetings and thanks for the hints, Phil

Signature · 15 lines
Viprinet GmbH
Mainzer Str. 43
55411 Bingen am Rhein
Germany

Zentrale:     +49-6721-49030-0
Durchwahl:    +49-6721-49030-134
Fax:          +49-6721-49030-209

phil.sutter at viprinet.com
http://www.viprinet.com

Sitz der Gesellschaft: Bingen am Rhein
Handelsregister: Amtsgericht Mainz HRB40380
Gesch?ftsf?hrer: Simon Kissel
#18 lore raw

Re: FW: [PATCH] af_packet: flush complete kernel cache in packet_sendmsg

Quoted 9 lines · Phil Sutter

So far we haven't noticed problems in that direction. I just tried some explicit test: having tcpdump print local timestamps (not the pcap-ones) on every received packet, activating icmp_echo_ignore_all and pinging the host on a dedicated line. I expected to sometimes see a second difference between the two timestamps, as like with sending from time to time a packet should get "lost" in the cache, and then occur to userspace after the next one arrived. Maybe my test is broken, or RX is indeed unaffected.

You will need high traffic rate. If interested, you could try pktgen(with varying packet-load). Keep the packet-payload under 1500 bytes (don't send jumbo frames) unless you have the following fix:
commit cc9f01b246ca8e4fa245991840b8076394f86707

Your Tx path is working because flush_cache_call gets triggered before flush_dcache_page. On the Rx path, since you don't have that workaround, you will eventually(it's just a matter of time) see this problem.

Or, delete your patch and try this workaround (in __packet_get/set_status) and you may be able to cover both Tx and Rx paths.

net/packet/af_packet.c +0 -0
index 2ea3d63..35d71dc 100644
@@ -412,11 +412,19 @@ static void __packet_set_status(struct

packet_sock *po, void *frame, int status)

        switch (po->tp_version) {
        case TPACKET_V1:
                h.h1->tp_status = status;

- flush_dcache_page(pgv_to_page(&h.h1->tp_status));
+ #ifndef ENABLE_CACHEPROB_WORKAROUND
+ flush_dcache_page(pgv_to_page(&h.h1->tp_status));
+ #else
+ kw_extra_cache_flush();
+ endif

                break;
        case TPACKET_V2:
                h.h2->tp_status = status;

- flush_dcache_page(pgv_to_page(&h.h2->tp_status));
+ #ifndef ENABLE_CACHEPROB_WORKAROUND
+ flush_dcache_page(pgv_to_page(&h.h2->tp_status));
+ #else
+ kw_extra_cache_flush();
+ #endif

                break;
        case TPACKET_V3:
        default:

@@ -437,13 +445,19 @@ static int __packet_get_status(struct packet_sock *po, void *frame)

        smp_rmb();

+ kw_extra_cache_flush();
+

        h.raw = frame;
        switch (po->tp_version) {
        case TPACKET_V1:

- flush_dcache_page(pgv_to_page(&h.h1->tp_status));
+ #ifndef ENABLE_CACHEPROB_WORKAROUND
+ flush_dcache_page(pgv_to_page(&h.h1->tp_status));
+ #endif

                return h.h1->tp_status;
        case TPACKET_V2:

- flush_dcache_page(pgv_to_page(&h.h2->tp_status));
+ #ifndef ENABLE_CACHEPROB_WORKAROUND
+ flush_dcache_page(pgv_to_page(&h.h2->tp_status));
+ #endif

                return h.h2->tp_status;
        case TPACKET_V3:
        default:
Quoted 1 line

Greetings and thanks for the hints, Phil

Chetan Loke

#11 lore raw

Re: [PATCH] af_packet: flush complete kernel cache in packet_sendmsg

In reply to Ben Hutchings
Quoted 11 lines · Ben Hutchings
Quoted 6 lines · Phil Sutter

This flushes the cache before and after accessing the mmapped packet buffer. It seems like the call to flush_dcache_page from inside __packet_get_status is not enough on Kirkwood (or ARM in general).
---
I know this is far from an optimal solution, but it's in fact the only working one I found.

[...]

This is ridiculous. If flush_dcache_page() isn't doing everything it should, you need to fix that.

It does do everything it should - which is to perform maintanence on page cache pages. It flushes the kernel mapping of the page. It also flushes the userspace mappings of the page which it finds by walking the mmap list via the associated struct page. It does not touch vmalloc mappings because it has no way to know whether they exist or not.

It doesn't do so much for anonymous pages - to do so would only duplicate what flush_anon_page() does at the very same callsites. Plus the mmap list isn't available for such pages so there's no way to find out what userspace addresses to flush.

If the AF_PACKET buffers are created from anonymous pages and it's using flush_dcache_page(), it's using the wrong interface.

#12 lore raw

Re: [PATCH] af_packet: flush complete kernel cache in packet_sendmsg

Hi,

Quoted 20 lines · Russell King - ARM Linux
Quoted 11 lines · Ben Hutchings
Quoted 6 lines · Phil Sutter

This flushes the cache before and after accessing the mmapped packet buffer. It seems like the call to flush_dcache_page from inside __packet_get_status is not enough on Kirkwood (or ARM in general).
---
I know this is far from an optimal solution, but it's in fact the only working one I found.

[...]

This is ridiculous. If flush_dcache_page() isn't doing everything it should, you need to fix that.

It does do everything it should - which is to perform maintanence on page cache pages. It flushes the kernel mapping of the page. It also flushes the userspace mappings of the page which it finds by walking the mmap list via the associated struct page. It does not touch vmalloc mappings because it has no way to know whether they exist or not.

Quoted 4 lines

It doesn't do so much for anonymous pages - to do so would only duplicate what flush_anon_page() does at the very same callsites. Plus the mmap list isn't available for such pages so there's no way to find out what userspace addresses to flush.

Indeed very interesting information, thanks a lot!

The code in question uses __get_free_pages(), and if that fails uses vmalloc() (see alloc_one_pg_vec_page() for reference). Both code paths show result in the same faulty behaviour.

Quoted 2 lines

If the AF_PACKET buffers are created from anonymous pages and it's using flush_dcache_page(), it's using the wrong interface.

So, in order to fix this, which alternative would you suggest? Quite a lot of work has been done regarding memory allocation, so I guess changing that side is a no-go.

Greetings, Phil

Signature · 15 lines
Viprinet GmbH
Mainzer Str. 43
55411 Bingen am Rhein
Germany

Zentrale:     +49-6721-49030-0
Durchwahl:    +49-6721-49030-134
Fax:          +49-6721-49030-209

phil.sutter at viprinet.com
http://www.viprinet.com

Sitz der Gesellschaft: Bingen am Rhein
Handelsregister: Amtsgericht Mainz HRB40380
Gesch?ftsf?hrer: Simon Kissel
#19 lore raw

Re: FW: [PATCH] af_packet: flush complete kernel cache in packet_sendmsg

In reply to chetan loke
Quoted 12 lines · chetan loke
Quoted 9 lines · Phil Sutter

So far we haven't noticed problems in that direction. I just tried some explicit test: having tcpdump print local timestamps (not the pcap-ones) on every received packet, activating icmp_echo_ignore_all and pinging the host on a dedicated line. I expected to sometimes see a second difference between the two timestamps, as like with sending from time to time a packet should get "lost" in the cache, and then occur to userspace after the next one arrived. Maybe my test is broken, or RX is indeed unaffected.

Quoted 4 lines

You will need high traffic rate. If interested, you could try pktgen(with varying packet-load). Keep the packet-payload under 1500 bytes (don't send jumbo frames) unless you have the following fix:
commit cc9f01b246ca8e4fa245991840b8076394f86707

Hmm. I don't really get your point here: with higher traffic rates, the bug should be even harder to identify. Assuming the same behaviour as for TX, of course. There are no packets lost, just not immediately transmitted (or never, if it's the last packet to be sent). This is how it goes:
1) userspace places packet into TX_RING, calls sendto()
2) kernel does not see the packet, reads TP_STATUS_AVAILABLE for the given field from the cache
3) userspace places second packet into TX_RING (after the first one, since it knows it's there)
4) something happens that makes caches flush
5) userspace calls sendto()
5) kernel sees two packets to be transmitted, sends them out

So analogous for RX, this should mean:
1) tcpdump runs pcap_loop() (which, according to strace, calls poll() with a timeout of 1s)
2) kernel receives packet, puts it into RX_RING, sets POLLIN
3) tcpdump's poll() returns, but an unmodified RX_RING is seen
4) something happens that makes caches flush
5) (2) happens again
6) tcpdump's poll() returns, two packets are seen in RX_RING

My tests on TX-side show that this "something that makes caches flush" actually happens quite frequently. But nevertheless, when receiving a packet once a second, I'm expecting to occasionally see no packet in two seconds, and then two in the following. The higher the packet rate, the harder it should be to notice this phenomenon.

Quoted 4 lines

Your Tx path is working because flush_cache_call gets triggered before flush_dcache_page. On the Rx path, since you don't have that workaround, you will eventually(it's just a matter of time) see this problem.

So you say if I called flush_cache_all() _after_ flush_dcache_page() it wouldn't work?

Quoted 3 lines

Or, delete your patch and try this workaround (in __packet_get/set_status) and you may be able to cover both Tx and Rx paths.

Oh great, thanks a lot for improving my ugly hacks! :)

Greetings, Phil

#13 lore raw

Re: [PATCH] af_packet: flush complete kernel cache in packet_sendmsg

In reply to Phil Sutter
Quoted 28 lines · Phil Sutter

Hi,

Quoted 20 lines · Russell King - ARM Linux
Quoted 11 lines · Ben Hutchings
Quoted 6 lines · Phil Sutter

This flushes the cache before and after accessing the mmapped packet buffer. It seems like the call to flush_dcache_page from inside __packet_get_status is not enough on Kirkwood (or ARM in general).
---
I know this is far from an optimal solution, but it's in fact the only working one I found.

[...]

This is ridiculous. If flush_dcache_page() isn't doing everything it should, you need to fix that.

It does do everything it should - which is to perform maintanence on page cache pages. It flushes the kernel mapping of the page. It also flushes the userspace mappings of the page which it finds by walking the mmap list via the associated struct page. It does not touch vmalloc mappings because it has no way to know whether they exist or not.

Quoted 4 lines

It doesn't do so much for anonymous pages - to do so would only duplicate what flush_anon_page() does at the very same callsites. Plus the mmap list isn't available for such pages so there's no way to find out what userspace addresses to flush.

Quoted 5 lines

Indeed very interesting information, thanks a lot!

The code in question uses __get_free_pages(), and if that fails uses vmalloc() (see alloc_one_pg_vec_page() for reference). Both code paths show result in the same faulty behaviour.

So, what you're wanting is cache coherency between vmalloc() and userspace. There is no API in the kernel to do that, and you'll see the same failures of this interface not only on ARM but also other architectures with virtual caches.

It sounds like we need an API to flush the cache using both the userspace address, plus the kernel side address be that in the direct map or the vmalloc map areas.

Or maybe the right solution is to simply disable AF_PACKET MMAP support for virtual cached architectures - it may be that adding cache flushing calls makes the thing too expensive and the benefits of mmap over normal read/write are lost.

#14 lore raw

Re: [PATCH] af_packet: flush complete kernel cache in packet_sendmsg

Quoted 9 lines · Russell King - ARM Linux
Quoted 3 lines

The code in question uses __get_free_pages(), and if that fails uses vmalloc() (see alloc_one_pg_vec_page() for reference). Both code paths show result in the same faulty behaviour.

So, what you're wanting is cache coherency between vmalloc() and userspace. There is no API in the kernel to do that, and you'll see the same failures of this interface not only on ARM but also other architectures with virtual caches.

Quoted 8 lines

It sounds like we need an API to flush the cache using both the userspace address, plus the kernel side address be that in the direct map or the vmalloc map areas.

Or maybe the right solution is to simply disable AF_PACKET MMAP support for virtual cached architectures - it may be that adding cache flushing calls makes the thing too expensive and the benefits of mmap over normal read/write are lost.

OK, that's horrible. Of course we depend on just this combination to work flawlessly, i.e. PACKET_MMAP && VIVT. :(

Another userspace-interface I'm working on uses a different solution:
memory is allocated in userspace and accessed from kernelspace using get_user_pages(). I did not explicitly search for the earlier described fault pattern, but we didn't notice any problem with this approach on the very same hardware either. I already see myself writing TPACKET_V3. ;)

What do you think?

Greetings, Phil

Signature · 15 lines
Viprinet GmbH
Mainzer Str. 43
55411 Bingen am Rhein
Germany

Zentrale:     +49-6721-49030-0
Durchwahl:    +49-6721-49030-134
Fax:          +49-6721-49030-209

phil.sutter at viprinet.com
http://www.viprinet.com

Sitz der Gesellschaft: Bingen am Rhein
Handelsregister: Amtsgericht Mainz HRB40380
Gesch?ftsf?hrer: Simon Kissel