CAN messages being lost on i.MX25 with flexcan
Hello,
I'm using Voipac i.MX25 module with flexcan, kernel 2.6.38.9.
I'm sending the data at 250kbps, around 1100 msgs/sec. When I enable the canbus interface (canconfig can0 start), CPU load is higher, that is understandable, there are many interrupts. I'm not doing anything else then using recvmsg (or recvmmsg which is little better), but some messages are still lost (around 1500 messages lost from 467 000 being send from another source). When I start doing something (like "cat /proc/interrupts", or write to file), many more messages are lost.
Do you have any idea how to fix this? I need to make some CAN messages logger and I cannot lose any message (idealy :) So I made big buffer in my program so that I don't need to write the messages into the file while "recording" is enabled, after "recording" is switched off, I write the buffer into the file, but that is still not good enough. Is there any way how to write to some buffer directly in flexcan driver (the best would be in the IRQ routine) and then read messages from this buffer in my program?
Or are just interrupts lost when doing something else in the system and I cannot fix this? Or can I somehow specify that "can rx interrupts" has highest prioroty?
Thanks,
Martin
Quoted 10 lines · Martin Kozusky
Hello,
I'm using Voipac i.MX25 module with flexcan, kernel 2.6.38.9. I'm sending the data at 250kbps, around 1100 msgs/sec. When I enable the canbus interface (canconfig can0 start), CPU load is higher, that is understandable, there are many interrupts. I'm not doing anything else then using recvmsg (or recvmmsg which is little better), but some messages are still lost (around 1500 messages lost from 467 000 being send from another source). When I start doing something (like "cat/proc/interrupts", or write to file), many more messages are lost.
Quoted 11 lines
Do you have any idea how to fix this? I need to make some CAN messages logger and I cannot lose any message (idealy :) So I made big buffer in my program so that I don't need to write the messages into the file while "recording" is enabled, after "recording" is switched off, I write the buffer into the file, but that is still not good enough. Is there any way how to write to some buffer directly in flexcan driver (the best would be in the IRQ routine) and then read messages from this buffer in my program?
Or are just interrupts lost when doing something else in the system and I cannot fix this? Or can I somehow specify that "can rx interrupts" has highest prioroty?
Do you already know where the packets are dropped (get lost). Maybe your user space app is simply not faster enough to process them in time. You can check that with the candump option "-d".
Wolfgang.
Dne 19.4.2012 12:54, Wolfgang Grandegger napsal(a):
Quoted 23 lines · Martin Kozusky
Quoted 10 lines
Hello,
I'm using Voipac i.MX25 module with flexcan, kernel 2.6.38.9. I'm sending the data at 250kbps, around 1100 msgs/sec. When I enable the canbus interface (canconfig can0 start), CPU load is higher, that is understandable, there are many interrupts. I'm not doing anything else then using recvmsg (or recvmmsg which is little better), but some messages are still lost (around 1500 messages lost from 467 000 being send from another source). When I start doing something (like "cat/proc/interrupts", or write to file), many more messages are lost.Quoted 11 lines
Do you have any idea how to fix this? I need to make some CAN messages logger and I cannot lose any message (idealy :) So I made big buffer in my program so that I don't need to write the messages into the file while "recording" is enabled, after "recording" is switched off, I write the buffer into the file, but that is still not good enough. Is there any way how to write to some buffer directly in flexcan driver (the best would be in the IRQ routine) and then read messages from this buffer in my program?
Or are just interrupts lost when doing something else in the system and I cannot fix this? Or can I somehow specify that "can rx interrupts" has highest prioroty?
Quoted 3 lines
Do you already know where the packets are dropped (get lost). Maybe your user space app is simply not faster enough to process them in time. You can check that with the candump option "-d".
Hello Wolfgang,
I don't know where it gets lost. But I'm just making select() and recvmmsg in my app now. Now when I'm trying candump -d, it shows messages and between that:
can0 18FFC33D [8] 4A E6 00 00 03 00 00 F6 can0 1CEFB8F3 [8] 34 42 23 3D 8C 01 00 00 DROPCOUNT: dropped 245 CAN frames on 'can0' socket (total drops 30886) can0 18FF0AEF [8] 70 76 46 7C A7 9E 2A 6A can0 18FF0AF0 [8] B0 7E FB 7D A7 DC 27 69 can0 18FEBF0B [8] F5 03 79 81 69 74 6E 79
I've sent around 40 000 messages.
Does this help somehow to identify where the problem is?
I found out that if I (in my program) don't printf every frame, then dropcount is better, so I think when candump would only show dropps and not every frame then it would also get better.
This is my /proc/interrupts
root@vmx25 /opt$ cat /proc/interrupts
CPU0
0: 0 - spi_imx 5: 32827 - IMX-uart 9: 129 - sdhci 13: 0 - spi_imx 25: 5 - imxdi_rtc 33: 12168 - mxc_nand 34: 1 - mxc-sdma 37: 109275 - ehci_hcd:usb1 43: 1624971 - can0 46: 0 - mxcadc 54: 195357 - i.MX Timer Tick 164: 0 - ESDHCI card 0 detect
Err: 0
I have another problem - when I send messages and nobody is acking them (no other device on CANbus), system stops responding until I put another device onto CANbus, but I think I should start new thread with this.
Martin
Quoted 6 lines
Wolfgang.
--
To unsubscribe from this list: send the line "unsubscribe linux-can" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 40 lines · Martin Kozusky
Dne 19.4.2012 12:54, Wolfgang Grandegger napsal(a):
Quoted 23 lines · Martin Kozusky
Quoted 10 lines
Hello,
I'm using Voipac i.MX25 module with flexcan, kernel 2.6.38.9. I'm sending the data at 250kbps, around 1100 msgs/sec. When I enable the canbus interface (canconfig can0 start), CPU load is higher, that is understandable, there are many interrupts. I'm not doing anything else then using recvmsg (or recvmmsg which is little better), but some messages are still lost (around 1500 messages lost from 467 000 being send from another source). When I start doing something (like "cat/proc/interrupts", or write to file), many more messages are lost.Quoted 11 lines
Do you have any idea how to fix this? I need to make some CAN messages logger and I cannot lose any message (idealy :) So I made big buffer in my program so that I don't need to write the messages into the file while "recording" is enabled, after "recording" is switched off, I write the buffer into the file, but that is still not good enough. Is there any way how to write to some buffer directly in flexcan driver (the best would be in the IRQ routine) and then read messages from this buffer in my program?
Or are just interrupts lost when doing something else in the system and I cannot fix this? Or can I somehow specify that "can rx interrupts" has highest prioroty?Quoted 3 lines
Do you already know where the packets are dropped (get lost). Maybe your user space app is simply not faster enough to process them in time. You can check that with the candump option "-d".
Hello Wolfgang,
I don't know where it gets lost. But I'm just makingselect()and recvmmsg in my app now.
Now when I'm trying candump -d, it shows messages and between that:can0 18FFC33D [8] 4A E6 00 00 03 00 00 F6 can0 1CEFB8F3 [8] 34 42 23 3D 8C 01 00 00 DROPCOUNT: dropped 245 CAN frames on 'can0' socket (total drops 30886) can0 18FF0AEF [8] 70 76 46 7C A7 9E 2A 6A can0 18FF0AF0 [8] B0 7E FB 7D A7 DC 27 69 can0 18FEBF0B [8] F5 03 79 81 69 74 6E 79
Quoted 2 lines
I've sent around 40 000 messages.
Does this help somehow to identify where the problem is?
Yes, you are loosing messages because your app is not faster enough.
Quoted 3 lines
I found out that if I (in my program) don't printf every frame, then dropcount is better, so I think when candump would only show dropps and not every frame then it would also get better.
printf takes time, of course. Therefore everything which makes your app reading the messages faster will help including increasing the scheduling priority of the process/thread. Also increasing the size of the receive buffer may help. Try candump with the "-r" option.
Quoted 16 lines
This is my
/proc/interruptsroot@vmx25 /opt$ cat
/proc/interrupts
CPU00: 0 - spi_imx 5: 32827 - IMX-uart 9: 129 - sdhci 13: 0 - spi_imx 25: 5 - imxdi_rtc 33: 12168 - mxc_nand 34: 1 - mxc-sdma 37: 109275 - ehci_hcd:usb1 43: 1624971 - can0 46: 0 - mxcadc 54: 195357 - i.MX Timer Tick 164: 0 - ESDHCI card 0 detect
Quoted 1 line
Err: 0
In a first place, I don't think your are loosing messages because they are not read-out faster enough from the CAN controller.
Quoted 3 lines
I have another problem - when I send messages and nobody is acking them (no other device on CANbus), system stops responding until I put another device onto CANbus, but I think I should start new thread with this.
What kernel version do you use? Do you use the flexcan driver from that kernel? The problem is bus errors coming at high rate.
Wolfgang.
Dne 19.4.2012 13:44, Wolfgang Grandegger napsal(a):
Quoted 46 lines · Martin Kozusky
Quoted 40 lines
Dne 19.4.2012 12:54, Wolfgang Grandegger napsal(a):
Quoted 23 lines · Martin Kozusky
Quoted 10 lines
Hello,
I'm using Voipac i.MX25 module with flexcan, kernel 2.6.38.9. I'm sending the data at 250kbps, around 1100 msgs/sec. When I enable the canbus interface (canconfig can0 start), CPU load is higher, that is understandable, there are many interrupts. I'm not doing anything else then using recvmsg (or recvmmsg which is little better), but some messages are still lost (around 1500 messages lost from 467 000 being send from another source). When I start doing something (like "cat/proc/interrupts", or write to file), many more messages are lost.Quoted 11 lines
Do you have any idea how to fix this? I need to make some CAN messages logger and I cannot lose any message (idealy :) So I made big buffer in my program so that I don't need to write the messages into the file while "recording" is enabled, after "recording" is switched off, I write the buffer into the file, but that is still not good enough. Is there any way how to write to some buffer directly in flexcan driver (the best would be in the IRQ routine) and then read messages from this buffer in my program?
Or are just interrupts lost when doing something else in the system and I cannot fix this? Or can I somehow specify that "can rx interrupts" has highest prioroty?Quoted 3 lines
Do you already know where the packets are dropped (get lost). Maybe your user space app is simply not faster enough to process them in time. You can check that with the candump option "-d".
Hello Wolfgang,
I don't know where it gets lost. But I'm just makingselect()and recvmmsg in my app now.
Now when I'm trying candump -d, it shows messages and between that:can0 18FFC33D [8] 4A E6 00 00 03 00 00 F6 can0 1CEFB8F3 [8] 34 42 23 3D 8C 01 00 00 DROPCOUNT: dropped 245 CAN frames on 'can0' socket (total drops 30886) can0 18FF0AEF [8] 70 76 46 7C A7 9E 2A 6A can0 18FF0AF0 [8] B0 7E FB 7D A7 DC 27 69 can0 18FEBF0B [8] F5 03 79 81 69 74 6E 79Quoted 2 lines
I've sent around 40 000 messages.
Does this help somehow to identify where the problem is?Yes, you are loosing messages because your app is not faster enough.
Quoted 8 lines
Quoted 3 lines
I found out that if I (in my program) don't printf every frame, then dropcount is better, so I think when candump would only show dropps and not every frame then it would also get better.
printf takes time, of course. Therefore everything which makes your app reading the messages faster will help including increasing the scheduling priority of the process/thread. Also increasing the size of the receive buffer may help. Try candump with the "-r" option.
I tried -r 10000000
and the result was that dropping started some time later then it would normaly. From 467 000 packets, total drops was 103064. That is better ratio than before. But few times, my system was stucked for a while, then it recovered and continued to work. So I will try to use this buffer size in my program, without printf and I will see how many packets I lose.
Is there a way how to get actual used size of the buffer?
Quoted 21 lines
Quoted 15 lines
This is my
/proc/interruptsroot@vmx25 /opt$ cat
/proc/interrupts
CPU00: 0 - spi_imx 5: 32827 - IMX-uart 9: 129 - sdhci 13: 0 - spi_imx 25: 5 - imxdi_rtc 33: 12168 - mxc_nand 34: 1 - mxc-sdma 37: 109275 - ehci_hcd:usb1 43: 1624971 - can0 46: 0 - mxcadc 54: 195357 - i.MX Timer TickQuoted 2 lines
164: 0 - ESDHCI card 0 detect
Err: 0In a first place, I don't think your are loosing messages because they are not read-out faster enough from the CAN controller.
Quoted 6 lines
Quoted 3 lines
I have another problem - when I send messages and nobody is acking them (no other device on CANbus), system stops responding until I put another device onto CANbus, but I think I should start new thread with this.
What kernel version do you use? Do you use the flexcan driver from that kernel? The problem is bus errors coming at high rate.
I'm using 2.6.35.9 Preempt, the flexcan.c driver is original.
Martin
Quoted 6 lines
Wolfgang.
--
To unsubscribe from this list: send the line "unsubscribe linux-can" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 64 lines · Martin Kozusky
Dne 19.4.2012 13:44, Wolfgang Grandegger napsal(a):
Quoted 54 lines · Martin Kozusky
Quoted 48 lines
Dne 19.4.2012 12:54, Wolfgang Grandegger napsal(a):
Quoted 30 lines · Martin Kozusky
Quoted 20 lines
Hello,
I'm using Voipac i.MX25 module with flexcan, kernel 2.6.38.9. I'm sending the data at 250kbps, around 1100 msgs/sec. When I enable the
canbus interface (canconfig can0 start), CPU load is higher, that is understandable, there are many interrupts. I'm not doing anything else then using recvmsg (or recvmmsg which is little better), but some messages are still lost (around 1500 messages lost from 467 000 being send from another source). When I start doing something (like "cat/proc/interrupts", or write to file), many more messages are lost.Do you have any idea how to fix this? I need to make some CAN messages logger and I cannot lose any message (idealy :) So I made big buffer in
my program so that I don't need to write the messages into the file while "recording" is enabled, after "recording" is switched off, I write
the buffer into the file, but that is still not good enough. Is there any way how to write to some buffer directly in flexcan driverQuoted 8 lines
(the best would be in the IRQ routine) and then read messages from this
buffer in my program?
Or are just interrupts lost when doing something else in the system and
I cannot fix this? Or can I somehow specify that "can rx interrupts" has
highest prioroty?Quoted 4 lines
Do you already know where the packets are dropped (get lost). Maybe your
user space app is simply not faster enough to process them in time. You can check that with the candump option "-d".Hello Wolfgang,
I don't know where it gets lost. But I'm just makingselect()and recvmmsg in my app now.
Now when I'm trying candump -d, it shows messages and between that:can0 18FFC33D [8] 4A E6 00 00 03 00 00 F6 can0 1CEFB8F3 [8] 34 42 23 3D 8C 01 00 00 DROPCOUNT: dropped 245 CAN frames on 'can0' socket (total drops 30886) can0 18FF0AEF [8] 70 76 46 7C A7 9E 2A 6A can0 18FF0AF0 [8] B0 7E FB 7D A7 DC 27 69 can0 18FEBF0B [8] F5 03 79 81 69 74 6E 79Quoted 2 lines
I've sent around 40 000 messages.
Does this help somehow to identify where the problem is?Yes, you are loosing messages because your app is not faster enough.
Quoted 8 lines
Quoted 3 lines
I found out that if I (in my program) don't printf every frame, then dropcount is better, so I think when candump would only show dropps and not every frame then it would also get better.
printf takes time, of course. Therefore everything which makes your app reading the messages faster will help including increasing the scheduling priority of the process/thread. Also increasing the size of the receive buffer may help. Try candump with the "-r" option.
Quoted 8 lines
I tried -r 10000000
and the result was that dropping started some time later then it would normaly. From 467 000 packets, total drops was 103064. That is better ratio than before. But few times, my system was stucked for a while, then it recovered and continued to work. So I will try to use this buffer size in my program, without printf and I will see how many packets I lose.
If other activities delay your app too much, your are in trouble. Proper thread priorities may help.
Quoted 1 line
Is there a way how to get actual used size of the buffer?
getsockopt?
Quoted 21 lines
Quoted 21 lines
Quoted 15 lines
This is my
/proc/interruptsroot@vmx25 /opt$ cat
/proc/interrupts
CPU00: 0 - spi_imx 5: 32827 - IMX-uart 9: 129 - sdhci 13: 0 - spi_imx 25: 5 - imxdi_rtc 33: 12168 - mxc_nand 34: 1 - mxc-sdma 37: 109275 - ehci_hcd:usb1 43: 1624971 - can0 46: 0 - mxcadc 54: 195357 - i.MX Timer TickQuoted 2 lines
164: 0 - ESDHCI card 0 detect
Err: 0In a first place, I don't think your are loosing messages because they are not read-out faster enough from the CAN controller.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
I have another problem - when I send messages and nobody is acking them (no other device on CANbus), system stops responding until I put another device onto CANbus, but I think I should start new thread with this.
What kernel version do you use? Do you use the flexcan driver from that kernel? The problem is bus errors coming at high rate.
I'm using 2.6.35.9 Preempt, the
flexcan.cdriver is original.
IIRC, that kernel version did not include the flexcan driver. It entered with 2.6.36. Anyway, maybe the following patch helps:
http://patchwork.ozlabs.org/patch/130761/
Wolfgang.
Quoted 4 lines
Quoted 2 lines
What kernel version do you use? Do you use the flexcan driver from that kernel? The problem is bus errors coming at high rate.
I'm using 2.6.35.9 Preempt, the
flexcan.cdriver is original.
We use flexcan on an iMX25 device. The vanilla flexcan driver (we use kernel 3.0, but same issue also seen on 2.6.37) has some serious bugs that occasionally caused high CPU utilization or even total crashes on our platform. If I remember correctly, I found some IRQs which were not serviced correctly, causing the ISR to be triggered continuously.
We fixed it by pulling the flexcan.c driver from Wolfgangs linux-can-next tree. I don't recall, if you could just replace the original flexcan.c directly, or if some integration were required.
Jesper Pedersen,
Inntrasys
Quoted 9 lines · Jesper Pedersen
Quoted 4 lines
Quoted 2 lines
What kernel version do you use? Do you use the flexcan driver from that kernel? The problem is bus errors coming at high rate.
I'm using 2.6.35.9 Preempt, the
flexcan.cdriver is original.We use flexcan on an iMX25 device. The vanilla flexcan driver (we use kernel 3.0, but same issue also seen on 2.6.37) has some serious bugs that occasionally caused high CPU utilization or even total crashes on our platform. If I remember correctly, I found some IRQs which were not serviced correctly, causing the ISR to be triggered continuously.
We fixed it by pulling the
flexcan.cdriver from Wolfgangs linux-can-next tree. I don't recall, if you could just replace the originalflexcan.cdirectly, or if some integration were required.
This confirms that the patch http://patchwork.ozlabs.org/patch/130761/
is need for the i.MX25 as well. I already told Martin to give it a try.
Wolfgang.
Dne 19.4.2012 15:37, Wolfgang Grandegger napsal(a):
Quoted 74 lines · Martin Kozusky
Quoted 64 lines
Dne 19.4.2012 13:44, Wolfgang Grandegger napsal(a):
Quoted 54 lines · Martin Kozusky
Quoted 48 lines
Dne 19.4.2012 12:54, Wolfgang Grandegger napsal(a):
Quoted 30 lines · Martin Kozusky
Quoted 20 lines
Hello,
I'm using Voipac i.MX25 module with flexcan, kernel 2.6.38.9. I'm sending the data at 250kbps, around 1100 msgs/sec. When I enable the
canbus interface (canconfig can0 start), CPU load is higher, that is understandable, there are many interrupts. I'm not doing anything else then using recvmsg (or recvmmsg which is little better), but some messages are still lost (around 1500 messages lost from 467 000 being send from another source). When I start doing something (like "cat/proc/interrupts", or write to file), many more messages are lost.Do you have any idea how to fix this? I need to make some CAN messages logger and I cannot lose any message (idealy :) So I made big buffer in
my program so that I don't need to write the messages into the file while "recording" is enabled, after "recording" is switched off, I write
the buffer into the file, but that is still not good enough. Is there any way how to write to some buffer directly in flexcan driverQuoted 8 lines
(the best would be in the IRQ routine) and then read messages from this
buffer in my program?
Or are just interrupts lost when doing something else in the system and
I cannot fix this? Or can I somehow specify that "can rx interrupts" has
highest prioroty?Quoted 4 lines
Do you already know where the packets are dropped (get lost). Maybe your
user space app is simply not faster enough to process them in time. You can check that with the candump option "-d".Hello Wolfgang,
I don't know where it gets lost. But I'm just makingselect()and recvmmsg in my app now.
Now when I'm trying candump -d, it shows messages and between that:can0 18FFC33D [8] 4A E6 00 00 03 00 00 F6 can0 1CEFB8F3 [8] 34 42 23 3D 8C 01 00 00 DROPCOUNT: dropped 245 CAN frames on 'can0' socket (total drops 30886) can0 18FF0AEF [8] 70 76 46 7C A7 9E 2A 6A can0 18FF0AF0 [8] B0 7E FB 7D A7 DC 27 69 can0 18FEBF0B [8] F5 03 79 81 69 74 6E 79Quoted 2 lines
I've sent around 40 000 messages.
Does this help somehow to identify where the problem is?Yes, you are loosing messages because your app is not faster enough.
Quoted 8 lines
Quoted 3 lines
I found out that if I (in my program) don't printf every frame, then dropcount is better, so I think when candump would only show dropps and not every frame then it would also get better.
printf takes time, of course. Therefore everything which makes your app reading the messages faster will help including increasing the scheduling priority of the process/thread. Also increasing the size of the receive buffer may help. Try candump with the "-r" option.
Quoted 8 lines
I tried -r 10000000
and the result was that dropping started some time later then it would normaly. From 467 000 packets, total drops was 103064. That is better ratio than before. But few times, my system was stucked for a while, then it recovered and continued to work. So I will try to use this buffer size in my program, without printf and I will see how many packets I lose.
Quoted 2 lines
If other activities delay your app too much, your are in trouble. Proper thread priorities may help.
So I tried my app with bigger receive buffer and saving data into memory and it looks like no messages are lost now :) (but when I write them into file, they are lost, but I can use the temp memory storage, so I can live with that :)
Quoted 32 lines
Quoted 1 line
Is there a way how to get actual used size of the buffer?
getsockopt?
Quoted 28 lines
Quoted 27 lines
Quoted 15 lines
This is my
/proc/interruptsroot@vmx25 /opt$ cat
/proc/interrupts
CPU0
0: 0 -spi_imx
5: 32827 - IMX-uart
9: 129 - sdhci13: 0 - spi_imx 25: 5 - imxdi_rtc 33: 12168 - mxc_nand 34: 1 - mxc-sdma 37: 109275 - ehci_hcd:usb1 43: 1624971 - can0 46: 0 - mxcadc 54: 195357 - i.MX Timer TickQuoted 2 lines
164: 0 - ESDHCI card 0 detect
Err: 0In a first place, I don't think your are loosing messages because they are not read-out faster enough from the CAN controller.
Quoted 3 lines
I have another problem - when I send messages and nobody is acking them (no other device on CANbus), system stops responding until I put another device onto CANbus, but I think I should start new thread with this.
What kernel version do you use? Do you use the flexcan driver from that kernel? The problem is bus errors coming at high rate.
Quoted 6 lines
Quoted 1 line
I'm using 2.6.35.9 Preempt, the
flexcan.cdriver is original.IIRC, that kernel version did not include the flexcan driver. It entered with 2.6.36. Anyway, maybe the following patch helps:
http://patchwork.ozlabs.org/patch/130761/
It seems that it helped :) when nobody is ACKing, it doesn't hang.
Thank you,
Martin
Quoted 6 lines
Wolfgang.
--
To unsubscribe from this list: send the line "unsubscribe linux-can" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 80 lines · Martin Kozusky
Dne 19.4.2012 15:37, Wolfgang Grandegger napsal(a):
Quoted 79 lines · Martin Kozusky
Quoted 77 lines
Dne 19.4.2012 13:44, Wolfgang Grandegger napsal(a):
Quoted 63 lines · Martin Kozusky
Quoted 52 lines
Dne 19.4.2012 12:54, Wolfgang Grandegger napsal(a):
Quoted 33 lines · Martin Kozusky
Quoted 23 lines
Hello,
I'm using Voipac i.MX25 module with flexcan, kernel 2.6.38.9. I'm sending the data at 250kbps, around 1100 msgs/sec. When I enable the
canbus interface (canconfig can0 start), CPU load is higher, that is understandable, there are many interrupts. I'm not doing anything else
then using recvmsg (or recvmmsg which is little better), but some messages are still lost (around 1500 messages lost from 467 000 being
send from another source). When I start doing something (like "cat/proc/interrupts", or write to file), many more messages are lost.Do you have any idea how to fix this? I need to make some CAN messages
logger and I cannot lose any message (idealy :) So I made big buffer in
my program so that I don't need to write the messages into the file while "recording" is enabled, after "recording" is switched off, I write
the buffer into the file, but that is still not good enough. Is there any way how to write to some buffer directly in flexcan driverQuoted 8 lines
(the best would be in the IRQ routine) and then read messages from this
buffer in my program?
Or are just interrupts lost when doing something else in the system and
I cannot fix this? Or can I somehow specify that "can rx interrupts" has
highest prioroty?Quoted 5 lines
Do you already know where the packets are dropped (get lost). Maybe your
user space app is simply not faster enough to process them in time. You
can check that with the candump option "-d".Hello Wolfgang,
I don't know where it gets lost. But I'm just makingselect()and recvmmsg in my app now.
Now when I'm trying candump -d, it shows messages and between that:can0 18FFC33D [8] 4A E6 00 00 03 00 00 F6 can0 1CEFB8F3 [8] 34 42 23 3D 8C 01 00 00 DROPCOUNT: dropped 245 CAN frames on 'can0' socket (total drops 30886) can0 18FF0AEF [8] 70 76 46 7C A7 9E 2A 6A can0 18FF0AF0 [8] B0 7E FB 7D A7 DC 27 69 can0 18FEBF0B [8] F5 03 79 81 69 74 6E 79Quoted 2 lines
I've sent around 40 000 messages.
Does this help somehow to identify where the problem is?Yes, you are loosing messages because your app is not faster enough.
Quoted 4 lines
I found out that if I (in my program) don't printf every frame, then dropcount is better, so I think when candump would only show dropps and
not every frame then it would also get better.Quoted 4 lines
printf takes time, of course. Therefore everything which makes your app reading the messages faster will help including increasing the scheduling priority of the process/thread. Also increasing the size of the receive buffer may help. Try candump with the "-r" option.
I tried -r 10000000
and the result was that dropping started some time later then it would normaly. From 467 000 packets, total drops was 103064. That is better ratio than before. But few times, my system was stucked for a while, then it recovered and continued to work. So I will try to use this buffer size in my program, without printf and I will see how many packets I lose.
Quoted 7 lines
Quoted 2 lines
If other activities delay your app too much, your are in trouble. Proper thread priorities may help.
So I tried my app with bigger receive buffer and saving data into memory and it looks like no messages are lost now :) (but when I write them into file, they are lost, but I can use the temp memory storage, so I can live with that :)
You are obviously hitting the resource limits. Do you need -rt? It makes your system slower.
Quoted 34 lines
Quoted 34 lines
Quoted 1 line
Is there a way how to get actual used size of the buffer?
getsockopt?
Quoted 30 lines
Quoted 29 lines
Quoted 15 lines
This is my
/proc/interruptsroot@vmx25 /opt$ cat
/proc/interrupts
CPU0
0: 0 -spi_imx
5: 32827 - IMX-uart
9: 129 - sdhci13: 0 - spi_imx 25: 5 - imxdi_rtc 33: 12168 - mxc_nand 34: 1 - mxc-sdma 37: 109275 - ehci_hcd:usb1 43: 1624971 - can0 46: 0 - mxcadc 54: 195357 - i.MX Timer TickQuoted 2 lines
164: 0 - ESDHCI card 0 detect
Err: 0In a first place, I don't think your are loosing messages because they are not read-out faster enough from the CAN controller.
Quoted 5 lines
I have another problem - when I send messages and nobody is acking them
(no other device on CANbus), system stops responding until I put another
device onto CANbus, but I think I should start new thread with this.What kernel version do you use? Do you use the flexcan driver from that kernel? The problem is bus errors coming at high rate.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
I'm using 2.6.35.9 Preempt, the
flexcan.cdriver is original.IIRC, that kernel version did not include the flexcan driver. It entered with 2.6.36. Anyway, maybe the following patch helps:
http://patchwork.ozlabs.org/patch/130761/It seems that it helped :) when nobody is ACKing, it doesn't hang.
Good. What does the CAN statistics show ($ ip -d -s link show can0)?
Wolfgang.
Dne 20.4.2012 8:04, Wolfgang Grandegger napsal(a):
Quoted 84 lines · Martin Kozusky
Quoted 83 lines
Dne 19.4.2012 15:37, Wolfgang Grandegger napsal(a):
Quoted 81 lines · Martin Kozusky
Quoted 69 lines
Dne 19.4.2012 13:44, Wolfgang Grandegger napsal(a):
Quoted 63 lines · Martin Kozusky
Quoted 52 lines
Dne 19.4.2012 12:54, Wolfgang Grandegger napsal(a):
Quoted 33 lines · Martin Kozusky
Quoted 23 lines
Hello,
I'm using Voipac i.MX25 module with flexcan, kernel 2.6.38.9. I'm sending the data at 250kbps, around 1100 msgs/sec. When I enable the
canbus interface (canconfig can0 start), CPU load is higher, that is understandable, there are many interrupts. I'm not doing anything else
then using recvmsg (or recvmmsg which is little better), but some messages are still lost (around 1500 messages lost from 467 000 being
send from another source). When I start doing something (like "cat/proc/interrupts", or write to file), many more messages are lost.Do you have any idea how to fix this? I need to make some CAN messages
logger and I cannot lose any message (idealy :) So I made big buffer in
my program so that I don't need to write the messages into the file while "recording" is enabled, after "recording" is switched off, I write
the buffer into the file, but that is still not good enough. Is there any way how to write to some buffer directly in flexcan driverQuoted 8 lines
(the best would be in the IRQ routine) and then read messages from this
buffer in my program?
Or are just interrupts lost when doing something else in the system and
I cannot fix this? Or can I somehow specify that "can rx interrupts" has
highest prioroty?Quoted 5 lines
Do you already know where the packets are dropped (get lost). Maybe your
user space app is simply not faster enough to process them in time. You
can check that with the candump option "-d".Hello Wolfgang,
I don't know where it gets lost. But I'm just makingselect()and recvmmsg in my app now.
Now when I'm trying candump -d, it shows messages and between that:can0 18FFC33D [8] 4A E6 00 00 03 00 00 F6 can0 1CEFB8F3 [8] 34 42 23 3D 8C 01 00 00 DROPCOUNT: dropped 245 CAN frames on 'can0' socket (total drops 30886) can0 18FF0AEF [8] 70 76 46 7C A7 9E 2A 6A can0 18FF0AF0 [8] B0 7E FB 7D A7 DC 27 69 can0 18FEBF0B [8] F5 03 79 81 69 74 6E 79Quoted 2 lines
I've sent around 40 000 messages.
Does this help somehow to identify where the problem is?Yes, you are loosing messages because your app is not faster enough.
Quoted 4 lines
I found out that if I (in my program) don't printf every frame, then dropcount is better, so I think when candump would only show dropps and
not every frame then it would also get better.Quoted 4 lines
printf takes time, of course. Therefore everything which makes your app reading the messages faster will help including increasing the scheduling priority of the process/thread. Also increasing the size of the receive buffer may help. Try candump with the "-r" option.
Quoted 8 lines
I tried -r 10000000
and the result was that dropping started some time later then it would normaly. From 467 000 packets, total drops was 103064. That is better ratio than before. But few times, my system was stucked for a while, then it recovered and continued to work. So I will try to use this buffer size in my program, without printf and I will see how many packets I lose.
If other activities delay your app too much, your are in trouble. Proper thread priorities may help.
Quoted 7 lines
Quoted 4 lines
So I tried my app with bigger receive buffer and saving data into memory and it looks like no messages are lost now :) (but when I write them into file, they are lost, but I can use the temp memory storage, so I can live with that :)
You are obviously hitting the resource limits. Do you need -rt? It makes your system slower.
I don't know what steps are necessary to make the kernel (and can driver) -rt :( I will first try to finish my app and see how it behaves.
Quoted 37 lines
Quoted 37 lines
Quoted 37 lines
Quoted 2 lines
Is there a way how to get actual used size of the buffer?
getsockopt?
Quoted 31 lines
Quoted 21 lines
Quoted 17 lines
This is my
/proc/interruptsroot@vmx25 /opt$ cat
/proc/interrupts
CPU0
0: 0 -spi_imx
5: 32827 - IMX-uart
9: 129 - sdhci
13: 0 -spi_imx
25: 5 -imxdi_rtc
33: 12168 -mxc_nand
34: 1 - mxc-sdma
37: 109275 - ehci_hcd:usb1
43: 1624971 - can0
46: 0 - mxcadc
54: 195357 - i.MX Timer Tick
164: 0 - ESDHCI card 0 detect
Err: 0In a first place, I don't think your are loosing messages because they are not read-out faster enough from the CAN controller.
Quoted 8 lines
Quoted 5 lines
I have another problem - when I send messages and nobody is acking them
(no other device on CANbus), system stops responding until I put another
device onto CANbus, but I think I should start new thread with this.What kernel version do you use? Do you use the flexcan driver from that kernel? The problem is bus errors coming at high rate.
I'm using 2.6.35.9 Preempt, the
flexcan.cdriver is original.
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
IIRC, that kernel version did not include the flexcan driver. It entered with 2.6.36. Anyway, maybe the following patch helps:
http://patchwork.ozlabs.org/patch/130761/It seems that it helped :) when nobody is ACKing, it doesn't hang.
Good. What does the CAN statistics show ($ ip -d -s link show can0)?
Now it shows (after some time of experimenting, it was disconnected from canbus for a while)
2: can0: <NOARP,ECHO> mtu 16 qdisc pfifo_fast state DOWN mode DEFAULT qlen 10
link/can
can state STOPPED (berr-counter tx 0 rx 0) restart-ms 0 bitrate 250000 sample-point 0.857
tq 285 prop-seg 5 phase-seg1 6 phase-seg2 2 sjw 1
flexcan: tseg1 4..16 tseg2 2..8 sjw 1..4 brp 1..256 brp-inc 1 clock 66500000
re-started bus-errors arbit-lost error-warn error-pass bus-off
0 0 0 1 0 0
RX: bytes packets errors dropped overrun mcast
4758311 602831 212 0 212 0
TX: bytes packets errors dropped carrier collsns
73 73 0 0 0 0
Martin
Quoted 8 lines
Wolfgang.
--
To unsubscribe from this list: send the line "unsubscribe linux-can" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 1 line · Martin Kozusky
Dne 20.4.2012 8:04, Wolfgang Grandegger napsal(a):
...
Quoted 5 lines
Quoted 2 lines
You are obviously hitting the resource limits. Do you need -rt? It makes your system slower.
I don't know what steps are necessary to make the kernel (and can driver) -rt :( I will first try to finish my app and see how it behaves.
Forget my comment. I was thinking that you are already using a -rt kernel (with the real-time preemption patch), but that's obviously not the case. Sorry for the noise.
...
Quoted 5 lines
Quoted 1 line
Good. What does the CAN statistics show ($ ip -d -s link show can0)?
Now it shows (after some time of experimenting, it was disconnected from canbus for a while)
Quoted 12 lines
2: can0: <NOARP,ECHO> mtu 16 qdisc
pfifo_faststate DOWN mode DEFAULT qlen 10
link/can
can state STOPPED (berr-counter tx 0 rx 0) restart-ms 0 bitrate 250000 sample-point 0.857
tq 285 prop-seg 5 phase-seg1 6 phase-seg2 2 sjw 1
flexcan: tseg1 4..16 tseg2 2..8 sjw 1..4 brp 1..256 brp-inc 1 clock 66500000
re-started bus-errors arbit-lost error-warn error-pass bus-off
0 0 0 1 0 0
RX: bytes packets errors dropped overrun mcast
4758311 602831 212 0 212 0
There have been some RX FIFO overflow errors, which means messages are lost by the hardware.
Wolfgang.