Adding IPv4 multicast support

Hello everyone,

I'm working on adding multicast support to barebox (for device discovery) and wanted to know if there's interest in having that included in upstream.

_______________________________________________
barebox mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/barebox

Can be network device discovery realized using picotcp entities?

I'll probably just push the patches to a github repository until picotcp lands.

Quoted 3 lines · On Sun, 13 Jul 2014 12:11:17 +0200 Colin Leitner…

Hello everyone,

I'm working on adding multicast support to barebox (for device discovery) and wanted to know if there's interest in having that included in upstream.

There is an initiative on integrating picotcp into barebox.

picotcp is a free TCP/IP stack implementation, see
https://github.com/tass-belgium/picotcp

AFAIK latest picotcp-enbled barebox anounced here:
http://lists.infradead.org/pipermail/barebox/2014-June/019723.html

Can be network device discovery realized using picotcp entities?

To Daniele: how things stand with picotcp tftp support?

Quoted 4 lines

_______________________________________________
barebox mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/barebox

--
--
Best regards,
Antony Pavlov

Quoted 18 lines · Antony Pavlov

Quoted 3 lines · On Sun, 13 Jul 2014 12:11:17 +0200 Colin Leitner…

Hello everyone,

I'm working on adding multicast support to barebox (for device discovery) and wanted to know if there's interest in having that included in upstream.

There is an initiative on integrating picotcp into barebox.

picotcp is a free TCP/IP stack implementation, see
https://github.com/tass-belgium/picotcp

AFAIK latest picotcp-enbled barebox anounced here:
http://lists.infradead.org/pipermail/barebox/2014-June/019723.html

Can be network device discovery realized using picotcp entities?

To Daniele: how things stand with picotcp tftp support?

TFTP is integrated in masterbranch, documented and tested. PicoTCP app can act as TFTP server as well as client. Supports put/get operations. Tested against tftp(d)-hpa.

barebox uses POSIX file interface for working with tftp.

We have to provide these basic functions (see fs/tftp.c):

static struct fs_driver_d tftp_driver = {

        .open      = tftp_open,
        .read      = tftp_read,
        .write     = tftp_write,
        .close     = tftp_close,

/* Other functions for unlink, truncate,
mkdir, rmdir, stat, opendir, lseek, create are not so important. */
};

So **fs/tftp.c code is driven** by user code that
uses file interface for accessing to the files
on tftp-server.

AFAI understand picotcp tftp support uses event-driven interface so **user code is driven** by picotcp tftp support code.

How we can use picotcp tftp support for realization of POSIX file interface?

Quoted 19 lines · Antony Pavlov

barebox uses POSIX file interface for working with tftp.

We have to provide these basic functions (see fs/tftp.c):

static struct fs_driver_d tftp_driver = {

        .open      = tftp_open,
        .read      = tftp_read,
        .write     = tftp_write,
        .close     = tftp_close,

/* Other functions for unlink, truncate,
mkdir, rmdir, stat, opendir, lseek, create are not so important. */
};

So **fs/tftp.c code is driven** by user code that
uses file interface for accessing to the files
on tftp-server.

Quoted 5 lines

AFAI understand picotcp tftp support uses event-driven interface so **user code is driven** by picotcp tftp support code.

How we can use picotcp tftp support for realization of POSIX file interface?

Hi Antony,

I will be able to provide such an interface by using a similar approach to what you used for ping (so via net_poll() routine called in a loop), assuming that your posix-like interface expects blocking calls for read/write operations.

The loop will also be interruptible via CTRL+C.

I would like to know more about this approach though, are you implementing client only functionality (get/put) or do you want to be able to listen for client "connections" as well? How is the connection initiated (I see there is a tftp_probe function that's not included in the API group)?

Thanks,

/d

Quoted 4 lines

I will be able to provide such an interface by using a similar approach to what you used for ping (so via net_poll() routine called in a loop), assuming that your posix-like interface expects blocking calls for read/write operations.

Alas! We can't use this approach for tftp because tftp is a FILESYSTEM in barebox.

Quoted 5 lines · Antony Pavlov

Quoted 4 lines

I will be able to provide such an interface by using a similar approach to what you used for ping (so via net_poll() routine called in a loop), assuming that your posix-like interface expects blocking calls for read/write operations.

Quoted 1 line

Alas! We can't use this approach for tftp because tftp is a FILESYSTEM in barebox.

Then again, I'd like to know if your FS implementation actually needs blocking call, and in case, where is the code supposed to block. Does barebox have some kind of support for multiple threads, or a default event loop where background operations can be added? Or are the FS calls non blocking?

Sorry for asking dumb questions, I am not a barebox developer and I am just trying to figure out what is your execution model. There certainly is a way to integrate my TFTP implementation as soon as I realize what is your model: as for instance we have blocking POSIX socket calls implemented with and without an OS infrastructure, and we are able to realize blocking calls on any systems, being baremetal or multithtreaded.

Thanks

/D

Quoted 9 lines · On Tue, 15 Jul 2014 12:57:21 +0200 Daniele Lacamera <[email protected]>…

Quoted 6 lines · Antony Pavlov

Quoted 4 lines

I will be able to provide such an interface by using a similar approach to what you used for ping (so via net_poll() routine called in a loop), assuming that your posix-like interface expects blocking calls for read/write operations.

Alas! We can't use this approach for tftp because tftp is a FILESYSTEM in barebox.

Quoted 5 lines

Then again, I'd like to know if your FS implementation actually needs blocking call, and in case, where is the code supposed to block. Does barebox have some kind of support for multiple threads, or a default event loop where background operations can be added? Or are the FS calls non blocking?

AFAIK barebox does not support threads.
Also all filesystem calls are blocking.

Tftp user code know nothing about network stuff. User code just use read and write for acessing file data, no matter which driver (ramdisk, SATA, MTD, I2C or network) is used for actual data transfer.

What would you need for "network discovery"? Currently picotcp
- mDNS (at the moment over v4 only)

Quoted 12 lines · Antony Pavlov

Quoted 9 lines · On Tue, 15 Jul 2014 12:57:21 +0200 Daniele Lacamera <[email protected]>…

Quoted 6 lines · Antony Pavlov

Quoted 4 lines

I will be able to provide such an interface by using a similar approach to what you used for ping (so via net_poll() routine called in a loop), assuming that your posix-like interface expects blocking calls for read/write operations.

Alas! We can't use this approach for tftp because tftp is a FILESYSTEM in barebox.

Quoted 9 lines

Quoted 5 lines

Then again, I'd like to know if your FS implementation actually needs blocking call, and in case, where is the code supposed to block. Does barebox have some kind of support for multiple threads, or a default event loop where background operations can be added? Or are the FS calls non blocking?

AFAIK barebox does not support threads.
Also all filesystem calls are blocking.

Then for me it is still not clear *where* a filesystem call is blocking. On a multi-threaded system, the calls is supposed to block somewhere in thread_sleep() functions. On a single thread execution model, we will need to tick the TCP/IP stack in the background while the call is "busy". Suppose that you issue a read call to any network filesystem (in this case TFTP issuing a get command). The file needs to be retreived by the TCP/IP stack, so I guess that the only implementation that makes sense would be something like:

while (callback_has_not_been_called) {
   if(ctrlc())
     break;
   pico_stack_tick();
   // maybe sleep here
}

What you stated earlier, i.e.

Quoted 3 lines

Tftp user code know nothing about network stuff. User code just use read and write for acessing file data, no matter which driver (ramdisk, SATA, MTD, I2C or network) is used for actual data transfer.

does not make much sense to me, as the network is required to access remote files, so the stack needs to tick constantly under the hood if you want to receive packets while your read call is blocked waiting for data.

User does not need to know that her read call is going through the network, but the TFTP fs module (or another underlying "driver") is supposed to access the picotcp API properly to retrieve the data needed when I/O operations are running. The intermediate layer (the new fs/tftp.c) is supposed to issue a stack tick whenever it is suspended waiting for network events.

Finally, I will assume that the use case needed is TFTP working in client mode, issuing GET/PUT commands upon open posix calls.

I can provide to you (and the list) an example implementation, which I will develop on top of your latest picotcp branch and test via sandbox/tuntap. In the meanwhile, any comment on the topic by barebox developers are more than welcome.

Quoted 6 lines · Antony Pavlov

Quoted 4 lines

I will be able to provide such an interface by using a similar approach to what you used for ping (so via net_poll() routine called in a loop), assuming that your posix-like interface expects blocking calls for read/write operations.

Alas! We can't use this approach for tftp because tftp is a FILESYSTEM in barebox.

Then again, I'd like to know if your FS implementation actually needs blocking call, and in case, where is the code supposed to block. Does barebox have some kind of support for multiple threads, or a default event loop where background operations can be added? Or are the FS calls non blocking?

Sorry for asking dumb questions, I am not a barebox developer and I am just trying to figure out what is your execution model. There certainly is a way to integrate my TFTP implementation as soon as I realize what is your model: as for instance we have blocking POSIX socket calls implemented with and without an OS infrastructure, and we are able to realize blocking calls on any systems, being baremetal or multithtreaded.

Right now the network users register to a udp port and provide a handler which is called whenever a packet to this port is received. The prototype for this function is:

struct net_connection *net_udp_new(IPaddr_t dest, uint16_t dport,

                rx_handler_f *handler, void *ctx);

Then network users can send packets on this connection:

int net_udp_send(struct net_connection *con, int len);

The function returns after the packet has been sent.

The network user has to keep the ball rolling by calling

void net_poll(void);

in a loop. This function will call into the network drivers receive function and dispatch the received packets. ARP packets are handled internally, the UDP packets are passed to the registered handlers.

The handlers usually will send answers to received packets (so a tftp client will send an ack here or request the next packet).

Usually the loop calling net_poll() also has some functionality to detect progress and will send the last packet again if it was lost.

Hope that explains the networking model in barebox.

Quoted 21 lines · Sascha Hauer

Right now the network users register to a udp port and provide a handler which is called whenever a packet to this port is received. The prototype for this function is:

struct net_connection *net_udp_new(IPaddr_t dest, uint16_t dport,

                rx_handler_f *handler, void *ctx);

Then network users can send packets on this connection:

int net_udp_send(struct net_connection *con, int len);

The function returns after the packet has been sent.

The network user has to keep the ball rolling by calling

void net_poll(void);

in a loop. This function will call into the network drivers receive function and dispatch the received packets. ARP packets are handled internally, the UDP packets are passed to the registered handlers.

Quoted 8 lines

The handlers usually will send answers to received packets (so a tftp client will send an ack here or request the next packet).

Usually the loop calling net_poll() also has some functionality to detect progress and will send the last packet again if it was lost.

Hope that explains the networking model in barebox.

Hi Sascha,

Thanks a lot for this clarification. The mechanism you described is the same as the native execution model of PicoTCP, and looking around in the code it seems that looping around net_poll() was in fact the way to go.

to Antony: I will improve TFTP first, by allowing multiple sessions at the same time. I will keep you posted on the progress.

Quoted 33 lines · On Wed, 16 Jul 2014 08:48:18 +0200 Daniele Lacamera <[email protected]>…

Quoted 21 lines · Sascha Hauer

Right now the network users register to a udp port and provide a handler which is called whenever a packet to this port is received. The prototype for this function is:

struct net_connection *net_udp_new(IPaddr_t dest, uint16_t dport,

                rx_handler_f *handler, void *ctx);

Then network users can send packets on this connection:

int net_udp_send(struct net_connection *con, int len);

The function returns after the packet has been sent.

The network user has to keep the ball rolling by calling

void net_poll(void);

in a loop. This function will call into the network drivers receive function and dispatch the received packets. ARP packets are handled internally, the UDP packets are passed to the registered handlers.

Quoted 8 lines

The handlers usually will send answers to received packets (so a tftp client will send an ack here or request the next packet).

Usually the loop calling net_poll() also has some functionality to detect progress and will send the last packet again if it was lost.

Hope that explains the networking model in barebox.

Hi Sascha,

Quoted 7 lines

Thanks a lot for this clarification. The mechanism you described is the same as the native execution model of PicoTCP, and looking around in the code it seems that looping around net_poll() was in fact the way to go.

to Antony: I will improve TFTP first, by allowing multiple sessions at the same time. I will keep you posted on the progress.

I see a barebox-related publication with nice pictures on the picotcp website :)

http://www.picotcp.com/barebox-on-top-of-picotcp

We are awaiting improved TFTP...

--
Best regards,
Antony Pavlov

I see a barebox-related publication with nice pictures on the picotcp website :)

http://www.picotcp.com/barebox-on-top-of-picotcp

We are awaiting improved TFTP...

Michele just pushed a TFTP which provides support for multiple sessions. API is slightly changed, to keep track of the sessions themselves. Please refer to the example in test/examples/tftp.c, or to the updated user manual.

Let us know if we can provide any support to help you progress with the integration.

1. Do we really need this '#ifdef PICO_SUPPORT_UDP' BEFORE (!) guard macro in pico_dhcp_client.h?

2. New pico_tftp_start_{r,t}x programming interface can be used for realization legacy standalone 'tftp' command.
But I have no idea how to use this interface for filesystem-over-tftp (it's the modern barebox' tftp realization)!

Barebox filesystem interface needs something like this programming interface:

struct pico_tftp_session *s;

  s = pico_tftp_new_session_rx(addr, port, family, filename);
  ...
  filesize = pico_tftp_get_file_size(s);
  ...
  pico_tftp_receive(s, buf, len);
  ...
  pico_tftp_receive(s, buf, len);
  ... ... ...
  pico_tftp_receive(s, buf, len);
  ...
  pico_tftp_close(s);

I mean that receiving process has to be controlled by barebox, not by picotcp. Because a new portion of data is received by filesystem user initiative, not by network stack initiative.

Quoted 4 lines · Antony Pavlov

1. Do we really need this '#ifdef PICO_SUPPORT_UDP' BEFORE (!) guard macro in pico_dhcp_client.h?

I moved these in latest PicoTCP commit. Thanks.

Quoted 23 lines

2. New pico_tftp_start_{r,t}x programming interface can be used for realization legacy standalone 'tftp' command.
But I have no idea how to use this interface for filesystem-over-tftp (it's the modern barebox' tftp realization)!

Barebox filesystem interface needs something like this programming interface:

struct pico_tftp_session *s;

  s = pico_tftp_new_session_rx(addr, port, family, filename);
  ...
  filesize = pico_tftp_get_file_size(s);
  ...
  pico_tftp_receive(s, buf, len);
  ...
  pico_tftp_receive(s, buf, len);
  ... ... ...
  pico_tftp_receive(s, buf, len);
  ...
  pico_tftp_close(s);

Quoted 3 lines

I mean that receiving process has to be controlled by barebox, not by picotcp. Because a new portion of data is received by filesystem user initiative, not by network stack initiative.

Thanks for the comments. A few from my side:

- pico_tftp_receive is non-blocking, and will return immediately if no data is available. You always have to be bound to some RX event to be sure that your next receive() call is successful, so in some way you still want it network-bound.

- We can implement the get_file_size, but not all servers will support it. Michele has a strategy for developing more advanced TFTP features, he will post on it later

regards,

Quoted 4 lines · Antony Pavlov

1. Do we really need this '#ifdef PICO_SUPPORT_UDP' BEFORE (!) guard macro in pico_dhcp_client.h?

I moved these in latest PicoTCP commit. Thanks.

Quoted 23 lines

2. New pico_tftp_start_{r,t}x programming interface can be used for realization legacy standalone 'tftp' command.
But I have no idea how to use this interface for filesystem-over-tftp (it's the modern barebox' tftp realization)!

Barebox filesystem interface needs something like this programming interface:

struct pico_tftp_session *s;

  s = pico_tftp_new_session_rx(addr, port, family, filename);
  ...
  filesize = pico_tftp_get_file_size(s);
  ...
  pico_tftp_receive(s, buf, len);
  ...
  pico_tftp_receive(s, buf, len);
  ... ... ...
  pico_tftp_receive(s, buf, len);
  ...
  pico_tftp_close(s);

Quoted 3 lines

I mean that receiving process has to be controlled by barebox, not by picotcp. Because a new portion of data is received by filesystem user initiative, not by network stack initiative.

Thanks for the comments. A few from my side:

- pico_tftp_receive is non-blocking, and will return immediately if no data is available. You always have to be bound to some RX event to be sure that your next receive() call is successful, so in some way you still want it network-bound.

- We can implement the get_file_size, but not all servers will support it. Michele has a strategy for developing more advanced TFTP features, he will post on it later