picotcp barebox integration

picotcp barebox integration

[Antony Pavlov]

Hi!

What is current picotcp barebox integration status?

AFAIR latest picotcp-barebox integration discussion thread is
http://lists.infradead.org/pipermail/barebox/2014-September/020732.html

-- 
Best regards,
  Antony Pavlov

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: picotcp barebox integration

[Daniele Lacamera]

Hi all,

Michele has been busy with another assignment lately, but he has also
been working on the features you requested. He will update you on the
status of his TFTP branch soon.

Thanks

/d

On Fri, Nov 21, 2014 at 3:25 PM, Antony Pavlov <antonynpavlov@gmail.com> wrote:
> Hi!
>
> What is current picotcp barebox integration status?
>
> AFAIR latest picotcp-barebox integration discussion thread is
> http://lists.infradead.org/pipermail/barebox/2014-September/020732.html
>
> --
> Best regards,
>   Antony Pavlov

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: picotcp barebox integration

[Michele Di Pede]

Hi all,
I'm been very busy but I plan to work on the TFPT during the coming holidays
to complete the it.

Regards

On Fri, Nov 28, 2014 at 8:38 PM, Daniele Lacamera
<daniele.lacamera@tass.be> wrote:
> Hi all,
>
> Michele has been busy with another assignment lately, but he has also
> been working on the features you requested. He will update you on the
> status of his TFTP branch soon.
>
> Thanks
>
> /d
>
> On Fri, Nov 21, 2014 at 3:25 PM, Antony Pavlov <antonynpavlov@gmail.com> wrote:
>> Hi!
>>
>> What is current picotcp barebox integration status?
>>
>> AFAIR latest picotcp-barebox integration discussion thread is
>> http://lists.infradead.org/pipermail/barebox/2014-September/020732.html
>>
>> --
>> Best regards,
>>   Antony Pavlov

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: picotcp barebox integration

[DI PEDE Michele]

Hi all,
HEAD of development branch contains the required API for a smooth integration 
with Barebox.
User manual contains all the information for use them. There is also a small 
demo TFTP client that show how to use the application driven interface 
test/test_tftp_app_client.c. To build it just do make clean; make test; cd 
test; make.
Now should be easy to integrate out stack in Barebox code.
Let me know how the integration effort proceed. We could provide proper 
support from our side to speed up things.

If you have any question please don't esitate to contact us

Best regards
Michele Di Pede

Friday 19 December 2014 18:28:30, Michele Di Pede wrote:
> Hi all,
> I'm been very busy but I plan to work on the TFPT during the coming holidays
> to complete the it.
> 
> Regards
> 
> On Fri, Nov 28, 2014 at 8:38 PM, Daniele Lacamera
> 
> <daniele.lacamera@tass.be> wrote:
> > Hi all,
> > 
> > Michele has been busy with another assignment lately, but he has also
> > been working on the features you requested. He will update you on the
> > status of his TFTP branch soon.
> > 
> > Thanks
> > 
> > /d
> > 
> > On Fri, Nov 21, 2014 at 3:25 PM, Antony Pavlov <antonynpavlov@gmail.com> 
wrote:
> >> Hi!
> >> 
> >> What is current picotcp barebox integration status?
> >> 
> >> AFAIR latest picotcp-barebox integration discussion thread is
> >> http://lists.infradead.org/pipermail/barebox/2014-September/020732.html
> >> 
> >> --
> >> Best regards,
> >> 
> >>   Antony Pavlov

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: picotcp barebox integration

[Antony Pavlov]

On Wed, 28 Jan 2015 14:35:19 +0100
DI PEDE Michele <michele.dipede@altran.com> wrote:

> Hi all,
> HEAD of development branch contains the required API for a smooth integration 
> with Barebox.
> User manual contains all the information for use them. There is also a small 
> demo TFTP client that show how to use the application driven interface 
> test/test_tftp_app_client.c. To build it just do make clean; make test; cd 
> test; make.
> Now should be easy to integrate out stack in Barebox code.
> Let me know how the integration effort proceed. We could provide proper 
> support from our side to speed up things.

I have tryed to integrate picotcp tftp support into barebox.
Latest picotcp development branch commit is

    commit 0a3b361848efe20259cb3ca5e79d73a95d8f476c
    Author: Daniele Lacamera <daniele.lacamera@tass.be>
    Date:   Wed Jan 28 13:55:23 2015 +0100
    
        Added link status call to the device driver interface
    
IMHO the tftp interface introduced in this picotcp development branch
can't be easely integrated into barebox tftp filesystem support code.

I'll illustrate this on sample barebox tftp file download session:

    barebox user setup network parameters and run 'tftp <filename>' command;
   
    the tftp commands mounts tftp-server rootdir to barebox "/.tftp_tmp_path";
    and start file copy operation (/.tftp_tmp_path/<filename> -> /<filename>).

    copy operation consists of standard calls (very schematically):

      srcfd = open(<source file>, O_RDONLY);
      ...
      dstfd = open(<destination file>, O_WRONLY | O_CREAT | O_TRUNC);
      ...
      stat(srcfd, ...); /* just determine source file size if possible */
      ...
      while (r = read(srcfd, buffer, 4096)) {
          write(dstfd, buffer, r);
          ...
      }

      so reading data block from tftp-server is made by initiative of barebox command.
      The barebox tftp support code realize all standard calls (open, stat, read, write ...).

      But let's examine picotcp tftp API. Picotcp tftp downloading file code consist of

           priv->session = pico_tftp_session_setup(&tpriv->server_ip4, PICO_PROTO_IPV4);
           ...
           pico_tftp_start_rx(priv->session, short_be(PICO_TFTP_PORT), filename, cb_tftp_rx_opt, priv);

       But pico_tftp_start_rx() tries to download WHOLE FILE from tftp-server.
       I see no means to suspend data transfer initiated by  pico_tftp_start_rx(),
       so I have no means to realize necessary read() and write() calls.

Sascha! Please comment this message.

> If you have any question please don't esitate to contact us
> 
> Best regards
> Michele Di Pede
> 
> Friday 19 December 2014 18:28:30, Michele Di Pede wrote:
> > Hi all,
> > I'm been very busy but I plan to work on the TFPT during the coming holidays
> > to complete the it.
> > 
> > Regards
> > 
> > On Fri, Nov 28, 2014 at 8:38 PM, Daniele Lacamera
> > 
> > <daniele.lacamera@tass.be> wrote:
> > > Hi all,
> > > 
> > > Michele has been busy with another assignment lately, but he has also
> > > been working on the features you requested. He will update you on the
> > > status of his TFTP branch soon.
> > > 
> > > Thanks
> > > 
> > > /d
> > > 
> > > On Fri, Nov 21, 2014 at 3:25 PM, Antony Pavlov <antonynpavlov@gmail.com> 
> wrote:
> > >> Hi!
> > >> 
> > >> What is current picotcp barebox integration status?
> > >> 
> > >> AFAIR latest picotcp-barebox integration discussion thread is
> > >> http://lists.infradead.org/pipermail/barebox/2014-September/020732.html
> > >> 
> > >> --
> > >> Best regards,
> > >> 
> > >>   Antony Pavlov


-- 
-- 
Best regards,
  Antony Pavlov

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: picotcp barebox integration

[DI PEDE Michele]

On Sat 28 Feb 2015 21:01:01
 Antony Pavlov <antonynpavlov@gmail.com> wrote:
> On Wed, 28 Jan 2015 14:35:19 +0100
> 
> DI PEDE Michele <michele.dipede@altran.com> wrote:
> > Hi all,
> > HEAD of development branch contains the required API for a smooth
> > integration with Barebox.
> > User manual contains all the information for use them. There is also a
> > small demo TFTP client that show how to use the application driven
> > interface test/test_tftp_app_client.c. To build it just do make clean;
> > make test; cd test; make.
> > Now should be easy to integrate out stack in Barebox code.
> > Let me know how the integration effort proceed. We could provide proper
> > support from our side to speed up things.
> 
> I have tryed to integrate picotcp tftp support into barebox.
Unfortunately I made a mistake in my commit (test/test_tftp_app_client.c was 
missing due to the wrong insertion of the binary executable); I do apologize.
It contains the call sequence you need for the integration in Barebox. Please 
take a look at test/test_tftp_app_client.c

[...]
> IMHO the tftp interface introduced in this picotcp development branch
> can't be easely integrated into barebox tftp filesystem support code.
Please take a look at the application driven API.

> 
> I'll illustrate this on sample barebox tftp file download session:
> 
>     barebox user setup network parameters and run 'tftp <filename>' command;
> 
>     the tftp commands mounts tftp-server rootdir to barebox
> "/.tftp_tmp_path"; and start file copy operation
> (/.tftp_tmp_path/<filename> -> /<filename>).
> 
>     copy operation consists of standard calls (very schematically):
> 
>       srcfd = open(<source file>, O_RDONLY);
>       ...
>       dstfd = open(<destination file>, O_WRONLY | O_CREAT | O_TRUNC);
>       ...
>       stat(srcfd, ...); /* just determine source file size if possible */
>       ...
>       while (r = read(srcfd, buffer, 4096)) {
>           write(dstfd, buffer, r);
>           ...
>       }
> 
>       so reading data block from tftp-server is made by initiative of
> barebox command. The barebox tftp support code realize all standard calls
> (open, stat, read, write ...).
> 
>       But let's examine picotcp tftp API. Picotcp tftp downloading file code
> consist of
> 
>            priv->session = pico_tftp_session_setup(&tpriv->server_ip4,
> PICO_PROTO_IPV4); ...
>            pico_tftp_start_rx(priv->session, short_be(PICO_TFTP_PORT),
> filename, cb_tftp_rx_opt, priv);
> 
>        But pico_tftp_start_rx() tries to download WHOLE FILE from
> tftp-server. I see no means to suspend data transfer initiated by 
> pico_tftp_start_rx(), so I have no means to realize necessary read() and
> write() calls.
You tried to use the wrong API.
In order to integrate our stack you should use the application driven API. It 
is documented in user manual in section 3.12 (TFTP). In order to receive a 
file the sequence should be:
session = pico_tftp_app_setup(&server_address, short_be(PICO_TFTP_PORT), 
PICO_PROTO_IPV4, &synchro);
ret = pico_tftp_app_start_rx(session, filename);
ret = pico_tftp_get_file_size(session, &file_size);
if (ret)
printf("Information about file size has not been received"\n);

while (file_is_not_complete) {
        len = pico_tftp_get(session, buf, PICO_TFTP_PAYLOAD_SIZE);
        ....
}

The example contained in test/test_tftp_app_client.c demonstrate how to use 
the interface for application driven behaviour.

Best regards
Michele Di Pede


_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: picotcp barebox integration

[Antony Pavlov]

On Sun, 1 Mar 2015 01:29:56 +0100
DI PEDE Michele <michele.dipede@altran.com> wrote:

> On Sat 28 Feb 2015 21:01:01
...
> Please take a look at the application driven API.

> In order to integrate our stack you should use the application driven API. It 
> is documented in user manual in section 3.12 (TFTP). In order to receive a 
> file the sequence should be:
> session = pico_tftp_app_setup(&server_address, short_be(PICO_TFTP_PORT), 
> PICO_PROTO_IPV4, &synchro);
> ret = pico_tftp_app_start_rx(session, filename);
> ret = pico_tftp_get_file_size(session, &file_size);
> if (ret)
> printf("Information about file size has not been received"\n);
> 
> while (file_is_not_complete) {
>         len = pico_tftp_get(session, buf, PICO_TFTP_PAYLOAD_SIZE);
>         ....
> }
> 
> The example contained in test/test_tftp_app_client.c demonstrate how to use 
> the interface for application driven behaviour.

Hmm. I have tried pico_tftp_get() and now I have several questions:

1. pico_tftp_get() has 3rd argument 'int32_t len'.
This 'len' argument is actually unused. Can we just drop it?


2. here is pico_tftp_get() example fragment:

        len = pico_tftp_get(session, buf, PICO_TFTP_PAYLOAD_SIZE);
        if (len < 0) {
            fprintf(stderr, "Failure in pico_tftp_get\n");
            close(fd);
            countdown = 1;
            continue;
        }
        ret = write(fd, buf, len);

pico_tftp_get() copies data to pointer buf. But the first four bytes are not raw file
data but tftp header fields.
E.g. I receiving GPL text via pico_tftp_get():

first tftp data block:

00000000: 00 03 00 01 09 09 20 20 20 20 47 4e 55 20 47 45    ......    GNU GE
00000010: 4e 45 52 41 4c 20 50 55 42 4c 49 43 20 4c 49 43    NERAL PUBLIC LIC
00000020: 45 4e 53 45 0a 09 09 20 20 20 20 20 20 20 56 65    ENSE...       Ve
...
000001d0: 65 72 61 6c 20 50 75 62 6c 69 63 0a 4c 69 63 65    eral Public.Lice
000001e0: 6e 73 65 20 69 73 20 69 6e 74 65 6e 64 65 64 20    nse is intended
000001f0: 74 6f 20 67 75 61 72 61 6e 74 65 65                to guarantee

second tftp data block:

00000000: 00 03 00 02 65 65 64 6f 6d 20 74 6f 20 73 68 61    ....eedom to sha
00000010: 72 65 20 61 6e 64 20 63 68 61 6e 67 65 20 66 72    re and change fr
00000020: 65 65 0a 73 6f 66 74 77 61 72 65 2d 2d 74 6f 20    ee.software--to
00000030: 6d 61 6b 65 20 73 75 72 65 20 74 68 65 20 73 6f    make sure the so
00000040: 66 74 77 61 72 65 20 69 73 20 66 72 65 65 20 66    ftware is free f

Here is a original GPL file dump, you can see that substring ' your fr'
is missed in data received by pico_tftp_get():

000001d0:  20 50 75 62 6c 69 63 0a  4c 69 63 65 6e 73 65 20   Public.License 
000001e0:  69 73 20 69 6e 74 65 6e  64 65 64 20 74 6f 20 67  is intended to g
000001f0:  75 61 72 61 6e 74 65 65  20 79 6f 75 72 20 66 72  uarantee your fr <<<<<<
00000200:  65 65 64 6f 6d 20 74 6f  20 73 68 61 72 65 20 61  eedom to share a <<<<<<
00000210:  6e 64 20 63 68 61 6e 67  65 20 66 72 65 65 0a 73  nd change free.s
00000220:  6f 66 74 77 61 72 65 2d  2d 74 6f 20 6d 61 6b 65  oftware--to make
   
So data block written to file by
    ret = write(fd, buf, len);
is corrupted.

Could you please check this?


3. there is the find_session_by_socket() function. It uses special tftp_sessions list.
Can we simply use priv field in 'struct pico_socket' for this purpose?


4. How I can determine "End Of File reached" situation after pico_tftp_get() call?


-- 
Best regards,
  Antony Pavlov

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

RE: picotcp barebox integration

[DI PEDE Michele]

On Sun 1 Mar 2015 16:54:25,
Antony Pavlov <antonynpavlov@gmail.com>  wrote:
> On Sun, 1 Mar 2015 01:29:56 +0100
> 
> DI PEDE Michele <michele.dipede@altran.com> wrote:
> > On Sat 28 Feb 2015 21:01:01
> 
> ...
> 
> > Please take a look at the application driven API.
> > 
> > In order to integrate our stack you should use the application driven API.
> > It is documented in user manual in section 3.12 (TFTP). In order to
> > receive a file the sequence should be:
> > session = pico_tftp_app_setup(&server_address, short_be(PICO_TFTP_PORT),
> > PICO_PROTO_IPV4, &synchro);
> > ret = pico_tftp_app_start_rx(session, filename);
> > ret = pico_tftp_get_file_size(session, &file_size);
> > if (ret)
> > printf("Information about file size has not been received"\n);
> > 
> > while (file_is_not_complete) {
> > 
> >         len = pico_tftp_get(session, buf, PICO_TFTP_PAYLOAD_SIZE);
> >         ....
> > 
> > }
> > 
> > The example contained in test/test_tftp_app_client.c demonstrate how to
> > use
> > the interface for application driven behaviour.
> 
> Hmm. I have tried pico_tftp_get() and now I have several questions:
> 
> 1. pico_tftp_get() has 3rd argument 'int32_t len'.
> This 'len' argument is actually unused. Can we just drop it?
No, current payload size is fixed to 512 as stated in TFTP specification but we plan to add also RFC 2348 to our features. Code to handle it is almost there, I need just some time to clean it up and perform validation test. We will release it soon.

> 
> 2. here is pico_tftp_get() example fragment:
> 
>         len = pico_tftp_get(session, buf, PICO_TFTP_PAYLOAD_SIZE);
>         if (len < 0) {
>             fprintf(stderr, "Failure in pico_tftp_get\n");
>             close(fd);
>             countdown = 1;
>             continue;
>         }
>         ret = write(fd, buf, len);
> 
> pico_tftp_get() copies data to pointer buf. But the first four bytes are not
> raw file data but tftp header fields.
> E.g. I receiving GPL text via pico_tftp_get():
> 
> first tftp data block:
> 
> 00000000: 00 03 00 01 09 09 20 20 20 20 47 4e 55 20 47 45    ......    GNU
> GE 00000010: 4e 45 52 41 4c 20 50 55 42 4c 49 43 20 4c 49 43    NERAL
> PUBLIC LIC 00000020: 45 4e 53 45 0a 09 09 20 20 20 20 20 20 20 56 65   
> ENSE...       Ve ...
> 000001d0: 65 72 61 6c 20 50 75 62 6c 69 63 0a 4c 69 63 65    eral
> Public.Lice 000001e0: 6e 73 65 20 69 73 20 69 6e 74 65 6e 64 65 64 20   
> nse is intended 000001f0: 74 6f 20 67 75 61 72 61 6e 74 65 65              
>  to guarantee
> 
> second tftp data block:
> 
> 00000000: 00 03 00 02 65 65 64 6f 6d 20 74 6f 20 73 68 61    ....eedom to
> sha 00000010: 72 65 20 61 6e 64 20 63 68 61 6e 67 65 20 66 72    re and
> change fr 00000020: 65 65 0a 73 6f 66 74 77 61 72 65 2d 2d 74 6f 20   
> ee.software--to 00000030: 6d 61 6b 65 20 73 75 72 65 20 74 68 65 20 73 6f  
>  make sure the so 00000040: 66 74 77 61 72 65 20 69 73 20 66 72 65 65 20 66
>    ftware is free f
> 
> Here is a original GPL file dump, you can see that substring ' your fr'
> is missed in data received by pico_tftp_get():
> 
> 000001d0:  20 50 75 62 6c 69 63 0a  4c 69 63 65 6e 73 65 20   Public.License
> 000001e0:  69 73 20 69 6e 74 65 6e  64 65 64 20 74 6f 20 67  is intended to
> g 000001f0:  75 61 72 61 6e 74 65 65  20 79 6f 75 72 20 66 72  uarantee
> your fr <<<<<< 00000200:  65 65 64 6f 6d 20 74 6f  20 73 68 61 72 65 20 61 
> eedom to share a <<<<<< 00000210:  6e 64 20 63 68 61 6e 67  65 20 66 72 65
> 65 0a 73  nd change free.s 00000220:  6f 66 74 77 61 72 65 2d  2d 74 6f 20
> 6d 61 6b 65  oftware--to make
> 
> So data block written to file by
>     ret = write(fd, buf, len);
> is corrupted.
> 
> Could you please check this?
This is a BUG. Unfortunately pico_tftp_get returns the raw data but it is expected to returns only payload data. Thanks for signalling it. I've fixed the bug in the development branch. The commit is

commit 82fac1df278ab98cf0c46644de0fd608e341e5dd
Author: Michele Di Pede <michele.di.pede@tass.be>
Date:   Sun Mar 1 16:28:51 2015 +0100

    pico_tftp_get now return only payload data (issue #221)

> 
> 3. there is the find_session_by_socket() function. It uses special
> tftp_sessions list. Can we simply use priv field in 'struct pico_socket'
> for this purpose?
> 
> 
> 4. How I can determine "End Of File reached" situation after pico_tftp_get()
> call?
On receiver side you detect the End Of File reached when the size of the received packed less than the PACKET SIZE (it could be even 0). For this reason when you are the sender you have to send an extra packet with a len of 0 for payload (TFTP protocol rules).
Section 3.12.16 of the user manual show how to detect the EOF condition.
Currently this rule is explained only in the general description of TFTP (intro of 3.12) and in the example code. I'll make the description of TFTP a bit more verbose.

The lines that follow the snippet you showed above (extracted by test_tftp_app_client.c) are:

    if (ret < 0) {
        fprintf(stderr, "Error in write\n");
        pico_tftp_abort(session, TFTP_ERR_EXCEEDED, "File write error");
        close(fd);
        countdown = 1;
        continue;
    }
    printf("Written %" PRId32 " bytes to file (synchro=%d)\n", len, *synchro);

    if (len != PICO_TFTP_PAYLOAD_SIZE) {
        close(fd);
        printf("Transfer complete!\n");
        countdown = 1;
    }

First if block show how to signal an error to the remote side. Last if block how to recognize that the file transfer has been completed.

> 
> -- 
> Best regards,
>   Antony Pavlov

Best regards,
Michele Di Pede
_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: picotcp barebox integration

[Antony Pavlov]

Hi!

I have just published latest picotcp-enabled barebox.
Please see my picotcp.20150331 branch in my github barebox repo
(https://github.com/frantony/barebox/tree/picotcp.20150331).

This version is based on latest barebox-next and latest picotcp-development.
Here is barebox picotcp network features:

       * network interfaces and the 'ifconfig' command to rule them;
       * the 'route' command to see current picotcp route table;
       * the 'picoping' command to use picotcp ping support (also now barebox responds
         to ping requests);
       * the 'dhclient' command to use picotcp dhcp client;
(new)  * picotcp-based tftp filesystem support.


Here is note/todo list:

  1. tftp write operation is not supported;

  2. with picotcp we can't easely use 'tftp' command with old syntax
     (no $<current ethernet>.serverip anymore);

  3. just now picotcp-based tftp file transfer is 5 times slower than
     old realisation (partialy it is my bad, e.g. extra memory copy operation
     is used for code simplification);

  4. picotcp has dns support, but just now it is unused in barebox;
 
  5. no nfs support in picotcp (should we can keep current barebox nfs support on
     top of barebox UDP support?).

-- 
Best regards,
  Antony Pavlov

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox