Question about RATP protocol limitation during handshake in barebox

Question about RATP protocol limitation during handshake in barebox

[Jules Maselbas]

Hi,

I am facing some issues with the RATP protocol that has been ironically
unreliable lately. I've looked at the implementation in barebox and in
ratp-barebox-cli host tool. I think I am starting to understand the last
issue I am facing: sometimes the host tool fails to initialize the RATP
link.

    host                                             barebox
 1. CLOSED                                           LISTEN
 2. SYN_SENT    --> <CTL=SYN>                    --> SYN_RECEIVED
 3.             ...             <CTL=SYN,ACK,AN> <--
 4. ESTABLISHED <-- <CTL=SYN,ACK,AN>             ...     
 5.             --> <CTL=ACK,AN,SN>              ...
 6.             ...             <CTL=SYN,ACK,AN> <-- (retransmit)
 7.             ..               <CTL=ACK,AN,SN> --> ESTABLISHED
 8. CLOSED (C2) <-- <CTL=SYN,ACK,AN>             ...
 9.             --> <CTL=AN,SN,RST>              --> (connection reset)

The Figure above illustrate the issue: during the handshake barebox has
a chance to resend the SYN-ACK packet however, upon reception, the host
will move into the ESTABLISHED state after the first packet and respond
with an ACK packet. When the host receive the duplicated SYN-ACK packet
it will execute behavior C2: "we assume that the other end crashed and
has attempted to open a new connection"; and close the connection.

I think it is important to note that I have locally applied the patch:
`[RFC PATCH] ratp: Fix retransmission for header-only packets`
https://lore.barebox.org/barebox/20230123102752.15444-1-jmaselbas@kalray.eu/T/#u
This issue might be revealed by the patch above, because now every
packets can be resent, but I think this issue is in the very nature of
RATP protocol and I cannot think of a good/simple way of fixing this.

I am tempted to simply tune the RTO (retransmit time out) of barebox to
be less aggressive during the handshake.

I would like to know your insight on this :)

Thanks
-- Jules


--- debug log from ratp-barebox-cli on host:
425089.225208 [ratp 000000000000000] debug: initialization requested
425089.225265 [ratp 000000000000000] debug: /dev/ttyACM0: opening tty
425089.225801 [ratp 000000000000000] debug: /dev/ttyACM0: tty open
425089.226008 [ratp 000000000000000] debug: initialization done
425089.226035 [ratp 000000000000000] debug: link active open requested...
425089.226045 [ratp 000000000000000] debug: sending SYN...
425089.226065 [ratp 000000000000000] debug: raw_send 01:80:ff:80
425089.226106 [ratp 000000000000000] debug: message sent    : [-- --- -- -- --- --- --- syn] len: 255: 01:80:ff:80
425089.226130 [ratp 000000000000000] info : state update: closed -> syn-sent (ok)
425089.226251 [ratp 140419952396032] debug: private thread started
425089.376918 [ratp 140419952396032] debug: message received: [-- --- an -- --- --- ack syn] len: 255: 01:c4:ff:3c
425089.376950 [ratp 140419952396032] debug: [behavior B]
425089.376961 [ratp 140419952396032] debug: stop_retransmission_logic
425089.376975 [ratp 140419952396032] debug: last packet acknowledged in 150ms (SRTT: 190ms, RTO: 285ms)
425089.376995 [ratp 140419952396032] info : state update: syn-sent -> established (ok)
425089.377034 [ratp 140419952396032] debug: raw_send 01:4c:00:b3
425089.377073 [ratp 140419952396032] debug: message sent ack: [-- --- an sn --- --- ack ---] len: 0  : 01:4c:00:b3
425089.377115 [ratp 140419952396032] debug: message received: [-- --- an -- --- --- ack syn] len: 255: 01:c4:ff:3c
Sending PING...
425089.377129 [ratp 140419952396032] debug: [behavior C2]
425089.377269 [ratp 140419952396032] debug: raw_send 01:1c:00:e3
425089.377307 [ratp 140419952396032] debug: message sent    : [-- --- an sn rst --- --- ---] len: 0  : 01:1c:00:e3
425089.377323 [ratp 140419952396032] debug: stop_retransmission_logic
425089.377344 [ratp 140419952396032] info : state update: established -> closed (connection reset)
error: couldn't send PING: invalid transition
425089.377462 [ratp 000000000000000] debug: shutdown requested
425089.377567 [ratp 140419952396032] debug: private thread finished
425089.378662 [ratp 000000000000000] debug: [/dev/ttyACM0] closed tty
425089.378708 [ratp 000000000000000] debug: shutdown requested

--- debug log from barebox:
ratp: Establish...
ratp:                       recv>-- --- -- -- --- --- --- syn< len: 255
ratp:  state LISTEN
ratp: ratp_behaviour_a
ratp: send>-- --- an -- --- --- ack syn< len: 255
ratp: state LISTEN     -> SYN_RECEIVED
ratp: ratp_poll: retransmit
ratp: resend>-- --- an -- --- --- ack syn< len: 255
ratp:                       recv>-- --- an sn --- --- ack ---< len: 0
ratp:  state SYN_RECEIVED
ratp: ratp_behaviour_c1
ratp: ratp_behaviour_d1
ratp: ratp_behaviour_e
ratp: ratp_behaviour_f1
ratp: ratp_behaviour_h1
ratp: state SYN_RECEIVED -> ESTABLISHED
ratp:                       recv>-- --- an sn rst --- --- ---< len: 0
ratp:  state ESTABLISHED
ratp: ratp_behaviour_c2
ratp: ratp_behaviour_d2
ratp: connection reset
ratp: ratp_behaviour_e
ratp: ratp_behaviour_f2
ratp: send>-- eor an sn --- --- ack ---< len: 5

Re: Re: Question about RATP protocol limitation during handshake in barebox

[John Watts]

Hi,

I had a read of the RATP spec at https://www.rfc-editor.org/rfc/rfc916

The goal is to detect one end trying to start a fresh connection while another
is established but manages to be a bit too broad by specifying that any SYN
is an error rather than a SYN without an ACK.

There seems to be two solutions here:

- Don't re-send SYN packets
- Break the spec and check if ACK is set before terminating the connection

John.

Re: Re: Question about RATP protocol limitation during handshake in barebox

[Jules Maselbas]

Hi John,

On Wed, Jan 25, 2023 at 08:59:49AM +1100, John Watts wrote:
> Hi,
> 
> I had a read of the RATP spec at https://www.rfc-editor.org/rfc/rfc916
> 
> The goal is to detect one end trying to start a fresh connection while another
> is established but manages to be a bit too broad by specifying that any SYN
> is an error rather than a SYN without an ACK.
> 
> There seems to be two solutions here:
> 
> - Don't re-send SYN packets
> - Break the spec and check if ACK is set before terminating the connection
Yes, i think that's the best thing to do, to ingnore SYN-ACK packets,
new connections should only start with a SYN.

Thanks
-- Jules 

Draft of an errata for RFC916

[Jules Maselbas]

Hi, 

Here is a revised version of my previous mail, I might propose this
as an errata for RFC916, any comments and insights would be greatly
appreciated.


Side A                                             Side B
1. CLOSED                                          LISTEN
2. [OPEN request]
   SYN_SENT    -->  <SN=0><CTL=SYN>            --> SYN_RECEIVED
3. ESTABLISHED <--  <SN=0><AN=1><CTL=SYN,ACK>  <--
4.             -->  <SN=1><AN=0><CTL=ACK>      ...
5.             ...  <SN=0><AN=1><CTL=SYN,ACK>  <-- (retransmit)
6. (packet sent by A at 4. finally arrives to B)
               ...                             --> ESTABLISHED
7. (packet resent by B at 5. finally arrives to A)
   CLOSED (C2) <--                             ...
8.             -->  <SN=1><AN=1><CTL=RST>      --> (connection reset)

The Figure above illustrate the current issue RATP can face during the
three-way handshake, and how behavior C2 can cause a connection to be
closed immediately after being established.

In the Figure above, side A try to establish a connection with side B,
which is in the LISTEN state.  Commented line:
 4. side A receive the SYN-ACK packet from B and respond an ACK packet
    and move to the ESTABLISHED state.  Meanwhile;
 5. side B hasn't received yet the ACK from side A and decide to
    retransmit the SYN-ACK packet;
 6. side B finally receive the ACK from side A and move to the
    ESTABLISHED state;
 7. side A finally receive the duplicated SYN-ACK packet from side B,
    execute behavior C2: the received packet doesn't have the expected
    SN and has the SYN flag set, thus respond by sending a legal reset.
 8. side B receive the reset and close the connection.

One solution could be to tweak the initial RTO value of side B in order
to prevent sending a duplicated SYN-ACK packet, however the initial RTT
value is likely inaccurate during the handshake.  This solution seems a
bit brittle.

The second solution would be to consider that a host has crashed only if
the packet received has the SYN flag set but not the ACK flag.  The
rational is that the first step during handshake is to send a packet
only containing the SYN flag, however a packet containing both ACK and
SYN flags must have come after the initial handshake exchange and can
be considered as a duplicated and be dropped.

I propose the following errata to the RFC916:
[Page 29]
- If SYN was set we assume that the other end crashed and has
- attempted to open a new connection.  We respond by sending a
- legal reset:
+ If the SYN flag was set but the ACK was not set then we assume
+ that the other end crashed and has attempted to open a new connection.
+ We respond by sending a legal reset:

[Page 30]
- If neither RST, FIN, nor SYN flags were set it is assumed that
- this packet is a duplicate of one already received.  Send an
- ACK back:
+ If neither RST nor FIN flags were set, or if SYN and ACK flags
+ were set, it is assumed that this packet is a duplicate of one
+ already received.  Send an ACK back:

Thanks
-- Jules

Re: Re: Draft of an errata for RFC916

[John Watts]

Hi Jules,

The looks good to me. Did you implement this in Barebox and test it?

John.

Re: Re: Draft of an errata for RFC916

[Jules Maselbas]

Hi John

I've not implemented this in Barebox but in ratp-barebox-cli, which is
the tool i use to comunicate with barebox, and it is the one initiating
the connection. However I also tweaked a few timeouts for barebox to
properly work. I've not sent these tweaks yet.

Thanks,
-- Jules