QUIC QUIC has numerous extension points, including the version number that occupies the second through fifth bytes of every long header <del> (see RFC8999).  If experimental versions are rare, and QUIC version 1 constitutes the vast majority of QUIC traffic, there is the potential for middleboxes to ossify on the version bytes always being </del>  0x00000001. <del> Furthermore, version 1 Initial packets are encrypted with keys derived from a universally known salt, which allow observers to inspect the contents of these packets, which include the TLS Client Hello and Server Hello messages.  Again, middleboxes may ossify on </del>  the version 1 key derivation and packet formats. Finally QUIC-VN provides two mechanisms for endpoints to negotiate

, , , , , .
Just a style thing (and something we should try an capture in a QUIC style guide).
There a bit of inconsistency through the document. Personally I prefer the QUIC version N style.
is there a missing word here?
Nope, extra word, thanks
I find this awkward to read in isolation, it requires converting the RFC 9000 types definition format into a different encoding to realize there is a change. A clearer alternative could be "All version 2 Long Packet Types are different: Initial: 0b01 0-RTT: 0b10 Handshake 0b11 Retry: 0b00
Starting with This might be a matter of style but to me the term "changes" imply that you're modifying the original, not creating something new. How about we term these as differences instead? The the opening paragraph of Section 3 could be something like "QUIC version 2 endpoints MUST implement the QUIC version 1 specification as described in ], ], and ]. However, the following differences apply in version 2."
This might have been important earlier in the document lifetime but it seems to stick out now.
This could be misread to imply the salt is universally known and universally applicable, which I don't think is your intention. In fact, I find the construction of this entire paragraph a bit awkward. Can we rephrase this for better clarity? Something like "In QUIC version 1, Initial packets are encrypted with the version-specific salt as described in Section 5.2 of [QUIC-TLS]. Protecting Initial packets in this way allows observers to inspect their contents, which includes the TLS Clent Hello or Server Hello messages. Again, there is the potential for middleboxes to ossify on the 1 key derivation and packet formats."
Couple of nits but otherwise LGTM

QUIC QUIC has numerous extension points, including the version number that occupies the second through fifth bytes of every long header  <ins> (see QUIC-INVARIANTS).  If experimental versions are rare, and QUIC version 1 constitutes the vast majority of QUIC traffic, there is the potential for middleboxes to ossify on the version bytes always being </ins> 0x00000001.  <ins> In QUIC version 1, Initial packets are encrypted with the version- specific salt as described in Section 5.2 of QUIC-TLS.  Protecting Initial packets in this way allows observers to inspect their contents, which includes the TLS Client Hello or Server Hello messages.  Again, there is the potential for middleboxes to ossify on </ins> the version 1 key derivation and packet formats. Finally QUIC-VN provides two mechanisms for endpoints to negotiate


to implement version negotiation to protect against downgrade attacks. <del> I-D.duke-quic-version-aliasing is a more robust, but much more complicated, proposal to address these ossification problems.  By design, it requires incompatible version negotiation.  QUICv2 enables exercise of compatible version negotiation mechanism. </del> 2. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",

to implement version negotiation to protect against downgrade attacks.  2. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",


3. QUIC version 2 endpoints MUST implement the QUIC version 1 <del> specification as described in QUIC, QUIC-TLS, and RFC9002, with the following changes. </del>  3.1. <del> The version field of long headers is 0x709a50c4. </del>  3.2. <del> Initial packets use a packet type field of 0b01. 0-RTT packets use a packet type field of 0b10.  Handshake packets use a packet type field of 0b11.  Retry packets use a packet type field of 0b00. </del>  3.3.

3. QUIC version 2 endpoints MUST implement the QUIC version 1  <ins> specification as described in QUIC, QUIC-TLS, and RFC9002.  However, the following differences apply in version 2. </ins> 3.1.  <ins> The Version field of long headers is 0x709a50c4. </ins> 3.2.  <ins> All version 2 long header packet types are different.  The Type field values are: Initial: 0b01 0-RTT: 0b10 Handshake: 0b11 Retry: 0b00 </ins> 3.3.


TLS session tickets and NEW_TOKEN tokens are specific to the QUIC version of the connection that provided them.  Clients SHOULD NOT use <del> a session ticket or token from a QUICv1 connection to initiate a QUICv2 connection, or vice versa. </del>  Servers MUST validate the originating version of any session ticket or token and not accept one issued from a different version.  A

TLS session tickets and NEW_TOKEN tokens are specific to the QUIC version of the connection that provided them.  Clients SHOULD NOT use  <ins> a session ticket or token from a QUIC version 1 connection to initiate a QUIC version 2 connection, or vice versa. </ins> Servers MUST validate the originating version of any session ticket or token and not accept one issued from a different version.  A


6. QUIC version 2 provides protection against some forms of <del> ossification.  Devices that assume that all long headers will contain encode version 1, or that the version 1 Initial key derivation formula will remain version-invariant, will not correctly process version 2 packets. </del>  However, many middleboxes such as firewalls focus on the first packet in a connection, which will often remain in the version 1 format due

6. QUIC version 2 provides protection against some forms of  <ins> ossification.  Devices that assume that all long headers will encode version 1, or that the version 1 Initial key derivation formula will remain version-invariant, will not correctly process version 2 packets. </ins> However, many middleboxes such as firewalls focus on the first packet in a connection, which will often remain in the version 1 format due


This version of QUIC provides no change from QUIC version 1 relating to the capabilities available to applications.  Therefore, all Application Layer Protocol Negotiation (ALPN) (RFC7301) codepoints <del> specified to operate over QUICv1 can also operate over this version of QUIC.  In particular, both the "h3" I-D.ietf-quic-http and "doq" I-D.ietf-dprive-dnsoquic ALPNs can operate over QUICv2. </del>  All QUIC extensions defined to work with version 1 also work with version 2.

This version of QUIC provides no change from QUIC version 1 relating to the capabilities available to applications.  Therefore, all Application Layer Protocol Negotiation (ALPN) (RFC7301) codepoints  <ins> specified to operate over QUIC version 1 can also operate over this version of QUIC.  In particular, both the "h3" I-D.ietf-quic-http and "doq" I-D.ietf-dprive-dnsoquic ALPNs can operate over QUIC version 2. </ins> All QUIC extensions defined to work with version 1 also work with version 2.


version of QUIC.  In particular, both the "h3" I-D.ietf-quic-http and "doq" I-D.ietf-dprive-dnsoquic ALPNs can operate over QUIC version 2. <del> All QUIC extensions defined to work with version 1 also work with version 2. </del>  8.

Though this would likely be monstrously inadvisable, I am fairly confident that I could construct an interoperable (or at least deployable) extension to QUIC that works in QUICv1 but not QUICv2 or vice versa. It's OK to establish expectations, but some softening of the statement might help.
This seems very accurate

version of QUIC.  In particular, both the "h3" I-D.ietf-quic-http and "doq" I-D.ietf-dprive-dnsoquic ALPNs can operate over QUIC version 2.  <ins> Unless otherwise stated, all QUIC extensions defined to work with version 1 also work with version 2. </ins> 8.


1. <del> QUIC QUIC has numerous extension points, including the version number that occupies the second through fifth bytes of every long header (see QUIC-INVARIANTS).  If experimental versions are rare, and QUIC version 1 constitutes the vast majority of QUIC traffic, there is the potential for middleboxes to ossify on the version bytes always being 0x00000001. </del>  In QUIC version 1, Initial packets are encrypted with the version- specific salt as described in QUIC-TLS.  Protecting Initial packets

here (URL) it is better that we call it QUIC version 1 not just QUIC. As later it is referred as QUIC version 1.here (URL) I would add reference to VN draft instead of calling it "that document"here (URL) i would add "packet" after "Initial" just to be clear, as we have now a initial version in VN draft.
Thanks, fixed here URL

1.  <ins> QUIC version 1QUIC has numerous extension points, including the version number that occupies the second through fifth bytes of every long header (see QUIC-INVARIANTS).  If experimental versions are rare, and QUIC version 1 constitutes the vast majority of QUIC traffic, there is the potential for middleboxes to ossify on the version bytes always being 0x00000001. </ins> In QUIC version 1, Initial packets are encrypted with the version- specific salt as described in QUIC-TLS.  Protecting Initial packets


and validate the version_information transport parameter specified in QUIC-VN to prevent version downgrade attacks. <del> Note that version 2 meets that document's definition of a compatible </del>  version with version 1, and version 1 is compatible with version 2. Therefore, servers can use compatible negotiation to switch a connection between the two versions.  Endpoints that support both

and validate the version_information transport parameter specified in QUIC-VN to prevent version downgrade attacks.  <ins> Note that version 2 meets the QUIC-VN definition of a compatible </ins> version with version 1, and version 1 is compatible with version 2. Therefore, servers can use compatible negotiation to switch a connection between the two versions.  Endpoints that support both


If the server sends a Retry packet, it MUST use the original version. The client ignores Retry packets using other versions.  The client <del> MUST NOT use a different version in the subsequent Initial that contains the Retry token.  The server MAY encode the QUIC version in its Retry token to validate that the client did not switch versions, and drop the packet if it switched. </del>  QUIC version 2 uses the same transport parameters to authenticate the Retry as QUIC version 1.  After switching to a negotiated version

If the server sends a Retry packet, it MUST use the original version. The client ignores Retry packets using other versions.  The client  <ins> MUST NOT use a different version in the subsequent Initial packet that contains the Retry token.  The server MAY encode the QUIC version in its Retry token to validate that the client did not switch versions, and drop the packet if it switched. </ins> QUIC version 2 uses the same transport parameters to authenticate the Retry as QUIC version 1.  After switching to a negotiated version


3.1. <del> The Version field of long headers is 0x709a50c4.  This randomly chosen value is meant to avoid creating patterns in specified version numbers. </del>  This version number will not change in subsequent versions of this draft, unless there is a behavior change that breaks compatibility.

As requested, all new version number, salts, etc. for QUICv2. Verification of the test vectors would be especially valuable.
Woof. Thank you very much for quickly checking, and sorry I botched every part of that. All fixed now, I hope.
The new test vectors works well with my QUIC implementation in Haskell.
Thanks Kazu!
WFM. Not validated with a real build, but a script. See URL for the script that I used.

3.1.  <ins> The Version field of long headers is 0x6b3343cf.  This was generated by taking the first four bytes of the sha256sum of "QUICv2 version number". </ins> This version number will not change in subsequent versions of this draft, unless there is a behavior change that breaks compatibility.


The salt used to derive Initial keys in QUIC-TLS changes to:  3.3.2. The labels used in QUIC-TLS to derive packet protection keys

The salt used to derive Initial keys in QUIC-TLS changes to: <ins> This is the first 20 bytes of the sha256sum of "QUICv2 salt". </ins> 3.3.2. The labels used in QUIC-TLS to derive packet protection keys


The key and nonce used for the Retry Integrity Tag (QUIC-TLS) change to:  4. QUIC version 2 is not intended to deprecate version 1.  Endpoints

The key and nonce used for the Retry Integrity Tag (QUIC-TLS) change to: <ins> The secret is the sha256sum of "QUICv2 retry secret".  The key and nonce are derived from this secret with the labels "quicv2 key" and "quicv2 iv", respectively. </ins> 4. QUIC version 2 is not intended to deprecate version 1.  Endpoints


2.1.2.  An encoder MUST NOT insert an entry into the dynamic table (or <del> duplicate an existing entry) if doing so would evict an entry with unacknowledged references.  For header blocks that might rely on the newly added entry, the encoder can use a literal representation. </del>  To ensure that the encoder is not prevented from adding new entries, the encoder can avoid referencing entries that are close to eviction.

A dynamic table entry that has not been acknowledged by the decoder must not be evicted by the encoder, even if there has never been any references to it. This is already alluded to in the current spec at URL by the phrase "acknowledged by the decoder", but since "acknowledged" is used interchangeably for acknowledging the insertion instruction and acknowledging references, a reader could understand this to mean "all references acknowledged". In addition, at three other places eviction of entries with unacknowledged references are banned, with unambigously no ban on eviction of unacknowledged entries with no references. I argue that more explicit language would be beneficial. This issue has already been discussed at URL and below. Note that avoiding this kind of corruption could also be achieved by banning an encoder from inserting entries that it does not immediately reference, but this is probably way too restrictive. For example, an encoder might wish to achieve best latency and speculatively improve future compression ratio at the expense of potentially wasting current bandwidth by encoding header fields as literals while inserting them at the same time for potiential future use. This is already alluded to in the spec at multiple places. Below is a specific scenario when this issue might cause corruption: Suppose MaxEntries is 5. Suppose that there is a fresh connection, with an empty dynamic table, and there is a header list to be sent that has two header fields which fit in the dynamic table together. Suppose one encoder implementation inserts these two header fields into the dynamic table, and encodes the header list using two Indexed Header Field instructions, referring to absolute indices 1 and 2. This header block will have an abstract Largest Reference value of 2, which is encoded as 3 on the wire. A spec compliant decoder should have no difficultly correctly decoding this header block. Suppose another encoder implementation starts by adding 10 unique entries, all different from the two header fields in the header list to be sent, to the dynamic table, but does not emit references to any of them. Of course in the end only the last couple will stay in the dynamic table, the rest will be evicted. This can be considered spec compliant depending on how one interprets the spec, see above. Suppose that after this, this encoder implementation encodes our header list by inserting both header fields into the dynamic table, with absolute indices 11 and 12, and using two Indexed Header Field instructions on the request stream. The abstract Largest Reference value is 12, which is encoded as 3 on the wire since 2 * MaxEntries == 10. Now suppose that out of the 12 instructions on the encoder stream, only the first two arrive before all the request stream data are received. The remaining 10 instructions on the encoder stream are delayed on the wire. Now a spec compliant decoder sees the same header block on the request stream, bit by bit, as above, and two insertions on the encoder stream. It will emit dynamic table entires 1 and 2, which are different from the encoded header list.
You should really create a new issue for these things. Because this comment relates only to the issue, not the proposed fix. I haven't read the proposed changes, though I will if you think that there is a fix in there that is worth keeping after this. Anyhow, the example is invalid here: Because: And you can only ever fit 5 entries in that table.
That's exactly the issue: "it has first been acknowledged by the decoder" is ambigous. "Acknowledged" is being used throughout the text with two different meanings, most often to refer to references, not insertions. The ban on evicting an entry the insertion of which has not been acknowledged, that you are quoting, only occurs once out of four different locations talking about when an entry can be evicted. The other three places clearly only ban evicting an entry with unacknowledged references (note the use of the same word), giving the impression that acknowledgement of the insertion is not necessary before eviction. Since the design is correct if you read it right, but wording is not as clear as I think it should be, I consider this issue editorial. The change I propose in this PR is meant to be an improvement on clarity.
OK, having read the changes, I agree with them. They consolidate the fix by creating two conditions that cause an entry to be "blocking".
Rebased on . Please let me know if anybody can come up with a term better than "blocking" which is confusing given that streams can be "blocked".
NAME Thank you for your feedback.
Thanks for this. I don't have a better name for blocking entries right now, but I will keep thinking on it as I agree it could confuse people even more.
This looks reasonable.

2.1.2. <ins> A dynamic table entry is considered blocking and cannot be evicted until its insertion has been acknowledged and there are no outstanding unacknowledged references to the entry.  In particular, a dynamic table entry that has never been referenced can still be blocking. A blocking entry is unrelated to a blocked stream, which is a stream that a decoder cannot decode as a result of references to entries that are not yet available.  Any encoder that uses the dynamic table has to keep track of blocked entries, whereas blocked streams are optional. </ins> An encoder MUST NOT insert an entry into the dynamic table (or  <ins> duplicate an existing entry) if doing so would evict a blocking entry.  In this case, the encoder can send literal representations of header fields. </ins> To ensure that the encoder is not prevented from adding new entries, the encoder can avoid referencing entries that are close to eviction.


Determining which entries are too close to eviction to reference is an encoder preference.  One heuristic is to target a fixed amount of available space in the dynamic table: either unused space or space <del> that can be reclaimed by evicting unreferenced entries.  To achieve </del>  this, the encoder can maintain a draining index, which is the smallest absolute index in the dynamic table that it will emit a reference for.  As new entries are inserted, the encoder increases

Determining which entries are too close to eviction to reference is an encoder preference.  One heuristic is to target a fixed amount of available space in the dynamic table: either unused space or space  <ins> that can be reclaimed by evicting non-blocking entries.  To achieve </ins> this, the encoder can maintain a draining index, which is the smallest absolute index in the dynamic table that it will emit a reference for.  As new entries are inserted, the encoder increases


Knowledge that a header block with references to the dynamic table has been processed permits the encoder to evict entries to which no <del> unacknowledged references remain, regardless of whether those references were potentially blocking (see blocked-insertion).  When a </del>  stream is reset or abandoned, the indication that these header blocks <del> will never be processed serves a similar function; see stream- cancellation. </del>  The decoder chooses when to emit Insert Count Increment instructions (see insert-count-increment).  Emitting an instruction after adding

Knowledge that a header block with references to the dynamic table has been processed permits the encoder to evict entries to which no  <ins> unacknowledged references remain (see blocked-insertion).  When a </ins> stream is reset or abandoned, the indication that these header blocks  <ins> will never be processed serves a similar function (see stream- cancellation). </ins> The decoder chooses when to emit Insert Count Increment instructions (see insert-count-increment).  Emitting an instruction after adding


Before a new entry is added to the dynamic table, entries are evicted from the end of the dynamic table until the size of the dynamic table is less than or equal to (table capacity - size of new entry) or <del> until the table is empty.  The encoder MUST NOT evict a dynamic table entry unless it has first been acknowledged by the decoder. </del>  If the size of the new entry is less than or equal to the dynamic table capacity, then that entry is added to the table.  It is an

Before a new entry is added to the dynamic table, entries are evicted from the end of the dynamic table until the size of the dynamic table is less than or equal to (table capacity - size of new entry) or  <ins> until the table is empty.  The encoder MUST NOT evict a blocking dynamic table entry (see blocked-insertion). </ins> If the size of the new entry is less than or equal to the dynamic table capacity, then that entry is added to the table.  It is an


connection error of type "HTTP_QPACK_ENCODER_STREAM_ERROR". Reducing the dynamic table capacity can cause entries to be evicted <del> (see eviction).  This MUST NOT cause the eviction of entries with outstanding references (see reference-tracking).  Changing the capacity of the dynamic table is not acknowledged as this instruction does not insert an entry. </del>  4.4.

connection error of type "HTTP_QPACK_ENCODER_STREAM_ERROR". Reducing the dynamic table capacity can cause entries to be evicted  <ins> (see eviction).  This MUST NOT cause the eviction of blocking entries (see blocked-insertion).  Changing the capacity of the dynamic table is not acknowledged as this instruction does not insert an entry. </ins> 4.4.


4.2.4. The CANCEL_PUSH frame (type=0x3) is used to request cancellation of a <del> server push prior to the push stream being created.  The CANCEL_PUSH </del>  frame identifies a server push by Push ID (see frame-push-promise), encoded as a variable-length integer.

Not sure if this is enough, but it enacts Brad's suggested fix.
Yup, that's what I had in mind.
Why is this limited to being prior to the push stream being created, given that you specify how the server handles it if it has created the stream.
I don't read this as saying that you have to avoid it. The point is that you also have to reset/STOPSENDING on the stream if it exists, so there is no real point to using CANCELPUSH in addition.
Sounds like the intent is that CANCELPUSH should be used between the client receiving a PUSHPROMISE and before receiving the stream (i.e. when it is ambiguous as to whether the stream has been created by the server from the clients perspective). I think replacing "created" with "received" would make this more clear.

4.2.4. The CANCEL_PUSH frame (type=0x3) is used to request cancellation of a  <ins> server push prior to the push stream being received.  The CANCEL_PUSH </ins> frame identifies a server push by Push ID (see frame-push-promise), encoded as a variable-length integer.


acknowledgement is received that establishes loss or delivery of packets. <del> When an ACK frame is received that establishes loss after a threshold number of consecutive PTOs (pto_count is more than kPersistentCongestionThreshold, see cc-consts-of-interest), the network is considered to be experiencing persistent congestion, and the sender's congestion window MUST be reduced to the minimum congestion window (kMinimumWindow).  This response of collapsing the congestion window on persistent congestion is functionally similar to a sender's response on a Retransmission Timeout (RTO) in TCP RFC5681. </del>  7.6.

Follow-up from
Consider the following sequence of packets being sent In our old loss recovery code (before unifying TLP and RTO, ), we used the following logic for determining if an RTO was not spurious, which would then lead to collapse of the congestion window: In the example above would be packet 14. This means that to count as a verified RTO, an ACK must acknowledge 15 or 16, but not any packets smaller or equal than 14. With , we now collapse the congestion window when is larger than 2 and we receive an ACK which leads us to declare any packet lost. In the example above this means any ACK received after sending packet 15 which declares a packet lost would suffice. Even worse, this logic already triggers if we receive the ACK immediately after sending 15, i.e. when the peer didn't even have a chance to receive it. This seems a lot more aggressive than what we had before before. Is this intentional?
This is unintentional. I belIeve Jana is intending to fix this in PR , so please take a look at that.
If I understand correctly, is a fix for and only makes the description of the constant and the pseudo code consistent.
I believe the text is now correct, but the pseudocode is still incorrect in the way you describe. I think we should introduce a largestsentbefore_congestion variable as a replacement, but I'm open to other solutions.
I'm removing design. I consider this a bug fix, but still very important.
See which I believe fixes the text.
I've been using to mark things that will result in changes that affect implementations. This would seem to fit that description. Whether you consider it worth discussing is different. If it isn't worth discussing, maybe it will just be fixed by and everything will be fine without discussion in Tokyo. WIthout the label, this won't make the change log, and that might be a problem.
Fair point, I think fixes the text, but I'll re-label this as design to ensure it's included in the changelogs.
The spurious RTO detection calls an RTO spurious if any unacked packets sent prior to the RTO are acked after the RTO. The implication here is that any packet that was received but not acked can cause the RTO to be labeled spurious, even if there were more packets that were in fact lost. This is not always right. Consider the following example: Packets 5, 6, 7 are sent Packet 5 is received, but 6 and 7 are dropped Ack of 5 is dropped TLPs (packets 8 and 9) are sent and dropped RTO (packet 10) is sent and received ACK acks 10 and 5 Since 5 was sent prior to RTO, RTO is marked spurious. A crisp definition of spurious RTO may be useful to continue. Here's what I think is sensible: A spurious RTO is an unnecessary RTO. If the RTO timer had not gone off, things would have been fine. Based on this definition, clearly the RTO was necessary in the example, and therefore not spurious. We need to change the mechanism for detecting spurious RTOs, and I'll propose the following. An RTO is spurious if an ack is received after the RTO fires that acks only packets sent prior to the RTO. In other words, RTO is spurious if ack is received after the RTO, and (instead of , as the draft currently states).
This was recently changed because Lars ran into an issue() where the RTO'd packet was acknowledged for the first time in the same ACK as the pre-RTO packets. In that case, I think it was due to an overly aggressive MinRTO and some pretty coarse grained timers, but I think we want to declare that RTO spurious, since everything would have been fine if it hadn't been sent?
NAME : does not change the semantics substantively of what was in the draft. The intent was always to declare spurious RTO if (packet prior to RTO was newly acked). I'm proposing that we change that to (no packet sent after RTO was newly acked).
In light of , I think this is OBE?
Yes, this is OBE. Closing.

acknowledgement is received that establishes loss or delivery of packets.  <ins> When an ACK frame is received that establishes loss of all in-flight packets sent prior to a threshold number of consecutive PTOs (pto_count is more than kPersistentCongestionThreshold, see cc- consts-of-interest), the network is considered to be experiencing persistent congestion, and the sender's congestion window MUST be reduced to the minimum congestion window (kMinimumWindow).  This response of collapsing the congestion window on persistent congestion is functionally similar to a sender's response on a Retransmission Timeout (RTO) in TCP RFC5681. </ins> 7.6.


Consequently, a Version Negotiation packet consumes an entire UDP datagram.  See version-negotiation for a description of the version negotiation process.

This keeps things simple.
I have no idea how that happened. Forced it.
Are we ok with a server sending both a VN packet and a Retry in response to a single UDP datagram?
NAME Does that happen? IIUC VN is sent only when there's a version mismatch. Retry is sent only when there's no version mismatch.
NAME A client could coalesce two packets of different versions. As I've said on the issue, I still feel uneasy with the fact that a client could coalesce around 70 packets into a single datagram, and I'd prefer more rigid rules to solve this problem.
NAME I'd argue that a client can't do that, because the draft states "Senders MUST NOT coalesce QUIC packets for different connections into a single UDP datagram", OTOH I agree that there's nothing that requires the server to drop following QUIC packets with different versions within a combined datagram. As I stated on URL, I (still) think the approach in this PR would require having minimal state. Would you mind elaborating (possibly on the issue) why you think differently?
Sure, the issue here comes from multi-threaded processing, and reusing packet buffers. I have one thread (or rather, one go-routine) that reads UDP datagrams from the network, splits the coalesced packets and parses the unencrypted part of the header, and then passes the QUIC packets to their respective connections (which are running in a separate go-routine each). In order to reduce allocations, I reuse the buffers I read my packets into. When handling coalesced packets, this means I have to wait until all coalesced parts have been processed before reusing the buffer. Therefore, I can't just parse the first coalesced packet, pass it on for handling in another threat, and then continue parsing. This would create a race condition, since the handling of the packet might be executed before the parsing is finished, and the packet buffer would be returned already. So I have to split the coalesced packets first, and then handle them. This wouldn't be a problem at all if a coalesced packet just had a few parts, as it's supposed to be. But since we're maximizing flexibility for no reason, an attacker might send me datagram consisting of around 70 packets, which I would all have to completely parse.
NAME Thank you for elaborating. I think that we should be dispatching each UDP datagram (instead of each QUIC packet) to the respective connections. That's how load balancers are expected to work, and what you are implementing is essentially a load balancer. The benefit of dispatching at the datagram level is that you'd have less things to do on the receiving thread. I understand your point on flexibility. However, in my view, the flexibility is not only how you coalesce, but the redundancy in all the long header fields. We have deliberately chosen the design because it is clear and easy to analyze (especially in regard to encryption). My preference goes to retaining the clarity by avoiding introducing restrictions.
I think this PR addresses the amplification problem directly, by tying the number of response datagrams to the number of incoming datagrams, so I am in favor of this PR as a spec solution. NAME : quic-go can still choose to drop datagrams with a large (> 3?) coalesced packets, which might help you (slightly). I agree with NAME also that since all coalesced packets are supposed to go to the same connection, the implementation would be better aligned if it were to pass the datagram along to the connection instead of the individual packets... but that's an impl choice, and it's certainly not the only way to do this.
I'm merging this. Please reopen the issue if you think this ought to be discussed further, especially in Tokyo.

Consequently, a Version Negotiation packet consumes an entire UDP datagram. <ins> A server MUST NOT send more than one Version Negotiation packet in response to a single UDP datagram. </ins> See version-negotiation for a description of the version negotiation process.


A server MAY send Retry packets in response to Initial and 0-RTT packets.  A server can either discard or buffer 0-RTT packets that it receives.  A server can send multiple Retry packets as it receives <del> Initial or 0-RTT packets. </del>  A client MUST accept and process at most one Retry packet for each connection attempt.  After the client has received and processed an

A server MAY send Retry packets in response to Initial and 0-RTT packets.  A server can either discard or buffer 0-RTT packets that it receives.  A server can send multiple Retry packets as it receives  <ins> Initial or 0-RTT packets.  A server MUST NOT send more than one Retry packet in response to a single UDP datagram. </ins> A client MUST accept and process at most one Retry packet for each connection attempt.  After the client has received and processed an


4. <del> As discussed above, frames are carried on QUIC streams and used on control streams, request streams, and push streams.  This section describes HTTP framing in QUIC.  For a comparison with HTTP/2 frames, see h2-frames. </del>  4.1.

So a friendly pointer to detail that could be easily misinterpreted or missed? It seems like a table footnote is required but I'm not sure how to do that in Markdown.
While each frame's description indicates the stream type(s) on which it is allowed, there isn't one unified place to indicate which frames are expected on which streams. QPACK subdivides the instruction space per stream, which has the nice property that you can't put something on the wrong stream. While that's overkill for H3 (too dramatic a departure from H2, multiple IANA registries, etc.), having a table to indicate which frames are permitted on which streams would be helpful.
I like the table idea for both. Want a PR?
Text always welcome.
I'll submit two PRs in next day or so.
Minor nit; otherwise good. Given that we have two frames that fall into this camp, might it be worth doing something like "(*) = only as the first frame"?

4.  <ins> HTTP frames are carried on QUIC streams, as described in stream- mapping.  HTTP/3 defines three stream types: control stream, request stream, and push stream.  This section describes HTTP/3 frame formats and the streams types on which they are permitted; see stream-frame- mapping for an overiew.  A comparison between HTTP/2 and HTTP/3 frames is provided in h2-frames. </ins> 4.1.


mapping for an overiew.  A comparison between HTTP/2 and HTTP/3 frames is provided in h2-frames.  4.1. All frames have the following format:

This is my stab at adding the indication discussed in .
Lgtm. An minor Improvement might be to highlight in the new sentence that "specific guidance is provided in the relevant section" but that might be obvious. I'm happy without anything further.

mapping for an overiew.  A comparison between HTTP/2 and HTTP/3 frames is provided in h2-frames. <ins> Certain frames can only occur as the first frame of a particular stream type; these are indicated in stream-frame-mapping with a (1). Specific guidance is provided in the relevant section. </ins> 4.1. All frames have the following format:


packets arrive.  If a peer could timeout within an Probe Timeout (PTO, see Section 6.2.2 of QUIC-RECOVERY), it is advisable to test for liveness before sending any data that cannot be retried safely.  10.3.

What data can and cannot be retried safely?
That's not something that QUIC knows. HTTP does though. As advice, I don't think that this needs much more expansion, but I'd be happy to take a PR.
I'm not so sure this is a HTTP-only issue. Unlike TLS 1.3, there is QUIC data being sent under encryption, and that might have side effects.
LGTM, one nit. ("using applications" is fine, it's just a bit awkward.)

packets arrive.  If a peer could timeout within an Probe Timeout (PTO, see Section 6.2.2 of QUIC-RECOVERY), it is advisable to test for liveness before sending any data that cannot be retried safely. <ins> Note that it is likely that only applications or application protocols will know what information can be retried. </ins> 10.3.


can be mounted using spoofed source addresses.  In determining this limit, servers only count the size of successfully processed packets. <del> Clients MUST pad UDP datagrams that contain only Initial packets to at least 1200 bytes.  Once a client has received an acknowledgment </del>  for a Handshake packet it MAY send smaller datagrams.  Sending padded datagrams ensures that the server is not overly constrained by the amplification restriction.

This change does two things: Clarify that if UDP datagram contains Initial packets, it must be at least 1200 bytes in size; and Rule out padding the UDP datagram with junk (the "padding packets" part) as NAME suggested earlier. Issue is only about (1). Not that I am against (2), but this is just so that we are clear.
I think that the reinterpretation of the text here has introduced the second problem NAME mentions. This also has the unfortunate consequence of disallowing small acknowledgment datagrams (for HelloRetryRequest in particular). This might need to be more carefully phrased.
As NAME said, this is an editorial change, so I'm going to merge. The substantive one is in NAME
NAME -- you're right, but I think this was unintentional earlier. I think if we want to make a case for allowing junk, that should be discussed separately. The proposed text captures what I believe our intent was.
states: This seemingly says that the first UDP datagram that has two coalesced packets, Initial and 0-RTT, can be smaller than 1200 bytes. I don't believe that's the intention.
Agreed, I believe the "only" needs to removed unless I'm missing some other intent.
I don't think that is sufficient. The intent of this was to allow the client to send Initial packets that acknowledge server Initial packets without padding to 1200.
Given PADDING is supposed to count towards bytes in flight, but it's not ack-eliciting, a special case for ACK-only packets makes sense. I'd be inclined to specify ACK-only packets as the exception, opposed to basing it on receiving an ACK of Handshake packets or some other mechanism.
As stated in URL, I prefer to stop requiring the use of full-sized packets once the client observes a response from server. Reasons: QUIC stacks would have the logic to change rules based on if the peer's address has been validated due to the 3-packet rule on the server side. It's just about using the same logic for padding on the client side. Implementations that prefer to pad all the Initial packets it sends can do so anyways. Regardless of what we decide, the server side logic would be indifferent; i.e. create new connection context only when observing a full-sized Initial packet.
tl;dr: we should always pad to 1200 for UDP datagrams that contain Initial packets. The reason we pad to 1200 is to provide a server with plenty of inbound bytes so that it can send outbound bytes prior to path validation. Padding to 1200 has sub-optimal performance in certain cases. In particular, it makes datagrams containing a simple ACK enormous. So the original thought was to try to find a case where we could allow for smaller datagrams to be sent. I think that in both alternative designs we have a problem with the interaction with our DoS defense. And while those problems might only exist in the abstract for now, I think that the best answer is to pad to 1200 always. (I will suggest rewording the text separately.) An ACK-only Initial packet is sent at the tail of the Initial phase. That is the obvious candidate for consideration here. But there are other cases where small Initial packets might be sent. The looser requirement NAME suggests is less than ideal for HelloRetryRequest. A client can receive a HelloRetryRequest and be forced to send another ClientHello. We can't assume here that the server is using the HelloRetryRequest to validate the client address (though it certainly can). If the ClientHello message is small, the number of bytes that a server can send is still limited. If the client doesn't pad, the server can still count the original ClientHello toward its budget, but it doesn't get significant additional budget from the client. And that budget doesn't increase when packets are lost. So it could be that the server is stuck with 3x whatever it gets in ACKs, which is not a lot. It's worse if you consider the possibility of the ServerHello being enormous (hello PQ crypto). At this point, the server won't have validated the client address and so is still limited in what it can send. Ideally, we want the packets to be big so that the server gets more budget for sending and isn't limited to sending tiny packets. But if all the client sends is small ACK-only packets (under either proposed amendment, either what NAME or NAME describe), the increased allowance is very small. Based on this, I would suggest that we reluctantly say that all Initial-carrying datagrams need to be 1200 bytes or more. For the case of the ACK at the tail, most of the designs proposed for key discard will result in that packet not being sent. If it is sent, there are plenty of cases where it will be a packet with Initial (ACK), Handshake (ACK + ...Finished) and 1-RTT data, so the requirement is met without wasting bytes at all. But for ACK of big ServerHello messages or HelloRetryRequest, the overhead is regrettably necessary.
SGTM. I agree this greatly incentivizes good coalescing, but otherwise I don't see any downsides. And simple is good. Even when suggesting the special case for ACK-only packets, it felt like a premature optimization.
Works for me. It's a corner case anyways. Unless HRR is involved, client will not send an Initial in response to the server's Initial, assuming we decide to continue dropping the Initial keys when the client receives a handshake packet.
A potential friendly amendment, but then we should proceed. I would say after confirming with chairs, because there was a pretty strong design decision involved here, but technically this is only an editorial change.

can be mounted using spoofed source addresses.  In determining this limit, servers only count the size of successfully processed packets.  <ins> Clients MUST ensure that UDP datagrams containing Initial packets are sized to at least 1200 bytes, adding padding to packets in the datagram as necessary.  Once a client has received an acknowledgment </ins> for a Handshake packet it MAY send smaller datagrams.  Sending padded datagrams ensures that the server is not overly constrained by the amplification restriction.


for responses in request-response.  Due to reordering, data on a push stream can arrive before the corresponding PUSH_PROMISE, in which case both the associated client request and the pushed request <del> headers are unknown.  Clients which receive a new push stream with an </del>  as-yet-unknown Push ID can buffer the stream data in expectation of the matching PUSH_PROMISE.  A client can use stream flow control (see section 4.1 of QUIC-TRANSPORT) to limit the amount of data a server

As discussed on Slack, currently the text makes no mention of the case where the PUSHPROMISE arrives behind (some of) the data on the push stream itself. However, it does mention this situation for the DUPLICATEPUSH frame. This proposed change recommends buffering pushed data for an unknown PUSHID and using flow control to prevent this from being a problem. I am unsure if more text is needed (e.g., what happens if the PUSHID is never resolved (which would mean a misbehaving server)), but I think the proposed change should suffice. This is also related to issue . Thanks to NAME for his help.
LGTM. Non-normative language seems to be the best form of guidance here
the push id that links a push stream to a push promise is now an unframed varint that comes right after the stream preface. this is right now the only unframed data (except made for the unidirectional stream prefaces) in the specs and I am proposing to make that a frame as well for consistency and to simplify implementation and allow code reuse. motivation: the specs already define frames that only carry a single varint (mostly push related) max push id duplicate push cancel push and goaway! here instead we decided to leave it unframed just because we can since it is at the beginning of the stream cost: two extra bytes per push stream the text will need to specify that on a push stream this new frame (PUSH_ID ?) MUST be the first one, otherwise it is a protocol error. I am happy to put up a PR, in case we reach consensus
The varint type and length of a frame is also unframed. So you are asking to trade one unframed varint for two. Is your assertion that this is unique and difficult code to write, or that the unframed content is somehow difficult to manage?
This seems to be going back to some of the early discussion we had on a thread I can't find. I'm not against discussing things again because we have more experience. However, I don't see a compelling reason for the change. My implementation needs to parse unframed data because of the stream header and for QPACK streams. I think this change might make my implementation more complex. Reading multiple varints requires more work: more checks to see if the stream has enough data, more checks to ensure only 1 frame sent at a specific time (this is the same as SETTINGS)
NAME those are in the frame header -- so they are part of the frame. I am not sure what your point is here then. Here I am assuming that an implementation has generic code to parse frames, and that parsing anything that is not part of a frame requires special handling. While this is implementation specific and there can definitely be counter examples, from an abstraction and consistency point of view I see no reason for doing it the way we are doing it now. (If we discount the unidirectional stream preface) I believe there should be two type of streams: framed (control, request, and push streams) and unframed (QPACK, and potentially extension streams). The push stream right now is a hybrid, that only becomes a framed stream after receiving that first PUSH_ID. If I am recalling correctly when we discussed potentially having DATA frames with zero-length to transition to unframed data, it was also considered a bit unusual. Here I am arguing that we are sort of doing the reverse here where we have an unframed stream that then transitions to framed. I think this generates complexity. And really the benefit we get is saving two bytes and one frame type. NAME I do vaguely remember that discussion and not 'wasting' frame types may have been one of the arguments when we made that decision, but that is not a problem now that we have a 62bit space for frame types.
I don't care about the cost, I care about things like having to deal with PUSH_ID (or whatever) frames in places other than the start of a push stream. And I don't believe that it is difficult to take the special-case code for pulling a varint off a partially-available stream and use that for push ID. We've not had too much trouble with that so far in our implementation.
it is definitely not difficult, but it is a special case and as any special case forces an increased complexity in the possible codepaths, even if not that big in this case. I am making my argument for the sake of consistency mostly, and for the fact that I believe it would also help to better define error handling. With the current text, if I receive something bogus on a push stream (after the preface) I'll have to keep the stream around (because the push stream may come out of order) waiting for a PUSH_PROMISE with that ID that may never come. So I believe that framing could help error handling as well. I do not have super strong feelings about this proposal but at the same time I'd like us to try and recall what the motivations are for the current text and for making this special since it was a while back and personally I can't recall what they were.
This sounds like the case NAME is looking at. I'd argue this is a good reason for the current design, frame processing should only take place once the PUSH_PROMISE is received. But I'd like to understand how framing could help avoid errors.
I was not trying to solve the same problem as here, but I was mentioning the fact that I believe framing would slightly help in a subset of cases. Take a server with a buggy push implementation, rather than a malicious one. Something that doesn't send the PUSHID at all. On the client you'll start seeing a flow of data and you can't do anything better than parsing the first few bytes of that as a PUSHID, and basically see random IDs. With Framing, if a server forgets to send the PUSHID, it is clear at the receiver that it is a protocol error. That being said there are many other classes of bugs that may trigger the same behavior even with frames, so this is minor. NAME now, that seems like an implementation dependent claim :) as says (and I like it) a client should buffer any data after receiving the PUSHID and before it has a PUSH PROMISE for it. and that is sufficient. no mention of frames there.
Right now in draft 19 we have: Parse varint - stream type Parse varint - push ID Parse varint - frame type (HEADERS). The assertion then is, if step 2 gets skipped by the sender, the receiver will parse the HEADERS frame type as a varint expecting it to be the push ID (value 0x1, with no restriction on min or max encoding). It will then parse the HEADERS frame length as a varint expecting it to be the HEADERS frame type (length is unlikely to be 1). Doesn't the client just blow up at that point?
It was just an example. And as I said I don't feel strongly about the error handling nit. But if you must, yes if you assume headers is the first frame after the pushid then yes you would read 0x1 which is conveniently a server initiated bidirectional stream which is not allowed anyway, so you could reject it immediately. But I find this just fortuitous. For example, one could make a counter argument: why are we framing SETTINGS at all given that it is the first message on the control stream ? We also defined it quite clearly in this table what to do with it [Certain frames can only occur as the first frame of a particular stream type; these are indicated in Table 1 with a (1).] URL Do you think there is zero value in doing that for the PUSHID as well? I personally think there is some value. Anyway, I think we are diverting this the wrong route. All I am looking for here is a good explanation for why it is defined like it is now, and I am failing to find it. As far as the implementation goes it makes little difference, and I guess it is more of a consistency problem for me. That naked varint on the wire triggers my OCD :) So instead of going back and forth like this, perhaps NAME can shed some light on the good reasons why it is this way.
+1 to levelling up, I'll take acknowledge my guilt here. But before I move on: If your at the stage of parsing QUIC STREAM frame payload, you should already know exactly what the stream ID is. Some historic points I remember: In Paris MT presented some ideas for unidirecitonal streams - URL This relates to issue URL and PR URL Push and control streams were identified then. And we were happy with reserving specific stream IDs for certain things. All non-reserved streams were server push streams. Subsequent data on the stream only makes logical sense to handle if you know what push it relates to, so those bytes go first. Roll forward some, QPACK design requires additional streams so those get reserved to. Around Kista, the topic of unidirectional stream types comes up. Without types, we make uncoordinated unidirecitonal stream extensions quite a difficult thing. So add more bytes at the start of stream and remove reservations. The SETTINGS frame can contain a set of parameters, indeed it SHOULD contain at least one GREASE setting. A single SETTINGS frame that describes the length of all parameters is quite useful, you know when the peer has finished sending all of them. I don't see as much value for PUSH_PROMISE because you're just framing a single integer that describes its own length already.
oh gosh yeah, I confused PUSH_ID with a stream id. scratch that completely
One can imagine a scenario where we'd want to place something before the SETTINGS frame on the control stream. On the other hand, it's less likely we'd ever want anything other than the Push ID to begin the push stream.
Discussed in London; no.
Seems like a good editorial improvement.

for responses in request-response.  Due to reordering, data on a push stream can arrive before the corresponding PUSH_PROMISE, in which case both the associated client request and the pushed request  <ins> headers are unknown.  Clients that receive a new push stream with an </ins> as-yet-unknown Push ID can buffer the stream data in expectation of the matching PUSH_PROMISE.  A client can use stream flow control (see section 4.1 of QUIC-TRANSPORT) to limit the amount of data a server


19.4. An endpoint uses a RESET_STREAM frame (type=0x04) to abruptly <del> terminate a stream. </del>  After sending a RESET_STREAM, an endpoint ceases transmission and retransmission of STREAM frames on the identified stream.  A receiver

Saying it "terminates a stream" dangerously makes it sound like it has the same meaning as HTTP/2, which is false.
Thanks for all these little patches Martin, they are a real help.

19.4. An endpoint uses a RESET_STREAM frame (type=0x04) to abruptly  <ins> terminate the sending part of a stream. </ins> After sending a RESET_STREAM, an endpoint ceases transmission and retransmission of STREAM frames on the identified stream.  A receiver


The final size is the amount of flow control credit that is consumed by a stream.  Assuming that every contiguous byte on the stream was sent once, the final size is the number of bytes sent.  More <del> generally, this is one higher than the largest byte offset sent on the stream. </del>  For a stream that is reset, the final size is carried explicitly in a RESET_STREAM frame.  Otherwise, the final size is the offset plus the

The transport draft states the following (Section 4.4): This verbiage, introduced in PR , is ambiguous. Byte offset may refer to the Offset field in the STREAM frame; in this case, the definition above is incorrect. What we want to say is that the final size is one higher than the offset of the byte with the largest offset sent on the stream. If no bytes were sent, the final size is zero.

The final size is the amount of flow control credit that is consumed by a stream.  Assuming that every contiguous byte on the stream was sent once, the final size is the number of bytes sent.  More  <ins> generally, this is one higher than the offset of the byte with the largest offset sent on the stream, or zero if no bytes were sent. </ins> For a stream that is reset, the final size is carried explicitly in a RESET_STREAM frame.  Otherwise, the final size is the offset plus the


Endpoints MUST NOT send this extension in a TLS connection that does not use QUIC (such as the use of TLS with TCP defined in TLS13).  A <del> fatal unsupported_extension alert MUST be sent if this extension is received when the transport is not QUIC. </del>  8.3.

No point in levying requirements on them.
Section 8.2 says: However, this directly contradicts TLS 1.3 URL, which says: In order to say this, this document would need to Update 8446, and even then it doesn't make sense because the presumptive point of this rule is to detect attempts to negotiate QUIC with non-QUIC TLS endpoints, which don't need to conform to this RFC (and may predate it).
I don't think QUIC can tell anyone what do with regard to any protocol that is not QUIC. And since QUIC v1 carries its own TLS record wrapping it makes even less sense.
I agree with the problem statement, however I wonder if this opens a session ticket obtained using TLS13-over-TCP to be used in QUIC. Is that what we want?
NAME why would this text have an impact on that? Couldn't I negotiate TLS 1.3 over TCP w/o extension and then send the resume over QUIC w/ extension?
The point here is to insist on the alert if the extension is understood. This wasn't intended to supersede 8446.
NAME My bad. I was confused with the earlier versions of the QUIC-TLS draft that had the transport_parameters extension in the session tickets.
NAME I think you could phrase that, but is it useful, given that we can't rely on it?
Probably not. I'm OK with culling the text.
This might not be possible if a cloud load balancer only forwards QUIC with TLS and terminates TLS otherwise.

Endpoints MUST NOT send this extension in a TLS connection that does not use QUIC (such as the use of TLS with TCP defined in TLS13).  A  <ins> fatal unsupported_extension alert MUST be sent by an implementation that supports this extension if the extension is received when the transport is not QUIC. </ins> 8.3.


maximum value for a Push ID that the server can use in a PUSH_PROMISE frame.  Consequently, this also limits the number of push streams that the server can initiate in addition to the limit set by the QUIC <del> MAX_STREAM_ID frame. </del>  The MAX_PUSH_ID frame is always sent on the control stream.  Receipt of a MAX_PUSH_ID frame on any other stream MUST be treated as a

Makes two minor fixes in the GOAWAY text, both editorial: Correctly references the MAXSTREAMS frame instead of the MAXSTREAM_ID frame Notes that servers need not (and cannot) send a GOAWAY if the client has used up all the possible stream IDs; the client simply won't be making any more requests.
This is a whatever...

maximum value for a Push ID that the server can use in a PUSH_PROMISE frame.  Consequently, this also limits the number of push streams that the server can initiate in addition to the limit set by the QUIC  <ins> MAX_STREAMS frame. </ins> The MAX_PUSH_ID frame is always sent on the control stream.  Receipt of a MAX_PUSH_ID frame on any other stream MUST be treated as a


of the connection.  An endpoint that completes a graceful shutdown SHOULD use the HTTP_NO_ERROR code when closing the connection.  6.3. An HTTP/3 implementation can immediately close the QUIC connection at

of the connection.  An endpoint that completes a graceful shutdown SHOULD use the HTTP_NO_ERROR code when closing the connection. <ins> If a client has consumed all available bidirectional stream IDs with requests, the server need not send a GOAWAY frame, since the client is unable to make further requests. </ins> 6.3. An HTTP/3 implementation can immediately close the QUIC connection at


permitted; receipt of a second stream which claims to be a control stream MUST be treated as a connection error of type HTTP_WRONG_STREAM_COUNT.  The sender MUST NOT close the control <del> stream.  If the control stream is closed at any point, this MUST be treated as a connection error of type HTTP_CLOSED_CRITICAL_STREAM. </del>  A pair of unidirectional streams is used rather than a single bidirectional stream.  This allows either peer to send data as soon

Technically design, but barely.
Originally posted by NAME in URL There are three reasonable places to blow up: When you decide to stop reading a critical stream (i.e. when STOPSENDING is sent) When you discover the peer has stopped reading a critical stream (i.e. when STOPSENDING is received) When you discover the peer has closed a critical stream (when RESETSTREAM or FIN is received) My inclination would be to advise against FIN/SS/R_S on a critical stream and mandate connection termination on receipt of any of them.
Completely agree. This seems like the simplest and most obvious solution.
SGTM mike
Note that this PR does NOT address , just a comment that came up there.
Re-reading this, I think the existing text is pretty good; it just needs a little reframing. The existing requirement is to error out "If the control stream is closed at any point," without reference to whether it was closed because the peer send a FIN/RESET on their control stream or you obeyed a STOP_SENDING on yours. I think I'll add a requirement not to request closure of a control stream; do you think we need more elucidation here?
nits, but LGTM

permitted; receipt of a second stream which claims to be a control stream MUST be treated as a connection error of type HTTP_WRONG_STREAM_COUNT.  The sender MUST NOT close the control  <ins> stream, and the receiver MUST NOT request that the sender close the control stream.  If either control stream is closed at any point, this MUST be treated as a connection error of type HTTP_CLOSED_CRITICAL_STREAM. </ins> A pair of unidirectional streams is used rather than a single bidirectional stream.  This allows either peer to send data as soon


The DUPLICATE_PUSH frame carries a single variable-length integer that identifies the Push ID of a resource that the server has <del> previously promised (see frame-push-promise). </del>  This frame allows the server to use the same server push in response to multiple concurrent requests.  Referencing the same server push

Skirts the editorial/design line. Previously, DUPLICATEPUSH referred to a Push ID in a previously-sent PUSHPROMISE frame; that frame would have caused a connection error on arrival. This says that the error should be sent regardless of which frame arrives first. May need reconciling with .
Two questions: Should DUPLICATEPUSH respect max push ID limit? PUSHPROMISE says the following: provided in a MAXPUSHID frame (Section 4.2.8) and MUST NOT use the same Push ID in multiple PUSHPROMISE frames What should a client do if it sees a DUPLICATEPUSH for an ID of a PUSHPROMISE that it never received? This could be caused by a race condition, so it seems sensible to allow this to clients to support this without error. We should probably document that like we did for other racy things. E.g. a Push ID that has not yet been mentioned by a PUSHPROMISE frame.
(2) is covered in : For (1), yes it should -- the DUPLICATE might show up first, but the implication is that the ID has already been used in a PUSHPROMISE frame. If that would be invalid, we should be explicit that the DUPLICATEPUSH is also invalid.
SGTM
I think that we have a frame name problem, otherwise, this is fine.I think some wires got crossed in the suggested changes. Your comment points out that this might need reconciliation with but it seems to borrow the text directly from that PR. The outcome is that this PR is inconsistent with itself because at line the same error with PUSHPROMISE causes a MALFORMEDFRAME. I think the best path is to make this change be a MALFORMED_FRAME error and reconcile on if the group want that change.

The DUPLICATE_PUSH frame carries a single variable-length integer that identifies the Push ID of a resource that the server has  <ins> previously promised (see frame-push-promise), though that promise might not be received before this frame.  A server MUST NOT use a Push ID that is larger than the client has provided in a MAX_PUSH_ID frame (frame-max-push-id).  A client MUST treat receipt of a DUPLICATE_PUSH that contains a larger Push ID than the client has advertised as a connection error of type HTTP_MALFORMED_FRAME. </ins> This frame allows the server to use the same server push in response to multiple concurrent requests.  Referencing the same server push


A TCP connection error is signaled with QUIC RESET_STREAM frame.  A proxy treats any error in the TCP connection, which includes receiving a TCP segment with the RST bit set, as a stream error of <del> type HTTP_CONNECT_ERROR (http-error-codes).  Correspondingly, a proxy MUST send a TCP segment with the RST bit set if it detects an error with the stream or the QUIC connection. </del>  4.3.

LGTM
S 5.2. I don't believe this is possible on UNIX. What system call do I use to generate RST? Neither close() nor shutdown() does it.
You set SO_LINGER to {1, 0}, then close.
I don't think there's a question here about the -http draft, is there?
The question is if it's possible on standard OSes. Sounds like it is on Unix. How about Windows? NAME
This is the same requirement as in HTTP/2. A "MUST (but we know you probably won't)" ?
Well, that was a joke in 6919, so probably we shouldn't do that.
This is not that. We know that some implementations use TCP half-closed in HTTP. Some even rely on it to the point that things break if you don't do it. Not that things don't break for those people, but they have a narrow set of peers that they interact with. Of course, whether that set of endpoints is ALSO the same set of endpoints using CONNECT is hard to know.
Note that this particular issue is not about half-closed. In half-closed, you generate a FIN and then keep receiving data. This is about sending an RST.
Perhaps the text should be "SHOULD", and if the platform doesn't allow it, to "close the connection".
There's risk with that too, though. We encountered issues where the TCP connection was closed cleanly instead of aborted, and the client interpreted a truncated response as the (cacheable) response. It gets ugly. If you can't force a RST on the other side, you perhaps shouldn't be implementing CONNECT.
Discussed in Tokyo; MUST close the connection, SHOULD close with a RST.

A TCP connection error is signaled with QUIC RESET_STREAM frame.  A proxy treats any error in the TCP connection, which includes receiving a TCP segment with the RST bit set, as a stream error of  <ins> type HTTP_CONNECT_ERROR (http-error-codes).  Correspondingly, if a proxy detects an error with the stream or the QUIC connection, it MUST close the TCP connection.  If the underlying TCP implementation permits it, the proxy SHOULD send a TCP segment with the RST bit set. </ins> 4.3.


bytes associated with an HTTP request or response payload. DATA frames MUST be associated with an HTTP request or response.  If <del> a DATA frame is received on either control stream, the recipient MUST </del>  respond with a connection error (errors) of type HTTP_WRONG_STREAM. 7.2.2.

Use 'on a control stream' instead of 'on either control stream'. This is consistent with the changes made by .

bytes associated with an HTTP request or response payload. DATA frames MUST be associated with an HTTP request or response.  If  <ins> a DATA frame is received on a control stream, the recipient MUST </ins> respond with a connection error (errors) of type HTTP_WRONG_STREAM. 7.2.2.


HTTP/3 endpoint is authoritative for other origins without an explicit signal.  A server that does not wish clients to reuse connections for a particular origin can indicate that it is not authoritative for a request by sending a 421 (Misdirected Request) status code in

in what I believe is the simplest way possible -- point to 8164 and require that be successfully retrieved prior to sending any scheme other than .
It would be good if it could be clarified what will happen if someone needs/tries to send a HTTP request with an http:// uri over H3. I primarily see this to occur when you have a case where you have an HTTP proxy that receives an HTTP request and then have an HTTP/3 connection with the authorative server. If the expectation is that all requests are upgraded by the HTTP/3 sending side, then that needs to be made explicit.
I might propose this comes down to a discovery problem for the proxy. What method(s) do you envisage the proxy used to decide to make an HTTP/3 connection to the authorative server? This seems related to the discussion in HTTP WG - URL If Alt-Svc is being used, the flow might be that the proxy attempts to make a cleartext HTTP/1.1 connection to the authoritative server and does an upgrade, as described in RFC 7838, to HTTP/3. In which case I don't think any action is required. If the proxy wants to opportunistically upgrade via some happy eyeballs approach described on URL, we might want to provide guidance that either 1) it shouldn't be done for client requests with http:// or 2) that it should always be done.
This is another variant of -- HTTP/2 and HTTP/3 explicitly include the scheme and the authority with all requests, so the server knows what it's being asked for. The question is how the client decides that a QUIC endpoint is authoritative for this particular URL. RFC 7838 describes how an https://-schemed request might come to be sent to an HTTP/3 endpoint and the resolution of the issue was to ask the HTTP WG to define other ways. Adding RFC 8164 as well, an http://-schemed resource might also be requested over HTTP/3; it's up to the HTTPbis group whether they want to enable a direct upgrade there as well. I don't currently see any text required for this, but if you have suggestions, I'm happy to review or expand the discussion to the list.
I'd rather duplicate this to and rely on for this. We were not very careful in that document to avoid mention of the specific HTTP version, but the technique is perfectly portable between versions. We only really intended to put HTTP/1.x off limits.
Perhaps "not very careful" was careful enough -- it says "such as HTTP/2," and mandates that the protocol needs a way to carry the scheme. HTTP/3 does, so we're covered. Do we think a reference to 8164 is warranted in the section about finding an HTTP/3 endpoint?
By "not very careful" I was mainly referring to the title :) As for the text, I don't know. It will depend on what else it says.
Discussed in London; reference 8164 and require the same check when sending a scheme other than "https."

HTTP/3 endpoint is authoritative for other origins without an explicit signal. <ins> Prior to making requests for an origin whose scheme is not "https," the client MUST ensure the server is willing to serve that scheme. If the client intends to make requests for an origin whose scheme is "http", this means that it MUST obtain a valid "http-opportunistic" response for the origin as described in RFC8164 prior to making any such requests.  Other schemes might define other mechanisms. </ins> A server that does not wish clients to reuse connections for a particular origin can indicate that it is not authoritative for a request by sending a 421 (Misdirected Request) status code in


An optional reference to a specification that describes the use of the setting.  The entries in the following table are registered by this document. Additionally, each code of the format "0x1f * N + 0x21" for integer

I went with "Default" because that and "Initial Value" now have specific, related (but distinct) usages in the context of pre-SETTINGS settings. Swapping them seems like it would create unnecessary confusion.
Yeah that makes sense on a reread of section 7.2.5.2
Not a new issue but: RFC 7540 has a column for . I think it makes sense that HTTP/3 does similar. Originally posted by NAME in URL

An optional reference to a specification that describes the use of the setting. <ins> The value of the setting unless otherwise indicated.  SHOULD be the most restrictive possible value. </ins> The entries in the following table are registered by this document. Additionally, each code of the format "0x1f * N + 0x21" for integer


To entities other than its intended recipient, a stateless reset will appear to be a packet with a short header.  For the stateless reset <del> to appear as a valid QUIC packet and be smaller than the received packet, the Unpredictable Bits field needs to include at least 46 bits of data (or 6 bytes, less the two fixed bits).  To ensure the stateless reset packet is not smaller than other packets received on the connection, an endpoint SHOULD also ensure the total packet length is at least the minimum chosen CID length plus 22 bytes.  22 bytes allows for 1 type byte, 4 packet number and data bytes, 16 bytes for AEAD expansion, and an extra byte to allow the peer to send a smaller stateless reset than the packet it receives. The Stateless Reset Token corresponds to the minimum expansion of the packet protection AEAD.  More unpredictable bytes might be necessary if the endpoint could have negotiated a packet protection scheme with a larger minimum AEAD expansion. An endpoint SHOULD NOT send a stateless reset that is significantly larger than the packet it receives.  Endpoints MUST discard packets that are too small to be valid QUIC packets.  With the set of AEAD functions defined in QUIC-TLS, packets that are smaller than 21 bytes are never valid. </del>  Endpoints MUST send stateless reset packets formatted as a packet with a short header.  However, endpoints MUST treat any packet ending

PR had a mistake in it, and some very confusing language. The rules for generating stateless resets was all mixed up in some complementary rules about generating packets that might trigger stateless resets. This attempts to split out the rules for generating stateless resets (minimum size is 5 + 16 = 21), and for generating packets that might trigger a stateless reset (min_cid + 22).
Do we need discussion on the fixed QUIC header bit in the triggering packet and outgoing reset packet?
NAME if you can lay out what you think the security concerns are in another issue, that would helpful. I realize that there are plenty of compromises in this design, which make this less than ideal. But first I'd like to try to understand where you think this is falling short.
Do we need discussion on the fixed QUIC header bit in the triggering packet and outgoing reset packet? Originally posted by NAME in URL
Maybe I'm misunderstanding the question, but isn't the spec is completely clear on that? A stateless reset has the first bit unset and the second bit set.
Agreed, I think this is already clear and I don't see any need to change it.
Thanks both. I just wanted to check and didn't feel like I had the brain cells to answer that. I don't like relying on the image only, but the text says "two fixed bits", and stateless reset - unlike Version Negotiation - is version-specific. So I'm closing with no action.
I believe this PR addresses the stated issue, which is that the text is unclear. However, I am still concerned about the security properties of this design.

To entities other than its intended recipient, a stateless reset will appear to be a packet with a short header.  For the stateless reset  <ins> to appear as a valid QUIC packet, the Unpredictable Bits field needs to include at least 38 bits of data (or 5 bytes, less the two fixed bits). A minimum size of 21 bytes does not guarantee that a stateless reset is difficult to distinguish from other packets if the recipient requires the use of a connection ID.  To prevent a resulting stateless reset from being trivially distinguishable from a valid packet, all packets sent by an endpoint SHOULD be padded to at least 22 bytes longer than the minimum connection ID that the endpoint might use.  An endpoint that sends a stateless reset in response to packet that is 43 bytes or less in length SHOULD send a stateless reset that is one byte shorter than the packet it responds to. These values assume that the Stateless Reset Token is the same as the minimum expansion of the packet protection AEAD.  Additional unpredictable bytes are necessary if the endpoint could have negotiated a packet protection scheme with a larger minimum expansion. An endpoint MUST NOT send a stateless reset that is three times or more larger than the packet it receives to avoid being used for amplification.  reset-looping describes additional limits on stateless reset size. Endpoints MUST discard packets that are too small to be valid QUIC packets.  With the set of AEAD functions defined in QUIC-TLS, packets that are smaller than 21 bytes are never valid. </ins> Endpoints MUST send stateless reset packets formatted as a packet with a short header.  However, endpoints MUST treat any packet ending


endpoint that occasionally uses different connection IDs might introduce some uncertainty about this. <del> Finally, the last 16 bytes of the packet are set to the value of the Stateless Reset Token. </del> This stateless reset design is specific to QUIC version 1.  An endpoint that supports multiple versions of QUIC needs to generate a stateless reset that will be accepted by peers that support any

endpoint that occasionally uses different connection IDs might introduce some uncertainty about this.  This stateless reset design is specific to QUIC version 1.  An endpoint that supports multiple versions of QUIC needs to generate a stateless reset that will be accepted by peers that support any


delay SHOULD NOT be considered an RTT sample. Until the server has validated the client's address on the path, the <del> amount of data it can send is limited, as specified in Section 8.1 of QUIC-TRANSPORT.  Data at Initial encryption MUST be retransmitted before Handshake data and data at Handshake encryption MUST be retransmitted before any ApplicationData data.  If no data can be sent, then the PTO alarm MUST NOT be armed until data has been received from the client. </del>  Since the server could be blocked until more packets are received from the client, it is the client's responsibility to send packets to

This is implied in some spots, but never stated with normative language.
Under "Handshakes and New Paths", it says "Data at Initial encryption MUST be retransmitted before Handshake data and data at Handshake encryption MUST be retransmitted before any ApplicationData". I moved that down to this section, since I think it fits slightly better here.
In terms of packet size, this is similar to what the old Handshake text recommended and what PTO should have said all along, as it aligns well with TCP's TLP and RTO behavior.
The previous text read like it only described retransmission of lost data, and this now reads like it only describes retransmission in PTOs. Are both intended? Is there a restriction anywhere that retransmitted data must not be sent at a higher encryption level than it was originally sent, or similar?
All of this text is in the PTO section, so only applies to PTO. I think transport is usually the best place to talk about where data is retransmitted, but it might be worth an editorial note that data is never retransmitted at a different packet number space than the original one.
Thanks for opening other issues, NAME I'm going to merge this, since I think it's a net betterment over what we have. We can continue the other discussions on the other issues/PRs.
At the New York QUIC interim, we dealt with the problem of the server being blocked by the amplification factor by making the client arm the PTO until it knew the server had validated the path, even if it had nothing to send. From the recovery : When reviewing URL, I realized that if we allowed the server to send a PING-only packet when the PTO fired and the amplification factor had been reached, then that would elicit a padded Initial ACK frame in response, which also fixes the issue. NAME though that was a simpler solution than the current approach and this is a bit of an edge case, so simple is probably good. The original issue was
In a strict sense, this means that a client can use N bytes to trigger the server to send 3N + epsilon bytes, but this fits in nicely with Probes generally not being subject to sending limits. It is somewhat gross but more elegant that a client timer with nothing in flight.
I think this might be concerning, though I could well be wrong. Consider the case of an attacker acting as a client, using the IP address of a victim as the source address. The attacker can spoof an Initial packet that carries a Client Hello and another Initial packet that carries an ACK for the server's Initial packet that carried the Server Hello. An attacker can build the correct second packet assuming that the server uses the DCID of the client's Initial packet as it's CID (or the CID is zero length), and that the server's first packet number can be guessed. If that ACK contains a delay that indicate to the server that the RTT is very small (e.g., 1ms), then the PTO could be as small as that value. Then, if the server's connection timeout is say 1000ms, the server might send 1000 packets to the victim before dropping the connection. Am I missing something?
PTO always uses exponential backoff, so if the first PTO was 1ms, that'd be ~10 PTOs, not 1000. In terms of total bytes sent, 10 packets with a 1 byte payload are still smaller than a single full-sized packet.
Isn't this concern why we designed the Retry mechanism in the first place? If we are concerned about address spoofing attacks, the simple solution might be to generalize retry. As in, "if the server can predict that its first flight will contain more than 3 times the amount of data received with the client's Initial packets, then it SHOULD verify the source address using Retry."
My concern is that the current anti-deadlock mechanism is fairly awkward and rarely needed, which makes me concerned it will be incorrectly implemented. By sending a PING only packet, the server is essentially informing the client it has something to send, but can't send it due to the amplification limit. The deadlock happens when the client's Initial is acknowledged, but the client doesn't receive the server's Initial, so it can't decrypt any handshake packets. In this state, the client has nothing to send. It can happen even if the server's first flight fits into 3 packets. But it's expected that typically servers will bundle their ACK of the client's Initial with the server Initial, and this will be rare in practice.
NAME Fair. Due to the risk of amplification attacks, the server is probably not willing to retransmit Initial packets on timer. In that case, the current spec is a bit too lose. It should say that "If the sender is not willing to retransmit Initial packets due to protection against amplification attacks, it MUST NOT acknowledge the client's initial packet." So basically, either the server verifies the client's address with the Retry/Token mechanism, or it MUST NOT acknowledge client initial packets before continuity can be verified by obtaining an ACK from the client.
That's an interesting suggestion, but I think it require some non-trivial changes to the server's logic in addition to not sending ACKs? Something similar to what's described in . It may be worth opening up an issue for allowing the server to not arm a timer until the path has been validated, because I think that's a wider issue than just this PR?
I agree that having the server repeat a PING on timer may also work, and that the ACK of the randomly chosen Server's CID would make a reasonable address ownership test. ACK of the PING would naturally unlock any transmission that is pending address ownership. Kazuho objects that this his hackable if the chosen Server's CID is guessable. We don't have a "non-guessable" requirement in the spec. In that case, the natural fix is the challenge/response mechanism.
NAME Ah. Thank you for pointing that out. Though it seems that the amount of the data that the server can send could go like 4x the amount of data that the client has sent rather than the current 3x. To paraphrase, I think what we might be trading here is the initial amount of data the server can send vs. the capability of sending PINGs. My view is that the former is a scarce resource, and that I'd prefer having the more space in the former even if requires us to have some complexity.
NAME An Initial ping is maybe 45 bytes. So 10 pings makes the amplification ~3.4. I was under the impression that the 3 multiplier was semi-arbitrary, so I'm unconcerned about a fractional increase.
I have a couple of thoughts. First, this change is not necessarily an overall simplification. It makes things slightly simpler at the client, but it adds about as much code to the server. Additional code has to be added at the server to the PTO response to only send a PING and not a retransmission when the path is not yet validated. Second, I think it does open up an interesting attack vector that is currently absent. NAME notes that there is amplification, but I don't think byte amplification is the interesting one here, since the total load is going to be < 1 MTU. The interesting amplification here is packet amplification, which is a useful attack since UDP and packet processing costs at endpoints / clients is a relatively high cost. And you can get a fair number of packets back from the server for spoofing a couple of packets, in spite of the exponential backoff. I'll grant that the attack is a bit of a stretch, and does require that the attacker guess the server's CID and packet number. Though, as NAME pointed out to me offline, we allow the server to use a zero-length CID. I don't think this change is necessary. And on balance, I don't think this change is a good one.
I do like the idea of sending a PING frame from the server as a signal to the client that it is blocked. I think we can suggest that as an optimization that a clever server might implement under the 3x limit, or when it reaches the 3x limit.
NAME While endpoints can do that, I am not sure if it is a good idea to endorse such a behavior in the specification. The reason we have so far been fine with having the 3x limit being applied until the Handshake packets are exchanged is because we assume that the size of ServerHello would not be that large compared to the size of the ClientHello, and therefore that the chance of hitting 3x limit before path validation is minimal. It's a hack. If we are to address the corner cases that the server hits after sending 3x amount of data, then I think we should fix the hack than putting another hack (i.e. the PING that signals 3x limit) above the existing hack. I mean, if we think that defining a way that allows the server to unblock the 3x limit is important, we should be considering allowing the use of PATHCHALLENGE / PATHRESPONSE frames in Initial packets.
I believe this is simpler to implement assuming one is implementing both a client and server. I already have special case code to send a PING when there are bytes in flight but nothing to send, so this is very similar. I agree it's not necessary, but I find the simplification appealing, and when we first discussed this problem, no one suggested this solution, so I thought it was worth writing up. If we keep the existing text that the client has to keep sending, I'm not sure doing this on the server side is a valuable optimization, so I'd prefer to stick with one mechanism or the other. To me, the main negative is the server sending extra datagrams. And maybe that's enough to stick with what we have.
When a PTO fires, the endpoint is supposed to try and retransmit something if there is data to retransmit. In this case, you'd need to special-case it so that the server doesn't in fact retransmit but sends a PING instead. That's what I meant by adding more code.
It also SHOULD send a PING only packet if there are bytes in flight but nothing to send.
Not new data, any data. This was deliberate. See Section 5.3.1: "When the PTO timer expires, and there is new or previously sent unacknowledged data, it MUST be sent." The point of this text was to send data if available, new or previously sent. In the case that we are discussing the server is supposed to send a PING, but it might have data to send.
Discussed in ZRH. NAME to decide if he wants to pursue this, potentially discuss with NAME and/or NAME
I realized there is one major benefit of keeping the existing design, which is the client is guaranteed to have at least one RTT measurement in this circumstance, whereas the server may not. For this reason, I'm inclined to close this with no action unless others feel strongly otherwise.
NAME came up with the same point simultaneously, so let's close this with no action.
There are multiple mentions of spurious retransmission in the recovery draft, but no text on what detecting this may require from an implementation. Maybe there should be? "Spuriously declaring packets as lost leads to unnecessary retransmissions and may result in degraded performance due to the actions of the congestion controller upon detecting loss. Implementations that detect spurious retransmissions and increase the reordering threshold in packets or time MAY choose to start with smaller initial reordering thresholds to minimize recovery latency." From comments on
Some discussion of implementation strategies here would absolutely be helpful; it doesn't obviously fall out of Quinn's current architecture.
NAME and NAME indicated my effort at adding more text wasn't that useful, so I'm closing this.
As of URL, if the client's Initial is lost, there is no required behavior that leads to it being retransmitted. Because the client has received no ACKs, no is set, and no loss is detected, leading to the client only sending probe packets, which are not required to bear any particular data ("A probe packet MAY carry retransmitted unacknowledged data") so long as they are ACK-eliciting. The server, having no connection state, cannot do anything with these packets. Quinn probes are currently a single PING frame (plus padding, when appropriate) for simplicity, so implementing the new logic immediately broke our tests. If recovery logic now requires probe packets to bear specific data to function correctly in certain circumstances, this should be specified explicitly.
I also hit this issue too. After several hours tries and error, I came to the conclusion that implementation should mark the in-flight Initial packet lost in this case, and resend it. Draft-22 has explicit statement that implementation should not mark in-flight crypto packet lost, but draft-23 does not have it anymore. BTW, draft-23 does not allow PING in Initial or Handshake packet.
Why not retransmit the data without declaring the in-flight packet lost?
I think it is rather implementation detail to internally mark in-flight packet lost but keep it in order to catch late ACK.
There's a related problem where if the server's first flight goes missing, the only response the new logic has is to send a probe at unspecified, i.e. highest known, encryption level. The server will generally have 1-RTT keys at this point, so it sends a 1-RTT probe, which the client, not having received anything prior, cannot decrypt and drops. As above, this was previously handled by explicitly tracking whether any data was in-flight, but that got removed.
The packets should not be marked as lost, but data needs to be retransmitted before sending a PING. I sent out a PR , feel free to make further suggestions there.
I'm marking this design because it adds some normative language, though I'll note that it doesn't change what I originally intended the text to say.

delay SHOULD NOT be considered an RTT sample. Until the server has validated the client's address on the path, the  <ins> amount of data it can send is limited to three times the amount of data received, as specified in Section 8.1 of QUIC-TRANSPORT.  If no data can be sent, then the PTO alarm MUST NOT be armed. </ins> Since the server could be blocked until more packets are received from the client, it is the client's responsibility to send packets to


ack-eliciting packets, to avoid an expensive consecutive PTO expiration due to a single lost datagram. <del> It is possible that the sender has no new or previously-sent data to send.  As an example, consider the following sequence of events: new </del>  application data is sent in a STREAM frame, deemed lost, then retransmitted in a new packet, and then the original transmission is <del> acknowledged.  In the absence of any new application data, a PTO timer expiration now would find the sender with no new or previously- sent data to send. When there is no data to send, the sender SHOULD send a PING or other ack-eliciting frame in a single packet, re-arming the PTO timer. </del>  Alternatively, instead of sending an ack-eliciting packet, the sender MAY mark any packets still in flight as lost.  Doing so avoids

ack-eliciting packets, to avoid an expensive consecutive PTO expiration due to a single lost datagram.  <ins> When the PTO timer expires, and there is new or previously sent unacknowledged data, it MUST be sent.  Data that was previously sent with Initial encryption MUST be sent before Handshake data and data previously sent at Handshake encryption MUST be sent before any ApplicationData data. It is possible the sender has no new or previously-sent data to send. As an example, consider the following sequence of events: new </ins> application data is sent in a STREAM frame, deemed lost, then retransmitted in a new packet, and then the original transmission is  <ins> acknowledged.  When there is no data to send, the sender SHOULD send a PING or other ack-eliciting frame in a single packet, re-arming the PTO timer. </ins> Alternatively, instead of sending an ack-eliciting packet, the sender MAY mark any packets still in flight as lost.  Doing so avoids