Question:
Most of the text was correct in that we talk about making a request of a resource. Where the text refers to making requests of a server, those usages seem to be correct.
Section 5 details HTTP usage. It seems it's technically correct but the section closes with a sequence of paragraphs that jump back and forth between statements about how an Oblivious Gateway Resource treats requests or responses. Perhaps these could be consolidated to present requests first consistently. I.e. swap the final two paragraphs around
mnot recently made a PR (now merged) that replaced "requests of a server" with "requests to an origin server" and that feels both grammatically and technically more correct. However, there are still few cases peppered through the doc that use "requests of a server". I'd kindly ask the editors to revisit these and consider if they could be made more consistent with the above

6. In this design, a client wishes to make a request of a server that is <del> authoritative for the Target Resource.  The client wishes to make this request without linking that request with either: </del> <ins> authoritative for a Target Resource.  The client wishes to make this request without linking that request with either: </ins> The identity at the network and transport layer of the client (that is, the client IP address and TCP or UDP port number the


Question:
I presume the idea here is to add jitter to the system (different delays for different requests) rather than compound delays on a single request? Even if not, this paragraph could probably benefit from some language fixups. There's a missing word and "might not be what some clients want" could benefit from a rephrasing to make it stronger.
clearer, thanks

analysis could be used to match messages that are forwarded by the relay. <del> A relay could, as part of its function, add delays in order to increase the anonymity set into which each message is attributed. This could latency to the overall time clients take to receive a response, which might not be what some clients want. </del> <ins> A relay could, as part of its function, delay requests before forwarding them.  Delays might increase the anonymity set into which each request is attributed.  Any delay also increases the time that a client waits for a response, so delays SHOULD only be added with the consent - or at least awareness - of clients. </ins> A relay that forwards large volumes of exchanges can provide better privacy by providing larger sets of messages that need to be matched.


Question:
This starts by moving the good text from to the "Errors" section, where I realized that it duplicated some of that language. It was more complete, so I merged the two. I then added some language about what happens to errors prior to decapsulation. This doesn't define an error signaling format. Though that might be useful, it is also a lot more work and I'm leery of scope creep in this document. My view: if you get an error, you have screwed up somewhere. You probably want to have a human look at what is going on. The privacy consequences of automated responses are difficult to work out properly. Hence this proposed change.
This is an improvement but still insufficient with respect to configuration mismatch signaling.
Let's start with this one and I'll look at addressing the open issue in a follow-up.
Thanks, I think this is clear enough now.

response to the POST request made to that resource. Errors detected by the after successfully removing encapsulation and <del> errors detected by the MUST be sent in an . </del> <ins> errors detected by the MUST be sent in an .  This might be because the request is malformed or the does not produce a response.  In either case the can generate a response with an appropriate error status code (such as 400 (Bad Request) or 504 (Gateway Timeout); see HTTP and HTTP, respectively).  This response is encapsulated in the same way as a successful response. Errors in the encapsulation of requests mean that responses cannot be encapsulated.  This includes cases where the is incorrect or outdated.  The can generate and send a response with an error status to the .  This response MAY be forwarded to the or treated by the as a failure.  If a receives a response that is not an , this could indicate that the client configuration used to construct the request is incorrect or out of date. </ins> 6.


Question: From NAME Interesting that this is the first time OHTTP appear in the text, excluding pseudocode and mediatype - please expand (or define somewhere else, for example in terminology).

4.6. <del> The encrypted payload of an OHTTP request and response is a binary HTTP message BINARY.  The and agree on this encrypted payload type by specifying the media type "message/bhttp" in the HPKE info string and HPKE export context string for request and response encryption, respectively. </del> <ins> The encrypted payload of an Oblivious HTTP request and response is a binary HTTP message BINARY.  The and agree on this encrypted payload type by specifying the media type "message/bhttp" in the HPKE info string and HPKE export context string for request and response encryption, respectively. </ins> Future specifications may repurpose the encapsulation mechanism described in this document.  This requires that the specification


Question:
Switches to an 8-bit key identifier here as well. This format now doesn't need varints. Yay. Implemented this in my ohttp library and it's fine.
Splitting this from . It seems likely that a single KEM key will support multiple KDF or AEAD options (NSS will soon support 3 of the former, and 2 of the latter, maybe 3 of each). Having a configuration that includes a single set of (KEM, key, and key identifier) with a list of KDF+AEAD pairs, akin to ECH, would make it easier to support that sort of deployment. Without that, key identifiers need to carry information about the KDF and AEAD. To support this, the format needs to be adjusted to carry the KDF and AEAD identifier. In doing so, we probably want to encode the choice of KDF and AEAD in the key schedule or AAD accordingly.
Should the key ID length be explicitly authenticated, e.g., by including it in the AAD, or should the AEAD implicitly authenticate things? Unclear!
Should we even bother with an AAD at all? The best reason is that the keyID determines which key the server uses, which is information that factors into decisions. (A related question might be why doesn't HPKE take variables as input so that things like key identifiers can be integrated into the key schedule.)
My take: yes. I need to think through the implications of no additional authentication at all, and in the mean time, I'd be more comfortable with simply authenticating everything possible. I think it was discussed, but it makes the interface a mess. And since applications can accomplish the same thing by shoving what they want authenticated into the AAD, it was voted down.
Yeah, you and I both know that putting stuff in the AAD isn't a real substitute for having it in the key schedule. I completely understand how this would mess with the purity of the design, but TLS managed by wrapping its labels better (which also addresses a known problem with HKDF not being injective).
Fixed by virtue of making the key ID fixed length in .

This draft includes pseudocode that uses the functions and conventions defined in HPKE. <del> This draft uses the variable-length integer encoding from Section 16 of QUIC.  Encoding and decoding variable-length integers to a sequence of bytes are described using the functions "vencode()" and "vdecode()".  The function "len()" takes the length of a sequence of bytes. </del> <ins> Encoding an integer to a sequence of bytes in network byte order is described using the function "encode(n, v)", where "n" is the number of bytes and "v" is the integer value.  The function "len()" returns the length of a sequence of bytes. </ins> Formats are described using notation from Section 1.3 of QUIC.


Question:
Switches to an 8-bit key identifier here as well. This format now doesn't need varints. Yay. Implemented this in my ohttp library and it's fine.
Splitting this from . It seems likely that a single KEM key will support multiple KDF or AEAD options (NSS will soon support 3 of the former, and 2 of the latter, maybe 3 of each). Having a configuration that includes a single set of (KEM, key, and key identifier) with a list of KDF+AEAD pairs, akin to ECH, would make it easier to support that sort of deployment. Without that, key identifiers need to carry information about the KDF and AEAD. To support this, the format needs to be adjusted to carry the KDF and AEAD identifier. In doing so, we probably want to encode the choice of KDF and AEAD in the key schedule or AAD accordingly.
Should the key ID length be explicitly authenticated, e.g., by including it in the AAD, or should the AEAD implicitly authenticate things? Unclear!
Should we even bother with an AAD at all? The best reason is that the keyID determines which key the server uses, which is information that factors into decisions. (A related question might be why doesn't HPKE take variables as input so that things like key identifiers can be integrated into the key schedule.)
My take: yes. I need to think through the implications of no additional authentication at all, and in the mean time, I'd be more comfortable with simply authenticating everything possible. I think it was discussed, but it makes the interface a mess. And since applications can accomplish the same thing by shoving what they want authenticated into the AAD, it was voted down.
Yeah, you and I both know that putting stuff in the AAD isn't a real substitute for having it in the key schedule. I completely understand how this would mess with the purity of the design, but TLS managed by wrapping its labels better (which also addresses a known problem with HKDF not being injective).
Fixed by virtue of making the key ID fixed length in .

5.1. <del> Clients encapsulate a request "request" with an HPKE public key "pkR", whose Key Identifier is "keyID" as follows: </del> <ins> Clients encapsulate a request "request" using values from a key configuration: the key identifier from the configuration, "keyID", the public key from the configuration, "pkR", and a selected combination of KDF, identified by "kdfID", and AEAD, identified by "aeadID". The client then constructs an encapsulated request, "enc_request", as follows: </ins> Compute an HPKE context using "pkR", yielding "context" and encapsulation key "enc". <del> Encrypt (seal) "request" with "keyID" as associated data using </del> <ins> Construct associated data, "aad", by concatenating the values of "keyID", "kdfID", and "aeadID", as 8-, 16- and 16-bit integers respectively, each in network byte order. Encrypt (seal) "request" with "aad" as associated data using </ins> "context", yielding ciphertext "ct". <del> Concatenate the length of "keyID" as a variable-length integer, "keyID", "enc", and "ct", yielding an Encapsulated Request "enc_request".  Note that "enc" is of fixed-length, so there is no ambiguity in parsing "enc" and "ct". </del> <ins> Concatenate the three values of "aad", "enc", and "ct", yielding an Encapsulated Request "enc_request". Note that "enc" is of fixed-length, so there is no ambiguity in parsing this structure. </ins> In pseudocode, this procedure is as follows: Servers decrypt an Encapsulated Request by reversing this process. Given an Encapsulated Request "enc_request", a server: <del> Parses "enc_request" into "keyID", "enc", and "ct" (indicated using the function "parse()" in pseudocode).  The server is then able to find the HPKE private key, "skR", corresponding to "keyID".  If no such key exists, the server returns an error; see errors. </del> <ins> Parses "enc_request" into "keyID", "kdfID", "aeadID", "enc", and "ct" (indicated using the function "parse()" in pseudocode).  The server is then able to find the HPKE private key, "skR", corresponding to "keyID". a.  If "keyID" does not identify a key, the server returns an error. b.  If "kdfID" and "aeadID" identify a combination of KDF and AEAD that the server is unwilling to use with "skR", the server returns an error. </ins> Compute an HPKE context using "skR" and the encapsulated key "enc", yielding "context". <del> Construct additional associated data, "aad", as the Key Identifier "keyID" from "enc_request". </del> <ins> Construct additional associated data, "aad", from "keyID", "kdfID", and "aeadID" or as the first five bytes of "enc_request". </ins> Decrypt "ct" using "aad" as associated data, yielding "request" or an error on failure.  If decryption fails, the server returns an


Question:
This came out of , where we realized that the relay padding is useful, but needs to employ different methods than client or gateway. The relay can't use binary HTTP padding; it has to insert stuff at the HTTP layer. Include citations for padding in those documents. Note that these citations are slightly misleading. Most relays won't have access at the level necessary to add that padding. More likely, the relay will need to do as NAME originally suggested, namely adding fields of an appropriate size. But we don't have good citations for that, so I'm cheaping out. (Note that RFC 9112, Section 7.1.1 talks about using chunk extensions for padding, but I'm going to pretend that doesn't exist; it's even less usable than the HTTP/2 and HTTP/3 options.)
Take it or leave it!

Padding is a capability provided by binary HTTP messages; see BINARY. If the encapsulation method described in this document is used to protect a different message type (see repurposing), that message <del> format might need to include padding support. </del> <ins> format might need to include padding support.  can also use padding for the same reason, but need to operate at the HTTP layer since they cannot manipulate binary HTTP messages; for example, see HTTP2 or HTTP3). </ins> 6.3.


Question:
Comment by NAME In How does a client know this? Preconfiguration ? Are we talking ms? sec? minutes? Any advise to implementers ?
It can't be less than 2s, because the granularity of the field doesn't permit detecting anything finer than 1s. I've done what I can.
This still does not seem to address: Maybe just cut off the “unless” part if there is no signal for this? Paul
NAME also included URL which I think should address your concern:
Thanks. That commit does address the issue, mostly. I guess it moves it do an unspecified provisioning layer :P Paul

6.5.1. SHOULD include a "Date" header field in Encapsulated Requests, unless <del> the does not use "Date" for anti-replay purposes. </del> <ins> the has prior knowledge that indicates that the does not use "Date" for anti-replay purposes. </ins> Though HTTP requests often do not include a "Date" header field, the value of this field might be used by a server to limit the amount of


Question:
Comment by NAME In How does a client know this? Preconfiguration ? Are we talking ms? sec? minutes? Any advise to implementers ?
It can't be less than 2s, because the granularity of the field doesn't permit detecting anything finer than 1s. I've done what I can.
This still does not seem to address: Maybe just cut off the “unless” part if there is no signal for this? Paul
NAME also included URL which I think should address your concern:
Thanks. That commit does address the issue, mostly. I guess it moves it do an unspecified provisioning layer :P Paul

might need to allow for the time it takes requests to arrive from the , with a time window that is large enough to allow for differences in clocks.  Insufficient tolerance of time differences could result in <del> valid requests being unnecessarily rejected. </del> <ins> valid requests being unnecessarily rejected.  Beyond allowing for multiple round trip times - to account for retransmission - network delays are unlikely to be significant in determining the size of the window, unless all potential are known to have excellent time- keeping.  A specific window size might need to be determined experimentally. </ins> MUST NOT treat the time window as secret information.  An attacker can actively probe with different values for the "Date" field to


Question:
This isn't anything that special, but this is a case where a bunch of our established defenses don't work.
The new changes LGTM. Feel free to merge assuming you're done with edits!
This means that servers need anti-replay protections, especially if they are performing non-idempotent operations.

corresponds to the key identifier, but the encapsulated request cannot be successfully decrypted using the key. <ins> 8.4. Encapsulated requests can be copied and replayed by the oblivious proxy resource.  The design of oblivious HTTP does not assume that the oblivious proxy resource will not replay requests.  In addition, if a client sends an encapsulated request in TLS early data (see Section 8 of TLS and RFC8470), a network-based adversary might be able to cause the request to be replayed.  In both cases, the effect of a replay attack and the mitigations that might be employed are similar to TLS early data. A client or oblivious proxy resource MUST NOT automatically attempt to retry a failed request unless it receives a positive signal indicating that the request was not processed or forwarded.  The HTTP/2 REFUSED_STREAM error code (Section 8.1.4 of RFC7540), the HTTP/3 H3_REQUEST_REJECTED error code (Section 8.1 of QUIC-HTTP), or a GOAWAY frame (in either protocol version) are all sufficient signals that no processing occurred.  Connection failures or interruptions are not sufficient signals that no processing occurred. The anti-replay mechanisms described in Section 8 of TLS are generally applicable to oblivious HTTP requests.  Servers can use the encapsulated keying material as a unique key for identifying potential replays. The mechanism used in TLS for managing differences in client and server clocks cannot be used as it depends on being able to observe previous interactions.  Oblivious HTTP explicitly prevents such linkability.  Applications can still include an explicit indication of time to limit the span of time over which a server might need to track accepted requests.  Clock information could be used for client identification, so reduction in precision or obfuscation might be necessary. The considerations in RFC8470 as they relate to managing the risk of replay also apply, though there is no option to delay the processing of a request. Limiting requests to those with safe methods might not be satisfactory for some applications, particularly those that involve the submission of data to a server.  The use of idempotent methods might be of some use in managing replay risk, though it is important to recognize that different idempotent requests can be combined to be not idempotent. Idempotent actions with a narrow scope based on the value of a protected nonce could enable data submission with limited replay exposure.  A nonce might be added as an explicit part of a request, or, if the oblivious request and target resources are co-located, the encapsulated keying material can be used to produce a nonce. The server-chosen "response_nonce" field ensures that responses have unique AEAD keys and nonces even when requests are replayed. </ins> 9. Please update the "Media Types" registry at


Question:
This was vague enough that an example might help.
Yeah, this seems good. Do we want to explicitly mention cookies?

resource; see key-configuration. Clients MUST NOT include identifying information in the request that <del> is encapsulated. </del> <ins> is encapsulated.  Identifying information includes cookies COOKIES, authentication credentials or tokens, and any information that might reveal client-specific information such as account credentials. </ins> Clients cannot carry connection-level state between requests as they only establish direct connections to the proxy responsible for the


Question:
This makes the choice of a 1:1 mapping clear: avoid being used as an open proxy through explicit configuration of your ACLs. cc NAME
The proxy has a fixed mapping. It's resource maps to just one request resource. That's a deliberate choice that we should probably be much more up front about.
Yes, I think this makes sense, but it took me a while to realize it when implementing. Having a section on mapping state on the proxy would be useful.
Vittorio says: This relates in particular to the potential for generic services with open destination (not a capability in the proposal) to be used for nefarious purposes (spamming, bot net command and control, hiding illegal activity, etc...).

HPKE algorithms that are used with that key. The identity of an oblivious proxy resource that will forward <del> encapsulated requests and responses to the oblivious request resource. </del> <ins> encapsulated requests and responses to a single oblivious request resource.  See proxy-state for more information about the mapping between oblivious proxy and oblivious request resources. </ins> This information allows the client to make a request of an oblivious target resource without that resource having only a limited ability


Question:
This makes the choice of a 1:1 mapping clear: avoid being used as an open proxy through explicit configuration of your ACLs. cc NAME
The proxy has a fixed mapping. It's resource maps to just one request resource. That's a deliberate choice that we should probably be much more up front about.
Yes, I think this makes sense, but it took me a while to realize it when implementing. Having a section on mapping state on the proxy would be useful.
Vittorio says: This relates in particular to the potential for generic services with open destination (not a capability in the proposal) to be used for nefarious purposes (spamming, bot net command and control, hiding illegal activity, etc...).

10. <ins> This section discusses various operational and deployment considerations. 10.1. </ins> Using Oblivious HTTP adds both cryptographic and latency to requests relative to a simple HTTP request-response exchange.  Deploying proxy services that are on path between clients and servers avoids adding


Question:
This makes the choice of a 1:1 mapping clear: avoid being used as an open proxy through explicit configuration of your ACLs. cc NAME
The proxy has a fixed mapping. It's resource maps to just one request resource. That's a deliberate choice that we should probably be much more up front about.
Yes, I think this makes sense, but it took me a while to realize it when implementing. Having a section on mapping state on the proxy would be useful.
Vittorio says: This relates in particular to the potential for generic services with open destination (not a capability in the proposal) to be used for nefarious purposes (spamming, bot net command and control, hiding illegal activity, etc...).

similar system ODoH found that deploying proxies close to servers was most effective in minimizing additional latency. <ins> 10.2. This protocol assumes a fixed, one-to-one mapping between the Oblivious Proxy Resource and the Oblivious Request Resource.  This means that any encapsulated request sent to the Oblivious Proxy Resource will always be forwarded to the Oblivious Request Resource. This constraint was imposed to simplify proxy configuration and mitigate against the Oblivious Proxy Resource being used as a generic proxy for unknown Oblivious Request Resources.  The proxy will only forward for Oblivious Request Resources that it has explicitly configured and allowed. It is possible for a server to be configured with multiple Oblivious Proxy Resources, each for a different Oblivious Request Resource as needed. 10.3. </ins> Oblivious HTTP might be incompatible with network interception regimes, such as those that rely on configuring clients with trust anchors and intercepting TLS connections.  While TLS might be


Question:
(which didn't require any non-editorial changes; the review is done now)
Also call out invariants (esp. in header structure and handshake) explicitly: v2 may/will break functionality outside the invariants.
Document is currently in in with final invariants; do one last pass to make sure everything documented in Section 17 as expose information appears here (which is basically the same check).

2.1. QUIC packets may have either a long header, or a short header.  The <del> first bit of the QUIC header indicates which type of header is present. </del> <ins> first bit of the QUIC header us the Header Form bit, and indicates which type of header is present. </ins> The long header exposes more information.  It is used during connection establishment, including version negotiation, retry, and


Question:
(which didn't require any non-editorial changes; the review is done now)
Also call out invariants (esp. in header structure and handshake) explicitly: v2 may/will break functionality outside the invariants.
Document is currently in in with final invariants; do one last pass to make sure everything documented in Section 17 as expose information appears here (which is basically the same check).

The following information is exposed in QUIC packet headers: <del> demux bit: the second most significant bit of the first octet every QUIC packet of the current version is set to 1, for demultiplexing with other UDP-encapsulated protocols. </del> <ins> "fixed bit": the second most significant bit of the first octet most QUIC packets of the current version is currently set to 1, for demultiplexing with other UDP-encapsulated protocols. </ins> latency spin bit: the third most significant bit of first octet in the short packet header.  The spin bit is set by endpoints such


Question:
(which didn't require any non-editorial changes; the review is done now)
Also call out invariants (esp. in header structure and handshake) explicitly: v2 may/will break functionality outside the invariants.
Document is currently in in with final invariants; do one last pass to make sure everything documented in Section 17 as expose information appears here (which is basically the same check).

end-to-end RTT.  See spin-usage for further details. header type: the long header has a 2 bit packet type field <del> following the Header Form bit.  Header types correspond to stages of the handshake; see Section 17.2 of QUIC-TRANSPORT. </del> <ins> following the Header Form and fixed bits.  Header types correspond to stages of the handshake; see Section 17.2 of QUIC-TRANSPORT for details. </ins> version number: the version number present in the long header, and identifies the version used for that packet.  Note that during


Question:
(which didn't require any non-editorial changes; the review is done now)
Also call out invariants (esp. in header structure and handshake) explicitly: v2 may/will break functionality outside the invariants.
Document is currently in in with final invariants; do one last pass to make sure everything documented in Section 17 as expose information appears here (which is basically the same check).

subsequent connection attempt.  The length of the token is explicit in both cases. <del> Retry and Version Negotiation packets are not encrypted or obfuscated in any way.  For other kinds of packets, other information in the packet headers is cryptographically obfuscated: </del> <ins> Retry (Section 17.2.5 of QUIC-TRANSPORT) and Version Negotiation (Section 17.2.1 of QUIC-TRANSPORT) packets are not encrypted or obfuscated in any way.  For other kinds of packets, other information in the packet headers is cryptographically obfuscated: </ins> <del> packet number: Most packets (with the exception of Version Negotiation and Retry packets) have an associated packet number; however, this packet number is encrypted, and therefore not of use to on-path observers.  The offset of the packet number is encoded in the header for packets with long headers, while it is implicit </del> <ins> packet number: All packets except Version Negotiation and Retry packets have an associated packet number; however, this packet number is encrypted, and therefore not of use to on-path observers.  The offset of the packet number is encoded in the header for packets with long headers, while it is implicit </ins> (depending on Destination Connection ID length) in short header packets.  The length of the packet number is cryptographically obfuscated.


Question:
This document should mention unidirectional streams Some words about the meaning of an abort signal vs. a clean close would be useful, both ingress and egress. Maximum stream bytes = 2^62-1. Saying "there are special streams for QUIC" is inaccurate (it's just one stream now, conceptually), and I would not bother to describe the CRYPTO frames this way.
Also: stream aborts are always unidirectional. what can/should applications assume about stream IDs? are they accessible, and does the app have any control over them?

QUIC's stream multiplexing feature allows applications to run multiple streams over a single connection, without head-of-line blocking between streams, associated at a point in time with a single <del> five-tuple.  Stream data is carried within Frames, where one (UDP) packet on the wire can carry one of multiple stream frames. Stream can be independently open and closed, gracefully or by error. If a critical stream for the application is closed, the application can generate respective error messages on the application layer to inform the other end or the higher layer and eventually indicate QUIC to reset the connection.  QUIC, however, does not need to know which streams are critical, and does not provide an interface to exceptional handling of any stream.  There are special streams in QUIC that are used for control on the QUIC connection, however, these streams are not exposed to the application. </del> <ins> five-tuple.  Stream data is carried within Frames, where one QUIC packet on the wire can carry one or multiple stream frames. Streams can be unidirectional or bidirectional, and a stream may be initiated either by client or server.  Only the initiator of a unidirectional stream can send data on it.  Due to offset encoding limitations, a stream can carry a maximum of 2^62-1 bytes in each direction.  In the presently unlikely event that this limit is reached by an application, the stream can simply be closed and replaced with a new one. Streams can be independently opened and closed, gracefully or by error.  An application can gracefully close the egress direction of a stream by instructing QUIC to send a FIN bit in a STREAM frame.  It cannot gracefully close the ingress direction without a peer- generated FIN, much like in TCP.  However, an endpoint can abruptly close either the ingress or egress direction; these actions are fully independent of each other. If a stream that is critical for an application is closed, the application can generate respective error messages on the application layer to inform the other end and/or the higher layer, and eventually indicate QUIC to reset the connection.  QUIC, however, does not need to know which streams are critical, and does not provide an interface for exceptional handling of any stream. </ins> Mapping of application data to streams is application-specific and <del> described for HTTP/s in QUIC-HTTP.  In general data that can be </del> <ins> described for HTTP/3 in QUIC-HTTP.  In general, data that can be </ins> processed independently, and therefore would suffer from head of line blocking if forced to be received in order, should be transmitted <del> over different streams.  If the application requires certain data to be received in order, the same stream should be used for that data. </del> <ins> over separate streams.  If the application requires certain data to be received in order, that data should be sent on the same stream. </ins> If there is a logical grouping of data chunks or messages, streams can be reused, or a new stream can be opened for each chunk/message. If one message is mapped to a single stream, resetting the stream to


Question:
This document should mention unidirectional streams Some words about the meaning of an abort signal vs. a clean close would be useful, both ingress and egress. Maximum stream bytes = 2^62-1. Saying "there are special streams for QUIC" is inaccurate (it's just one stream now, conceptually), and I would not bother to describe the CRYPTO frames this way.
Also: stream aborts are always unidirectional. what can/should applications assume about stream IDs? are they accessible, and does the app have any control over them?

currently open and currently used streams to the application to make the mapping of data to streams dependent on this information. <del> Further, streams have a maximum number of bytes that can be sent on one stream.  This number is high enough (2^64) that this will usually not be reached with current applications.  Applications that send chunks of data over a very long period of time (such as days, months, or years), should rather utilize the 0-RTT session resumption ability provided by QUIC, than trying to maintain one connection open. </del> <ins> While a QUIC implementation must necessarily provide a way for an application to send data on separate streams, it does not necessarily expose stream identifiers to the application (see e.g.  QUIC-HTTP section 6) either at the sender or receiver end, so applications should not assume access to these identifiers. </ins> 4.1.


Question: However, we also use the terms Server/Client Hello datagram to cover all QUIC Packets of the first two datagrams sent. This term is not used in the transport draft (anymore) and I find it confusing given the TLS terminology. Should we maybe use a different term?

frames. The Client Hello datagram exposes version number, source and <del> destination connection IDs, and information in the TLS Client Hello message, including any TLS Server Name Indication (SNI) present, in the clear.  The QUIC PADDING frame shown here may be present to ensure the Client Hello datagram has a minimum size of 1200 octets, to mitigate the possibility of handshake amplification.  Note that the location of PADDING is implementation-dependent, and PADDING frames may not appear in the Initial packet in a coalesced packet. The Server Hello datagram exposes version number, source and destination connection IDs, and information in the TLS Server Hello message. </del> <ins> destination connection IDs in the clear.  Information in the TLS Client Hello frame, including any TLS Server Name Indication (SNI) present, is obfuscated using the Initial secret.  The QUIC PADDING frame shown here may be present to ensure the Client Hello datagram has a minimum size of 1200 octets, to mitigate the possibility of handshake amplification.  Note that the location of PADDING is implementation-dependent, and PADDING frames may not appear in the Initial packet in a coalesced packet. The Server Hello datagram also exposes version number, source and destination connection IDs and information in the TLS Server Hello message which is obfuscated using the Initial secret. </ins> The Initial Completion datagram does not expose any additional information; however, recognizing it can be used to determine that a


Question:
addresses
In "Connection ID and Rebinding": "Even when encrypted, a scheme could embed the unencrypted length of the Connection ID in the Connection ID itself, instead of remembering it, e.g. by using the first few bits to indicate a certain size of a well-known set of possible sizes with multiple values that indicate the same size but are selected randomly." I almost went and filed an issue against quic-lb to do this instead of just encoding a plaintext length like it does currently. But is this a good recommended practice? ISTM that long headers would provide a bit of a Rosetta Stone for this mapping, making it not particularly effective for the complexity.
The idea was not to conceal the length but to avoid likability by identifying linked connection based on the existence of a certain length field. This might look slightly more random but I guess you also have to change your mapping more time to time. So not sure. I'm fine to remove that half sentence and maybe add a sentence about likability instead...?
Yes, I would prefer deleting the quoted text, as I think it's a solution that adds complexity for little apparent benefit.
Okay, will do that change after your PR has landed.

The best way to obscure an encoding is to appear random to observers, which is most rigorously achieved with encryption.  Even when encrypted, a scheme could embed the unencrypted length of the <del> Connection ID in the Connection ID itself, instead of remembering it, e.g. by using the first few bits to indicate a certain size of a well-known set of possible sizes with multiple values that indicate the same size but are selected randomly. </del> <ins> Connection ID in the Connection ID itself, instead of remembering it. </ins> QUIC_LB further specified possible algorithms to generate Connection IDs at load balancers.


Question:
Re SNI and ALPN: we can certainly point at the work in progress in TLS that will encrypt this information, but IMO the document should describe the current state of the world. I've just added something pointing to the SNI encryption document; while it seems that the tunneling technique described there could pretty clearly be applied to ALPN fronting as well, is there a reference for encrypting ALPN we can use here that doesn't leave details as an exercise for the reader?
I would actually prefer to describe the wire image of TLS and associated protocols in a separate document and just point to it because that's actually not only applicable for quic
I don't have a reference for encrypting ALPN. We don't have one. Maybe I should write one... On the plus side, it's something that benefits from the same techniques as SNI (and less likely to cause deployment issues, though that is only because we haven't ossified around it as much). As for the general point about now vs. later. One of the things we've tried to do is be very clear about what the invariants are, or what promises we are making. It's not the same thing as true ossification defense, but if we make it clear what we are committing to, maybe we won't have people relying on stuff we might want to change.

3.2. <del> Connection establishment requires cleartext packets and is using a TLS handshake on stream 0.  Therefore it is detectable using heuristics similar to those used to detect TLS over TCP. 0-RTT connection may additional also send data packets, right after the Client Initial with the TLS client hello.  These data may be reordered in the network, therefore it may be possible that 0-RTT Protected data packets are seen before the Client Initial packet. </del> <ins> Connection establishment uses Initial, Handshake, and Retry packets containing a TLS handshake on Stream 0.  Connection establishment can therefore be detected using heuristics similar to those used to detect TLS over TCP.  A client using 0-RTT connection may also send data packets in 0-RTT Protected packets directly after the Initial packet containing the TLS Client Hello.  Since these packets may be reordered in the network, note that 0-RTT Protected data packets may be seen before the Initial packet.  Note that only clients send Initial packets, so the sides of a connection can be distinguished by QUIC packet type in the handshake. </ins> 3.3. <ins> The cleartext TLS handshake may contain Server Name Indication (SNI) RFC6066, by which the client reveals the name of the server it intends to connect to, in order to allow the server to present a certificate based on that name.  It may also contain information from Application-Layer Protocol Negotiation (ALPN) RFC7301, by which the client exposes the names of application-layer protocols it supports; an observer can deduce that one of those protocols will be used if the connection continues. Work is currently underway in the TLS working group to encrypt the SNI in TLS 1.3 TLS-ENCRYPT-SNI, reducing the information available in the SNI to the name of a fronting service, which can generally be identified by the IP address of the server anyway.  If used with QUIC, this would make SNI-based application identification impossible through passive measurement. 3.4. </ins> The QUIC Connection ID (see rebinding) is designed to allow an on- path device such as a load-balancer to associate two flows as identified by five-tuple when the address and port of one of the


Question:
Re SNI and ALPN: we can certainly point at the work in progress in TLS that will encrypt this information, but IMO the document should describe the current state of the world. I've just added something pointing to the SNI encryption document; while it seems that the tunneling technique described there could pretty clearly be applied to ALPN fronting as well, is there a reference for encrypting ALPN we can use here that doesn't leave details as an exercise for the reader?
I would actually prefer to describe the wire image of TLS and associated protocols in a separate document and just point to it because that's actually not only applicable for quic
I don't have a reference for encrypting ALPN. We don't have one. Maybe I should write one... On the plus side, it's something that benefits from the same techniques as SNI (and less likely to cause deployment issues, though that is only because we haven't ossified around it as much). As for the general point about now vs. later. One of the things we've tried to do is be very clear about what the invariants are, or what promises we are making. It's not the same thing as true ossification defense, but if we make it clear what we are committing to, maybe we won't have people relying on stuff we might want to change.

should be treated as opaque; see sec-loadbalancing for caveats regarding connection ID selection at servers. <del> 3.4. </del> <ins> 3.5. </ins> The QUIC does not expose the end of a connection; the only indication to on-path devices that a flow has ended is that packets are no


Question:
Re SNI and ALPN: we can certainly point at the work in progress in TLS that will encrypt this information, but IMO the document should describe the current state of the world. I've just added something pointing to the SNI encryption document; while it seems that the tunneling technique described there could pretty clearly be applied to ALPN fronting as well, is there a reference for encrypting ALPN we can use here that doesn't leave details as an exercise for the reader?
I would actually prefer to describe the wire image of TLS and associated protocols in a separate document and just point to it because that's actually not only applicable for quic
I don't have a reference for encrypting ALPN. We don't have one. Maybe I should write one... On the plus side, it's something that benefits from the same techniques as SNI (and less likely to cause deployment issues, though that is only because we haven't ossified around it as much). As for the general point about now vs. later. One of the things we've tried to do is be very clear about what the invariants are, or what promises we are making. It's not the same thing as true ossification defense, but if we make it clear what we are committing to, maybe we won't have people relying on stuff we might want to change.

Changes to this behavior are currently under discussion: see https://github.com/quicwg/base-drafts/issues/602. <del> 3.5. </del> <ins> 3.6. </ins> Round-trip time of QUIC flows can be inferred by observation once per flow, during the handshake, as in passive TCP measurement; this


Question:
Re SNI and ALPN: we can certainly point at the work in progress in TLS that will encrypt this information, but IMO the document should describe the current state of the world. I've just added something pointing to the SNI encryption document; while it seems that the tunneling technique described there could pretty clearly be applied to ALPN fronting as well, is there a reference for encrypting ALPN we can use here that doesn't leave details as an exercise for the reader?
I would actually prefer to describe the wire image of TLS and associated protocols in a separate document and just point to it because that's actually not only applicable for quic
I don't have a reference for encrypting ALPN. We don't have one. Maybe I should write one... On the plus side, it's something that benefits from the same techniques as SNI (and less likely to cause deployment issues, though that is only because we haven't ossified around it as much). As for the general point about now vs. later. One of the things we've tried to do is be very clear about what the invariants are, or what promises we are making. It's not the same thing as true ossification defense, but if we make it clear what we are committing to, maybe we won't have people relying on stuff we might want to change.

Changes to this behavior are currently under discussion: see https://github.com/quicwg/base-drafts/issues/631. <del> 3.6. </del> <ins> 3.7. </ins> All QUIC packets carry packet numbers in cleartext, and while the protocol allows packet numbers to be skipped, skipping is not


Question:
Re SNI and ALPN: we can certainly point at the work in progress in TLS that will encrypt this information, but IMO the document should describe the current state of the world. I've just added something pointing to the SNI encryption document; while it seems that the tunneling technique described there could pretty clearly be applied to ALPN fronting as well, is there a reference for encrypting ALPN we can use here that doesn't leave details as an exercise for the reader?
I would actually prefer to describe the wire image of TLS and associated protocols in a separate document and just point to it because that's actually not only applicable for quic
I don't have a reference for encrypting ALPN. We don't have one. Maybe I should write one... On the plus side, it's something that benefits from the same techniques as SNI (and less likely to cause deployment issues, though that is only because we haven't ossified around it as much). As for the general point about now vs. later. One of the things we've tried to do is be very clear about what the invariants are, or what promises we are making. It's not the same thing as true ossification defense, but if we make it clear what we are committing to, maybe we won't have people relying on stuff we might want to change.

observation point and the receiver ("downstream loss") cannot be reliably estimated. <del> 3.7. </del> <ins> 3.8. </ins> QUIC explicitly exposes which side of a connection is a client and which side is a server during the handshake.  In addition, the


Question:
The point of is a little odd, because it seems to imply that ALL operations require access to the information that is integrity- or confidentiality-protected. That's clearly not the case, I think, as it is merely pointing out that if you want to modify data, then you have to be an endpoint. I think that this might be better phrased as: Note: This drops the last sentence of the original paragraph. If proxying is out of scope, then it doesn't help to talk about it so much. The point about cooperation should do.
Created a PR based on your proposed text but added some of the information about proxies to make it more clear what kind of proxies we are talking about.

and a description of how common network management practices will be impacted by QUIC. <del> Since QUIC's wire image WIRE-IMAGE is integrity protected and not modifiable on path, in-network operations are not possible without terminating the QUIC connection, for instance using a back-to-back proxy.  Proxy operations are not in scope for this document.  A proxy can either explicit identify itself as providing a proxy service, or may share the TLS credentials to authenticate as the server and (in some cases) client acting as a front-facing instance for the endpoint itself. </del> <ins> Since QUIC's wire image WIRE-IMAGE is integrity-protected, in-network operations that depend on modification of data are not possible without the cooperation of an endpoint.  Network operation practices that alter data are only possible if performed as a QUIC endpoint; this might be possible with the introduction of a proxy which authenticates as an endpoint.  Proxy operations are not in scope for this document. </ins> Network management is not a one-size-fits-all endeavour: practices considered necessary or even mandatory within enterprise networks


Question:
This is a proposal by Gorry to address issue
This works for me.
Should TAPS be cited as an informational ref to an API that is being discussed by the IETF, and which could apply to QUIC - at least to define the primitives needed in an API, and has quite some insight into protocol racing/fallback as described in applicability?
I'd be happy to do so ;-) however we avoided in the end to talk to much about specific of the interfaces or needed nobs. So not sure it fits or where to fit it.... where would you want to say something?
I think the WG should consider writing a short section on this. It's important enough that we have a WG developing relevant specs for how to build APIs and the need for this type of fallback/racing.
I'm not sure the QUIC WG has enough experience to say anything on the matter. The fallback discussion is about how (Application Mapping + QUIC) could fallback to (alternative application mapping + TCP). For HTTP/3 this isn't a straight fallback - if QUIC fails then a different HTTP version over TCP is required. This has consequences for applications that use HTTP/3 as a substrate (MASQUE, WebTransport) which rely on features only it provides like unreliable DATAGRAM. I think the Applicability spec might need to say more here. What I'm trying to highlight is that there isn't a straight shootout between TCP and QUIC. If TAPS frames things like that then I don't know that it helps Applicability.
taps allows you to use the same interface for multiple transports and also provides you a way to state requirements (hard or soft) to restricts the selection of possible transports. In that sense for some application TCP and QUIC might be not make a real difference as long as all features requested are support, e.g reliability and in-order delivery. Still not sure what/if we want to say something in applicability statement...?
NAME we were trying to not focus on http in this document. But I do agree that the fact that there is a different version needed for http to fallback could still be interesting. How about adding as sentence like the add the end of the first paragraph in section 2: OLD In the case of HTTP, this fallback is TLS 1.3 over TCP. NEW In the case of HTTP, this fallback is TLS 1.3 over TCP. Note that for HTTP a fallback to TCP also requires the use of a different version of HTTP, as HTTP/3 has been optimised for use with QUIC, e.g., by removing multiplexing from the HTTP layer and used the provided mechanism within QUIC. We could even add something more like: "Other applications that are optimised for QUIC might need similar adaption in order to be able to function on TCP." However, not sure that adds much... What do you think?
I'm still not sure that helps. And I'm trying not to derail Gorry's OP. The main grounding I have in my understanding is HTTP. But I appreciate the Applicability document is general purpose, so apologies. Many HTTP applications operate at the semantic level. HTTP semantics are version-independent and get profiled into versions that have a wire-syntax, which depends on different properties of a transport. This could be an optimised use of QUIC features, or it could be an outright requirement for those features. There isn't an interoperable notion of "I want to do HTTP/1.1 over a reliable in-order transport, TAPS selected QUIC for me, great I'll use that" nor "I want to do HTTP/3 over a reliable, multiplexed, in-order transport, TAPS selected SCTP for me". The transport fallback behaviour for HTTP semantics is quite straightforward. Pick a different HTTP version. This could lend itself to a race like scenario. However, there are applications that use HTTP as a substrate and things like MASQUE and WebTransport place requirements on the use of features only available in HTTP/3 - I doubt anyone will try to backport features of QUIC to TCP in order to satisfy the requirements of layers above.
I was thinking of something like: /The IETF TAPS specifications [URL] describe a system in which multiple protocols can be provided below a common API. and describe some of the implications of fall back, which specifically precludes fallback to insecure protocols or to weaker versions of secure protocols. / Perhaps inserted after this: /These applications must operate, perhaps with impaired functionality, in the absence of features provided by QUIC not present in the fallback protocol./ ... and before mentioning fallback to TLS over TCP.
NAME you are right that QUIC provides features that TCP doesn't provide and if you use those features you cannot just straight-away replace one with the other. However, a simple one-to-one replacement not implicated when talking about falling back. Rather than trying to back-port features to TCP, what you need to do is implement that feature in your application instead (as H2 does for multiplexing). But this is also true if you try to use TCP and fall back to UDP but need reliability; then you have to implement it in the application layer. I was trying to be brief in my proposed text and maybe the word "optimise" is not the right one. We can of course say more. Do you want to give it a try and propose some text if you think it would be helpful to say more?
Thanks. I'll have a think on some text. The important thing is that H2 wasn't designed as a fallback for H3, it is its own thing. There are few features that an application desires that it couldn't also achieve by just using HTTP/1.1 and multiple connections for multiplexing.
May I ask what these features are? E.g., the ability to send unreliable datagrams should not matter in the best effort Internet (reliable transfer, when not needed, is just slower). In RFC 8923, we made an effort to identify exactly what MUST be exposed to an application because there's no such fall back. This covers TCP, MPTCP, UDP, UDP-Lite, SCTP and LEDBAT; alas, it excludes QUIC, since this work pre-dates its finalization (by far, despite the date: it spent a long time in the RFC Editor queue). So now I'm curious what QUIC adds to this list.
mainly multiplexing I would say but that probably covered by SCTP
The main feature that QUIC provides to the HTTP substrate is that it is a secure transport that can work on the Internet and is easily integrated with WebPKI. These are meta-features in a way. Having parity of the deployment and security models allows more straightforward rationalisation when designing applications and e.g. Web APIs on top of this substrate.
Thanks! About muxing: is a certain prioritization behavior guaranteed (not just a wish) when muxing with QUIC? SCTP could do that but we don't currently give such guarantees in the TAPS interface. About security: right... that's what I expected. TAPS isn't going to fall back to a less secure version of what you wish for... so I guess "mostly encrypted header" isn't a part of any API but as you say it's a meta-feature - it's implicit in the choice of QUIC, and when that's desired, falling back wouldn't happen, a TAPS system should always only give you QUIC. I think what's missing is a QUIC API that makes these wishes explicit...
Personally I'd say, for an HTTP application, that QUIC packet header protection is less of a concern. More that QUIC provides transport security on-par with TCP+TLS. Often an application can use the same TLS and crypto libraries for both. Fallback from QUIC to TCP can be a downgrade. But an application that does this can (and should) be aware of the security properties of the transport it is using. With HTTP, imagine a client that connects to URL and receives . This advertises the availability of QUIC and HTTP/3. If that transport connection fails for some reason, then there is not a viable fallback to TCP+TLS. This is an unlikely scenario but the specs allow for it to happen.
I would also have thought that encrypting data is more important - but surely having fall-back "under the hood" and just being aware of what you got (this can be queried in TAPS) is nicer than having to take care of it in the application. Fall-backs can get complex: e.g., when multi-homing, considering IPv6 ... a TAPS system can have a rich set to choose from and implement it all efficiently while shielding applications from this trouble - the application is still in control of things and can express limitations, as well as query what it got in case it did give the system some freedom. This HTTP case of preventing a fall-back... well, it is what it is :) No fall-back in this case, clearly.
I remain to be convinced that TAPS API will address the way that applications (I am familiar with) intend to use and interact with QUIC.
I think we're drifting towards a discussion of should a mapping document be written for TAPS to QUIC... which is a different topic. My observation was only that TAPS architecture is something the IETF is working upon, and it has documents that describe how to make an API that with functions and racing/fallback in a very different way to sockets. The guidance about what is an acceptable "alternative" connection appear common, at least enough to say there is a similarity and cite it.
The suggestion for a short informational reference seems good to me.
There is already PR
We merged PR . Is there anything we want to add or can this issue be closed?
The merged PR addresses all of my concerns. I'll let NAME be the judge whether his original issue has been addressed.
That looked good to me.
Thanks all!
o/ lgtm

fall back to some other transport protocol.  In the case of HTTP, this fallback is TLS 1.3 over TCP. <ins> The IETF TAPS specifications I-D.ietf-taps-arch describe a system with a common API for multiple protocols and some of the implications of fallback between these different protocols, specifically precluding fallback to insecure protocols or to weaker versions of secure protocols. </ins> An application that implements fallback needs to consider the security consequences.  A fallback to TCP and TLS 1.3 exposes control information to modification and manipulation in the network.  Further


Question:
Thanks to Mike for noticing this.
This is true, but there is nothing inherent to the design that prevents RESETSTREAM (and STOPSENDING) from using a different code space from CONNECTION_CLOSE. Though we have the policy in HTTP/3 that any stream error can become a connection error, the same isn't necessarily true of other protocols. Maybe we cab just avoid mentioning this possibility..,
I don't think I have a strong opinion on removing or keeping this sentence. Would it be any better if we say: "Application error codes are recommended to be defined from a single space that applies to all three frame types." ? Or is there any other guidance we can or want to give to application designer about the use of error codes?

The CONNECTION_CLOSE frame provides an optional reason field, that can be used to append human-readable information to an error code. <del> Note that QUIC RESET_STREAM and STOP_SENDING frames also include an error code, but no reason string.  Application error codes are expected to be defined from a single space that applies to all three frame types. </del> <ins> RESET_STREAM and STOP_SENDING frames also include an error code, but no reason string. </ins> Alternatively, a QUIC connection can be silently closed by each endpoint separately after an idle timeout.  If enabled as indicated


Question:
proposed by NAME
Plan is to update the PR to point to the IANA registry instead.
Added a ref to the iana registration sec in the transport draft. Should be ready to go now!

and identifies the version used for that packet.  During Version Negotiation (see version and Section 17.2.1 of QUIC-TRANSPORT), the version number field has a special value (0x00000000) that <del> identifies the packet as a Version Negotiation packet.  Upon time of publishing of this document, QUIC versions that start with 0xff implement IETF drafts.  QUIC version 1 uses version 0x00000001. Operators should expect to observe packets with other version numbers as a result of various Internet experiments and future standards. </del> <ins> identifies the packet as a Version Negotiation packet.  QUIC version 1 uses version 0x00000001.  Operators should expect to observe packets with other version numbers as a result of various Internet experiments, future standards, and greasing.  All deployed versions are maintained in an IANA registry (see Section 22.2 of QUIC-TRANSPORT). </ins> source and destination connection ID: short and long packet headers carry a destination connection ID, a variable-length field


Question:
This mainly wiggles the existing text around and tries to fully qualify transport vs application to help explain the ways an implementation might do things, which is independent of how an application protocol might be designed (for example H3 doesn't require incremental processing but most people come to a realizatopm that it is the best way to do things).
The new text is good and clear, but it doesn't really explain that the when credit is released at the receiver, could we say a little more: it takes 1/2 RTT for cerdit to reach the sender, that causes new data to be released, for a large RTT, this means that credit has to be sufficient to avoid blocking or to be sent promptly, otherwise transfers will stall for periods of an RTT. (I know TCP does this stuff for the app and grows the window, but QUIC I suspect needs something to happen to release the credit). I know people closer to the Apps can write this sentence better than me; from my perspective I just see the effects when an app blocks or release credit in big chunks... over a large path RTT and the network performance suffers.
Thanks Gorry. I think I agree. How about I tweak "Timely updates can improve performance" paragraph to say "Timing of updates can affect performance". I can add a mention that the extension of credits on the data recever side is enacted by emitting MAXDATA/MAXSTREAM_DATA frames, which take at least a 1/2 RTT to reach the data sender side. I think the useful additional applicability statement to be made based on the above is that the data consumer read rate may be disjoint from the delivery rate and responsiveness of signalling channel back. For a highly asymmetric link, the download rate can be application-limited well below the theoretical bandwidth simply because an application is spending too much share of the time reading packets and not giving enough time to emit QUIC packets. Does that sound ok, or am I missing your ask?
This is definitely heading in the direction I was hoping, it helps avoid people developing on a LAN and then discovering the effects on other paths later. Please add something.
I've reworked the paragraph with my attempt to address NAME comment. I've introduced the terms uplink and downlink, which I'm not totally happy with. If someone has a better suggestion I'm all ears.
Issue: Sect 4.3. Flow Control - we should have a subsection also encouraging timely credits This ID discusses flow control deadlock, whichI think is important. I suggest we should also note that timely release of flow control credit is also important (QUIC transport has some text to point to). This need is evident when the path has a large RTT, and there can be a large BDP, where the whole of operation of QUIC can be dominated by the way in which the application manages the flow credit. This might not be obvious, because the equivalent rwnd update in TCP is handled by the transport rather than the application.
The point is the audience for this draft is applications that use a quic library. I would assume that the actual credit handling is performed in the transport/in quic directly, usually without an interface for the application to directly impact it. Do you propose the application should have more direct impact on flow credit handling? I guess we can add a pointer to the right section in the transport draft but not sure we should/can say more here...?
I agree this seems mostly like a QUIC transport/library issue and I wouldn't expect the application to have much control here. I can imagine applications setting a max value?
I generally agree with Mirja and Ian. However, on a closer inspection of this section I think it would benefit from a crisper definition of what it means to read data vs. process data. Its a reasonable application mapping implementation design to read data out of the transport in order to release flow control but still buffer it (or just dropping it on the floor) in application space before processing that as application data.
NAME can you propose text?
I'm content that flow control is something the libraries need to get correct - the TCP people spent energy to get this automated, although QUIC's multi-streaming makes the trickier. Lucas text suggestion would improve things by making it clearer how the API is intended to be used.
NAME ?
I will add some text (just need to find the time).

4.4. <del> Flow control provides a means of managing access to the limited </del> <ins> QUIC flow control provides a means of managing access to the limited </ins> buffers endpoints have for incoming data.  This mechanism limits the amount of data that can be in buffers in endpoints or in transit on the network.  However, there are several ways in which limits can


Question:
This mainly wiggles the existing text around and tries to fully qualify transport vs application to help explain the ways an implementation might do things, which is independent of how an application protocol might be designed (for example H3 doesn't require incremental processing but most people come to a realizatopm that it is the best way to do things).
The new text is good and clear, but it doesn't really explain that the when credit is released at the receiver, could we say a little more: it takes 1/2 RTT for cerdit to reach the sender, that causes new data to be released, for a large RTT, this means that credit has to be sufficient to avoid blocking or to be sent promptly, otherwise transfers will stall for periods of an RTT. (I know TCP does this stuff for the app and grows the window, but QUIC I suspect needs something to happen to release the credit). I know people closer to the Apps can write this sentence better than me; from my perspective I just see the effects when an app blocks or release credit in big chunks... over a large path RTT and the network performance suffers.
Thanks Gorry. I think I agree. How about I tweak "Timely updates can improve performance" paragraph to say "Timing of updates can affect performance". I can add a mention that the extension of credits on the data recever side is enacted by emitting MAXDATA/MAXSTREAM_DATA frames, which take at least a 1/2 RTT to reach the data sender side. I think the useful additional applicability statement to be made based on the above is that the data consumer read rate may be disjoint from the delivery rate and responsiveness of signalling channel back. For a highly asymmetric link, the download rate can be application-limited well below the theoretical bandwidth simply because an application is spending too much share of the time reading packets and not giving enough time to emit QUIC packets. Does that sound ok, or am I missing your ask?
This is definitely heading in the direction I was hoping, it helps avoid people developing on a LAN and then discovering the effects on other paths later. Please add something.
I've reworked the paragraph with my attempt to address NAME comment. I've introduced the terms uplink and downlink, which I'm not totally happy with. If someone has a better suggestion I'm all ears.
Issue: Sect 4.3. Flow Control - we should have a subsection also encouraging timely credits This ID discusses flow control deadlock, whichI think is important. I suggest we should also note that timely release of flow control credit is also important (QUIC transport has some text to point to). This need is evident when the path has a large RTT, and there can be a large BDP, where the whole of operation of QUIC can be dominated by the way in which the application manages the flow credit. This might not be obvious, because the equivalent rwnd update in TCP is handled by the transport rather than the application.
The point is the audience for this draft is applications that use a quic library. I would assume that the actual credit handling is performed in the transport/in quic directly, usually without an interface for the application to directly impact it. Do you propose the application should have more direct impact on flow credit handling? I guess we can add a pointer to the right section in the transport draft but not sure we should/can say more here...?
I agree this seems mostly like a QUIC transport/library issue and I wouldn't expect the application to have much control here. I can imagine applications setting a max value?
I generally agree with Mirja and Ian. However, on a closer inspection of this section I think it would benefit from a crisper definition of what it means to read data vs. process data. Its a reasonable application mapping implementation design to read data out of the transport in order to release flow control but still buffer it (or just dropping it on the floor) in application space before processing that as application data.
NAME can you propose text?
I'm content that flow control is something the libraries need to get correct - the TCP people spent energy to get this automated, although QUIC's multi-streaming makes the trickier. Lucas text suggestion would improve things by making it clearer how the API is intended to be used.
NAME ?
I will add some text (just need to find the time).

Understanding what causes deadlocking might help implementations avoid deadlocks. <del> Large messages can produce deadlocking if the recipient does not process the message incrementally.  If the message is larger than the flow control credit available and the recipient does not release additional flow control credit until the entire message is received and delivered, a deadlock can occur.  This is possible even where stream flow control limits are not reached because connection flow control limits can be consumed by other streams. A common flow control implementation technique is for a receiver to extend credit to the sender as a the data consumer reads data.  In this setting, a length-prefixed message format makes it easier for the data consumer to leave data unread in the receiver's buffers and thereby withhold flow control credit.  If flow control limits prevent the remainder of a message from being sent, a deadlock will result. A length prefix might also enable the detection of this sort of deadlock.  Where protocols have messages that might be processed as a </del> <ins> The size and rate of transport flow control credit updates can affect performance.  Applications that use QUIC often have a data consumer that reads data from transport buffers.  Some implementations might have independent transport-layer and application-layer receive buffers.  Consuming data does not always imply it is immediately processed.  However, a common flow control implementation technique is to extend credit to the sender, by emitting MAX_DATA and/or MAX_STREAM_DATA frames, as data is consumed.  Delivery of these frames is affected by the latency of the back channel from the receiver to the data sender.  If credit is not extended in a timely manner, the sending application can be blocked, effectively throttling the sender. Large application messages can produce deadlocking if the recipient does not read data from the transport incrementally.  If the message is larger than the flow control credit available and the recipient does not release additional flow control credit until the entire message is received and delivered, a deadlock can occur.  This is possible even where stream flow control limits are not reached because connection flow control limits can be consumed by other streams. A length-prefixed message format makes it easier for a data consumer to leave data unread in the transport buffer and thereby withhold flow control credit.  If flow control limits prevent the remainder of a message from being sent, a deadlock will result.  A length prefix might also enable the detection of this sort of deadlock.  Where application protocols have messages that might be processed as a </ins> single unit, reserving flow control credit for the entire message atomically makes this style of deadlock less likely. <del> A data consumer can read all data as it becomes available to cause the receiver to extend flow control credit to the sender and reduce the chances of a deadlock.  However, releasing flow control credit might mean that the data consumer might need other means for holding a peer accountable for the state it keeps for partially processed messages. </del> <ins> A data consumer can eagerly read all data as it becomes available, in order to make the receiver extend flow control credit and reduce the chances of a deadlock.  However, such a data consumer might need other means for holding a peer accountable for the additional state it keeps for partially processed messages. </ins> Deadlocking can also occur if data on different streams is interdependent.  Suppose that data on one stream arrives before the


Question: Based on the discussion in webtrans at IETF-110, I guess we could be more explicit that QUIC streams do not guarantee to be received in any particular order.

A single stream provides ordering.  If the application requires certain data to be received in order, that data should be sent on <del> the same stream. </del> <ins> the same stream.  There is no guarantee of transmission, reception, or delivery order across streams. </ins> Multiple streams provide concurrency.  Data that can be processed independently, and therefore would suffer from head of line


Question:
It could be worth noting two general further things about DSCPs : (a) "When multiplexing multiple flows over a QUIC connection, the DSCP value selected should be the one associated with the highest priority requested for all multiplexed flows." (b) "If a packet enters a network segment that does not support the DSCP value, this could result in the connection not receiving the network treatment it expects. The DSCP value in this packet could also be remarked as the packet travels along the network path."
A sample PR: "More aspects of setting DSCPs"

use of DiffServ Code Points (DSCPs) RFC2475 as well as Equal-Cost Multi-Path (ECMP) routing, is applied on a per flow-basis (and not per-packet) and as such that all packets belonging to the same QUIC <del> connection get uniform treatment.  Using ECMP to distribute packets from a single flow across multiple network paths or any other non- uniform treatment of packets belong to the same connection could result in variations in order, delivery rate, and drop rate.  As feedback about loss or delay of each packet is used as input to the congestion controller, these variations could adversely affect performance. Depending on the loss recovery mechanism implemented, QUIC may be more tolerant of packet re-ordering than traditional TCP traffic (see packetnumber).  However, it cannot be known by the network which exact recovery mechanism is used and therefore reordering tolerance should be considered as unknown. </del> <ins> connection get uniform treatment. Using ECMP to distribute packets from a single flow across multiple network paths or any other non-uniform treatment of packets belong to the same connection could result in variations in order, delivery rate, and drop rate.  As feedback about loss or delay of each packet is used as input to the congestion controller, these variations could adversely affect performance.  Depending on the loss recovery mechanism implemented, QUIC may be more tolerant of packet re- ordering than traditional TCP traffic (see packetnumber).  However, the recovery mechanism used by a flow cannot be known by the network and therefore reordering tolerance should be considered as unknown. </ins> 4.9.


Question:
Separates for manageability and this PR for applicability
I actually find this sentence weak: "Therefore it is not recommended to use different DiffServ Code Points (DSCPs) {{?RFC2475}} for ... Considering that QUIC v1 as defined has a single congestion controller and recovery handler and don't treat different DSCP values as separate transport flows I would think that this should be stronger formulated. However, I could see that one could provide a bit more context to this DSCP and single QUIC connections vs multiple QUIC streams discussion by referencing Section 5.1 of URL and be explicit about the point that QUIC v1 is similar to SCTP in regards to the issues raised.
I think the new text do work.

12. <del> QUIC assumes that all packets of a QUIC connection, or at least with the same 5-tuple {dest addr, source addr, protocol, dest port, source port}, will receive similar network treatment since feedback about loss or delay of each packet is used as input to the congestion controller.  Therefore, it is not recommended to use different DiffServ Code Points (DSCPs) RFC2475 for packets belonging to the same connection.  If differential network treatment, e.g. by the use of different DSCPs, is desired, multiple QUIC connections to the same server may be used.  However, in general it is recommended to minimize the number of QUIC connections to the same server, to avoid increased overheads and, more importantly, competing congestion control. </del> <ins> QUIC, as defined in RFC9000, has a single congestion controller and recovery handler.  This design assumes that all packets of a QUIC connection, or at least with the same 5-tuple {dest addr, source addr, protocol, dest port, source port} that same the same DiffServ Code Point (DSCP) RFC2475, will receive similar network treatment since feedback about loss or delay of each packet is used as input to the congestion controller.  Therefore, packets belonging to the same connection should use a single DSCP.  Section 5.1 of RFC7657 provides a discussion of DiffServ interactions with datagram transport protocols RFC7657 (in this respect the interactions with QUIC resemble those of SCTP). When multiplexing multiple flows over a single QUIC connection, the selected DSCP value should be the one associated with the highest priority requested for all multiplexed flows. If differential network treatment is desired, e.g., by the use of different DSCPs, multiple QUIC connections to the same server may be used.  However, in general it is recommended to minimize the number of QUIC connections to the same server, to avoid increased overhead and, more importantly, competing congestion control. As in other uses of DiffServ, when a packet enters a network segment that does not support the DSCP value, this could result in the connection not receiving the network treatment it expects.  The DSCP value in this packet could also be remarked as the packet travels along the network path, changing the requested treatment. </ins> 13.


Question:
Suggesting a rewording to clarify various issues and correct some mistakes.
Previously scope of this text was agreed, but what appeared seems to me to have various NiTs: • Saying "every other packet is acknowledged" is a recommendation in RC9000. I suggest citing also. • The text seems to conflate endpoint processing optimisation, and network path performance optimisation: /constraint/ wasn't the correct word, but even /constrained/ was a value judgment on design traedoffs for any mobile/RF link - and many bandwidth-limited cabled link, which could be avoided by simply saying what is intended and saying either /which can impact performance of some types of network/ or /which can impact performance across some types of network/ ... depending on your viewpoint as an operator or application. • I think we can do better than URL, suggesting removing some and leaving might. This might be resolved in a PR .

7. QUIC version 1 without extensions uses an acknowledgment strategy <del> adopted from TCP.  That is, every other packet is acknowledged. However, generating and processing QUIC acknowledgments can consume significant resources, both in terms of processing costs and link utilization, especially on constraint networks.  Some applications might be able to improve overall performance by using alternative strategies that reduce the rate of acknowledgments. </del> <ins> adopted from TCP Section 13.2 of RFC9000).  That is, it recommends every other packet is acknowledged.  However, generating and processing QUIC acknowledgments consumes resources at a sender and receiver.  Acknowledgments also incur forwarding costs and contribute to link utilization, which can impact performance over some types of network.  Applications might be able to improve overall performance by using alternative strategies that reduce the rate of acknowledgments. </ins> 8.


Question:
One sentences added to address remaining bits for
NAME I didn't replace "five-tuple" with "connection" in this sentence in this PR: "QUIC’s stream multiplexing feature allows applications to run multiple streams over a single connection, without head-of-line blocking between streams, associated at a point in time with a single five-tuple. " because it already has connection in the sentence; so we could only remove the last half sentence, but I actually think there is a value in leaving this in because some application may actually care about the 5-tuple.
WFM. We should probably do an editorial pass to make sure we're using terms like "5-tuple" and "connection" and "path" properly everywhere anyway.
See also section on Streams in transport draft: "Stream offsets allow for the octets on a stream to be placed in order. An endpoint MUST be capable of delivering data received on a stream in order. Implementations MAY choose to offer the ability to deliver data out of order. There is no means of ensuring ordering between octets on different streams."
See PR
lgtm modulo editorial. circleci doesn't like this but can fix after it lands.

Mapping of application data to streams is application-specific and described for HTTP/s in QUIC-HTTP.  In general data that can be processed independently, and therefore would suffer from head of line <del> blocking, if forced to be received in order, should be transmitted over different streams.  If there is a logical grouping of those data chunks or messages, stream can be reused, or a new stream can be opened for each chunk/message.  If one message is mapped on a single stream, resetting the stream if the message is not needed anymore can be used to emulate partialreliability on a message basis.  If a QUIC receiver has maximum allowed concurrent streams open and the sender on the other end indicates that more streams are needed, it doesn't automatically lead to an increase of the maximum number of streams by the receiver.  Therefore it can be valuable to expose maximum number of allowed, currently open and currently used streams to the application to make the mapping of data to streams dependent on this information. </del> <ins> blocking if forced to be received in order, should be transmitted over different streams.  If the application requires certain data to be received in order, the same stream should be used for that data. If there is a logical grouping of data chunks or messages, streams can be reused, or a new stream can be opened for each chunk/message. If one message is mapped to a single stream, resetting the stream to expire an unacknowledged message can be used to emulate partial reliability on a message basis.  If a QUIC receiver has maximum allowed concurrent streams open and the sender on the other end indicates that more streams are needed, it doesn't automatically lead to an increase of the maximum number of streams by the receiver. Therefore it can be valuable to expose maximum number of allowed, currently open and currently used streams to the application to make the mapping of data to streams dependent on this information. </ins> Further, streams have a maximum number of bytes that can be sent on one stream.  This number is high enough (2^64) that this will usually


Question:
NAME Good point about being more explicit. I actually think is valuable to reference an existing encoding of public keys providing also key type and curve, like CREDX in URL Also, there is a discussion about format for public keys in COSE, which if it is specified probably will be referenced by EDHOC instead of CREDX. I would be good if there is a common format used in all these instances. For the time being I propose that we reference CRED_X with empty text string subject name, and keep track of the COSE activity.
We should consider replace the NIST reference with SECG [1] to harmonize with EDHOC [2]. [1] URL [2] URL (edit) ... but the commits in [3] mainly refers to the NIST spec. so I propose we keep it as is for the time being. [2] URL
It looks good to me; we can give more details on one point. For the two HKDF invocations in Section 2.3.1, we can define more explicitly how the public keys are expressed when concatenated to form IKM-Sender and IKM-Recipient. Given a public key, that can simply be: For OKP signature keys, the compressed 'x' coordinate, i.e. what would be expressed in the key parameter -2 of a corresponding COSE Key. For EC2 signature keys, the 'x' coordinate followed by the 'y' coordinate in this order, i.e. what would be expressed in the key parameters -2 and -3 of a corresponding COSE Key. This should be sufficient, i.e. it shouldn't be needed to have a more structured encoding of public keys providing also key type and curve, e.g. like done for CRED_X in URL

Context associated to endpoint X.  The Recipient Key is used as salt in the HKDF, when deriving the Pairwise Recipient Key. <del> The Shared Secret is computed as a static-static Diffie-Hellman shared secret NIST-800-56A, where the endpoint uses its private key and the public key of the other endpoint X.  The Shared Secret is used as Input Keying Material (IKM) in the HKDF. </del> <ins> IKM-Sender - the Input Keying Material (IKM) used in HKDF for the derivation of the Pairwise Sender Key - is the concatenation of the endpoint's own signature public key, endpoint X's signature public key from the Recipient Context, and the Shared Secret IKM-Recipient - the Input Keying Material (IKM) used in HKDF for the derivation of the Recipient Sender Key - is the concatenation of endpoint X's signature public key from the Recipient Context, the endpoint's own signature public key, and the Shared Secret. The Shared Secret is computed as a cofactor Diffie-Hellman shared secret, see Section 5.7.1.2 of NIST-800-56A, where the endpoint uses its private key and the public key of the other endpoint X. Note the requirement of validation of public keys in ssec-crypto- considerations.  For X25519 and X448, the procedure is described in Section 5 of RFC7748 using public keys mapped to Montgomery coordinates, see montgomery. </ins> info and L are as defined in Section 3.2.1 of RFC8613. If EdDSA asymmetric keys are used, the Edward coordinates are mapped to Montgomery coordinates using the maps defined in Sections 4.1 and 4.2 of RFC7748, before using the X25519 and X448 functions defined in <del> Section 5 of RFC7748. </del> <ins> Section 5 of RFC7748.  For further details, see montgomery. </ins> After establishing a partially or completely new Security Context (see ssec-sec-context-persistence and sec-group-key-management), the


Question:
NAME Good point about being more explicit. I actually think is valuable to reference an existing encoding of public keys providing also key type and curve, like CREDX in URL Also, there is a discussion about format for public keys in COSE, which if it is specified probably will be referenced by EDHOC instead of CREDX. I would be good if there is a common format used in all these instances. For the time being I propose that we reference CRED_X with empty text string subject name, and keep track of the COSE activity.
We should consider replace the NIST reference with SECG [1] to harmonize with EDHOC [2]. [1] URL [2] URL (edit) ... but the commits in [3] mainly refers to the NIST spec. so I propose we keep it as is for the time being. [2] URL
It looks good to me; we can give more details on one point. For the two HKDF invocations in Section 2.3.1, we can define more explicitly how the public keys are expressed when concatenated to form IKM-Sender and IKM-Recipient. Given a public key, that can simply be: For OKP signature keys, the compressed 'x' coordinate, i.e. what would be expressed in the key parameter -2 of a corresponding COSE Key. For EC2 signature keys, the 'x' coordinate followed by the 'y' coordinate in this order, i.e. what would be expressed in the key parameters -2 and -3 of a corresponding COSE Key. This should be sufficient, i.e. it shouldn't be needed to have a more structured encoding of public keys providing also key type and curve, e.g. like done for CRED_X in URL

2.3.2. <ins> 2.3.2.1. The y-coordinate of the other endpoint's Ed25519 public key is decoded as specified in Section 5.1.3 of RFC8032.  The Curve25519 u-coordinate is recovered as u = (1 + y) / (1 - y) (mod p) following the map in Section 4.1 of RFC7748.  Note that the mapping is not defined for y = 1, and that y = -1 maps to u = 0 which corresponds to the neutral group element and thus will result in a degenerate shared secret.  Therefore implementations MUST abort if the y-coordinate of the other endpoint's Ed25519 public key is 1 or -1 (mod p). The private signing key byte strings (= the lower 32 bytes used for generating the public key, see step 1 of Section 5.1.5 of RFC8032) are decoded the same way for signing in Ed25519 and scalar multiplication in X25519.  Hence, to compute the shared secret the endpoint applies the X25519 function to the Ed25519 private signing key byte string and the encoded u-coordinate byte string as specified in Section 5 of RFC7748. 2.3.2.2. The y-coordinate of the other endpoint's Ed448 public key is decoded as specified in Section 5.2.3. of RFC8032.  The Curve448 u-coordinate is recovered as u = y^2 * (d * y^2 - 1) / (y^2 - 1) (mod p) following the map from "edwards448" in Section 4.2 of RFC7748, and also using the relation x^2 = (y^2 - 1)/(d * y^2 - 1) from the curve equation. Note that the mapping is not defined for y = 1 or -1.  Therefore implementations MUST abort if the y-coordinate of the peer endpoint's Ed448 public key is 1 or -1 (mod p). The private signing key byte strings (= the lower 57 bytes used for generating the public key, see step 1 of Section 5.2.5 of RFC8032) are decoded the same way for signing in Ed448 and scalar multiplication in X448.  Hence, to compute the shared secret the endpoint applies the X448 function to the Ed448 private signing key byte string and the encoded u-coordinate byte string as specified in Section 5 of RFC7748. 2.3.3. </ins> When using any of its Pairwise Sender Keys, a sender endpoint including the 'Partial IV' parameter in the protected message MUST use the current fresh value of the Sender Sequence Number from its


Question:
NAME Good point about being more explicit. I actually think is valuable to reference an existing encoding of public keys providing also key type and curve, like CREDX in URL Also, there is a discussion about format for public keys in COSE, which if it is specified probably will be referenced by EDHOC instead of CREDX. I would be good if there is a common format used in all these instances. For the time being I propose that we reference CRED_X with empty text string subject name, and keep track of the COSE activity.
We should consider replace the NIST reference with SECG [1] to harmonize with EDHOC [2]. [1] URL [2] URL (edit) ... but the commits in [3] mainly refers to the NIST spec. so I propose we keep it as is for the time being. [2] URL
It looks good to me; we can give more details on one point. For the two HKDF invocations in Section 2.3.1, we can define more explicitly how the public keys are expressed when concatenated to form IKM-Sender and IKM-Recipient. Given a public key, that can simply be: For OKP signature keys, the compressed 'x' coordinate, i.e. what would be expressed in the key parameter -2 of a corresponding COSE Key. For EC2 signature keys, the 'x' coordinate followed by the 'y' coordinate in this order, i.e. what would be expressed in the key parameters -2 and -3 of a corresponding COSE Key. This should be sufficient, i.e. it shouldn't be needed to have a more structured encoding of public keys providing also key type and curve, e.g. like done for CRED_X in URL

sequences of one-to-one exchanges with servers in the group, by sending requests over unicast. <del> 2.3.3. </del> <ins> 2.3.4. </ins> If the pairwise mode is supported, the Security Context additionally includes Secret Derivation Algorithm, Secret Derivation Parameters


Question:
NAME Good point about being more explicit. I actually think is valuable to reference an existing encoding of public keys providing also key type and curve, like CREDX in URL Also, there is a discussion about format for public keys in COSE, which if it is specified probably will be referenced by EDHOC instead of CREDX. I would be good if there is a common format used in all these instances. For the time being I propose that we reference CRED_X with empty text string subject name, and keep track of the COSE activity.
We should consider replace the NIST reference with SECG [1] to harmonize with EDHOC [2]. [1] URL [2] URL (edit) ... but the commits in [3] mainly refers to the NIST spec. so I propose we keep it as is for the time being. [2] URL
It looks good to me; we can give more details on one point. For the two HKDF invocations in Section 2.3.1, we can define more explicitly how the public keys are expressed when concatenated to form IKM-Sender and IKM-Recipient. Given a public key, that can simply be: For OKP signature keys, the compressed 'x' coordinate, i.e. what would be expressed in the key parameter -2 of a corresponding COSE Key. For EC2 signature keys, the 'x' coordinate followed by the 'y' coordinate in this order, i.e. what would be expressed in the key parameters -2 and -3 of a corresponding COSE Key. This should be sufficient, i.e. it shouldn't be needed to have a more structured encoding of public keys providing also key type and curve, e.g. like done for CRED_X in URL

hold for Group OSCORE, about building the AEAD nonce and the secrecy of the Security Context parameters. <del> The EdDSA signature algorithm and the elliptic curve Ed25519 RFC8032 are mandatory to implement.  For endpoints that support the pairwise mode, the ECDH-SS + HKDF-256 algorithm specified in Section 6.3.1 of I-D.ietf-cose-rfc8152bis-algs and the X25519 curve RFC7748 are also mandatory to implement. </del> <ins> The EdDSA signature algorithm Ed25519 RFC8032 is mandatory to implement.  For endpoints that support the pairwise mode, the ECDH-SS + HKDF-256 algorithm specified in Section 6.3.1 of I-D.ietf-cose- rfc8152bis-algs and the X25519 algorithm RFC7748 are also mandatory to implement. </ins> Constrained IoT devices may alternatively represent Montgomery curves and (twisted) Edwards curves RFC7748 in the short-Weierstrass form