Consider using RFC3986 encode form-urlencoded

[chanjarster]

3.0.2 spec section Support for x-www-form-urlencoded Request Bodies:

To submit content using form url encoding via RFC1866, ...

And, RFC1866 section 8.2.1 The form-urlencoded Media Type:

The form field names and values are escaped: space characters are replaced by `+', and then reserved characters are escaped as per [URL]

Indicates that using encoding specified in RFC1738 and a special treatment on space character

While according to HTML 5.2 section 4.10.21.6 URL-encoded form data:

See the WHATWG URL specification for details on application/x-www-form-urlencoded. [URL] ...

Which says:

Note: URLs can be used in numerous different manners, in many differing contexts. For the purpose of producing strict URLs one may wish to consider [RFC3986] [RFC3987] ...

So, please consider using RFC3986 instead of RFC1738, RFC1738 is too old(December 1994).

[handrews]

I believe the only difference is that /, ?, and ';' are no longer reserved within the query string in RFC 3986, correct? (And I agree, it should reference RFC 3986).

[chanjarster]

I believe the only difference is that /, ?, and ';' are no longer reserved within the query string in RFC 3986, correct? (And I agree, it should reference RFC 3986).

Yes

[vinniefalco]

Does anyone have a reference to the BNF/grammar for the "key=value" pairs? I wrote this:

query-params    = query-param *( "&" query-param )
query-param     = key [ "=" value ]
key             = *qpchar
value           = *( qpchar / "=" )

but is there an official IETF document (or addendum) that states this?

[handrews]

@vinniefalco there is no IETF document. There was a draft, but it was abandoned. WHATWG addresses it in their URL "living standard", although I assume like all of their specs it's defined in terms of browser implementation state machines as opposed to ABNF or anything like that.

[vinniefalco]

Thanks! (I find the WHATWG "spec" exceptionally frustrating)

[handrews]

It's not clear to me whether we can change an RFC reference in a patch release, so for now I'm putting this in 3.2.

[karenetheridge]

FWIW, RFC3986 is what JSON Schema's "format": "uri-reference" uses (which appears in the openapi schema a few times).

[handrews]

@karenetheridge yeah I think the chances that anyone else even realizes the difference, much less implements it, is pretty slim. Arguably, since RFC3986 is formally listed as updating RFC 1738 (the URL encoding that RFC1866 references), it should be automatically understood that 3986 supersedes the outdated requirements.

I'm not sure whether we should do anything here- it's really getting into the weeds, and I think we can rely on the IETF's Update/Obsolete linkage. Maybe. I hope?

[handrews]

I noticed (with the discussion around urlencoding { and }) that our examples do not urlencode ~, which would be required by RFC 1738 but is not required by RFC 3986. Of course, we weren't urlencoding that example correctly anyway, but... I don't think anyone at any time meant we should urlencode ~ just because RFC 1738 is very indirectly referenced. We reference 3986 normatively elsewhere.

[handrews]

@OAI/tsc review request: Is it worth doing anything here? To summarize:

• We cite RFC 1886 (HTML 2.0) for application/x-www-form-urlencoded, presumably as it was the last IETF document to even sort-of address it
• RFC 1886 cites RFC 1738 (Uniform Resource Locators (URL)), the then-current URI/URL RFC, which has relatively restrictive rules for URL encoding
• RFC 1738 has long since been superseded by RFC 3986 (with others in between), which has somewhat more relaxed encoding rules (which we take advantage of in some of our examples, such as by not encoding ~ which RFC 3986 specifically calls out as being allowed but previously forbidden)

Do we need to specifically note that our reference to RFC 1886 should incorporate RFC 3986 rules rather than RFC 1738 rules, or is that to be expected automatically because RFC 3986 formally obsoletes RFC 1738 already?

In terms of practical impact, I need to know for OASComply whether to validate using the RFC 1738 rules or the RFC 3986 rules.

[ralfhandl]

The WhatWG URL spec requires ~ to be percent-encoded:

Note

The application/x-www-form-urlencoded percent-encode set contains all code points, except the ASCII alphanumeric, U+002A (*), U+002D (-), U+002E (.), and U+005F (_).

[mikekistler]

I would be in favor of updating the RFC reference in the 3.2. My meager understanding of this topic is that the RFC switch is not simply a "clarification" so I don't think it could be done in a patch version on earlier versions.