UriComponentsBuilder.fromUriString may not parse correctly when there is no path [SPR-11970]

[spring-projects-issues]

Kazuki Shimizu opened SPR-11970 and commented

e.g.) "http://example.com?..." and "http://example.com/?..." has different behavior.

For following example...
If not specify path in uri, string of up to '@' in the query parameters is recognized as a UserInfo.

Test code

	public static void main(String[] args) throws URISyntaxException,
			UnsupportedEncodingException {

		UriComponentsBuilder builder = UriComponentsBuilder.newInstance();
		builder.uri(new URI("http://example.com"));
		builder.queryParam("key1 ", "value1 ");
		builder.queryParam("key2+", "value2+");
		builder.queryParam("key3&", "value3&");
		builder.queryParam("key4=", "value4=");
		// parameters that not require encoding
		builder.queryParam(
				"key5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;",
				"value5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;");

		UriComponents uriComponents = builder.build().encode("UTF-8");
		String uriString = uriComponents.toUriString();

		System.out.println("----- Before -------");
		System.out.println("UriString :" + uriString);
		System.out.println("Scheme :" + uriComponents.getScheme());
		System.out.println("UserInfo :" + uriComponents.getUserInfo());
		System.out.println("Host :" + uriComponents.getHost());
		System.out.println("Port :" + uriComponents.getPort());
		System.out.println("Path :" + uriComponents.getPath());
		System.out.println("Query :" + uriComponents.getQuery());
		System.out.println("Fragment :" + uriComponents.getFragment());
		System.out.println("QueryParams :" + uriComponents.getQueryParams());

		builder = UriComponentsBuilder.fromUriString(uriString);
		uriComponents = builder.build();
		uriString = uriComponents.toUriString();

		System.out.println("----- After -------");
		System.out.println("UriString :" + uriString);
		System.out.println("Scheme :" + uriComponents.getScheme());
		System.out.println("UserInfo :" + uriComponents.getUserInfo());
		System.out.println("Host :" + uriComponents.getHost());
		System.out.println("Port :" + uriComponents.getPort());
		System.out.println("Path :" + uriComponents.getPath());
		System.out.println("Query :" + uriComponents.getQuery());
		System.out.println("Fragment :" + uriComponents.getFragment());
		System.out.println("QueryParams :" + uriComponents.getQueryParams());
	}

Console

----- Before -------
UriString :http://example.com?key1%20=value1%20&key2%2B=value2%2B&key3%26=value3%26&key4%3D=value4%3D&key5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;=value5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;
Scheme :http
UserInfo :null
Host :example.com
Port :-1
Path :null
Query :key1%20=value1%20&key2%2B=value2%2B&key3%26=value3%26&key4%3D=value4%3D&key5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;=value5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;
Fragment :null
QueryParams :{key1%20=[value1%20], key2%2B=[value2%2B], key3%26=[value3%26], key4%3D=[value4%3D], key5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;=[value5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;]}
----- After -------
UriString :http://example.com?key1%20=value1%20&key2%2B=value2%2B&key3%26=value3%26&key4%3D=value4%3D&key5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;=value5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;
Scheme :http
UserInfo :example.com?key1%20=value1%20&key2%2B=value2%2B&key3%26=value3%26&key4%3D=value4%3D&key5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:
Host :
Port :-1
Path :/
Query :!$'()*,;=value5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;
Fragment :null
QueryParams :{!$'()*,;=[value5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;]}

If specify path in uri, result are following.
Before and after is same. (this behavior is correct.)

----- Before -------
UriString :http://example.com/?key1%20=value1%20&key2%2B=value2%2B&key3%26=value3%26&key4%3D=value4%3D&key5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;=value5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;
Scheme :http
UserInfo :null
Host :example.com
Port :-1
Path :/
Query :key1%20=value1%20&key2%2B=value2%2B&key3%26=value3%26&key4%3D=value4%3D&key5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;=value5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;
Fragment :null
QueryParams :{key1%20=[value1%20], key2%2B=[value2%2B], key3%26=[value3%26], key4%3D=[value4%3D], key5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;=[value5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;]}
----- After -------
UriString :http://example.com/?key1%20=value1%20&key2%2B=value2%2B&key3%26=value3%26&key4%3D=value4%3D&key5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;=value5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;
Scheme :http
UserInfo :null
Host :example.com
Port :-1
Path :/
Query :key1%20=value1%20&key2%2B=value2%2B&key3%26=value3%26&key4%3D=value4%3D&key5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;=value5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;
Fragment :null
QueryParams :{key1%20=[value1%20], key2%2B=[value2%2B], key3%26=[value3%26], key4%3D=[value4%3D], key5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;=[value5abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-._~:@/?!$'()*,;]}

---

Affects: 3.2.9, 4.0.5

Backported to: 3.2.11

[spring-projects-issues]

Rossen Stoyanchev commented

The trigger for this seems to be (a) no path and (b) the presence of an "@" in the query or fragment. Most likely the USERINFO_PATTERN in UriComponentsBuilder needs to be updated.

[spring-projects-issues]

Kazuki Shimizu commented

"?" add as exclude character !?

Current

private static final String USERINFO_PATTERN = "([^@/]*)";

Modify candidate

private static final String USERINFO_PATTERN = "([^@/?]*)";

[spring-projects-issues]

Kazuki Shimizu commented

Hi Brian.

Thank you for resolved.
I see a4484bb.

I have a question.
Why do need to be ignore the '@' after '[' ?
If possible, i want to understand this reason.

[spring-projects-issues]

Brian Clozel commented

Hi Kazuki Shimizu

We're ignoring the following chars because:

• @ marks the end of the userinfo part of the URI
• / is a reserved char and is part of the path
• ? same thing for the query string (your case, actually)
• # for anchors
• [ this can be used for IPv6 addresses

[spring-projects-issues]

Kazuki Shimizu commented

Thank you for answer.

This means that is can be specify the '@' character in the IPv6 address ?
I was referring the rfc3986(http://www.ietf.org/rfc/rfc3986.txt), but i was not able to find the above specifications.
Where is written ?

Specifications of rfc3986

host          = IP-literal / IPv4address / reg-name

IP-literal    = "\[" ( IPv6address / IPvFuture  ) "\]"

IPvFuture     = "v" 1*HEXDIG "." 1*( unreserved / sub-delims / ":" )

IPv6address   = 6( h16 ":" ) ls32
                 / "::" 5( h16 ":" ) ls32
                 / [               h16 ] "::" 4( h16 ":" ) ls32
                 / [ *1( h16 ":" ) h16 ] "::" 3( h16 ":" ) ls32
                 / [ *2( h16 ":" ) h16 ] "::" 2( h16 ":" ) ls32
                 / [ *3( h16 ":" ) h16 ] "::"    h16 ":"   ls32
                 / [ *4( h16 ":" ) h16 ] "::"              ls32
                 / [ *5( h16 ":" ) h16 ] "::"              h16
                 / [ *6( h16 ":" ) h16 ] "::"

#           = 1*4HEXDIG

ls32          = ( h16 ":" h16 ) / IPv4address

IPv4address   = dec-octet "." dec-octet "." dec-octet "." dec-octet

dec-octet   = DIGIT ; 0-9
                  / %x31-39 DIGIT ; 10-99
                  / "1" 2DIGIT ; 100-199
                  / "2" %x30-34 DIGIT ; 200-249
                  / "25" %x30-35 ; 250-255

reg-name      = *( unreserved / pct-encoded / sub-delims )

unreserved    = ALPHA / DIGIT / "-" / "." / "_" / "~"

sub-delims    = "!" / "$" / "&" / "'" / "(" / ")" / "*" / "+" / "," / ";" / "="

pct-encoded = "%" HEXDIG HEXDIG

[spring-projects-issues]

Brian Clozel commented

No, it just means that the userinfo part "user:password" should not contain the following chars (@, /, ?, #, [), because they're linked with some other part of the URL.

Here is a complete example:

http://login:password@[1abc:2abc:3abc::5ABC:6abc%eth0]/resource?foo=bar#baz
                     ||                               |        |       |

Here I marked with | the chars we don't want to match as part of the userinfo pattern.

[spring-projects-issues]

Kazuki Shimizu commented

Thank you very*2 much for answer.
But i could not understand...(sorry...) .

In my understanding, URI parts that can be specify @(same value as user info separator) are a path or query or fragment.
If this understanding is correct, I thought that @ and [ is unnecessary as regex expression to extract user information.

private static final String USERINFO_PATTERN = "([^/?#]*)";

This understanding is wrong ?

[spring-projects-issues]

Rossen Stoyanchev commented

But the "@" is what separates the user info from the host. So you want to stop right there, no?