Skip to content

x/net/http2/hpack: encoder table search hashes the header name twice #80787

Description

@h2zi

What version of Go are you using (go version)?

$ go version
go version go1.26.5 darwin/arm64

golang.org/x/net at 8d10596 (v0.57.0 plus a few commits).

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

darwin/arm64, Apple M4.

What did you do?

Profiled an HTTP/2 server under sustained load from h2load (32 connections,
32 concurrent streams each, 1.7M req/s, 13-byte responses). hpack.Encoder.WriteField
accounts for 1.4% of total CPU, which is 28% of the CPU spent outside syscalls,
and 78% of that sits in headerFieldTable.search.

search consults two maps:

func (t *headerFieldTable) search(f HeaderField) (i uint64, nameValueMatch bool) {
	if !f.Sensitive {
		if id := t.byNameValue[pairNameValue{f.Name, f.Value}]; id != 0 {
			return t.idToIndex(id), true
		}
	}
	if id := t.byName[f.Name]; id != 0 {
		return t.idToIndex(id), false
	}
	return 0, false
}

A field whose name is in the table but whose value is not hashes the name twice
and the value once. That is the common case for a server: content-type,
content-length, date and server all match a static-table name and carry a
value that is not in the table. Encoder.searchTable then repeats the whole
thing for the dynamic table, so a single WriteField can reach four map lookups
and six string hashes.

What did you expect to see?

One hash per lookup. The values under a given name are few — the widest entry in
the static table is :status with seven — so once the name is found the value
can be settled by comparison.

What did you see instead?

Keying a single map on the name, with the entries for that name held in a slice,
is measurably faster on both the existing benchmark and one that mirrors a real
response header block:

goos: darwin
goarch: arm64
pkg: golang.org/x/net/http2/hpack
cpu: Apple M4
                             │     old      │                 new                 │
                             │    sec/op    │   sec/op     vs base                │
EncoderSearchTable-10          4.364µ ± ∞ ¹  3.538µ ± ∞ ¹  -18.93% (p=0.008 n=5)
EncoderWriteFieldResponse-10   176.9n ± ∞ ¹  144.5n ± ∞ ¹  -18.32% (p=0.008 n=5)

EncoderWriteFieldResponse encodes :status: 200, content-type,
content-length, date, server and an x-request-id.

Allocation stays at zero and the change removes 51 lines net, since one map
replaces two in the table, in buildMaps, in addEntry, in evictOldest and in
the generated static table.

I have a change ready and will send it for review.

Metadata

Metadata

Assignees

No one assigned

    Labels

    NeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions