Notes on HTTP headers

Technically they’re HTTP “fields”, since they can also be used in HTTP trailers, but who uses those? Most people just colloquially say “headers”.

The places to find more information are RFC 9110, which species HTTP semantics and MDN’s HTTP Headers page.

Retry-After

This is supposed to delay the user-agent, even after a redirect. (See RFC 9110’s entry on Retry-After.) I just tested this on Firefox 114 (see /redirectretryafter which should only redirect after 10 seconds), but it doesn’t work. Neither does GNOME Web (Epiphany) 43.1. Neither does curl (see curl issue #11447 and the curl docs/TODO file).

This is kind of annoying to me because it would be nice to display a message before just launching a person to another page. It means that you might have to use a <meta> redirect instead of a proper HTTP 3XX response. Or, you could redirect to a secondary page that delays a response and then redirects again. Or, you could delay finishing the content body (I think this might be the best workaround in the meantime). At least curl (without -L) will show you the message.

X-Robots-Tag

You can set this to values like “noindex, nofollow” to prevent Google (or other indexes) from indexing or following links in a page. This can sometimes be handy if you want to prevent indexing in non-HTML resources. I think if you just put a URL in robots.txt, pages can still be listed but not indexed for content. I might be wrong on that.

Google describes X-Robots-Tag at https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag.

X-Accel-Buffering

Set this to “no” if you don’t want nginx to buffer an HTTP response output. For example, you might be streaming a response slowly.

Random questions

Are response headers protected from fetch?

Like, I know that you can set cookies to be HTTP-only, but if some javascript made a fetch or XMLHttpRequest, could it access new cookies set in a header?

Looking at this MDN page on fetch response headers and this fetch demo (looking at the console) it appears like headers are accessible.

Ah, but looking at the MDN Set-Cookie page it mentions the fetch spec on forbidden response headers. MDN also has a (stub) page on forbidden response header names. It seems like only Set-Cookie is protected (looking at my "/forbiddenresponseheadertest1"), and Sec-* headers aren’t filtered out.

The fetch spec also mentions that Set-Cookie2 is filtered out, which, what the hell is that header? It looks like RFC 6265 mentions that Set-Cookie2 is obsolete. Cookie2 is specified in RFC29265, and maybe has to do with cross-origin (like different port or domain) cookies maybe?

To-dos

Things I should document