Transparent Negotiation - the Missing HTTP Feature

I have a story for you.

It is a tale of love and Acceptance. A tale of betrayal. A tale of adventure, of finding riches hidden beneath our feet, spanning the world and its many languages.

Once Upon A Time...

Once upon a time, on the Web, there was a humble server called www.example.com. It was a good server, never out of resources, always willing to give you what you asked for. It had content in many formats and languages. It could show you images and text and all kinds of multimedia. It could serve travelers from afar, speaking Hindi, German or Brazilian Portuguese.

Thanks to the magic of URI and HTTP, the magic of all things on the Web, example.com could serve the proper representation of each resource to each of its clients. A client would just come and say hello, example.com, could I please get information about today's weather? In French, if possible, or in English if you don't have any French. For the presentation, I'm not picky, really, text would do, or HTML, or even one of those image formats, what do you call them, PNG, JPG, or even that SVG special you told me about the other day. Thank you.. In the language of the Web, it went a bit like this:

GET /weather_today HTTP/1.1
Host: www.example.com
Accept-Language: fr, en;q=0.8
Accept: text/plain, text/html, image/svg+xml, image/png;q=0.9, image/jpeg;q=0.8
...

And www.example.com would harness the power of Content Negotiation, look at all the existing variants of the resource identified by http://www.example.com/weather_today, find a nice version in HTML with text in English, and make another happy customer.

The curse of the failed negotiation

But all is not rosy and peaceful on the Web, and sometimes the server has too many good choices and can't decide which would be the best for its customer. Sometimes comes a client with a difficult request, and the server has nothing that quite fits.

What is the server to do in such situations? The book of HTTP, one of the laws of the land, has some answers:

• When there are too many good choices, the server can admit he has too much choice (in Web lore, that is called a 300 Multiple Choices), or just decide to choose one arbitrary variant.

  Neither solution are very satisfying, for the server or its client. Choosing an arbitrary variant when there are many good choices is a pity — perhaps the client could have chosen better, by refining its criteria? And while in the case of a 300 Multiple Choices response, the server should give its client a list of the choices, often in an HTML representation, the client can't read that list, it has to go back and let its master, the almighty user, choose. Sometimes, the user would rather not choose, sometimes not; mostly, the user would rather have it all automatic, yet perfect. And sometimes, the client is a Ronin, a robot, a client without a user behind.

• Similarly, when there is no good choice, the server can apologize for not finding anything appropriate (that's called a 406 Not Acceptable), listing the existing variants for the user to choose, or it can silently pick an arbitrary fallback representation.

  Again, and for the same reasons as above, noone is really happy with the solution.

Oh, Content Negotiation in HTTP, how could you fail us like that? We placed so much of our hopes on you! Is there nothing you can do?

One day my Transparent Negotiation will come

Is there a solution, a savior, a hero? The HTTP specification mentions the possibility of its existence, even mentions its name: Transparent Negotiation, stating This specification does not define any mechanism for transparent negotiation, though it also does not prevent any such mechanism from being developed as an extension that could be used within HTTP/1.1.. There is hope still.

Some old adventurers of the Web claim that Transparent Negotiation is more than a myth, it exists — it has mostly been dormant and ignored for the past ten years. Indeed, it lives, and it has a notable solution to our problems: the Alternates HTTP Header.

Imagine that in addition to returning a user-readable list of possible variants in the case of a 300 Multiple Choices or 406 Not Acceptable, the server could also give that list to the client in a language the client-machine can understand, giving the client a chance to refine its request and pick a good representation.

Imagine that there was a machine-readable equivalent to the good practice that says that a web page should link to version in alternate languages, or to equivalent representations in other formats. That would make search engines so much more powerful when dealing with multilingual content. That would allow browsers to present the alternatives in a good UI.

You are imagining the potential benefits of Alternates and Transparent Negotiation.

(un)Happy Ever After?

This is not the end of our story. This is but the beginning. Will Transparent Negotiation save the day, or is it a bad guy in disguise, bringing more trouble than it would solve. Maybe it has been banished and forgotten for a reason. Maybe the Web Elders know. Maybe we should give it a second chance.

Time, and further developments of the Web will tell. In the meantime, Content Negotiation, a powerful yet fragile feature of the Web, seems well in need of a good sidekick.