Understanding the internet - HTTP
Lately, I have become more and more interested in the exact underpinnings of a particular 'magical' element in our lives: the Internet. I have gradually learned more and more and feel finally confident to start a blog series about it. The goal of this blog series is not only to educate you, the reader, but will also help myself to study more in the area of my own knowledge gaps. I will kick off this series with HTTP, otherwise known as the mysterious abbreviation which appears at the beginning of a URL. So let's get to it!
First things first. HTTP is the abbreviation for HyperText Transfer Protocol. Don't worry too much now about the exact meaning of these words, we will handle them one for one.
When we hear protocol, we can think of a few things. In diplomacy, protocol governs how important international actors should interact between each other. For example, before WWII, it was rude to look the Japanese emperor straight into the eyes. In some respects, it has a lot do do with the more common concept of etiquette: it is a set of courtesy rules which take the hierarchical and social positions of the people present into account. An easy example is the way how you address certain people: in the academic world, you should never ever address a professor with "Dear Mr." but instead use "Dear Professor". In the diplomatic world, you are supposed to address a female minister with "Dear Madam Minister" instead of "Dear Mrs.". Even in a more day-to-day context, etiquette/protocol is everywhere you look: you are expected to say 'Good morning' to your co-workers when you arrive; if you know that your conversation partner doesn't speak your language, you are supposed to switch to their language; if you notice that he keeps looking at his watch, you are expected to finish the conversation.
Protocol is also dependent on the situation (you will not care too much about etiquette in the McDonalds) and dependent on the culture (while bowing to greet somebody is quite normal in Japan, you will get strange looks when you do it in Belgium). Protocol can be verbal or non-verbal but will in almost all cases mean something: if for example, you don't look in the pre-war Japanese emperor's eyes, you acknowledge that he is the emperor and that you are 'lower' than him.
We can conclude that protocol determines how two people will interact. It structures the communication between them. It is not about what you say but more about how you say it. Beware! It is different from language! For example, the courtesy rule to start a letter with "Dear Mr." and end it with something like "Kind regards" can be translated to any language.
We have finally arrived to HTTP. As we have seen, the last P stands for Protocol. HTTP determines how computers communicate with each other on the internet. This has nothing to do with the language computers communicate in, they actually "speak" English. But as you might expect, they don't say "Hello, can you send me that webpage from CNN about that Trump article from yesterday?". This is because computers don't have a sense of context or nuance, they need exact instructions in a manner as compact as possible in order to succesfully fulfill a task. The rulebook that determines how they should interact is called the protocol.
We will see an example of 'how' they speak to each other (their protocol) in a moment. But first, I will teach you a bit of terminology:
Communication (or the transfer of information) is quite easy to grasp. After all, we do it every day ;) Typically, you have a person asking a question to somebody else. This other person (I will call him the receiver) then replies with the answer. A lot of possible answers exist: the receiver might give the answer directly, or the receiver might direct him/her to somebody else or the receiver might even flat-out say that he/she cannot answer the question.
For HTTP, this happens in exactly the same manner. Have a look at the following figure:
- A request occurs when the first computer asks a question to the second computer.
- A response occurs when the second computer replies to the first computer.
As you may notice, it is quite annoying to talk in terms of person asking question / person answering question and computer 1 / computer 2. Luckily, in computer science, there are two terms we can use for it.
- A computer performing a request (=asking a question) is typically called the client.
- The computer waiting for an incoming request in order to send back a response (=answering the question) is called the server. A server which does this using HTTP, is called a web server.
To make it visual, I updated the previous figure with these terms:
Please be aware that a server (or web server) isn't necessarily different from a normal computer. I met a lot of people who think that the term 'server' is used for a special breed of computer. This is not the case. 'Server' is defined in a purely functional way. If you install web server software on your pc (the two most popular are Apache (by the Apache Software Foundation) and IIS (by Microsoft)), your computer becomes a server.
Just as a web server needs software to be a web server, so also does a client need software to be a client. In this case, we are actually talking about a browser. A browser is just a program which translates everything you do on the internet into requests, and retranslates the responses back to a form you might understand.
Now we will finally start to talk about the protocol (=rules of interaction). A few verbs make up the core of the protocol. This is very important. If I ask my wife to give me a glass, I can use different verbs:
- Could you get me a glass of water, sweethart?
- Could you bring me a glass of water, sweethart?
- Could you give me a glass of water, sweethart?
Obviously, my wife will understand that she has to take an action which results in me having a glass of water. Unfortunately, a computer is not really at home in the intricacies of the English language and to be honest: its just not necessary.
So, at one point in time, some dusty guys came together and decided that computers would use the word GET to ask for something. Tadaaaam: we have our first and most commonly used verb (called method in HTTP lingo) on the internet.
GET: used to read or retrieve a resource
Every time you typ a URL into the address bar at the top of your browser and hit ENTER, your browser performs a GET request. I put resource in the definition because we are ofcourse not only talking about webpages. You can ask other computers for images, videos, excel files etc.
Ofcourse, there are other verbs besides GET. Be aware that some methods might require that you have permission to access that web server. For instance, not everybody can just DELETE files on a web server.
- POST: submits data to be processed to a specified resource
- DELETE: deletes the specified resource
- HEAD: same as GET but returns only some meta-information and not the real information
- OPTIONS: returns the HTTP methods that the server supports
An example GET Request
To keep promoting my blog, I will perform a GET request to ask for one of my articles ;)
GET http://www.economaniac.be/demand-and-supply/ HTTP/1.1
Accept-Encoding: gzip, deflate, sdch
We will go through it line by line:
- GET http://www.economaniac.be/demand-and-supply/ HTTP/1.1: I am asking to 'get' the demand-and-supply page on the www.economaniac.be website. I add HTTP/1.1 at the end to indicate that I 'speak' HTTP version 1.1.
After this first line, we encounter different HTTP Headers. These headers typically give more information about the request: this can be crucial or just nice-to-know information for the server to do his job to our satisfaction.
- Host: this is the domain name of the server. You can think of the page (demand-and-supply) as the house number of your webpage and the domain (www.economaniac.be) as the street name. This is ofcourse crucial, otherwise the demand will never end up with the correct server.
- User-Agent: a user-agent is software that is acting on behalf of a user. In our case, we are ofcourse talking about our browser. For my request, you can see that I was using Chrome. The number after that is the version number of Chrome that I am using.
- Accept: this specifies which media types I would like to receive in the response. The first possibility text/html means that I expect a text file with HTML code in it.
- Accept-Encoding: The data which is sent back is in most cases compressed. Compression reduces file size so the response can be transferred to you in a faster way. You are probably familiar with compression algorithms through working with .zip and .rar files. Ofcourse, there exist other compression algorithms. In our GET request, I specified gzip, deflate and sdch. It makes sense to communicate this to server in order to avoid receiving a response compressed with an algorithm we are unable to decompress.
- Accept-Language: your browser also sends some information specifying what language you (the user) speaks. This way, the server can send us the correct language version of the webpage (in cases that there are multiple versions of the page).
There are ofcourse a lot of different HTTP headers which can be added to a HTTP request. You can find a list of each of them with their meaning here.
An example Response
Our request will ofcourse result in a response. This is the response we might receive:
HTTP/1.1 200 OK
Date: Tue, 31 Jan 2017 13:52:24 GMT
Content-Type: text/html; charset=utf-8
The first line starts of quite easily. The server responds with HTTP/1.1, meaning he too speaks HTTP/1.1 and he agrees to proceed the conversation using that protocol version. On the same line, he sends 200 OK. This is called a status code and is used to communicate in super-short format what will happen with the request: does the resource exist? has it been moved to another place? does the user have enough permissions? Below, you will find a summary of the most common status codes with a word of explanation:
|Status code - digits||Human readable||Explanation|
|200||OK||The request succeeded, and the resulting resource (e.g. file or script output) is returned in the message body.|
|404||Not Found||The requested resource doesn't exist.|
|301||Moved Permanently||Moved Permanently|
|302||Moved Temporarily||Moved Temporarily|
|500||Server Error||An unexpected server error. The most common cause is a server-side script that has bad syntax, fails, or otherwise can't run correctly.|
Now, those 404 pages finally start to make sense, don't they?
Next are the HTTP headers of the response:
- Date: the server sends the current date back in the response.
- Server: similar as User-Agent, which contains information about the requestor's client software (=browser), the server indicates that it is running Apache web server software.
- Content-Length: length of the body of the reponse
- Content-Type: this is the counterpart to the Accept header in the request. It specifies what media type is returned. We requested text/html and we received test/html. Success!
- Connection: what should be done with the connection established between client and server?
- Content-Encoding: counterpart to Accept-Encoding. This indicates how the body is encoded.
There are ofcourse a lot of different HTTP headers which can be added to a HTTP response. You can find a list of each of them with their meaning here.