LuaSocket: HTTP support

HTTP

HTTP (Hyper Text Transfer Protocol) is the protocol used to exchange information between web-browsers and servers. The http namespace offers full support for the client side of the HTTP protocol (i.e., the facilities that would be used by a web-browser implementation). The implementation conforms to the HTTP/1.1 standard, RFC 2616.

The module exports functions that provide HTTP functionality in different levels of abstraction. From the simple string oriented requests, through generic LTN12 based, down to even lower-level if you bother to look through the source code.

To obtain the http namespace, run:

-- loads the HTTP module and any libraries it requires
local http = require("socket.http")

URLs must conform to RFC 1738, that is, an URL is a string in the form:

[http://][<user>[:<password>]@]<host>[:<port>][/<path>]

MIME headers are represented as a Lua table in the form:

headers = { field-1-name = field-1-value, field-2-name = field-2-value, field-3-name = field-3-value, ... field-n-name = field-n-value }

Field names are case insensitive (as specified by the standard) and all functions work with lowercase field names. Field values are left unmodified.

Note: MIME headers are independent of order. Therefore, there is no problem in representing them in a Lua table.

The following constants can be set to control the default behavior of the HTTP module:

PORT: default port used for connections;
PROXY: default proxy used for connections;
TIMEOUT: sets the timeout for all I/O operations;
USERAGENT: default user agent reported to server.

http.request(url [, body])
http.request{
  url = string,
  [sink = LTN12 sink,]
  [method = string,]
  [headers = header-table,]
  [source = LTN12 source],
  [step = LTN12 pump step,]
  [proxy = string,]
  [redirect = boolean,]
  [create = function]
}

The request function has two forms. The simple form downloads a URL using the GET or POST method and is based on strings. The generic form performs any HTTP method and is LTN12 based.

If the first argument of the request function is a string, it should be an url. In that case, if a body is provided as a string, the function will perform a POST method in the url. Otherwise, it performs a GET in the url

If the first argument is instead a table, the most important fields are the url and the simple LTN12 sink that will receive the downloaded content. Any part of the url can be overridden by including the appropriate field in the request table. If authentication information is provided, the function uses the Basic Authentication Scheme (see note) to retrieve the document. If sink is nil, the function discards the downloaded data. The optional parameters are the following:

method: The HTTP request method. Defaults to "GET";
headers: Any additional HTTP headers to send with the request;
source: simple LTN12 source to provide the request body. If there is a body, you need to provide an appropriate "content-length" request header field, or the function will attempt to send the body as "chunked" (something few servers support). Defaults to the empty source;
step: LTN12 pump step function used to move data. Defaults to the LTN12 pump.step function.
proxy: The URL of a proxy server to use. Defaults to no proxy;
redirect: Set to false to prevent the function from automatically following 301 or 302 server redirect messages;
create: An optional function to be used instead of socket.tcp when the communications socket is created.

In case of failure, the function returns nil followed by an error message. If successful, the simple form returns the response body as a string, followed by the response status code, the response headers and the response status line. The generic function returns the same information, except the first return value is just the number 1 (the body goes to the sink).

Even when the server fails to provide the contents of the requested URL (URL not found, for example), it usually returns a message body (a web page informing the URL was not found or some other useless page). To make sure the operation was successful, check the returned status code. For a list of the possible values and their meanings, refer to RFC 2616.

Here are a few examples with the simple interface:

-- load the http module
local io = require("io")
local http = require("socket.http")
local ltn12 = require("ltn12")

-- connect to server "www.cs.princeton.edu" and retrieves this manual
-- file from "~diego/professional/luasocket/http.html" and print it to stdout
http.request{ 
    url = "http://www.cs.princeton.edu/~diego/professional/luasocket/http.html", 
    sink = ltn12.sink.file(io.stdout)
}

-- connect to server "www.example.com" and tries to retrieve
-- "/private/index.html". Fails because authentication is needed.
b, c, h = http.request("http://www.example.com/private/index.html")
-- b returns some useless page telling about the denied access, 
-- h returns authentication information
-- and c returns with value 401 (Authentication Required)

-- tries to connect to server "wrong.host" to retrieve "/"
-- and fails because the host does not exist.
r, e = http.request("http://wrong.host/")
-- r is nil, and e returns with value "host not found"

And here is an example using the generic interface:

-- load the http module
http = require("socket.http")

-- Requests information about a document, without downloading it.
-- Useful, for example, if you want to display a download gauge and need
-- to know the size of the document in advance
r, c, h = http.request {
  method = "HEAD",
  url = "http://www.tecgraf.puc-rio.br/~diego"
}
-- r is 1, c is 200, and h would return the following headers:
-- h = {
--   date = "Tue, 18 Sep 2001 20:42:21 GMT",
--   server = "Apache/1.3.12 (Unix)  (Red Hat/Linux)",
--   ["last-modified"] = "Wed, 05 Sep 2001 06:11:20 GMT",
--   ["content-length"] = 15652,
--   ["connection"] = "close",
--   ["content-Type"] = "text/html"
-- }

Note: When sending a POST request, simple interface adds a "Content-type: application/x-www-form-urlencoded" header to the request. This is the type used by HTML forms. If you need another type, use the generic interface.

Note: Some URLs are protected by their servers from anonymous download. For those URLs, the server must receive some sort of authentication along with the request or it will deny download and return status "401 Authentication Required".

The HTTP/1.1 standard defines two authentication methods: the Basic Authentication Scheme and the Digest Authentication Scheme, both explained in detail in RFC 2068.

The Basic Authentication Scheme sends <user> and <password> unencrypted to the server and is therefore considered unsafe. Unfortunately, by the time of this implementation, the wide majority of servers and browsers support the Basic Scheme only. Therefore, this is the method used by the toolkit whenever authentication is required.

-- load required modules
http = require("socket.http")
mime = require("mime")

-- Connect to server "www.example.com" and tries to retrieve
-- "/private/index.html", using the provided name and password to
-- authenticate the request
b, c, h = http.request("http://fulano:[email protected]/private/index.html")

-- Alternatively, one could fill the appropriate header and authenticate
-- the request directly.
r, c = http.request {
  url = "http://www.example.com/private/index.html",
  headers = { authentication = "Basic " .. (mime.b64("fulano:silva")) }
}