On this page:
1.1 Why not just use net/ url?
1.2 Connections and requests
connect
disconnect
connect-uri
uri->scheme&host&port
uri&headers->path&header
start-request
1.3 Reading response headers
purify-port/ log-debug
redirect-uri
same-connection?
close-connection?
1.4 Reading response entities
1.4.1 Building blocks
read-entity/ transfer-decoding-port
entity-content-decoder/ ports
entity-content-decode-bytes
1.4.2 Conveniences
read-entity/ bytes
read-entity/ xexpr
1.5 call/ input-url style
call/ requests
call/ input-request
call/ output-request
1.6 net/ url impure port style
head-impure-port*
get-impure-port*
delete-impure-port*
put-impure-port*
post-impure-port*
http-ver
1.7 Date string conversion
seconds->gmt-string
seconds->gmt-8601-string
gmt-8601-string->seconds

1 Request

 (require (planet gh/http:1:=0/request))

1.1 Why not just use net/url?

Racket’s net/url functions are very convenient for doing simple requests using HTTP 1.0. Use them when you can.

Sometimes you need to use HTTP 1.1. For example you may need the speed of making multiple requests to a server over one persistent connection. Or you may want to use the Expect: 100-continue request header—so that if a server is going to reject or redirect your put or post request, it can inform you before you send it a potentially large amount of data for the entity.

This library is intended to help you with those kinds of applications. It provides basic building blocks to let you do any kind of HTTP request flow, with help for some of the gory details.

Also it supplies some higher-level support. The call/requests functions are in the style of call/input-url. These take care of some standard setup and teardown, including ensuring the ports are closed even if an exception is raised.

It even provides variations of net/url’s functions like get-impure-port. You can use those when you want to use Expect: 100-continue but do not need to make multiple requests over one connection.

1.2 Connections and requests

procedure

(connect scheme host port)  
input-port? output-port?
  scheme : (or/c "http" "https")
  host : string?
  port : exact-positive-integer?

procedure

(disconnect in out)  any

  in : input-port?
  out : output-port?
Begin and end an HTTP connection.

Examples:

> (define-values (in out) (connect "http" "www.google.com" 80))
> (disconnect in out)

procedure

(connect-uri uri)  
input-port? output-port?
  uri : string?
Given uri, uses uri->scheme&host&port and connect to connect.

Examples:

> (define-values (in out) (connect-uri "http://www.google.com/"))
> (disconnect in out)

procedure

(uri->scheme&host&port uri)  
(or/c "http" "https")
string?
exact-positive-integer?
  uri : string?
Splits uri into three elements. If the URI does not specify an element, then reasonable defaults are provided.

Examples:

> (uri->scheme&host&port "www.foo.com")

"http"

"www.foo.com"

80

> (uri->scheme&host&port "http://www.foo.com")

"http"

"www.foo.com"

80

> (uri->scheme&host&port "https://www.foo.com")

"https"

"www.foo.com"

443

> (uri->scheme&host&port "http://www.foo.com:8080")

"http"

"www.foo.com"

8080

procedure

(uri&headers->path&header uri heads)  
string? string?
  uri : string?
  heads : dict?
Splits uri into the path portion required for a request, as well as some headers. If heads doesn’t already contain Host or Date headers they will be added automatically.

Examples:

> (uri&headers->path&header "http://www.foo.com" '())

"/"

"Host: www.foo.com\r\nDate: Sat, 18 Aug 2012 16:41:59 GMT\r\n\r\n"

> (uri&headers->path&header
    "http://www.foo.com"
    (hash 'Date "Fri, 04 Nov 2011 22:16:34 GMT"))

"/"

"Host: www.foo.com\r\nDate: Fri, 04 Nov 2011 22:16:34 GMT\r\n\r\n"

> (uri&headers->path&header
     "http://www.foo.com/path/to/res#fragment?query=val"
     '())

"/path/to/res#fragment?query=val"

"Host: www.foo.com\r\nDate: Sat, 18 Aug 2012 16:41:59 GMT\r\n\r\n"

procedure

(start-request in    
  out    
  http-version    
  method    
  path    
  heads)  boolean?
  in : input-port?
  out : output-port?
  http-version : (or/c "1.0" "1.1")
  method : string?
  path : string?
  heads : string?
Starts an HTTP request. Returns a boolean?:

1.3 Reading response headers

procedure

(purify-port/log-debug in)  string?

  in : input-port?
This is exactly like purify-port from net/url but this logs the headers using log-debug when log-level? is 'debug.

The other functions in this library use log-debug, so using purify-port/log-debug will give a complete/consistent log output.

procedure

(redirect-uri h)  (or/c #f string?)

  h : string?
Given response headers, if the code is a 30x value for a redirect and if a Location header exists, return the value of the Location header. Otherwise return #f.

procedure

(same-connection? old new)  boolean?

  old : url?
  new : url?
Do old and new represent the same connection, i.e. the same scheme, host and port?

procedure

(close-connection? h)  boolean?

  h : string?
Do the response headers require the connection to be closed? For example is the HTTP version 1.0, or is there a Connection: close header?

1.4 Reading response entities

1.4.1 Building blocks

procedure

(read-entity/transfer-decoding-port in h)  input-port?

  in : input-port?
  h : string?
Transforms in to an input-port? that reads an HTTP entity in accordance with the Transfer-Encooding header, specifically the chunked transfer encoding. You can read from the port as if the entity were not transfer-encoded.

Used by read-entity/bytes.

procedure

(entity-content-decoder/ports heads)

  (input-port? output-port? -> any)
  heads : (or/c string? bytes?)

Note: I wanted this to return an input-port? like read-entity/transfer-decoding-port and to be named read-entity/content-decoding-port. Unfortunately, the gzip and deflate functions provided by Racket don’t seem to make that possible, as they use a copy-port pattern.

Examines the Content-Encoding header, if any, and returns a function (input-port? output-port? -> void?) which will copy the bytes and decode them. If there no content encoding specified, or an unsupported encoding, this simply returns copy-port.

procedure

(entity-content-decode-bytes bytes heads)  bytes?

  bytes : bytes?
  heads : (or/c string? bytes?)
If you have already read the entity into bytes?, you may use this to decode it based on the Content-Encoding header if any in heads.

1.4.2 Conveniences

procedure

(read-entity/bytes in h)  bytes?

  in : input-port?
  h : string?
Read the entity from in and return it as bytes?. This is a convenience wrapper for read-entity/transfer-decoding-port when you don’t mind keeping the entire entity in memory.

procedure

(read-entity/xexpr in h)  xexpr?

  in : input-port?
  h : string?
Read the entity from in to an xexpr?. This is a convenience wrapper for read-entity/bytes when you would like the entity parsed into an xexpr.

How the xexpr is created depends on the Content-Type response header present in h:

1.5 call/input-url style

Like call/input-url, these procedures help you make HTTP requests and ensure that the ports are closed even if there is an exception.

procedure

(call/requests scheme host port func)  any/c

  scheme : (or/c "http" "https")
  host : string?
  port : exact-positive-integer?
  func : (input-port? output-port? . -> . any/c)
Provides a way to make one or more requests to the same host connection, and ensure the ports are closed even if there is an exception.

Your func function is called with two arguments, the input and output ports. The value you return from func is returned as the value of call/requests.

Tip: Remember to call flush-output after writing data to out, otherwise the server may not get the data and respond, while you wait to read from in.

(define-values (scheme host port) (uri->scheme&host&port uri))
(call/requests
  scheme host port
  (lambda (in out)
    (define-values (path rh)
      (uri&headers->path&header uri '("Expect: 100-continue")))
    (define tx-data? (start-request in out method path rh "1.0"))
    (when tx-data?
      (display data out)
      (flush-output out)) ;Important!
    (define h (purify-port/log-debug in))
    (read-entity/bytes in h)))

For a single request, you may prefer to use call/input-request or call/output-request.

procedure

(call/input-request http-version    
  method    
  uri    
  heads    
  entity-reader    
  #:redirects redirects)  any/c
  http-version : (or/c "1.0" "1.1")
  method : string?
  uri : string?
  heads : dict?
  entity-reader : (input-port? string? . -> . any/c)
  redirects : 10
A wrapper around call/requests for the case where you want to make just one request, and there is no data to send (e.g. it is a HEAD, GET, or DELETE request).

Like call/requests, this guarantees the ports will be closed.

Your entity-reader function is called after the response header has been read (the header is passed to you as a string), giving you the ability to read the response entity from the supplied input-port?. For example you may read the response using read-entity/bytes or read-entity/xexpr.

If redirects is non-zero, up to that number of redirects will be followed automatically. Whether the response is a redirect is determined using redirect-uri. If the redirected location can be reached on the existing HTTP connection (as determined by close-connection? and same-connection?), then the existing connection will be used (which is faster than disconnecting and reconnecting.)

procedure

(call/output-request http-version    
  method    
  uri    
  data    
  data-length    
  heads    
  entity-reader    
  #:redirects redirects)  any/c
  http-version : (or/c "1.0" "1.1")
  method : string?
  uri : string?
  data : (or/c bytes? (output-port? . -> . void?))
  data-length : (or/c #f exact-nonnegative-integer?)
  heads : dict?
  entity-reader : (input-port? string? . -> . any/c)
  redirects : 10
A wrapper around call/requests for the case where you want to make just one request, and there is data to send (it is a PUT or POST request).

Like call/requests, this guarantees the ports will be closed.

The data and data-length arguments depend on whether you want to provide the data as bytes? or as a function that will write the data:

Your entity-reader function is called after the response header has been read (the header is passed to you as a string), giving you the ability to read the response entity from the supplied input-port?. For example you may read the response using read-entity/bytes or read-entity/xexpr.

If redirects is non-zero, up to that number of redirects will be followed automatically. Whether the response is a redirect is determined using redirect-uri. If the redirected location can be reached on the existing HTTP connection (as determined by close-connection? and same-connection?), then the existing connection will be used (which is faster than disconnecting and reconnecting.)

If you supply "1.1" for http-version and the request header Expect: 100-continue, then redirects or failures can be handled much more efficiently. If the server supports Expect: 100-continue, then it can respond with the redirect or failure before you transmit all of the entity data.

1.6 net/url impure port style

procedure

(head-impure-port* url heads)  input-port?

  url : url?
  heads : (listof string? '())

procedure

(get-impure-port* url heads)  input-port?

  url : url?
  heads : (listof string? '())

procedure

(delete-impure-port* url heads)  input-port?

  url : url?
  heads : (listof string? '())

procedure

(put-impure-port* url data heads)  input-port?

  url : url?
  data : bytes?
  heads : (listof string? '())

procedure

(post-impure-port* url data heads)  input-port?

  url : url?
  data : bytes?
  heads : (listof string? '())

parameter

(http-ver)  string?

(http-ver version)  void?
  version : string?
Variations of the net/url impure port functions, if you prefer that style for one-off requests.

These variations add the ability to do HTTP 1.1, and support for you including an Expect: 100-continue request header in heads. This allows a PUT or POST request to fail or redirect, before data is sent to the server.

To make these functions do HTTP 1.1 requests, you should set the http-ver parameter to "1.1".

Because HTTP 1.1 defaults to persistent connections, it would be nice for you to include a Connection: close request header in heads.

1.7 Date string conversion

procedure

(seconds->gmt-string [s])  string?

  s : exact-integer? = (current-seconds)

Examples:

> (seconds->gmt-string)

"Sat, 18 Aug 2012 16:41:59 GMT"

> (seconds->gmt-string 0)

"Thu, 01 Jan 1970 00:00:00 GMT"

procedure

(seconds->gmt-8601-string [style s])  string?

  style : (or/c 'plain 'T/Z 'T/.00Z) = 'T/Z
  s : exact-integer? = (current-seconds)

Examples:

> (define sc (current-seconds))
> (seconds->gmt-8601-string 'plain sc)

"2012-08-18 16:41:59"

> (seconds->gmt-8601-string 'T/Z sc)

"2012-08-18T16:41:59Z"

> (seconds->gmt-8601-string 'T/.000Z sc)

"2012-08-18T16:41:59.000Z"

procedure

(gmt-8601-string->seconds s)  exact-integer?

  s : string?

Examples:

> (gmt-8601-string->seconds "1970-01-01 00:00:00")

0

> (gmt-8601-string->seconds "1970-01-01T00:00:00Z")

0

> (gmt-8601-string->seconds "1970-01-01T00:00:00.000Z")

0