4. WebRepos

Overview

The fanr includes client and server implementations of repositories which may be used over a simple HTTP REST protocol.

WebRepoMod

The WebRepoMod class is a standard web module which allows you to publish a repository in your own web sites. This class wraps a backing store repository which is often a file repo and implements the server side REST protocol.

The WebRepoAuth class provides hooks for to integrate the repository into your web site's authentication and security model.

WebRepoMain

There is a super, simple implementation in the fanr pod which allows you to publish a file repo to the web using the following command line:

fan fanr::WebRepoMain /path/to/fileRepo -port 80

By default the server is public, or you can use the -username and -password options to protect it with a simple username/password combination.

REST Protocol

The following table specifies the URI namespace of the fanr REST protocol:

Method   Uri                       Operation
------   --------------------      ---------
GET      {base}/ping               ping meta-data
GET      {base}/find/{name}/{ver}  find pod
GET      {base}/query?{query}      pod query
POST     {base}/query              pod query
GET      {base}/pod/{name}/{ver}   pod download
POST     {base}/publish            publish pod
GET      {base}/auth?{username}    authentication info

The protocol defines a set of "Fanr-" headers for authentication and specifying options to operations. The payload of all operations is either a pod file or a JSON data structure. Errors are indicated with an HTTP status code and JSON error message.

The following sections details the various features of the protocol:

Authentication: authentication and digital signatures
Ping: ping a server's meta-data
Find: find exact match for pod name/version
Query: query the repository for set of pods
Read: download a pod for installation
Publish: upload a pod to add to the repository
Errors: error handling

REST Authentication

The following set of HTTP headers are defined to support authentication:

Fanr-Username: username string to identify authentication account
Fanr-SignatureAlgorithm: algorithm used to perform the digital signature of the headers
Fanr-SecretAlgorithm: algorithm used to determine the secret key used for the digital signature
Fanr-Signature: base64 encoded digital signature of headers
Fanr-Ts: timestamp formatted as DateTime string which is used a time based nonce for signatures

A digital signature of the request method, request URI (relative to authority), and "Fanr-" headers is used to verify the credentials of the user account. The process for signing a request is as follows:

Query "{base}/auth?{username}" to discover which algorithms are supported and acquire public salt for user (if needed)
Determine how to compute the secret key for the signing process which might be the password itself or a salted hash of the password
Create a normalized string representation of the request method, request URI, and "Fanr-" headers
Sign the headers from step 3 and add the "Fanr-Signature" header which the server can use to verify the user's credentials

The first step to the authentication process is to make a GET request to the "auth" URI with the username as a query string. The response will be a JSON data structure with a map of name/value pairs:

{
 "username":"someone",
 "salt":"3d98fe2bc7cd13e02344a76400e1c212",
 "secretAlgorithms":"PASSWORD,SALTED-HMAC-SHA1",
 "signatureAlgorithms":"HMAC-SHA1",
 "ts":"2011-07-13T14:50:01.865Z UTC"
}

The following keys are specified in the response:

username: username passed in URI query
salt: if salted hashes are used for secret, this is the salt to use for given user
secretAlgorithms: comma separated list of algorithms which server supports for computing the secret key used in signatures
signatureAlgorithms comma separated list of algorithms which server supports for computing the digital signature
ts current server time in DateTime format

The secret is either the user's password or some digest of the password. Two algorithms are supported

PASSWORD: the user's plain text password encoded as UTF-8 is used to create the digital signature (this option requires server to store the user's actual password)
SALTED-HMAC-SHA1: salted digest of password, computed as follows:
```
Buf().print("$username:$salt").hmac("SHA-1", password.toBuf)
```

The signature algorithm is how we use the secret to create a digital signature of the request. Only one algorithm is currently supported:

HMAC-SHA1: computed as follows:
```
normReq.hmac("SHA-1", secret).toBase64
```

The normalized request is a UTF-8 encoded string calculated as follows:

find all the headers which begin with "Fanr-", but excluding the "Fanr-Signature" header itself
normalize the header to lower case
sort the headers by key
encode HTTP method in upper case followed by "\n"
encode full the URI in lowercase followed by "\n"
encode each header as "{key}:{value}" followed by "\n"

Here is a full example:

// credentials
username: "bob"
password: "xyz"
salt: "7fff2a65234b8cb5a97d8e69e5dc3ef4"

// secret computation using SALTED-HMAC-SHA1
Buf().print("bob:7fff2a65234b8cb5a97d8e69e5dc3ef4").hmac("SHA-1", "xyz".toBuf)

// secret base64 encoded
"zAEESAZCtn/f1ahhAmXpoC1sZi4="

// normalized request string
GET
http://localhost/ping
fanr-secretalgorithm:SALTED-HMAC-SHA1
fanr-signaturealgorithm:HMAC-SHA1
fanr-ts:2011-07-13T15:14:42.671Z UTC
fanr-username:bob

// signature computation
normReq.hmac("SHA-1", secret)

// request headers with base64 digital signature
Fanr-Username: bob
Fanr-SecretAlgorithm: SALTED-HMAC-SHA1
Fanr-SignatureAlgorithm: HMAC-SHA1
Fanr-Ts: 2011-07-13T15:14:42.671Z UTC
Fanr-Signature: 0/dpJysIs8ajx8032WgmPIPrFD0=

REST Ping

The "ping" URI is used to query the server to check that is alive, test credentials, and query server metadata. The "ping" URI is always publicly accessible, although if "Fanr-Username" is specified then authentication will be checked and an error returned if credentials are invalid.

The ping response is a JSON map of string name/value pairs:

{"fanr.version":"1.0.59",
 "fanr.type":"fanr::WebRepo",
 "ts":"2011-07-13T11:39:03.256-04:00 New_York"}

Also see Repo.ping and WebRepoMod.pingMeta.

REST Find

The "find" URI is used to perform a an exact find.

Server side permission for the "find" URI is controlled by the WebRepoAuth.allowQuery method.

The find response is a JSON map containing the the pod metadata as string name/value pairs. If the pod is not found, then 404 is returned.

// request
GET http://localhost/find/xml/1.0.59

// response
{
  "pod.name":"xml",
  "pod.version":"1.0.59",
  "pod.depends":"sys 1.0",
  "pod.summary":"XML Parser and Document Modeling",
}

REST Query

The "query" URI is used to perform a repo query. The query string may be passed in the URI query parameter using a GET request or may be passed as the request body of a POST request.

Server side permission for the "query" URI is controlled by the WebRepoAuth.allowQuery method.

The query response is a JSON map containing the "pods" key which is a list of pod metadata as string name/value pairs.

// request
GET http://localhost/query?xml

// response
{"pods":[
  {
    "pod.name":"xml",
    "pod.version":"1.0.59",
    "pod.depends":"sys 1.0",
    "pod.summary":"XML Parser and Document Modeling",
  },
  {
    "pod.name":"xml",
    "pod.version":"1.0.58",
    "pod.depends":"sys 1.0",
    "pod.summary":"XML Parser and Document Modeling",
  }
]}

You can specify the "Fanr-NumVersions" header to limit the number of versions returned for each pod. The default version limit is three.

REST Read

The "pod" URI is used to download a specific version of a pod. The URI is formatted as following to identify the pod and its version:

// format
{base}/pod/{name}/{version}

// example
fanr/pod/acmeWidgets/1.3.67

Server side permission for the "pod" URI is controlled by the WebRepoAuth.allowRead method.

If the request is successful, then the request body will contain the pod file itself.

REST Publish

The "publish" URI is used to upload a new pod to the repository.

Server side permission for the "publish" URI is controlled by the WebRepoAuth.allowPublish method.

Publication is performed by POSTing a pod file to the "publish" URI. It is recommended to use the "Expect: 100-continue" header to verify permissions before actually posting the file. If the publication is successful, then a JSON data structure is returned with the pod metadata:

{"published": {
   "pod.name":"acmeWidgets",
   "pod.version":"1.3.68",
   "pod.depends":"sys 1.0, util 1.0",
  }
}

REST Errors

If an error is encountered during a HTTP request, then a 4xx or 5xx status code is returned with a JSON payload indicating the error message. The JSON message is formatted as follows:

{"err":"error message here"}

The following are common error status codes:

401: authentication was missing or invalid
403: request was authenticated, but user is not allowed to perform operation
404: invalid URI or resource not found
500: internal server error
501: HTTP method not supported for given URI

Fantom