CloudFlare Keyless SSL

This repository contains a reference implementation of CloudFlare's keyless
SSL server.

Protocol

The CloudFlare Keyless SSL client communicates to the server via a binary
protocol over a mutually authenticated TLS 1.2 tunnel. Messages are in binary
format and identified by a unique ID.

Messages consist of a fixed length header, and a variable length body. The
body of the message consists of a sequence of items in TLV (tag, length,
value) messages.

All messages with major version 1 will conform to the following
format. The minor version is currently set to 0 and is reserved for
communicating policy information.

Header:

0 - - 1 - - 2 - - 3 - - 4 - - - - 6 - - 7 - - 8, Maj, Min, Length, ID, Body, Body, <- 8 + Length

Item:

0 - - 1 - - 2 - - 3 - - 4 - - - - 6 - - 7 - - 8, Tag, Length, Data, Data, <- 3 + Length

All numbers are in network byte order (big endian).

The following tag values are possible for items:

0x01 - Certificate Digest,
0x02 - Server Name Indication,
0x03 - Client's IP address,
0x11 - Opcode,
0x12 - Payload,

A requests contains a header and the following items:

0x01 - length: 32 bytes, data: SHA256 of RSA modulus
0x02 - length: variable, data: SNI string
0x03 - length: 4 or 16 bytes, data: IPv4/6 address
0x11 - length: 1, data: opcode describing operation
0x12 - length: variable, data: payload to sign or encrypt

The following opcodes are supported in the opcode item:

0x01 - operation: RSA decrypt payload 
0x02 - operation: RSA sign MD5SHA1
0x03 - operation: RSA sign SHA1
0x04 - operation: RSA sign SHA224
0x05 - operation: RSA sign SHA256
0x06 - operation: RSA sign SHA384
0x07 - operation: RSA sign SHA512
0x08 - operation: RSA raw decrypt payload
0x12 - operation: ECDSA sign MD5SHA1
0x13 - operation: ECDSA sign SHA1
0x14 - operation: ECDSA sign SHA224
0x15 - operation: ECDSA sign SHA256
0x16 - operation: ECDSA sign SHA384
0x17 - operation: ECDSA sign SHA512

Responses contain a header with a matching ID and only two items:

0x11 - length: 1, data: opcode describing operation status
0x12 - length: variable, data: payload response

The following opcodes are supported in the opcode item:

0xF0 - operation: success, payload: modified payload
0xFF - operation: RSA decrypt payload, payload: 

On an error, these are the possible 1-byte payloads:

0x01 - cryptography failure
0x02 - key not found - no matching certificate ID
0x03 - read error - disk read failure
0x04 - version mismatch - unsupported version incorrect
0x05 - bad opcode - use of unknown opcode in request
0x06 - unexpected opcode - use of response opcode in request
0x07 - format error - malformed message
0x08 - internal error - memory or other internal error

Defines and further details of the protocol can be found in kssl.h

Key Management

The Keyless SSL server is a TLS server and therefore requires cryptographic
keys. All requests are mutually authenticated, so both the client and the
server need a TLS 1.2 compatible key pair. The client must present a client
certificate that can be verified against the CA that the keyless server is
configured to use.

The server will need a valid key and certificate pair in PEM format. The
following options are required and take a path to these files. These two
parameters set up the certificate (and associated private key) that will be
presented by the server when a client connects.

 --server-cert 
 --server-key

The private keys that this server is able to use should be stored in
PEM format in a directory denoted by the option:

--private-key-directory

In order to authenticate the client's certificate, a custom CA file is
required. This CA file available is provided by CloudFlare and specified
with:

--ca-file

Deploying

Installing

Source

Use Git to get the latest development version from our repository:

git clone https://github.com/cloudflare/keyless.git
cd keyless
make && make install

Alternatively you can just
download the
bleeding edge code directly.

Packages

Running

A typical invocation of keyless might look like:

keyless --port=2412 --server-cert=server-cert/cert.pem \
        --server-key=server-cert/key.pem               \
        --private-key-directory=keys                   \
        --ca-file=CA/cacert.pem                        \
        --pid-file=keyless.pid                         \
        --num-workers=4 --daemon --silent              \
        --user nobody:nobody

That runs the keyless server as a daemon process (and outputs the
parent PID in keyless.pid) after changing to the user nobody in
group nobody.

It sets up four workers (threads) which will process connections from
CloudFlare handling cryptographic requests using the private keys from
a directory called keys.

Command-line Arguments

This is the keyserver for Keyless SSL. It consists of a single binary file
'kssl_server' that has the following command-line options:

• --port (optional) The TCP port on which to listen for connections. These
  connections must be TLSv1.2. Defaults to 2407.
• --ip (optional) The IP address of the interface to bind to. If missing
  binds to all available interfaces.
• --ca-file Path to a PEM-encoded file containing the CA certificate used to
  sign client certificates presented on connection.
• --server-cert, --server-key Path to PEM-encoded files containing the
  certificate and private key that are used when a connection is made to the
  server. These must be signed by an authority that the client side recognizes
  (e.g. the same CA as --ca-file).
• --private-key-directory Path to a directory containing private keys which
  the keyserver provides decoding service against. The key files must end with
  ".key" and be PEM-encoded. There should be no trailing / on the path.
• --silent Prevents keyserver from producing any log output. Fatal start up
  errors are sent to stderr.
• --verbose Enables verbose logging. When enabled access log data is sent to
  the logger as well as errors.
• --num-workers (optional) The number of worker threads to start. Each
  worker thread will handle a single connection from a KSSL client. Defaults
  to 1.
• --pid-file (optional) Path to a file into which the PID of the
  keyserver. This file is only written if the keyserver starts successfully.
• --test (optional) Run through program start up and check that the keyless
  server is correctly configured. Returns 0 if good, 1 if an error.

The following options are not available on Windows systems:

• --user (optional) user and group to switch to. Can be in the form
  user:group or just user (in which case user:user is implied) (root
  only)
• --daemon (optional) Forks and abandons the parent process.
• --syslog (optional) Log lines are sent to syslog (instead of stdout or
  stderr).

Developing

Code Organization

The code is split into several files by function in order to enable
swapping with custom implementations.

kssl.h              contains the shared constants and structures
kssl_core.h         APIs for performing the keyless operation
kssl_helpers.h      APIs for serialization and parsing functions
kssl_private_key.h  APIs for storing and matching private keys
kssl_log.h          APIs for writing logs

keyless.c           Sample server implementation with OpenSSL and libuv
testclient.c        Client implementation with OpenSSL

The following files are reference implementations of the APIs above.

kssl_core.c         Implementation of v1.0 policy for keyless operation
kssl_helpers.c      Implementation of v1.0 serialization and parsing
kssl_private_key.c  Implementation of reading, storage and operations of
                    private keys using OpenSSL
kssl_log.c          Implementation of logging

Prerequisites

On Debian-based Linuxes:

sudo apt-get install gcc automake libtool
sudo apt-get install rubygems # only required for packages
sudo gem install fpm --no-ri --no-rdoc # only required for packages

On Centos:

sudo yum install gcc automake libtool
sudo yum install rpm-build rubybgems ruby-devel # only required for packages
sudo gem install fpm --no-ri --no-rdoc # only required for packages

On OS X (homebrew):

sudo gem install fpm

Makefile

The Makefile has the following useful targets:

• all - The default target that builds both the keyless server and the
  testclient
• clean - Deletes the keyless server, testclient and related object files
• install - Install the keyless server
• run - Runs the keyless server with a configuration suitable for testing
  (with the testclient)
• kill - Stops the keyless server started by 'make run'
• test - Runs the testclient against the keyless server
• release - Increment the minor version number and generate an updated
  RELEASE_NOTES with all changes to keyless since the last time a release was
  performed.
• package - build and make app package for specific OS. e.g. deb for Debian

Building

The Keyless SSL server implementation has two external dependencies,
OpenSSL and
libuv. These are open source and available
for most platforms. For ease of deployment and consistency these dependencies
are statically compiled by default.

For Unix-based systems, the server and test suite are built with a GNU make
makefile.

To build:

make

This will create the files o/testclient, o/keyless after downloading and
building OpenSSL and libuv.

To test:

make test

This runs the testclient against the keyless server using test
certificates and keys provided in the repository.

There is also a short version of the test suite that can be used to test that
the keyless server works (at all!):

make test-short

License

See the LICENSE file for details. Note: the license for this project is not
'open source' as described in the Open Source
Definition.