Abstract

Something that has long been missing from Fantom is an API for doing cryptography (crypto) operations. As the world has moved to a secure-by-default mentality, we need to address this gap. The following is a set of proposals for adding formal crypto support to Fantom.

Some high-level goals for the project include:

• generate and model public/private key pairs with ability to perform crypto operations such as sign/verify and encrypt/decrypt
• model certificates (X509)
• create key/trust stores
• a public API for creating server TLS sockets
• a public API for configuring Wisp for TLS

Proposal

In order to achieve these goals, there are a number of changes that we need to make. These fall largely into three categories:

1. new asn1 pod
2. new crypto pod
3. updated inet and wisp pods

ASN.1

We plan to add a new asn1 pod to Fantom. Many low-level crypto operations require a basic ASN.1 library for modeling components of public/private keys and generating CSRs etc. The entire PKI data model is essentially based on ASN.1. So the first step will be to add a new asn1 pod to Fantom.

The goal of this pod is primarily to deal with modeling ASN.1 objects and to provide a BER encoding/decoding of those objects. This will not be a "full" ASN.1 implementation, but will be geared towards what we need to support the crypto pod.

Work on this pod is done and will be posted to the git repo for review shortly. Here are some examples of working with the asn1 api:

seq := AsnColl.builder
  .add(Asn.oid("100.1.3"))
  .add(Asn.int(2, AsnTag.context(2).implicit)
  .seq
buf := BerWriter.toBuf(seq)
decoded := BerReader(buf.in).readObj

crypto

We propose adding a new crypto pod. This pod would only define an API for crypto operations - it would have no implementation. Crypto is complex enough that these operations should be delegated as much as possible to the native runtime. So we would also add a cryptoJava pod that contains an implementation of the various crypto APIs for Java.

Access to all crypto operations would be through the Crypto object. The final set of APIs is still in design, but would look something like this

const mixin Crypto
{
  static const Crypto cur := <default implementation for the runtime>

  ** CertFactory is used to load Certs encoded in a certain format (e.g. "X.509")
  abstract CertFactory certFactory(Str type)

  ** Get a Digest object for computing message digests
  abstract Digest digest(Str algorithm)

  ** Get an object for working with crypto keys
  abstract KeyTool keyTool()

  ** Get a keystore for storing keys
  abstract KeyStore keyStore(Str type)

  ... TBD ...
}

// load an x509 certificate
cert := Crypto.cur.certFactory("X.509").load(in)

// compute the SHA-256 message digest for the given data
buf := Crypto.cur.digest("SHA-256").update(block1).update(block2).digest

The initial goal for the crypto api is to have a public API for common crypto operations - especially those needed to support the enhancements we want to do to inet to make creating and working with TLS sockets easier.

inet

After we have a public API for crypto types we plan to make the following changes to the inet pod. We plan to redesign how sockets are obtained, while maintaining backwards compatibility (if possible, which we think it should be).

The primary goal is to have a SocketFactory API that can be used to obtain plain and tls sockets. Rather than creating a TcpSocket() manually, sockets should be obtained from factories moving forward.

This API is still undergoing design, but we foresee something like this

abstract const class TcpSocketFactory
{
  static const TcpSocketFactory plain := PlainTcpSocketFactory()

  ** Create a new `TcpSocket`. If 'wrap' is null, then a new, unbound
  ** socket will be returned. 
  ** If 'wrap' is not null, then it is implementation-dependent what the
  ** behavior is. For the default PlainSocketFactory, the passed-in socket
  ** is returned. For the default TlsSocketFactory, the passe-in socket will
  ** be "upgraded" to TLS
  abstract TcpSocket create(TcpSocket? wrap := null)
}

A TlSSocketFactory factory class will be provided that allows you to create TLS sockets. It can be configured to take a keystore to configure the underlying key and truststore for the socket.

Once these inet changes are in place we will update wisp so that it can be configured as either a "plain" server or "tls" server by passing in corresponding TcpSocketFactory implementations.

Feedback

Your feedback on these various proposals and changes is appreciated. The work is currently underway, so prompt feedback will enable us to make changes more easily. Thanks!

Hi Matthew,

Thanks for this, I've been doing a lot more work with certificates of late, so this would be a welcome addition. Although my uses are fairly high level and generally revolve around keyStores and certificate parsing.

The singleton approach to the Crypto instance seems a little odd to me, the static const Crypto cur, may I ask why? (I know there are good reasons - I'm just not clever enough to know what they are!)

With regards to inet, I've been using the TcpSocket.makeTls() ctor with a Java SSLContext quite successfully; usually using Java FFI and Interop to create the neccesary Context. But I do this to create client sockets that connect out, not for server sockets that listen for and accept connections.

The proposed factory method and wrapping mechanism seems rather awkward to me. There is obviously a very specific reason for wanting the TcpSocket? wrap parameter, but I don't know what you have in mind?

If create() really does need an argument on every invocation, then given TcpSocketFactory is an abstract, implementation specific, factory class - would it not be more useful / generic for create() to take an Obj? instead?

abstract TcpSocket create(Obj? arg := null)

And that's about all my 2 pence's worth - thanks for porting this over to the main Fantom distribution!

The singleton approach to the Crypto instance seems a little odd to me, the static const Crypto cur, may I ask why?

This allows for the crypto implementation to be pluggable, but accessible in a single, standard way. For example, this is the implementation right now, but could be enhanced to check a config property for override of the implementation:

** Get the installed crypto implementation for this runtime.
static const Crypto cur
static
{
  try
  {
    cur = Type.find("cryptoJava::JCrypto").make
    // But could do lookup based on Env.cur.runtime or check Pod.find("crypto").config("crypto.impl")
  }
  catch (Err err)
  {
    err.trace
    throw err
  }
}

The proposed factory method and wrapping mechanism seems rather awkward to me. There is obviously a very specific reason for wanting the TcpSocket? wrap parameter, but I don't know what you have in mind?

There could be a number of reasons - maybe you have a custom impl of TcpSocketFactory and want to do debug tracing for an existing socket. But the big reason is for server-side upgrade of plain sockets to TLS. We need to be able to listen and accept plain tcp sockets, and then "upgrade" them to TLS at a later time (i.e. in a separate actor so as not to block the listener).

I don't think we'd want that api to take an Obj? because the idea is

1. If the wrap param is null, create a new socket of the factory type.
2. Otherwise, create a new socket of the factory type that basically proxies the existing socket in some way.

You'd only ever be wrapping TcpSocket objects. What alternative types did you have in mind that you might pass to the create() method?

As outlined in this proposal, today I pushed a new asn1 pod to the Fantom git repository. This is the first phase of the proposal and a requirement for some of the crypto functionality we want to provide.

Hi Matthew,

Thanks for the explanations.

Re: Crypto singleton instance via static Crypto cur()

This allows for the crypto implementation to be pluggable, but accessible in a single, standard way.

For this, given it's a mixin, I tend to use static new make(). The implementation of both methods can be identical, it's more a matter of semantics as to whether the end user believes they are retrieving the same object or creating a new one. (In reality, the code could be doing either.)

For Crypto, given it's not really a disposable class, I think it's fine to keep the current cur() design if it is pluggable as you suggest. (I also know cur() a SkyFoundry design pattern and you've probably got lots of code using it right now'!)

Re: TcpSocketFactory.create() args

Wanting to upgrade existing sockets makes sense - thanks for explaining.

You'd only ever be wrapping TcpSocket objects. What alternative types did you have in mind that you might pass to the create() method?

Given TcpSocketFactory is abstract, it's really down to the implementation to decide what it needs in order to create() a socket. Maybe it needs SocketOptions, an SSLContext, certificate instances, or any other configuration / credentials it may require to conjure up a socket instance.

I guess a lot of this could be configured on the specific implementation, rather than through the create() method - I just mention it in case explicitly passing a TcpSocket instance could potentially be limiting.

But granted, I don't do a great deal of socket processing so I'm happy to take your lead! :)

Re: asn1

Looks great! :D Thanks for sharing!

Steve.

I also know cur() a SkyFoundry design pattern and you've probably got lots of code using it right now'!

That is actually a Fantom pattern we started with sys::TimeZone.cur and sys::Locale.cur.

Given TcpSocketFactory is abstract, it's really down to the implementation to decide what it needs in order to create() a socket.

In general those are the things you would configure on the factory, not the individual sockets.

Hello, I've just come to update my code that used the old TcpSocket.makeTls(javax.net.ssl.SSLContext ctx) ctor, but I'm struggling to workout what to replace it with?

I have an custom instance of javax.net.ssl.SSLContext, how would I use it to create an inet::TcpSocket?

Perhaps it's just late in the evening, but I'm not seeing an easy way to override or inject it into TcpSocket.upgradeTls().

I have an custom instance of javax.net.ssl.SSLContext, how would I use it to create an inet::TcpSocket?

Why are you making a custom javax.net.ssl.SSLContext?

The general way to make TLS socket is

socket := TcpSocket().upgradeTls

That will use the default platform (Java) key and trust stores internally when creating the TLS socket.

If you need a custom keystore or truststore, then you create a inet::SocketConfig instance with those fields configured

config := SocketConfig.cur.copy { 
  // you can set one or both of these
  it.keystore = myCustomKeyStore
  it.truststore = myCustomTrustStore
}

// create TLS socket using my custom socket configuration
socket := TcpSocket(config).upgradeTls

Hi Matthew,

I mostly create SSL Contexts for mutual 2-way SSL authentication.

I see now that the new crypto classes wrap all the usual Java stuff (e.g. KeyStores). I'll look into re-writing the code to use the new crypto pod.

Thanks for the code example, I missed the SocketConfig.copy() method with an it-block and was concerned there was no means to configure a SocketConfig at an instance level.

By the way, should SocketOptions now be deprecated, or are there reasons why we may still want / need to use both SocketConfig and SocketOptions?

Cheers!

By the way, should SocketOptions now be deprecated, or are there reasons why we may still want / need to use both SocketConfig and SocketOptions?

We'd eventually like to deprecate SocketOptions, but there is still some design work to do around that. There are cases where you want to modify the socket options after socket creation, and we will still need a way to do that. For now, SocketOptions is still the way to do that.