Using a Custom Certificate Trust Store on Android

As mentioned in a previous post, Android 4.0 (ICS) adds both a system UI and SDK API's that let you add certificates to the system trust store. On all previous version though, the system trust store is read-only and there is no way to add certificates on non-rooted devices. Therefore, if you want to connect to a server that is using a certificate not signed by one of the CA's included in the system trust store (including a self-signed one), you need to create and use a private trust store for the application. That is not particularly hard to do, but 'how to connect to a server with a self-signed certificate' is one of the most asked Android questions on StackOverflow, and the usual answer goes along the lines of 'simply trust all certificates and you are done'. While this will indeed let you connect, and might be OK for testing, it defeats the whole purpose of using HTTPS: your connection might be encrypted but you have no way of knowing who you are talking to. This opens the door to man-in-the-middle (MITM) attacks, and, needless to say, is bad practice. In this post we will explore how Android's HTTPS system works pre-ICS and show how to create and use a custom certificate trust store and a dynamically configurable TrustManager.

Some background: JSSE

Java, and by extension Android, implement SSL using a framework called Java Secure Socket Extension (JSSE). A discussion of how SSL and JSSE work is beyond the scope of this post, but you can find a shot introduction to SSL in the context of JSSE here. In brief, SSL provides both privacy and data integrity (i.e., an encrypted communications channel) and authentication of the parties involved. Authentication is implemented using public key cryptography and certificates. Each party presents their certificate, and if the other party trusts it, they negotiate a shared key to encrypt communications using the associated key pairs (public and private). JSSE delegates trust decisions to a TrustManager class, and authentication key selection to a KeyManager class. Each SSLSocket instance created via JSSE has access to those classes via the associated SSLContext (you can find a pretty picture here). Each TrustManager has a set of trusted CA certificates (trust anchors) and makes trust decisions based on those: if the target party's certificate is issued by one of the trusted CA's, it is considered trusted itself.

One way to specify the trust anchors is to add the CA certificates to a Java key store file, referred to as a 'trust store'. The default JSSE TrustManager is initialized using the system trust store which is generally a single key store file, saved to a system location and pre-populated with a set of major commercial and government CA certificates. If you want to change this, you need to create an appropriately configured TrustManager instance, either via a TrustManagerFactory, or by directly implementing the X509TrustManager interface. To make the general case where one just wants to use their own key store file to initialize the default TrustManager and/or KeyManager, JSSE provides a set of system properties to specify the files to use.

Android and javax.net.ssl.trustStore

If you want to specify your own system trust store file in desktop Java, it is just a matter of setting a value to the javax.net.ssl.trustStore system property when starting the program (usually using the -D JVM command line parameter). This property is also supported on Android, but things work a little differently. If you print the value of the property it will most likely be /system/etc/security/cacerts.bks, the system trust store file (pre-ICS; the property is not set on ICS). This value is used to intialize the default TrustManagerFactory, which in turn creates an X.509 certificate-based TrustManager. You can print the current trust anchors like this:

TrustManagerFactory tmf = TrustManagerFactory
                    .getInstance(TrustManagerFactory.getDefaultAlgorithm());
tmf.init((KeyStore) null);
X509TrustManager xtm = (X509TrustManager) tmf.getTrustManagers()[0];
for (X509Certificate cert : xtm.getAcceptedIssuers()) {
    String certStr = "S:" + cert.getSubjectDN().getName() + "\nI:"
                        + cert.getIssuerDN().getName();
    Log.d(TAG, certStr);
}

If you now use System.setProperty() to point the property to your own trust store file, and run the above code again, you will see that it outputs the certificates in the specified file. Check the 'Set javax.net.ssl.trustStore' checkbox and use the 'Dump trusted certs' button of the sample app to try it.

If we can change the set of trusted certificates using this property, connecting to a server using a custom certificate should be easy, right? It turns out this is not the case. You can try it yourself using the sample app: pressing 'Default Connect' will result in a 'Trust anchor for certificate path not found' error regardless of the state of the 'Set javax.net.ssl.trustStore' checkbox. A little further investigation reveals that the default SSLContext is already initialized with the system trust anchors and setting the javax.net.ssl.trustStore property does not change this. Why? Because Android pre-loads system classes, and by the time your application starts, the default SSLContext is already initialized. Of course, any TrustManager's you create after setting the property will pick it up (see above).

Using your own trust store: HttpClient

Since we can't use the 'easy way' on Android, we need to specify the trust store to use programmatically. This is not hard either, but first we need to create a key store file with the certificates we need. The sample project contains a shell script that does this automatically. All you need is a recent Bouncy Castle jar file and the openssl command (usually available on Linux systems).  Drop the jar and a certificate (in PEM format) in the script's directory and run it like this:

$ ./importcert.sh cacert.pem

This will calculate the certificate subject's hash and use it as the alias in a Bouncy Castle key store file (BKS format) created in the application's raw/ resource directory. The script deletes the key store file if it already exists, but you can easily modify it to append certificates instead. If you are not the command-line type, you can use the Portecle GUI utility to create the key store file.

Apache's HttpClient provides a convenient SSLSocketFactory class that can be directly initialized with a trust store file (and a key store file if client authentication is needed). All you need to do is to register it in the scheme registry to handle the https scheme:

KeyStore localTrustStore = KeyStore.getInstance("BKS");
InputStream in = getResources().openRawResource(R.raw.mytruststore);
localTrustStore.load(in, TRUSTSTORE_PASSWORD.toCharArray());

SchemeRegistry schemeRegistry = new SchemeRegistry();
schemeRegistry.register(new Scheme("http", PlainSocketFactory
                .getSocketFactory(), 80));
SSLSocketFactory sslSocketFactory = new SSLSocketFactory(trustStore);
schemeRegistry.register(new Scheme("https", sslSocketFactory, 443));
HttpParams params = new BasicHttpParams();
ClientConnectionManager cm = 
    new ThreadSafeClientConnManager(params, schemeRegistry);

HttpClient client = new DefaultHttpClient(cm, params);

Once initialized like this, the HttpClient instance will use our local trust store when verifying server certificates. If you need to use client authentication as well, just load and pass the key store containing the client's private key and certificate to the appropriate SSLSocketFactory constructor. See the sample project for details and use the 'HttpClient SSLSocketFactory Connect' button to test. Note that, when initialized like this, our HttpClient will use only the certificates in the specified file, completely ignoring the system trust store. Thus connections to say, https://google.com will fail. We will address this later.

Using your own trust store: HttpsURLConnection

Another popular HTTPS API on Android is HttpsURLConnection. Despite the not particularly flexible or expressive interface, apparently this is the preferred API from Android 2.3 (Gingerbread) and on. Whether to actually use is it is, of course, entirely up to you :) It uses JSSE to connect via HTTPS, so initializing it with our own trust and/or key store involves creating and initializing an SSLContext (HttpClient's SSLSocketFactory does this behind the scenes):

KeyStore trustStore = loadTrustStore();
KeyStore keyStore = loadKeyStore();

TrustManagerFactory tmf = TrustManagerFactory
                    .getInstance(TrustManagerFactory.getDefaultAlgorithm());
tmf.init(trustStore);

KeyManagerFactory kmf = KeyManagerFactory
                    .getInstance(KeyManagerFactory.getDefaultAlgorithm());
kmf.init(keyStore, KEYSTORE_PASSWORD.toCharArray());

SSLContext sslCtx = SSLContext.getInstance("TLS");
sslCtx.init(kmf.getKeyManagers(), tmf.getTrustManagers(), null);

URL url = new URL("https://myserver.com");
HttpsURLConnection urlConnection = (HttpsURLConnection) url
urlConnection.setSSLSocketFactory(sslCtx.getSocketFactory());

In this example we are using both a trust store and a key store, but if you don't need client authentication, you can just pass null as the first parameter of SSLContext.init().

Creating a dynamic TrustManager

As mentioned above, a TrustManager initialized with a custom trust store will only use the certificates in that store as trust anchors: the system defaults will be completely ignored. Sometimes this is all that is needed, but if you need to connect to both your own server and other public servers that use HTTPS (such as Twitter, for example), you will need to create two separate instances of HttpClient or HttpsURLConnection and switch between the two. Additionally, since the trust store is stored as an application resource, there is no way to add trusted certificates dynamically, you need to repackage the application to update the trust anchors. Certainly we can do better than that. The first problem is easily addressed by creating a custom TrustManager that delegates certificate checks to the system default one and uses the local trust store if verification fails. Here's how this looks like:

public class MyTrustManager implements X509TrustManager {

    private X509TrustManager defaultTrustManager;
    private X509TrustManager localTrustManager;

    private X509Certificate[] acceptedIssuers;

    public MyTrustManager(KeyStore localKeyStore) { 
      // init defaultTrustManager using the system defaults
      // init localTrustManager using localKeyStore
    }

    public void checkServerTrusted(X509Certificate[] chain, String authType)
            throws CertificateException {
        try {
            defaultTrustManager.checkServerTrusted(chain, authType);
        } catch (CertificateException ce) {
            localTrustManager.checkServerTrusted(chain, authType);
        }
    }
    
    //...
}

To address the second problem, we simply copy the trust store to internal storage when we first start the application and use that file to initialize our TrustManager's. Since the file is owned by the application, you can easily add and remove trusted certificates. To test modifying the trust store works, copy a certificate file(s) in DER format to the SD card (external storage) root and use the sample application's 'Add certs' and 'Remove certs' menus to add or remove it to/from the local trust store file. You can then verify the contents of the file by using the 'Dump trusted certs' button (don't forget to check 'Set javax.net.ssl.trustStore'). To implement this the app simply uses the JCE KeyStore API to add or remove certificates and save the trust store file:

CertificateFactory cf = CertificateFactory.getInstance("X509");
InputStream is = new BufferedInputStream(new FileInputStream(certFile));
X509Certificate cert = (X509Certificate) cf.generateCertificate(is);
String alias = hashName(cert.getSubjectX500Principal());
localTrustStore.setCertificateEntry(alias, cert);

FileOutputStream out = new FileOutputStream(localTrustStoreFile);
localTrustStore.store(out, TRUSTSTORE_PASSWORD.toCharArray());

Using our MyTrustManager with HttpsURLConnection is not much different than using the default one:

MyTrustManager myTrustManager = new MyTrustManager(localTrustStore);
TrustManager[] tms = new TrustManager[] { myTrustManager };
SSLContext sslCtx = SSLContext.getInstance("TLS");
context.init(null, tms, null);

HttpsURLConnection urlConnection = (HttpsURLConnection) url
                            .openConnection();
urlConnection.setSSLSocketFactory(sslCtx.getSocketFactory());

HttpClient's SSLSocketFactory doesn't let us specify a custom TrustManager, so we need to create our own SocketFactory. To make initialization consistent with that of HttpsURLConnection, we have it take an already initialized SSLContext as a parameter and use it to get a factory that lets us create SSL sockets as needed:

public class MySSLSocketFactory implements LayeredSocketFactory {

    private SSLSocketFactory socketFactory;
    private X509HostnameVerifier hostnameVerifier;

    public MySSLSocketFactory(SSLContext sslCtx,
            X509HostnameVerifier hostnameVerifier) {
        this.socketFactory = sslCtx.getSocketFactory();
        this.hostnameVerifier = hostnameVerifier;
    }

    //..

    @Override
    public Socket createSocket() throws IOException {
        return socketFactory.createSocket();
    }
}

Initializing an HttpClient instance is now simply a matter of registering our socket factory for the https scheme:

SSLContext sslContext = createSslContext();
MySSLSocketFactory socketFactory = new MySSLSocketFactory(
                       sslContext, new BrowserCompatHostnameVerifier());
schemeRegistry.register(new Scheme("https", sslSocketFactory, 443));

You can check that this actually works with the 'HttpClient Connect' and 'HttpsURLConnection Connect' buttons of the sample application. Both clients are using our custom TrustManager outlined above and trust anchors are loaded dynamically: adding and removing certificates via the menu will directly influence whether you can connect to the target server.

Summary

We've shown how the default TrustManager on pre-ICS Android devices works and how to set up both HttpClient and HttpsURLConnection to use a local (application-scoped) trust and/or key store. In addition, the sample app provides a custom TrustManager implementation that both extends the system one, and supports dynamically adding and removing application-specified trust anchors. While this is not as flexible as the system-wide trust store introduced in ICS, it should be sufficient for most applications that need to manage their own SSL trust store. Do use those examples as a starting point and please do not use any of the trust-all 'solutions' that pop up on StackOverflow every other day.