Secure file exchange with .NET Crypto API

• Download demo project - 819 KB

Introduction

I have recently found interest in online storage services like Skydrive, Dropbox or Box, just to mention a few. I've been working on secure protocols and cryptography many times in my engineer life and I decided to find a solution to a simple requirement: exchange documents with other people confidentially using those online storage. 

The requirements

Online storage allow you to store documents online and eventually share them with other users. Some of those storage services encrypt the data on their servers but most free one like Skydrive simply store data without any encryption. Even with services that encrypt the data locally when data are transferred on the internet they are no longer encrypted. So far only FileLocker seems to provide encryption during data transfer.

In order to create a secure exchange when sharing documents you need to have the following features:

• Data need to be encrypted where they are stored but although while transferring 
• The owner of the data must be able to select the recipients he wishes to share the data with
• The recipient should be able to control the integrity of the data 

Those are standard features you expect when securely exchanging data, so I added some other requirements to fit the constraint of an exchange of the data on the internet. 

• The overhead of the exchange information must be small 
• The exchange protocol must be robust 

A few years ago, I worked on a very interesting project, I had to implement from scratch WS-SECURECONVERSATION on a prototype of smart card running the .NET CLR. This security protocol is a perfect candidate for my exchange file format. Secure conversation is used to securely transport data between two end-points and is part of the WS-* specifications. Each end-point uses a certificate and they have to register each other to exchange data. In the case of this data exchange it's a bit different; a document owner may want to share the same document with several other persons he knows. In secure conversation, data are encrypted with symmetric algorithm like AES and the key is encrypted with the RSA public key of the recipient. Encrypted data are sent together with the encrypted AES key, then the recipient can decipher the key with his RSA private key and finally decipher the data. So if there are several recipients, the idea is to encrypt the AES key with the RSA key of each recipient and attach all the encrypted keys with the encrypted data. Each recipient can use his public key to get decryption key and get the data. In addition to that, the owner can sign the data before they are encrypted using his RSA private key. That way the recipients can be sure that the data were encrypted by the rightful owner and that they can trust them.  

The other part of the requirements I have is to keep the overhead of the exchange information as small as possible and that the protocol must be robust. By choosing RSA and AES for the encryption algorithm I'm sure that the security of the exchange will be extremely solid. I have decided to use JSON serialization to store the exchange information header with the encrypted data. JSON provides a very compact serialization which makes it a perfect candidate. It also has the advantage of being neutral and very flexible. Also I'm giving a C# implementation the whole code could be re-written using any language that supports JSON serialization and has the proper encryption API. 

The header contains the following information about the encrypted data:

• Original file name 
• MIME format of the file (optional) 
• Application that can open it (optional) 
• Encryption algorithm name. Default is AES 256. (optional)
• Signature with the (RSA) certificate of the user who encrypted the data, Base64 encoded. (optional)
• ID of the user who encrypted the data. 
• Encrypted key with the (RSA) public key of the recipient. There must be at least one if the file is to be shared. Base64 encoded. 

An example of the exchange header serialized with JSON is given below:

{
    "fileName":"file.txt",
    "mime":"text/plain",
    "application":"notepad.exe",
    "encryptionAlgorithm":"AES256",
    "userID":"file.owner@mailaddress.com",
    "signature":
    {
        "digest":"RG1NtSDQ2xZ1UwlTXZM1q25S23hGWNaDYTkxd5h ... jUz4jj8Bilpeb6ADrf2dYAxDg=",
        "algorithm":"http://www.w3.org/2000/09/xmldsig#rsa-sha1",
        "hash":"SHA1"
    },
    "encryptedKeys":
    [
        {
            "userID":"file.recipient@mailaddress.com",
            "encrypted":
            {
                "digest":"laOYGSZvKWxv5FxtCGxxttY1VkDbwLSmLtTd ... S32I5gYl0xSOP/P28HrFVFcwAxq1ElnM=",
                "algorithm":"RSA-PKCS1-KeyEx",
                "hash":"SHA1"
            }
        }
    ]
}

The size of the header depends on the number of recipient as the digest are by far the largest object to serialize. A typical header with two recipients and the signature is less than 1000 bytes.

In order to make this process really secure there are few rules to respect:

• Each file must be encrypted with a fresh symmetric key. In case of a leak of the encryption key, only that file could be deciphered  If the same key is used by the owner to encrypt several files, recipients that are not suppose to be able to decipher that file might be able to do it.
• The public keys must be exchanged in a trusted manner. Certificate are based on trust so when you get one from a user you want to share something depending on the level of confidentiality the certificate should be signed by a certificate authority. 
• Private keys MUST never be shared asthey are used to decipher data and emit signatures. The safest certificate stores currently available are smart cards or hardware security modules (HSM).

A sample implementation in C#

The purpose of this article is not to give a complete implementation of that exchange protocol but the attached solution contains a working set of classes that demonstrate how to implement it using some of the .NET cryptography APIs. A unit test validates the most important classes.

Below is the method that tests the complete process of encrypting the data, creating the header and generating the raw bytes to be stored in a file or transmitted on the network.

public void TestFileExchange()
{
    // Load the file to encrypt
    byte[] imgData = File.ReadAllBytes(IMG_FILE_NAME);

    AESEncryptor aesEncryptor = new AESEncryptor(PASSWORD);

    RSACryptoServiceProvider rsaProviderOfRecipient = new RSACryptoServiceProvider();
    RSAOAEPEncryptor rsaDigestEncrypt = new RSAOAEPEncryptor(rsaProviderOfRecipient);

    RSACryptoServiceProvider rsaProviderOfOwner = new RSACryptoServiceProvider();
    RSASHA1Signature rsaDigestSigned = new RSASHA1Signature(rsaProviderOfOwner);

    // Encrypt the file data, the key and sign the original file data
    EncryptedFile encryptFile = new EncryptedFile(imgData,
        new FileDescription(IMG_FILE_NAME, MIME_JPG, APP_SLIDESHOW, ALGO_AES),
        aesEncryptor,
        new Recipient[] { new Recipient(USER_ID_DEST1, rsaDigestEncrypt) },
        new Owner(USER_ID_SRCE, rsaDigestSigned));

    // Build an EncryptedFile instance from the encrypted content with header
    EncryptedFile encryptFileOut = new EncryptedFile(encryptFile.EncryptedContent);

    ExchangeDataHeader encryptedHeader = encryptFileOut.EncryptedHeader;

    // Process the encrypted DigestData to extract the AES key
    IDigestEncryptor encryptDigest = RSADigestFactory.CreateDigestData(
      encryptedHeader.EncryptedKeys.Where(k => k.UserID == USER_ID_DEST1).First().Encrypted, 
      rsaProviderOfRecipient) as IDigestEncryptor;
    byte[] decryptedKeyAndIV = encryptDigest.Decrypt();

    IEncryptProcess aesDecryptor = new AESEncryptor(decryptedKeyAndIV);
    byte[] decryptedFileData = aesDecryptor.DecryptData(encryptFileOut.EncryptedFileData);

    // Process the Signature DigestData
    IDigestSignature signDigest = RSADigestFactory.CreateDigestData(
      encryptedHeader.Signature, rsaProviderOfOwner) as IDigestSignature;
    bool verified = signDigest.Verify(decryptedFileData);
    Assert.IsTrue(verified);
} 

An interesting aspect of the header serialization is that JSON doesn't store any type information unlike a binary serialization or an XML serialization. This could appear to be a drawback of JSON but it is in fact a great advantage in term of flexibility and extensibility. 

The following class diagram shows the architecture of the digest data that is used to transport the encrypted AES key and the data signature. 

When either RSAOAEPEncryptor or RSASHA1Signature is serialized using JSON only the members of DigestData are serialized. The class DigestData contains information that are used by the DigestDactory class to recreate an instance of the correct class after deseriliazing it into a DigestData object.

I tried to use the factory during the deserialization but the JSON implementation I'm using doesn't seem to support this possibility. I would need to read some of the data and then use the factory to instantiate the correct class but it is not possible. The solution I adopted is to deserialized into a DigestData instance and then construct an object with the DigestData instance but of a "concrete" type.

The code of DigestData, RSABase, and RSAOAEPEncryptor is given below.

[Serializable]
public class DigestData
{
    [JsonProperty]
    private string algorithm;

    [JsonProperty]
    private string hash;

    [JsonProperty]
    protected string digest;

    protected DigestData()
    {
    }

    public DigestData(string algorithm, string hash)
    {
        this.algorithm = algorithm;
        this.hash = hash;
    }

    [JsonIgnore]
    public string Algorithm
    {
        get { return algorithm; }
    }

    [JsonIgnore]
    public string Hash
    {
        get { return hash; }
    }

    [JsonIgnore]
    public string Digest
    {
        get { return digest; }
    }
}

[Serializable]
public abstract class RSABase : DigestData
{
    [JsonIgnore]
    protected RSACryptoServiceProvider rsaCryptoProvider;

    protected RSABase(RSACryptoServiceProvider rsaCryptoProvider, string algorithm, string hash)
        : base(algorithm, hash)
    {
        this.rsaCryptoProvider = rsaCryptoProvider;
    }

    protected RSABase(DigestData digestData, RSACryptoServiceProvider rsaCryptoProvider)
        : base(digestData.Algorithm, digestData.Hash)
    {
        this.rsaCryptoProvider = rsaCryptoProvider;
        this.digest = digestData.Digest;
    }
}

/// <summary>
/// Implements the IDigestEncryptor interface for an RSA AOEP asymetric algorithm
/// </summary>
[Serializable]
public class RSAOAEPEncryptor : RSABase, IDigestEncryptor
{
    public RSAOAEPEncryptor(RSACryptoServiceProvider rsaCryptoProvider)
        : base(rsaCryptoProvider, rsaCryptoProvider.KeyExchangeAlgorithm, "SHA1")
    {
    }

    public RSAOAEPEncryptor(DigestData digestData, RSACryptoServiceProvider rsaCryptoProvider)
        : base(digestData, rsaCryptoProvider)
    {
    }

    public void Encrypt(byte[] data)
    {
        byte[] encrypted = rsaCryptoProvider.Encrypt(data, true);
        digest = Convert.ToBase64String(encrypted);
    }

    public byte[] Decrypt()
    {
        byte[] encrypted = Convert.FromBase64String(digest);

        return rsaCryptoProvider.Decrypt(encrypted, true);
    }

    [JsonIgnore]
    DigestData IDigestEncryptor.Digest
    {
        get { return this; }
    }
}

This sample factory uses the algorithm and hash algorithm names to reconstruct the proper concrete class that will be able to process the digest data.

/// <summary>
/// This class is used to create a typed DigestData object given one that was reconstructed
/// from a Json serialization
/// </summary>
public static class RSADigestFactory
{
    static RSADigestFactory()
    {                CreateRSADigestTypeDictionary();
    }

    const string 
        RSAOAEPAlgo = "RSA-PKCS1-KeyEx",
        RSASignatureAlgo = "http://www.w3.org/2000/09/xmldsig#rsa-sha1",
        SHA1 = "SHA1";

    private static Dictionary<string, Type> typePerAlgo = new Dictionary<string, Type>();

    public static DigestData CreateDigestData(DigestData digest, 
                  RSACryptoServiceProvider rsaCryptoProvider)
    {
        DigestData digestData = null;
                Type digestDataType = null;

        if (typePerAlgo.TryGetValue(GetAlgoHash(digest.Algorithm, digest.Hash), out digestDataType))
        {
            ConstructorInfo digestCstr = 
              digestDataType.GetConstructor(new Type[] { typeof(DigestData), 
              typeof(RSACryptoServiceProvider) });
            if (digestCstr != null)
            {
                digestData = digestCstr.Invoke(new object[] { digest, rsaCryptoProvider }) as DigestData;
            }        }

        return digestData;
    }

    private static void CreateRSADigestTypeDictionary()
    {
        // RSAAOEPEncrytor
        typePerAlgo.Add(GetRSAOAEPAlgoHash(), typeof(RSAOAEPEncryptor));

        // RSASHA1Signature
        typePerAlgo.Add(GetRSASignatureAlgoHash(), typeof(RSASHA1Signature));
    }

    private static string GetRSAOAEPAlgoHash()
    {                return GetAlgoHash(RSAOAEPAlgo, SHA1);
    }

    private static string GetRSASignatureAlgoHash()
    {
        return GetAlgoHash(RSASignatureAlgo, SHA1);
    }

    private static string GetAlgoHash(string algoName, string hashName)
    {
        SHA1Managed sha1 = new SHA1Managed();
        StringBuilder algoHash = new StringBuilder(algoName);
        algoHash.Append(hashName);

        return Convert.ToBase64String(sha1.ComputeHash(ASCIIEncoding.ASCII.GetBytes(algoHash.ToString())));}
}

The complete code of the ExchangeDataHeader is given in the demo solution and can be used freely in your projects. You will certainly have to extend and enhance it but the purpose of this article is to present the cryptography technique to securely exchange data.

Points of Interest 

Those who are familiar with key exchange using RSA key pairs would find this article trivial but if you are new to cryptography techniques this might be of some interest to you. RSA and AES algorithm are very robust algorithms that are largely used in financial applications or even for military purposes. In fact you use this type of exchange protocol certainly every day when you connect to your bank or a payment site using HTTPS.