How to Use Amazon Simple Storage Service (S3) in C++ with gSOAP

• Download aws-s3-gsoap-examples.zip - 19.1 KB

Introduction

This article will walk through the process of using gSOAP to connect with Amazon Web Services' Simple Storage Service (S3). We will create a bucket and then upload an object to the bucket using a streaming DIME attachment.

gSOAP (website/SourceForge) is a toolkit that auto-generates SOAP and XML data binding for C and C++. It simplifies the development of SOAP/XML web services, and in our case, the consumption of existing Web services.

Amazon S3 is a web service that is used to store files. They offer a SOAP API, so we can easily use the gSOAP toolkit to generate code to interact with AWS S3.

The gSOAP User Guide currently does not have any information on using gSOAP with AWS S3. The Developer Center has a small example on AWS S3, but it is not substantial and does not cover the common AWS S3 SOAP API functions.

Installation and Setup 

gSOAP

Refer to the relevant section on this page for instructions on how to install gSOAP on your system.

OpenSSL

You will need OpenSSL installed because Amazon requires all SOAP requests to be encrypted. You can read more about OpenSSL and how to install it at their website.

AWS, Access Keys, and Config File

An AWS S3 account is necessary in order to use the service. You'll also need to create access keys to send any requests to the AWS API. Keep these keys close by.

It's extremely recommended to use a config file to safely store your access keys. This way, you can read in the keys when necessary and you won't accidentally upload them to any version control systems. To generate a config file, we can use an AWS tool called the AWS Command Line Interface. Install it and run:

aws configure

Follow the prompts to enter your information. This tool will create a local file named credentials in a folder named .aws in your home directory (environment variable ~ on Unix systems or %USERPROFILE% on Windows systems). For more help on using the AWS CLI tool to create a local config file, refer to this page.

If you don't want to use the AWS CLI, you could also manually make a config file. It's best to store it in a directory not associated with any version control. Alternatively, you could use environment variables to store your access keys. If you choose one of these options, you'll have to change the parser in the Reading Keys from Config File section, as it assumes that the file has the format generated by the AWS CLI.

Generate C++ Data-bindings from the AWS S3 WSDL with gSOAP

It's recommended to make a new, empty directory for the files we are about to generate.

NOTE: In at least two cases (CreateBucket and CopyObject), Amazon's SOAP responses don't match their promised schema (this will hopefully be corrected soon). Because of this, it is necessary to edit the typemap.dat file to edit gSOAP's generated classes to ensure we can access every response from Amazon.

I recommended first copying the typemap.dat file from the gsoap/ directory into your new directory, and then add the following lines to it (anywhere):

_s3__CreateBucketResponse = $ s3__CreateBucketResult* CreateBucketResponse;
_s3__CopyObjectResponse = $ s3__CopyObjectResult* CopyObjectResponse;

Now, we will generate the C++ data bindings.

Step 1

Use gSOAP's wsd2lh tool on Amazon's S3 WSDL to generate a header file (we'll name it aws-s3.h).

wsdl2h -t typemap.dat -o aws-s3.h http://doc.s3.amazonaws.com/2006-03-01/AmazonS3.wsdl

Step 2

The second step in gSOAP's code generation is to use the soapcpp2 tool on the header file created from the wsdl2h tool.

soapcpp2 -C -j aws-s3.h

This generates client-side code (-C option) with C++ service proxies and objects (-j option) from the aws-s3.h header.

The generated aws-s3.h header file contains all of the class declarations for Amazon S3's service operation requests and responses, along with other Amazon S3 functions and gSOAP functions.

If you're interested, check out the auto-generated report on the aws-s3.h header for a list of classes and types. Or for more general information on how gSOAP works, visit the Developer Center or the User Guide.

Access Keys and Signature for AWS S3 

Reading Keys from Config File

In the Installation and Setup section, we stored our access keys in a file called credentials in the .aws folder of your home directory. We'll define this function to read in the keys from that file.

// Read access keys from file generated by AWS CLI
bool getAWSKeys(std::string path, std::string user, std::string &accessKey, std::string &secretKey) {
    std::ifstream credentialsFile(path.c_str());
    if (!credentialsFile.is_open())
        return false;
    
    std::string line;
    while (std::getline(credentialsFile, line)) {
        // Keep going until we get to the desired user
        if (line.find(user) == std::string::npos)
            continue;
        
        while (std::getline(credentialsFile, line)) {
            // Keep going until we get to the access key lines
            if (line.find("aws_access_key_id") == std::string::npos)
                continue;

            // Grab keys and trim whitespace
            size_t first, last;
            accessKey = line.substr(line.find_first_of('=')+1);
            first = accessKey.find_first_not_of(' ');
            if (first == std::string::npos)
                return false;
            last = accessKey.find_last_not_of(' ');
            accessKey.substr(first, last-first+1).swap(accessKey); 

            std::getline(credentialsFile, line);
            secretKey = line.substr(line.find_first_of('=')+1);
            first = secretKey.find_first_not_of(' ');
            if (first == std::string::npos)
                return false;
            last = secretKey.find_last_not_of(' ');
            secretKey.substr(first, last-first+1).swap(secretKey);
            
            return true;
        }

    }
    return false;
}

We can call this function to set local variables, keeping hard-coded access keys out of our source.

// Load AWS keys from file
std::string accessKey, secretKey;
std::string credentialsFile = "path_to_aws_credentials_file"; 
std::string user = "default";
if (!getAWSKeys(credentialsFile, user, accessKey, secretKey)) {
    std::cout << "Couldn't read AWS keys for user " << user 
              << " from file " << credentialsFile << '\n';
    return 0;
}

You must supply the path to the file containing your access keys. If you didn't use the --profile option when running aws configure, you can leave the user variable as default.

How to Construct a Signature

Amazon needs a specially-constructed signature for every SOAP request in order to authenticate and process the request. We will define the function soap_make_s3__signature in order to make creating a signature painless. Its inputs are a soap context, an operation name (like "CreateBucket"), and your AWS secret key. It returns a string with the base64-encoded version of the HMAC-SHA1 hashed string "AmazonS3" + OPERATION_NAME + Timestamp.

// Make base64-encoded, HMAC-SHA1 hashed signature for AWS S3
std::string soap_make_s3__signature(struct soap *soap, char const *operation, char const *key) {
    std::string signature = "AmazonS3";
    signature += operation;
    char UTCstamp[40]; //to hold ISO 8601 time format
    time_t now;
    time(&now);
    strftime(UTCstamp, sizeof UTCstamp, "%Y-%m-%dT%H:%M:%S.000Z", gmtime(&now));
    signature += UTCstamp;
    // Get the HMAC-SHA1 digest of the signature string
    unsigned char * digest;
    digest = HMAC(EVP_sha1(), key, strlen(key), 
    (unsigned char*)(signature.c_str()), 
    signature.length(), NULL, NULL);        
    char signatureBase64[20];
    // Convert the digest to base64
    soap_s2base64(soap, digest, signatureBase64, sizeof signatureBase64);
    return std::string(signatureBase64);
}

The functions HMAC and EVP_sha1 are from the OpenSSL libraries (which we'll link with in the Compiling section).

So we can simply call our new function and supply the necessary parameters...

std::string signature = soap_make_s3__signature(aws.soap,
                                                "...", // use an operation name
                                                secretKey.c_str());

...making it easy to create signatures whenever we need to make an API request. Note that we're using the secretKey variable that we created in the previous section, rather than using a hard-coded string.

Using gSOAP to Connect with AWS S3

Note: In these examples, we are going to use classes (AmazonS3SoapBindingProxy, _s3__CreateBucket, etc.) we generated with gSOAP's wsdl2h and soapcpp2 tools in the previous step. Again, if you are curious what generated classes are available or what member data they have, check the generated aws-s3.h file or the auto-generated report on it.

First Example: Creating a Bucket

For the first example, let's create a new bucket.

For gSOAP's generated code to work, we need to include the generated header file and namespace mapping file.

#include "soapAmazonS3SoapBindingProxy.h"
#include "AmazonS3SoapBinding.nsmap"

We're also going to add a convenient template function to allocate primitive values on gSOAP's managed heap using soap_malloc() with context soap. This isn't necessary, but will make the code cleaner and easier to manage.

// Make allocation of primitive values quick and easy:
template<class T>
T * soap_make(struct soap *soap, T val) {
    T *p = (T*)soap_malloc(soap, sizeof(T));
    *p = val;
    return p;
}

Now we are going to write the code to connect to S3. This is all going to be in our main function. We're going to create a proxy of auto-generated type AmazonS3SoapBindingProxy to invoke AWS S3 services. Then we'll create a CreateBucket request of auto-generated type _s3__CreateBucket and set its arguments.

// Create a proxy to invoke AWS S3 services
AmazonS3SoapBindingProxy aws(SOAP_XML_INDENT);

// Set the arguments of the CreateBucket service 
_s3__CreateBucket createBucketReq;
std::string bucketName = "BucketName";
createBucketReq.Bucket = bucketName;
createBucketReq.AWSAccessKeyId  = soap_new_std__string(aws.soap);
*createBucketReq.AWSAccessKeyId = accessKey;
createBucketReq.Timestamp       = soap_make(aws.soap, time(0));
createBucketReq.Signature       = soap_new_std__string(aws.soap);
 *createBucketReq.Signature     = soap_make_s3__signature(aws.soap,
                                                          "CreateBucket",
                                                          secretKey.c_str());

Make sure to change the bucketName string to your desired bucket name. For the Signature, we use the function we made in the How to Construct a Signature section of this article, with the operation name "CreateBucket". 

We're going to need to catch the response of our request, so we'll make an instance of the auto-generated class _s3__CreateBucketResponse.

// To store the result of the service
_s3__CreateBucketResponse createBucketRes;

Now, we are going to actually call the service.

// Create a bucket
if (aws.CreateBucket(&createBucketReq, createBucketRes)) {
    aws.soap_stream_fault(std::cerr);
}
/*
    NOTE: you must add the line:
       _s3__CreateBucketResponse = $ s3__CreateBucketResult* CreateBucketResponse;
    to the typemap.dat file because Amazon's response doesn't match
    their promised schema. This adds the variable CreateBucketResponse
    to the _s3__CreateBucketResponse class so we can access the response.
*/
else if (createBucketRes.CreateBucketResponse) {
    s3__CreateBucketResult &result = *createBucketRes.CreateBucketResponse;
    std::cout << "You are the owner of bucket'" << 
    result.BucketName << "'." << std::endl;
}

This will send createBucketReq to Amazon S3 (in proper SOAP format, thanks to gSOAP) and put the response in createBucketRes. You can see the new bucket in your AWS Management Console.

The comment above the success block is just a reminder of what was said in the note at the top of the Generate C++ data-binding from the AWS S3 WSDL section of this article.

And when we're all finished, make sure to call the proxy's destroy function.

// Delete all managed data
aws.destroy();

And that's all we need to do to create a bucket. The final .cpp file (we'll call it createbucket.cpp) will look like this:

/*
    createbucket.cpp
    Example AWS S3 CreateBucket service invocation
*/

#include "soapAmazonS3SoapBindingProxy.h"
#include "AmazonS3SoapBinding.nsmap"
#include <fstream>

// Make allocation of primitive values quick and easy:
template<class T>
T * soap_make(struct soap *soap, T val) {
    T *p = (T*)soap_malloc(soap, sizeof(T));
    *p = val;
    return p;
}

// Make base64-encoded, HMAC-SHA1 hashed signature for AWS S3
std::string soap_make_s3__signature(struct soap *soap, char const *operation, char const *key) {
    std::string signature = "AmazonS3";
    signature += operation;
    char UTCstamp[40]; //to hold ISO 8601 time format
    time_t now;
    time(&now);
    strftime(UTCstamp, sizeof UTCstamp, "%Y-%m-%dT%H:%M:%S.000Z", gmtime(&now));
    signature += UTCstamp;
    // Get the HMAC-SHA1 digest of the signature string
    unsigned char * digest;
    digest = HMAC(EVP_sha1(), key, strlen(key), 
    (unsigned char*)(signature.c_str()), 
    signature.length(), NULL, NULL);        
    char signatureBase64[20];
    // Convert the digest to base64
    soap_s2base64(soap, digest, signatureBase64, sizeof signatureBase64);
    return std::string(signatureBase64);
}

// Read access keys from file generated by AWS CLI
bool getAWSKeys(std::string path, std::string user, std::string &accessKey, std::string &secretKey) {
    std::ifstream credentialsFile(path.c_str());
    if (!credentialsFile.is_open())
        return false;
    
    std::string line;
    while (std::getline(credentialsFile, line)) {
        // Keep going until we get to the desired user
        if (line.find(user) == std::string::npos)
            continue;
        
        while (std::getline(credentialsFile, line)) {
            // Keep going until we get to the access key lines
            if (line.find("aws_access_key_id") == std::string::npos)
                continue;

            // Grab keys and trim whitespace
            size_t first, last;
            accessKey = line.substr(line.find_first_of('=')+1);
            first = accessKey.find_first_not_of(' ');
            if (first == std::string::npos)
                return false;
            last = accessKey.find_last_not_of(' ');
            accessKey.substr(first, last-first+1).swap(accessKey); 

            std::getline(credentialsFile, line);
            secretKey = line.substr(line.find_first_of('=')+1);
            first = secretKey.find_first_not_of(' ');
            if (first == std::string::npos)
                return false;
            last = secretKey.find_last_not_of(' ');
            secretKey.substr(first, last-first+1).swap(secretKey);
            
            return true;
        }

    }
    return false;
}

int main(int argc, char **argv) {    
    // Load AWS keys from file
    std::string accessKey, secretKey;
    // Use the path to your AWS credentials file
    std::string credentialsFile = (argc > 2 ? argv[2] : "path_to_aws_credentials_file"); 
    std::string user = "default";
    if (!getAWSKeys(credentialsFile, user, accessKey, secretKey)) {
        std::cout << "Couldn't read AWS keys for user " << user 
                  << " from file " << credentialsFile << '\n';
        return 0;
    }
    
    // Create a proxy to invoke AWS S3 services
    AmazonS3SoapBindingProxy aws(SOAP_XML_INDENT);

    // Create bucket
    
    // Set the arguments of the CreateBucket service operation
    _s3__CreateBucket createBucketReq;
    std::string bucketName = (argc > 1 ? argv[1] : "BucketName");
    createBucketReq.Bucket = bucketName;
    createBucketReq.AWSAccessKeyId  = soap_new_std__string(aws.soap);
    *createBucketReq.AWSAccessKeyId = accessKey;      
    createBucketReq.Timestamp       = soap_make(aws.soap, time(0));
    createBucketReq.Signature       = soap_new_std__string(aws.soap);
    *createBucketReq.Signature      = soap_make_s3__signature(aws.soap, 
                                                              "CreateBucket", 
                                                              secretKey.c_str());
                                                              
    // Store the result of the service
    _s3__CreateBucketResponse createBucketRes;

    // Create a bucket
    if (aws.CreateBucket(&createBucketReq, createBucketRes)) {
        aws.soap_stream_fault(std::cerr);
    }
    /*
        NOTE: you must add the line:
           _s3__CreateBucketResponse = $ s3__CreateBucketResult* CreateBucketResponse;
        to the typemap.dat file because Amazon's response doesn't match
        their promised schema. This adds the variable CreateBucketResponse
        to the _s3__CreateBucketResponse class so we can access the response.
    */
    else if (createBucketRes.CreateBucketResponse) {
        s3__CreateBucketResult &result = *createBucketRes.CreateBucketResponse;
        std::cout << "You are the owner of bucket '" << result.BucketName << "'." << std::endl;
    }

    // Delete all managed data
    aws.destroy();

    return 0;
}

Compiling the CreateBucket Example

With g++

To compile our program, we must use OpenSSL because the AWS S3 service requires HTTPS to ensure that the credentials are securely transmitted. This means we must link with the OpenSSL and OpenSSL crypto libraries. The command looks like:

g++ -DWITH_OPENSSL -o createbucket createbucket.cpp soapAmazonS3SoapBindingProxy.cpp \
soapC.cpp stdsoap2.cpp -lssl -lcrypto

You may have to fix the path to stdsoap2.cpp to be an absolute path to the stdsoap2.cpp file in your gsoap/ directory from the installation. You may also have to include a -I option with the path to the gsoap/import folder.

Compile the code, run createbucket, and a new bucket will be created.

In the final .cpp file, note that we check the command line arguments (argv) when setting credentialsFile and bucketName. This allows the program to be called with arguments:

./createbucket BucketName path_to_credentials_file

Second Example: Uploading a File with PutObject Using a DIME Attachment with Streaming

Now that we have created a bucket, let's upload a file into it.

AWS S3's SOAP API has two different operations for uploading objects: PutObjectInline and PutObject. The inline version expects the data directly included in the body of the SOAP message, while the non-inline version expects the data provided as a DIME attachment. Files over 1MB cannot be uploaded with the inline version, so this example will demonstrate how to use DIME attachments. An example (putobjectinline.cpp) for the PutObjectInline operation has also been provided in the aws-s3-gsoap-examples.zip file.

gSOAP supports DIME attachments with or without streaming. Without streaming, the full file is stored and retrieved in memory, which can be a problem with larger files/smaller devices. We will show how to use gSOAP's DIME attachments with streaming, although an example (putobject.cpp) without streaming can be found in the aws-s3-gsoap-examples.zip file.

We will use the same helper functions (soap_make, getAWSKeys, and soap_make_s3__signature) as we did in the CreateBucket example.

Our request will be of auto-generated class _s3__PutObject. It requires a bucket name, a key (to name the object we're going to upload), an access key, a timestamp, and a signature. We can also supply name-value metadata pairs (of type s3__MetadataEntry) to the Metadata container element, to be stored with the object, but this is optional. 

// Create a proxy to invoke AWS S3 services
AmazonS3SoapBindingProxy aws(SOAP_XML_INDENT);

// Set the arguments of the PutObject service operation
_s3__PutObject putObjectReq;
std::string bucketName = "BucketName";
std::string keyName = "KeyName";
putObjectReq.Bucket = bucketName;
putObjectReq.Key = keyName; 
putObjectReq.AWSAccessKeyId  = soap_new_std__string(aws.soap);
*putObjectReq.AWSAccessKeyId = accessKey;
putObjectReq.Timestamp       = soap_make(aws.soap, time(0));
putObjectReq.Signature       = soap_new_std__string(aws.soap);
*putObjectReq.Signature      = soap_make_s3__signature(aws.soap, 
                                                       "PutObject", 
                                                       secretKey.c_str());
    
s3__MetadataEntry *metadataEntry = soap_new_req_s3__MetadataEntry(aws.soap, 
                                                            "Metadata Name", 
                                                            "Metadata Value");
putObjectReq.Metadata.push_back(metadataEntry);

All that's left is to include the data we want to upload as a DIME attachment, encoded in base 64. Streaming DIME attachments with gSOAP is simple enough, but as a reminder, there is a non-streaming version included in the examples .zip folder.

An in-depth explanation of the callback functions needed to stream DIME attachments can be read in this section of the gSOAP User Guide, but this example will show an implementation. dime_read_open will simply return the handle (we'll open the file ourselves beforehand), dime_read will read in up to len bytes from the file and store it in a buffer managed by gSOAP, and dime_read_close will close the file.

void *dime_read_open(struct soap *soap, void *handle, const char *id, const char *type, const char *options) {
    return handle;
}

size_t dime_read(struct soap *soap, void *handle, char* buf, size_t len) {
    return fread(buf, 1, len, (FILE*)handle);
}

void dime_read_close(struct soap *soap, void *handle) {
    fclose((FILE*)handle);
}

Great. We need an xsd__base64Binary object to facilitate the streaming...

xsd__base64Binary *data = soap_new_xsd__base64Binary(aws.soap);

...and a FILE pointer to our file to pass as the handle between the callback functions:

FILE *fd = fopen("filename.jpg"), "rb");
if (fd == NULL) return 0;

We can only stream the data if we can get the size of the file (for which we will use fstat). If we know the size, then we set the proxy's soap struct's callback functions to the ones we just defined and assign the data object's __ptr and __size fields. These values will be used in the gSOAP's streaming callbacks that we defined earlier (the __ptr field is sent as the handle parameter of the dime_read_open function, and the __size field is used by the callback functions to know when they've finished).

struct stat sb;
if (!fstat(fileno(fd), &sb) && sb.st_size > 0) {
    // We can get the length of the file, so we can stream it

    // Set soap callbacks 
    aws.soap->fdimereadopen = dime_read_open;
    aws.soap->fdimereadclose = dime_read_close;
    aws.soap->fdimeread = dime_read;

    //__ptr must be non-NULL. This is the handle in the callbacks
    data->__ptr = (unsigned char*)fd;
    data->__size = sb.st_size;
}

If we can't read the size of the file, we need a backup plan. gSOAP supports HTTP chunking (which doesn't require a pre-known size), but we can't use that method as the PutObject API call expects a ContentLength element. So we'll naively read and set the data in data->__ptr until we reach the end, cutting off at an arbitrary size limit.

else {
    // We don't know the size, so buffer it
    int i;
    data->__ptr = (unsigned char*)soap_malloc(aws.soap, MAX_FILE_SIZE);
    for (i = 0; i < MAX_FILE_SIZE; i++) {
         int c;
         if ((c = fgetc(fd)) == EOF)
             break;
         data->__ptr[i] = c;
    }
    fclose(fd);
    data->__size = i;
}

Be sure to define MAX_FILE_SIZE.

#define MAX_FILE_SIZE    (1000000) // (1MB). Max size for when we must attach as non-streaming

Great! Now, to tell gSOAP to put the data object in the request as a DIME attachment, we need to set either its type or id field. gSOAP will know to stream the data because we set the callback functions earlier. We also need to call the gSOAP functions soap_set_dime with our soap context and soap_set_dime_attachment with the information we just constructed. Then we'll set the ContentLength argument of the request.

// Set DIME attachment with options
char type[] = "text/plain";
data->type = type;
soap_set_dime(aws.soap);
if (soap_set_dime_attachment(aws.soap, (char*)data->__ptr, data->__size, data->type, data->id, 0, data->options)) {
    aws.soap_stream_fault(std::cerr);
    return 0;
}
putObjectReq.ContentLength = data->__size;

Make sure to set data->type appropriately. You can also set data->id, but gSOAP will automatically create one if not. data->options controls the DIME-specific options field, which can be left blank or set with soap_dime_option...

data->options = soap_dime_option(aws.soap, 0, "Optional text");

Now the request is fully constructed! Now we just have to call the API and catch the response, similar to the CreateBucket example.

// Store the result of the service
_s3__PutObjectResponse putObjectRes;

// Put object
if (aws.PutObject(&putObjectReq, putObjectRes)) {
    aws.soap_stream_fault(std::cerr);
}
else if (putObjectRes.PutObjectResponse) {
    s3__PutObjectResult &result = *putObjectRes.PutObjectResponse;
    std::cout << "Object with key '" << putObjectReq.Key 
              << "' put into bucket '" << putObjectReq.Bucket << "'" << std::endl;
    std::cout << "\tEtag: " << result.ETag << std::endl;
    std::cout << "\tLast modified: " << soap_dateTime2s(aws.soap, result.LastModified) << std::endl;
}

Again, be sure to call aws.destroy() before ending the program.

The final .cpp file (we'll call it putobjectstreaming.cpp) will look like this:

/*
    putobjectstreaming.cpp
    Example AWS S3 PutObject service invocation
    with DIME attachment (streaming)
*/

#include "soapAmazonS3SoapBindingProxy.h"
#include "AmazonS3SoapBinding.nsmap"
#include <fstream>
#include <sys/stat.h>

#define MAX_FILE_SIZE    (1000000) // (1MB). Max size for when we must attach as non-streaming

// Make allocation of primitive values quick and easy:
template<class T>
T * soap_make(struct soap *soap, T val) {
    T *p = (T*)soap_malloc(soap, sizeof(T));
    *p = val;
    return p;
}

// Make base64-encoded, HMAC-SHA1 hashed signature for AWS S3
std::string soap_make_s3__signature(struct soap *soap, char const *operation, char const *key) {
    std::string signature = "AmazonS3";
    signature += operation;
    char UTCstamp[40]; //to hold ISO 8601 time format
    time_t now;
    time(&now);
    strftime(UTCstamp, sizeof UTCstamp, "%Y-%m-%dT%H:%M:%S.000Z", gmtime(&now));
    signature += UTCstamp;
    // Get the HMAC-SHA1 digest of the signature string
    unsigned char * digest;
    digest = HMAC(EVP_sha1(), key, strlen(key), 
    (unsigned char*)(signature.c_str()), 
    signature.length(), NULL, NULL);        
    char signatureBase64[20];
    // Convert the digest to base64
    soap_s2base64(soap, digest, signatureBase64, sizeof signatureBase64);
    return std::string(signatureBase64);
}

// Read access keys from file generated by AWS CLI
bool getAWSKeys(std::string path, std::string user, std::string &accessKey, std::string &secretKey) {
    std::ifstream credentialsFile(path.c_str());
    if (!credentialsFile.is_open())
        return false;
    
    std::string line;
    while (std::getline(credentialsFile, line)) {
        // Keep going until we get to the desired user
        if (line.find(user) == std::string::npos)
            continue;
        
        while (std::getline(credentialsFile, line)) {
            // Keep going until we get to the access key lines
            if (line.find("aws_access_key_id") == std::string::npos)
                continue;

            // Grab keys and trim whitespace
            size_t first, last;
            accessKey = line.substr(line.find_first_of('=')+1);
            first = accessKey.find_first_not_of(' ');
            if (first == std::string::npos)
                return false;
            last = accessKey.find_last_not_of(' ');
            accessKey.substr(first, last-first+1).swap(accessKey); 

            std::getline(credentialsFile, line);
            secretKey = line.substr(line.find_first_of('=')+1);
            first = secretKey.find_first_not_of(' ');
            if (first == std::string::npos)
                return false;
            last = secretKey.find_last_not_of(' ');
            secretKey.substr(first, last-first+1).swap(secretKey);
            
            return true;
        }

    }
    return false;
}

void *dime_read_open(struct soap *soap, void *handle, const char *id, const char *type, const char *options) {
    return handle;
}

size_t dime_read(struct soap *soap, void *handle, char* buf, size_t len) {
    return fread(buf, 1, len, (FILE*)handle);
}

void dime_read_close(struct soap *soap, void *handle) {
    fclose((FILE*)handle);
}

int main(int argc, char **argv) {
    // Load AWS keys from file
    std::string accessKey, secretKey;
    // Use the path to your AWS credentials file
    std::string credentialsFile = (argc > 4 ? argv[4] : "path_to_aws_credentials_file"); 
    std::string user = "default";
    if (!getAWSKeys(credentialsFile, user, accessKey, secretKey)) {
        std::cout << "Couldn't read AWS keys for user " << user 
                  << " from file " << credentialsFile << '\n';
        return 0;
    }
    
    // Create a proxy to invoke AWS S3 services
    AmazonS3SoapBindingProxy aws(SOAP_XML_INDENT);

    // Put object in bucket (streaming)

    // Set the arguments of the PutObject service operation
    _s3__PutObject putObjectReq;
    std::string bucketName = (argc > 1 ? argv[1] : "BucketName");
    std::string keyName = (argc > 2 ? argv[2] : "KeyName");
    putObjectReq.Bucket = bucketName;
    putObjectReq.Key = keyName; 
    putObjectReq.AWSAccessKeyId  = soap_new_std__string(aws.soap);
    *putObjectReq.AWSAccessKeyId = accessKey;
    putObjectReq.Timestamp       = soap_make(aws.soap, time(0));
    putObjectReq.Signature       = soap_new_std__string(aws.soap);
    *putObjectReq.Signature      = soap_make_s3__signature(aws.soap, 
                                                           "PutObject", 
                                                           secretKey.c_str());
    
    s3__MetadataEntry *metadataEntry = soap_new_req_s3__MetadataEntry(aws.soap, 
                                                                "Metadata Name", 
                                                                "Metadata Value");
    putObjectReq.Metadata.push_back(metadataEntry);

    // Set up gSOAP DIME streaming 
    xsd__base64Binary *data = soap_new_xsd__base64Binary(aws.soap);
    FILE *fd = fopen((argc > 3 ? argv[3] : "filename"), "rb");
    if (fd == NULL) return 0;
    struct stat sb;
    if (!fstat(fileno(fd), &sb) && sb.st_size > 0) {
        // We can get the length of the file, so we can stream it

        // Set soap callbacks 
        aws.soap->fdimereadopen = dime_read_open;
        aws.soap->fdimereadclose = dime_read_close;
        aws.soap->fdimeread = dime_read;

        //__ptr must be non-NULL. This is the handle in the callbacks
        data->__ptr = (unsigned char*)fd;
        data->__size = sb.st_size;
    }
    else {
        // We don't know the size, so buffer it
        int i;
        data->__ptr = (unsigned char*)soap_malloc(aws.soap, MAX_FILE_SIZE);
        for (i = 0; i < MAX_FILE_SIZE; i++) {
            int c;
            if ((c = fgetc(fd)) == EOF)
                break;
            data->__ptr[i] = c;
        }
        fclose(fd);
        data->__size = i;
    }
    
    // Set DIME attachment with options
    char type[] = "text/plain";
    data->type = type;
    soap_set_dime(aws.soap);
    if (soap_set_dime_attachment(aws.soap, (char*)data->__ptr, data->__size, data->type, data->id, 0, data->options)) {
        aws.soap_stream_fault(std::cerr);
        return 0;
    }
    putObjectReq.ContentLength = data->__size;

    // Store the result of the service
    _s3__PutObjectResponse putObjectRes;

    // Put object
    if (aws.PutObject(&putObjectReq, putObjectRes)) {
        aws.soap_stream_fault(std::cerr);
    }
    else if (putObjectRes.PutObjectResponse) {
        s3__PutObjectResult &result = *putObjectRes.PutObjectResponse;
        std::cout << "Object with key '" << putObjectReq.Key 
                  << "' put into bucket '" << putObjectReq.Bucket << "'" << std::endl;
        std::cout << "\tEtag: " << result.ETag << std::endl;
        std::cout << "\tLast modified: " << soap_dateTime2s(aws.soap, result.LastModified) << std::endl;
    }

    // Delete all managed data
    aws.destroy();
    
    return 0;
}

Compiling the PutObject (streaming) Example

With g++

g++ -DSOAP_MAXDIMESIZE=100000000 -DWITH_OPENSSL -o putobjectstreaming putobjectstreaming.cpp soapAmazonS3SoapBindingProxy.cpp \
soapC.cpp stdsoap2.cpp -lssl -lcrypto

See the section titled Compiling the CreateBucket Example for an explanation of the command.

Note that we added a new option -DSOAP_MAXDIMESIZE=100000000 to increase the default DIME size limit (set in stdsoap2.h) of 8MB to ~100MB. Increase this option to allow larger attachments, if necessary.

In the final .cpp file, note that we check the command line arguments (argv) when setting bucketName, keyName, the filename, and credentialsFile. This allows the program to be called with arguments:

./putobjectstreaming BucketName KeyName filename path_to_credentials_file

Other AWS S3 API Functions With Examples

Calling different API functions will simply involve using different classes and setting the appropriate parameters.

The full SOAP API documentation for AWS S3 can be found here. It lists the operations available. You can also check the generated aws-s3.h file or the auto-generated report on it to get a list of the available gSOAP classes and functions (which are auto-generated from Amazon's WSDL/XSD).

I've written working example programs for the usage of the following AWS S3 API functions:

• CreateBucket (createbucket.cpp) (shown in this article)
• ListBucket (listbucket.cpp)
• DeleteBucket (deletebucket.cpp)
• GetBucketAccessControlPolicy (setacp.cpp)
• SetBucketAccessControlPolicy (setacp.cpp)
• PutObjectInline (putobjectinline.cpp)
• PutObject (non-streaming: putobject.cpp, streaming: putobjectstreaming.cpp) (shown in this article)
• CopyObject (copyobject.cpp)
• GetObject (getobject.cpp)
• DeleteObject (deleteobject.cpp)

These are all included in the aws-s3-gsoap-examples.zip file.

This should be more than enough to create a functional client to interact with AWS S3, and to figure out how to use the other auto-generated classes from gSOAP to call the other API operations. It also would be possible to write a wrapper class to make interacting with AWS S3 even easier.

If you have any questions or suggestions, please leave a comment!