Manage AWS S3 Objects in C#

• Download source code - 18.8 MB

Introduction

In this article, I will show you how to create an S3 Bucket in Amazon Web Services and then access and manage its objects from your C# code.

Using the Code

Before you use this code, you will need to have an AWS account and know how to create an S3 bucket in AWS. I have provided a step-by-step guide that explains how to create an S3 bucket, add users by using IAM in AWS, and set up a group policy to regulate access to the S3 resources.

You can see my S3 tutorial here.

Start a new project in Visual Studio, then continue by adding AWS SDK to our project from the Nuget repository:

To maintain our objects and perform the download, upload, and delete, I have created separated classes:

S3BucketDelete: To delete objects (files and folders):

public class S3BucketDelete
	{
        public string bucketName { get; set; }
        public string fileName { get; set; }
        public string filePath { get; set; }

        public void DeleteFile()
        {
			try
			{
                var client = new AmazonS3Client(Amazon.RegionEndpoint.USWest2);

                DeleteObjectRequest delRequest = new DeleteObjectRequest
                {
                    BucketName = bucketName,
                    Key = Path.Combine(filePath, fileName)  // Delete file at the path
                };

                DeleteObjectResponse response = client.DeleteObject(delRequest);
            }catch(Exception x)
			{
                MessageBox.Show(x.Message);
			}
        }
        public void DeleteFolder()
        {
            var client = new AmazonS3Client(Amazon.RegionEndpoint.USWest2);

            DeleteObjectRequest delRequest = new DeleteObjectRequest
            {
                BucketName = bucketName,
                Key = Path.Combine(filePath, fileName) + "/" // Delete folder
            };

            DeleteObjectResponse response = client.DeleteObject(delRequest);
        }
    }

S3BucketDownload: To download objects into local destinations from the AWS S3:

public class S3BucketDownload
	{
		public string bucketName { get; set; }
        public string keyName { get; set; }
		public string filePath { get; set; }
		public string fileDestination { get; set; }
        public async Task DownoadFileAsync()
        {
            try
            {
                var client = new AmazonS3Client(Amazon.RegionEndpoint.USWest2);
                var fileTransferUtility = new TransferUtility(client);

                var request = new TransferUtilityDownloadRequest
                {
                    BucketName = bucketName,
                    FilePath = Path.Combine(filePath, keyName),
                  //  Key = Path.Combine(fileDestination, keyName),
                    Key = fileDestination+keyName,
                    //PartSize = 6291456, // 6 MB.
                    //ServerSideEncryptionMethod = ServerSideEncryptionMethod.AES256
                };

                await fileTransferUtility.DownloadAsync(request);
            }
            catch (AmazonS3Exception s3Exception)
            {
                MessageBox.Show(s3Exception.Message, "Error 102", 
                                MessageBoxButtons.OK, MessageBoxIcon.Error);
            }
            catch (Exception ex)
            {
                MessageBox.Show(ex.Message, "Error 103", 
                                MessageBoxButtons.OK, MessageBoxIcon.Error);
            }
        }
    }

S3BucketUpload: To upload files from the location sources into our S3:

public class S3BucketUploader
	{
        public string bucketName { get; set; }
        public string keyName { get; set; }
        public string filePath { get; set; }
        public string fileDestination { get; set; }

        public void UploadFile()
        {
          try
			{
                var client = new AmazonS3Client(Amazon.RegionEndpoint.USWest2);

                PutObjectRequest putRequest = new PutObjectRequest
                {
                    BucketName = bucketName,
                    Key = keyName,
                    FilePath = filePath,
                    ContentType = "text/plain"
                };

                PutObjectResponse response = client.PutObject(putRequest);
            }
            catch (Exception x)
            {
                MessageBox.Show(x.Message, "Error 101", 
                                MessageBoxButtons.OK, MessageBoxIcon.Error);
            }
        }
        public void UploadFileToFolder()
        {
			try
			{
                var client = new AmazonS3Client(Amazon.RegionEndpoint.USWest2);

                PutObjectRequest putRequest = new PutObjectRequest
                {
                    BucketName = bucketName,
                    Key = Path.Combine(fileDestination, keyName),
                    FilePath = filePath,
                    ContentType = "text/plain"
                };

                PutObjectResponse response = client.PutObject(putRequest);
            }catch(Exception x)
			{
                MessageBox.Show(x.Message, "Error 100", 
                                MessageBoxButtons.OK, MessageBoxIcon.Error);
            }
        }
        public  async Task UploadFileAsync()
        {
            try
            {
                var client = new AmazonS3Client(Amazon.RegionEndpoint.USWest2);
                var fileTransferUtility = new TransferUtility(client);

                var request = new TransferUtilityUploadRequest
                {
                    BucketName = bucketName,
                    FilePath = filePath,
                    Key = Path.Combine(fileDestination, keyName),
                    PartSize = 6291456, // 6 MB.
                    ServerSideEncryptionMethod = ServerSideEncryptionMethod.AES256
                };

                await fileTransferUtility.UploadAsync(request);
            }
            catch (AmazonS3Exception s3Exception)
            {
                MessageBox.Show(s3Exception.Message, "Error 102", 
                                MessageBoxButtons.OK, MessageBoxIcon.Error);                
            }
            catch (Exception ex)
            {
                MessageBox.Show(ex.Message, "Error 103", 
                                MessageBoxButtons.OK, MessageBoxIcon.Error);                
            }
        }
    }

S3BucketView: To view objects (files and folders) in S3:

public class S3BucketView
  {
      public string bucketName { get; set; }
      public ListObjectsResponse ListFolders()
      {
          //var client = new AmazonS3Client(Amazon.RegionEndpoint.USWest2);

          ListObjectsResponse response;
          ListObjectsResponse result ;
          IAmazonS3 client;
          try
          {
              using (client = new AmazonS3Client(Amazon.RegionEndpoint.USWest2))
              {
                  ListObjectsRequest listRequest = new ListObjectsRequest
                  {
                      BucketName = bucketName,
                  };
                  do
                  {
                      response = client.ListObjects(listRequest);

                      IEnumerable<s3object> folders =
                      response.S3Objects.Where(x => x.Key.EndsWith(@"/")
                                               && x.Size == 0);
                      result = new ListObjectsResponse();
                      foreach (S3Object x in folders)
                      {
                          result.S3Objects.Add(x);
                      }
                      if (response.IsTruncated)
                      {
                          listRequest.Marker = response.NextMarker;
                      }
                      else
                      {
                          listRequest = null;
                      }
                  } while (listRequest != null);
              }
          }catch (Exception x)
          {
              MessageBox.Show(x.Message, "Erorr 1");
              result = null;
          }

          return result;
      }
      public S3DirectoryInfo ListFiles(string folder)
      {
          var client = new AmazonS3Client(Amazon.RegionEndpoint.USWest2);
          var dir = new S3DirectoryInfo(client, bucketName, folder);

          ListObjectsRequest listRequest = new ListObjectsRequest
          {
              BucketName = bucketName,
              Prefix = folder
          };

          return dir;
      }
  }

Both local and remote files and folders are listed in listView and treeView objects. So, you can browse. The project’s app.config file is where we need to place our credentials to access the S3 objects so that AWS SDK can access them:

<appsettings> <add key="AWSProfileName" value="Username"> 
<add key="AWSAccessKey" value="Access Key"> <add key="AWSSecretKey" value="Secret Key">

In addition to this, I have defined another property to store the bucket name.

As soon as the app is launched, we prepare the files:

public PiBucket()
{
    InitializeComponent();
    PopulateTreeView();
    PopulateS3Tree();
}

Two ContextMenuStrip controls provide shortcuts to perform our functions

I have also added StatusBar and placed StatusLabels to show the selected file, current local path, S3 object address, and network status label.

To remove objects from the S3 folders, you have to check the item checkbox, and after the right-click, select the Delete option.

The Settings

I wanted this project to be as customizable as possible, so I decided to add the Settings screen so that I can switch between different AWS accounts and S3 objects:

As you can see, the Settings form provides two methods to store and restore the app settings values from the app.config file.

private void ShowValues()
		{
			try
			{
				textProfile.Text = ConfigurationManager.AppSettings.Get("AWSProfileName");
				textAccess.Text = ConfigurationManager.AppSettings.Get("AWSAccessKey");
				textSecret.Text = ConfigurationManager.AppSettings.Get("AWSSecretKey");
				textBucket.Text = Properties.Settings.Default.bucket;
			}
			catch (Exception x)
			{
				MessageBox.Show(x.Message, "Error 23", 
                                MessageBoxButtons.OK, MessageBoxIcon.Error);
			}
		}

And this method saves the changes to the app.config:

private void StoreValues()
		{
			try
			{
				if (!string.IsNullOrEmpty(textAccess.Text) && 
                    !string.IsNullOrEmpty(textBucket.Text) && 
                    !string.IsNullOrEmpty(textSecret.Text) && 
                    !string.IsNullOrEmpty(textProfile.Text))
				{
					Setconfig.UpdateAppSettings("AWSProfileName", textProfile.Text);
					Setconfig.UpdateAppSettings("AWSAccessKey", textAccess.Text);
					Setconfig.UpdateAppSettings("AWSSecretKey", textSecret.Text);
					Properties.Settings.Default.bucket = textBucket.Text;
					Properties.Settings.Default.Save();
					MessageBox.Show("Restart the software for the changes 
                    to take effect.", "Setting", MessageBoxButtons.OK, 
                    MessageBoxIcon.Information);
				}
			}catch(Exception x)
			{
				MessageBox.Show(x.Message, "Error 33", MessageBoxButtons.OK, 
                                MessageBoxIcon.Error);
			}
		}

Points of Interest

Once you figure out how everything works, you will improve this code and build your own. I wrote this code in a few hours, and I’m sure it has plenty of room for improvement.

History

This is the first version of the software and I hope to improve it with some new features such as access to multiple S3 objects from different AWS accounts.