Azure Blob C# Non-Trivial

Introduction

While I was working on integration of Azure function and blob storage for a client's problem, I needed help with the blob storage. Although there were many tutorials available on basic connectivity, writing and downloading from Azure storage, what almost all of these tutorials were missing was some of these following scenarios:

1. How to loop through all the blobs in a directory inside a container
2. How to append to an already existing blob
3. How to replicate a local folder structure along with files in a container

How to Loop Through All the Blobs in a Directory Inside a Container

Assuming that you already know that there are no directories as such in Azure blob storage, and that we use the path of the file to simulate the folder structure (/Test/fileName.txt - we will consider Test to be a folder), let’s move to looping the files.

Here is the method that I wrote for moving all the files from one Azure directory to another one:

public static async void MoveAllFiles(string sourceContainer, 
    string sourceDircetory, string destinationDirectory, string destinationContainer)
{
string storageConnection = "DefaultEndpointsProtocol=https;
    AccountName=abcappstorage;AccountKey=somelongkey;EndpointSuffix=core.windows.net";
CloudStorageAccount cloudStorageAccount = CloudStorageAccount.Parse(storageConnection);
var client = cloudStorageAccount.CreateCloudBlobClient();
var sContainer = client.GetContainerReference(sourceContainer);
CloudBlobDirectory sDirectory = sContainer.GetDirectoryReference(sourceDircetory);

//this will fetch all the items in the particular directory
var blobs = sDirectory.ListBlobsSegmentedAsync
              (false, BlobListingDetails.Metadata, 100, null, null, null).Result;
var dContainer = client.GetContainerReference(destinationContainer);

foreach (var blob in blobs.Results)
{
   var fileName = System.IO.Path.GetFileName(blob.Uri.LocalPath);

   if (fileName.EndsWith(".txt"))
   {
     var sourceBlob = sContainer.GetBlobReference(sourceDircetory + fileName);
     var destinationBlob = dContainer.GetBlobReference(destinationDirectory + fileName);
     await destinationBlob.StartCopyAsync(sourceBlob.Uri);
     await sourceBlob.DeleteIfExistsAsync();
   }
  }
}

You can call this method like this:

MoveAllFiles("sourceContainer","FirstFolder/SecondFolder/",
    "DestinationFirstFolder/DestinationSecondFolder/","destinationContainer");

How to Append to an Already Existing File

In order to be able to append to an existing file we need to create an append blob instead of a block blob. Here is a method that I wrote in order to write some log files on Azure blob storage:

public async static void AppendToBlob(string fileName, string blobContainer, string text)
{
string storageConnection = "DefaultEndpointsProtocol=https;
AccountName=abcappstorage;AccountKey=somelongkey;Endpoint Suffix=core.windows.net";
CloudStorageAccount cloudStorageAccount = CloudStorageAccount.Parse(storageConnection);

if (cloudStorageAccount != null)
{  
  CloudBlobClient client = cloudStorageAccount.CreateCloudBlobClient();
  CloudBlobContainer container = client.GetContainerReference(blobContainer);
  container.CreateIfNotExistsAsync();
  container.SetPermissionsAsync(new BlobContainerPermissions 
    { PublicAccess =  BlobContainerPublicAccessType.Container });

  CloudAppendBlob blob = container.GetAppendBlobReference(fileName);

   //the below code creates the blob if it doesn’t already exist.
   if (!await blob.ExistsAsync())
      blob.CreateOrReplaceAsync();
   
   blob.AppendTextAsync(text);
   }
  }

You can call this method like this:

AppendToBlob("azurefolder/logfile.txt","containerName","Some text");

How to Replicate a Local Folder Structure Along With Files in a Container

public static void UploadFolders(string sourceFolderPath)
{
 string storageConnection = "DefaultEndpointsProtocol=https;AccountName=abcStorage;
                              AccountKey=longkey;EndpointSuffix=core.windows.net";
 CloudStorageAccount cloudStorageAccount = CloudStorageAccount.Parse(storageConnection);

 if (cloudStorageAccount != null)
 {
   CloudBlobClient client = cloudStorageAccount.CreateCloudBlobClient();
   CloudBlobContainer container = client.GetContainerReference("containerName");
   container.CreateIfNotExistsAsync();
   container.SetPermissionsAsync(new BlobContainerPermissions 
          { PublicAccess =BlobContainerPublicAccessType.Container });

   var folders = System.IO.Directory.GetDirectories(sourceFolderPath);

   foreach (var folder in folders)
    {
     var files = System.IO.Directory.GetFiles(folder);
     foreach (var file in files)
     {
      var filename = System.IO.Path.GetFileName(file);
      var foldername =  System.IO.Path.GetFileName(Path.GetDirectoryName(file));
      string strFileName = foldername + "/" + filename;
      CloudBlockBlob blockblob = container.GetBlockBlobReference(strFileName);
      blockblob.UploadFromFileAsync(file);
      }
     }
  }
}

You can call this method like this:

UploadFolders("D:/SomeFolder/");

Conclusion

I have tried to keep it as simple as I can. Hope this has helped.

History

• 20^th March, 2019: Initial version