Click here to Skip to main content
15,114,470 members
Articles / Programming Languages / C#
Tip/Trick
Posted 29 Mar 2020

Tagged as

Stats

6.5K views
367 downloads
15 bookmarked

A Small Utility for Synchronization Between Two Folders

Rate me:
Please Sign up or sign in to vote.
5.00/5 (6 votes)
29 Mar 2020CPOL3 min read
A small command line utility for synchronization between two folders
A small command-line utility for synchronization of files and subfolders between two folders. It scans all the files and folders within both locations and synchronizes differences – copies missing files/folders (one-way or both), deletes missing and overwrites based on file size (only one way option).

Disclaimer

This utility is used to change the file/folder structure of selected folders on your disk(s).

It has been tested for all possible scenarios I could think of, but I would strongly advise to use it with caution, and also use readonly mode first - if readonly mode is used, then no changes are made to the file structure, the utility just lists the proposed changes.

This is especially advised if you are using delete and overwrite options with force mode, because in case of malfunctions and/or misusage, files and folders might get irreversibly deleted or overwritten.

The Usage

The utility is called like this:

folderSync mode folder1 folder2 -[options]

Mode, folder1 and folder2 are mandatory. Folder1 is always the reference (main) folder, which means that in one-way sync mode, folder2 is being updated based on folder1.

Two modes are available:

  • 1 = one way
  • 2 = two way

In one way mode only folder2 is updated based on folder1. Delete and overwrite options are only available in this mode.

In two way mode, both folders are updated only with the files and subfolders that are missing compared to the other folder.

All options are (as the name suggests) optional, the following are available:

  • d = delete files in folder2 that don't exist in folder1 (works only in one way sync)
  • s = Compare file sizes and overwrite files with different size (reference folder is folder1) (works only in one way sync)
  • f = Force delete/overwrite (don't ask user for confirmation for each file/subfolder)
  • r = Readonly (simulation mode – there is no actual copying, deletion or overwrite)

The Code

Parsing the Arguments

First, we are parsing the arguments of the call. The argument count and order is checked. Folders are read and checked if they exist.

Options are gathered and parsed with the method parseOption. parseOption is called for each option; options can be passed separately or combined, like this:

-d -s -r

-ds -r

-dsr

(These are all valid.)

C#
static IEnumerable<OptionEnum> parseOption(string option)
 {
     option = option.Replace("-", "");
     Dictionary<string, OptionEnum> _validParams = new Dictionary<string, OptionEnum>();
     _validParams.Add("d", OptionEnum.DELETE_FILES);
     _validParams.Add("s", OptionEnum.COMPARE_SIZES);
     _validParams.Add("f", OptionEnum.FORCE);
     _validParams.Add("r", OptionEnum.READONLY);
     OptionEnum retval;
     foreach (char c in option)
     {
         try
         {
             retval = _validParams[c.ToString()];
         }
         catch
         {
             throw new Exception("Invalid option: " + option);
         }
         yield return retval;
     }
 }

Collect Folder Content and Differences

Folders are scanned using System.IO.Directory.EnumerateDirectories and System.IO.Directory.EnumerateFiles methods.

C#
_content1_folders = Directory.EnumerateDirectories
(_folder1, "*", SearchOption.AllDirectories).Select(x => x.Replace(_folder1, "")).ToList();
_content1_files = Directory.EnumerateFiles
(_folder1, "*", SearchOption.AllDirectories).Select(x => x.Replace(_folder1, "")).ToList();

_content2_folders = Directory.EnumerateDirectories
(_folder2, "*", SearchOption.AllDirectories).Select(x => x.Replace(_folder2, "")).ToList();
_content2_files = Directory.EnumerateFiles
(_folder2, "*", SearchOption.AllDirectories).Select(x => x.Replace(_folder2, "")).ToList();

After that, they are compared for differences, first of all, they are compared for files and subfolders that are missing:

C#
if (_mode == ModeEnum.TWO_WAY)
{
    _missing1_folders = _content2_folders.Except(_content1_folders).ToList();
}
_missing2_folders = _content1_folders.Except(_content2_folders).ToList();
if (_mode == ModeEnum.TWO_WAY)
{
    _missing1_files = _content2_files.Except(_content1_files).ToList();
}
_missing2_files = _content1_files.Except(_content2_files).ToList();

After that, based on the selected options, we collect files/subfolders for deletion and overwrite:

C#
_delete_folders = _content2_folders.Except(_content1_folders).ToList();

_delete_files = _content2_files.Except(_content1_files).ToList();

_overwrite = _content1_files.Intersect(_content2_files).Where
(x => (new FileInfo(_folder1 + x)).Length != (new FileInfo(_folder2 + x)).Length).ToList();

Also, an additional step is required to ensure that the operations could be completed successfully – we need to check if there is enough space on the target partitions for the needed changes:

C#
//////////////////////
// CHECK DISK SPACE //
//////////////////////

if (_missing1_files.Count + _missing2_files.Count + _overwrite.Count > 0)
{
    try
    {
        log.Log("CHECKING DISK SPACE...", true);
        DriveInfo _di;
        long _size;
        // check disk space 1
        if (_mode == ModeEnum.TWO_WAY)
        {
_di = new DriveInfo(_folder1.Split(':')[0]);
_size = _missing1_files.Sum(x => new FileInfo(_folder2 + x).Length);
if (_di.AvailableFreeSpace <= _size)
    throw new Exception("ERROR: NOT ENOUGH SPACE ON DISK " + _folder1.Split(':')[0]);
        }
        _di = new DriveInfo(_folder2.Split(':')[0]);
        _size = _missing2_files.Sum
           (x => new FileInfo(_folder1 + x).Length); // size of files to be copied
        _size += _overwrite.Sum(x => new FileInfo(_folder1 + x).Length - 
        new FileInfo(_folder2 + x).Length); // difference in size of files to be overwritten
        if (_di.AvailableFreeSpace <= _size)
throw new Exception("ERROR: NOT ENOUGH SPACE ON DISK " + _folder2.Split(':')[0]);
    }
    catch (Exception ex)
    {
        log.Log(ex.Message, true);
        PressAnyKey();
        log.Close();
        return -1;
    }
    log.Log("DISK SPACE OK!", true);
    log.Log("");
}

Finally, based on the collected enumerables, we are doing the proposed changes:

  1. Files and folders to be copied are printed out (to be reviewed by the user)
  2. Missing folders are created first; missing folder lists are sorted alphabetically to make sure that the parent folders are created first:
    C#
    _missing1_folders.Sort(); // to ensure that parents are created before children
    _missing2_folders.Sort(); // to ensure that parents are created before children
  3. Next, missing files are copied
  4. If option was selected to delete files on folder2, they are deleted one by one. If force mode was not selected, each file for deletion has to be confirmed by the user. After the files are deleted, the folders marked for deletion are deleted in the same way.
  5. If the option to compare filesizes was chosen, then the files marked for overwrite shall be overwritten next. Also depending on force mode, each file needs to be confirmed by the user.

Logging

All the actions are being logged in a file; in the code, there is a LogClass defined which opens and closes the streams to the log file, and contains a method for logging:

C#
class LogClass
    {
        FileStream fs;
        StreamWriter sw;
        public LogClass(string filename)
        {
            if (File.Exists(filename))
                File.Delete(filename);
            fs = new FileStream("Log.txt", FileMode.OpenOrCreate);
            sw = new StreamWriter(fs);
        }

        public void Log(string text, bool writeToFile = false, 
                        bool timestamp = true, bool silent = false)
        {
            if (!silent)
                Console.WriteLine(text);
            if (writeToFile)
                sw.WriteLine(timestamp ? 
                (DateTime.Now.ToString("dd.MM.yyyy HH:mm:ss.fffffff") + "   ") : "" + text);
        }

        public void Close()
        {
            sw.Flush();
            sw.Close();
            fs.Close();
        }
    }

History

  • 29th March, 2020: Initial version

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Marijan Nikic
User Interface Analyst Raiffeisenbank Austria
Croatia Croatia
I acquired Masters degree in computing science at the Faculty of Electrical Engineering and Computing in Zagreb, Croatia in 2009. Following my studies, I got a job in a Croatian branch of Austrian-based CEE Raiffeisen Bank as an MIS (Management information system) analyst.
I have been working there since 2010, as an IT expert within the Controlling department, maintaining the Oracle's OFSA system, underlying interfaces and databases.
Throughout that time, I have worked with several different technologies, which include SQL & PL/SQL (mostly), postgres, Cognos BI, Apparo, Datastage, ODI, Jenkins, Qlik, ...
I am doing a lot of automation with scripting in batch / shell and VBscript (mostly) - data analysis and processing, automated DB imports and exports, Jenkins automation etc.
Privately, I was mostly doing Windows Forms and Console app tools in Visual Studio, C#.

Comments and Discussions

 
SuggestionDates Pin
Roger50031-Mar-20 7:07
MemberRoger50031-Mar-20 7:07 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.