Bitsup - Server Upload Utility Using BITS
4.70/5 (19 votes)
Jun 26, 2005
6 min read
98693
674
A file upload utility that uses BITS as the transfer engine.
Introduction
Bitsup (Bits Up) is a program designed to automate data transfer from a production machine to a backup server. The program uses the BITS Microsoft Windows service to perform the transfer of files or directories. The program is command line driven, and designed to be run from a task scheduler, batch file, or script.
The Bitsup program creates an upload job in the BITS Microsoft Windows Service. It then compresses the data to be transferred, adds the compressed file or directory to the upload job, and starts the job. At this point, the BITS service takes over and handles the transfer of the file to the server.
The Bitsup program will monitor the transfer and present a progress bar, or can be setup to "Fire And Forget" the job; submitting the job to BITS, then exiting. There is an /Admin switch that will start Bitsup in the Administration mode, which displays all the jobs in the transfer queue, and allows the jobs to be cancelled, paused, resumed or completed.
Background
BITS
Background Intelligent Transfer Service (BITS) is a Windows component that asynchronously transfers files in the foreground or background, throttles the transfers to preserve the responsiveness of other network applications, and will automatically resume file transfers when a computer is restarted and a network connection is re-established.
I think that one of the beautiful things about BITS is that it extends the HTTP protocol to transfer files, and because it uses port 80, it is often immune to firewall restrictions. Also important is the fact that the default setting in BITS is keep trying to send a file for 14 days!
BITSAdmin
BITSAdmin is a command-line tool that you can use to create download or upload jobs and monitor their progress. Bitsup is primarily a "BITSAdmin Wrapper" utility. Because the BITS API is old school COM, and I have had enough of that, thanks for asking, I decided that the quickest way to get an interface into BITS was to wrap the BITSAdmin tool.
BITSAdmin Download Page on MSDN.
Zip file compression
Bitsup uses the Sharp Zip Library, and Open Source C# Compression Library. The compression engine supports the ZIP, GZIP, BZIP2 and TAR formats. For the Bitsup program, the compression used is the ZIP format. The compression level can be set from 0 to 9. A compression value of 0 will simply store the file/directory uncompressed inside the zip file. A compression value of 9 will produce maximum compression but will take more time to create the zip file archive. The default value is 6, which can be changed by editing the Bitsup.exe.config file using a text editor, such as Notepad.
I found a nice C# class for wrapping the SharpZipLib on the internet, and converted it to VB.NET for this project. I added some events to the class so that I could display a progress bar in a Windows Form dialog. I have also exposed a property to set the compression level for the zip file.
Public Event FileProcessed(ByVal sFileName As String, _
ByVal iPercent As Integer)
Public Event DirProcessed(ByVal sDirName As String, _
ByVal iPercent As Integer)
Public Event ZipComplete()
Public Event ZipStarted()
The Compression Progress dialog
Using the code
The heart of Bitsup is the cShell class, which presents a shared method (GetProcessText()) for calling a process, and getting the output from the called process. Bitsup uses GetProcessText() to wrap the BITSAdmin command line tool, and present a graphical interface.
Public Class cShell
Public Shared Function GetProcessText(ByVal process As String, _
ByVal param As String, _
ByVal workingDir As String) _
As String
Dim p As Process = New Process
' this is the name of the process we want to execute
p.StartInfo.FileName = process
If Not (workingDir = "") Then
p.StartInfo.WorkingDirectory = workingDir
End If
p.StartInfo.Arguments = param
' need to set this to false to redirect output
p.StartInfo.UseShellExecute = False
p.StartInfo.RedirectStandardOutput = True
p.StartInfo.CreateNoWindow = True
' start the process
p.Start()
' read all the output
' here we could just read line by line and display it
' in an output window
Dim output As String = p.StandardOutput.ReadToEnd
' wait for the process to terminate
p.WaitForExit()
Return output
End Function
End Class
The Command Line interface
Bitsup uses a command line interface. If Bitsup is started without any command line arguments, a help screen will be displayed, showing the available options.
Errors encountered during the command line parse will be displayed at the top of the screen.
The Help screen
The Command Line format
bitsup /MODE [/TYPE] [filename|directory]
MODE = RESET | ADMIN | TRANSFER
TYPE = FILE | DIRECTORY
Mode of operation
There are three basic "modes of operation" that Bitsup understands. The MODE parameter tells Bitsup which of these three modes to start up in. The MODE parameter must be prefixed by a forward slash. The valid values for the MODE parameter are RESET, ADMIN, and TRANSFER.
The RESET mode parameter causes Bitsup to clear all the transfer jobs from the transfer queue. This will stop any uploads already in progress.
The ADMIN mode parameter will display a listing of all the jobs currently in the transfer queue. Job ID, Job Name, Job State, number of files transferred, Number of files total, Number of bytes transferred, Number of bytes total are displayed for each job.
The TRANSFER mode parameter causes Bitsup to transfer data to the server. If the TRANSFER mode is used, the next parameter must be the TYPE parameter, which tells Bitsup which type of transfer to perform.
Type of transfer
If the MODE parameter is set to TRANSFER, then the TYPE parameter must be set to the type of transfer. Bitsup was designed to be extendable to other types of data besides FILES and DIRECTORIES. In the place where I work, there are several systems that need to be backed up regularly to an offsite location. I designed Bitsup to handle this task. In this code base, I have stripped out the other system specific transfer types.
To add your own custom transfer types is a simple exercise. Thinking about it, I should have created a cTranferType object that was inheritable, and descended the FILE and DIRECTORY transfers. Maybe in version 2.0 that will happen. I did write this in one night, after all.
A FILE transfer type specifies the transfer of a single file. The next parameter would be the full path to the file, including a drive letter. The file will be compressed and transferred to the server.
A DIRECTORY transfer type specifies that Bitsup should compress the contents of a directory and all of its sub-directories, and transfer the compressed archive to the server.
Points of interest
The CodeProject template file, from which I created this page, asks the following question:
Did you learn anything interesting/funny/annoying while writing the code? Did you do anything particularly clever or wild or zany?
Thinking about this question I did come up with one item. I found that the Process object has a problem getting a handle to a program's standard output stream when Bitsup was started via a shortcut. The solution to this was to create a batch file calling Bitsup, and start the batch file from the shortcut.
Dim p As Process = New Process
Also of interest, at least to me, is that I wrote Bitsup in one night. I have since thought of several improvements that I could make, but I am undecided as to whether the time improving Bitsup might be better spent creating a true wrapper around the BITS API. For now, Bitsup does what it is supposed to do, which is to transfer files using the BITS service.
History
- 6/25/2005: Posted to CodeProject.