A DOS One-Liner: Running a Program Against a List of Files

Introduction

Yesterday, I needed to run a character mode program that takes a single file name or mask against all files in a directory tree. Since the program in question doesn't support directory recursion out of the box, and I didn't want to take the time to modify it, I devised an alternative strategy based on using a for loop to process a list of file names. Though I've used such loops in batch files for many years, and I was aware that they can be run as one-line commands, this was the first use case I've had that fairly begged for this approach.

Background

Using for loops to process lists is not exactly new, and I've been doing so for at least the last 20 years, more so in recent years than in the distant past. Since most of the command line programs that I use regularly support subdirectory recursion out of the box, I haven't had much need for this capability. Two things made this an outstanding use case.

1. The directory tree is quite large.
2. There were whole subdirectories that needed to be excluded.

Even if the comand line tool in question supported directory recursion, the requirement to exclude whole directories would have prevented its use, leaving me in search of another option. Enter the humble for loop, in the form of a one-line shell command.

Using the code

Thanks to the underrated /b switch, generating the preliminary file list is straightforward, 

dir F:\Praesidium_SVN\ALTPS\admin\*.js /b /s > F:\Praesidium\Armatus_Admin_EventRegistration\_Stories\DT_563\JavaScript_Inventory_ArmatusAdmin.TXTL

Let's take this apart.

• dir F:\Praesidium_SVN\ALTPS\admin\*.js /b /s is a straightforward request for a recursive directory listing. It uses two switches.
  • /b is the Bare Listing switch, which causes dir to list only the file names, one file per line.
  • /s is the Subdirectory Recursion switch, which cause dir to look through all subdirectories of the specified top level directory for names that match the file specification.
  • F:\Praesidium_SVN\ALTPS\admin\ names the top level directory.
  • *.js is the file name mask.
• > is the pipe directive, which redirects output that would have displayed on the command prompt window elsewhere. In our case, elsehwere is a file, though it could as well be the standard input (console input) of another program, such as more.exe.
• F:\Praesidium\Armatus_Admin_EventRegistration\_Stories\DT_563\BabelRC_Inventory_ArmatusAdmin.TXT is the name to give to a new file that receives the output of the dir command.

F:\Praesidium\Armatus_Admin_EventRegistration\_Stories\DT_563\JavaScript_Inventory_ArmatusAdmin_20180423_133355.TXT is a list of every JavaScript file in directory F:\Praesidium_SVN\ALTPS\admin and its subdirectories. However, since the application uses locally installed Node/JS packages, it has an enormous node_modules directory that is irrelevant, and adds significant processing time. (For reasons that have relatively little bearing on the topic at hand, there are actually two node_modules directories in this tree.) Pruning the list is straightforward; open your favorite text editor, and eliminate them from the list. Since the files are listed alphabedically by directory, finding and eliminating the unwanted directories is a cinch.

It's Showtime

for /f "tokens=*" %F in ( F:\Praesidium\Armatus_Admin_EventRegistration\_Stories\DT_563\JavaScript_Inventory_ArmatusAdmin.TXT ) do encoding show %F >> F:\Praesidium\Armatus_Admin_EventRegistration\_Stories\DT_563\JavaScript_Encodings_ArmatusAdmin.TXT

Let's take this apart.

• for /f "tokens=*" %F in sets up the for loop to process a text file composed of lines, each of which contains the name of a file.
  • /f tells for that the list it is receiving is a list of file names.
  • %F is a subsitution token, which is replaced in turn by each file name in the list in the command that comprises the body of the for loop. Note the use of a single % character, unlike the doulbe % used in batch files.
• ( F:\Praesidium\Armatus_Admin_EventRegistration\_Stories\DT_563\JavaScript_Inventory_ArmatusAdmin.TXT ) is the list, comprised in this case of the edited text file that I just described.
• do encoding show %F is the one-line body of the for loop.
  • encoding is the program or command to run. encoding is a compiled Python script that attempts to identify the text encoding of one or more text files.
  • show is a directive to encoding , which instructs the program to show the cureent encoding. The program offers other options for changing the encoding.
  • %F is the substitution token, which is replaced on each iteration with the name of a file to process.
• >> F:\Praesidium\Armatus_Admin_EventRegistration\_Stories\DT_563\JavaScript_Encodings_ArmatusAdmin.TXT is a slightly different output redirection instruction that sends the voluminous output of the loop into a text file for subsequent review, Note again the use of a single % character.
  • >> is a different redirecton symbol; which directs the shell to append to the specified file. Since the command repeats many times, this is required to preserve output from previous iterations. Since a regular redirection (>) creates a new file, it would preserve only the output of the last iteration.
  • The rest of the string is the name to give to the output file.

Hit Enter, sit back, and watch your DOS one-liner run.

Points of Interest

Encoding.exe, the program around which I built this wonderful one-liner, is worthy of attention in its own right, since it works really well, and can be handy for just such use cases as I faced, where I needed to verify that none of the files in two collecitons was UTF-8 encoded.

For many years, I've heard and read about Perl one-liners, and I've used some, and even written a few of my own. Along the same lines, I've read about PowerShell and Bash one-liners. However, this is the first of what I think qualfies as a DOS one-liner.

I put spaces around all special tokens, such as the parentheses that enclose the list and the pipe direcitve. Though technically optional, they make the command much easier to read and deciper.

Bear in mind that if the command that follows do is a batch (.CMD or .BAT) file, its name must be preceded by call. Otherwise, the first invocation would never return to the loop, and only the first file would get processed.

History

Tussday, 24 April 2018 marks the initial publication of this article.