Click here to Skip to main content
15,867,308 members
Articles / Programming Languages / C#

Source Code Super Search

Rate me:
Please Sign up or sign in to vote.
4.71/5 (22 votes)
10 Mar 2009CPOL8 min read 52.1K   1.5K   66   16
A simple solution for searching source code directories
Image 1

This information pertains to the v 0.9 release. The latest release v 1.0 is discussed in the 03-2009 Updates Section.

Introduction

This tool is a simple search utility I developed to search all my source code, regardless of language. The tool integrates with Visual Studio 2008, so you can kickoff a search while writing code. The utility will search any directory for files that match the extensions specified, and it will match search strings (supports Regular Expressions) inside the matched files. Results are presented in a clean treeview format, and allow you to take various actions with the results.

Background

I spend around 80% of my day in front of a computer screen writing code for customers, and a lot of that time is spent digging through old code that I've written in the past, trying to remember how I solved a problem previously so I can implement the solution in a current project. I wish I could put a precise time value on this exercise, but the closest value I can come up with is a lot. I determined early on that I needed a GREP type application that I could use to search a large directory hierarchy containing thousands of source files. There are a lot of Windows Grep programs out there, and I tried a few of them (the ones that were free). Ultimately, it proved too tedious and cumbersome to use many of them for my needs, and there were some features I really wanted specific to Visual Studio IDE programmers, such as the ability to open a *.sln solution file for a matched code file, that just isn't possible in a switchblade (one size fits all) type Grep-ping application.

Visual Studio's Find In Files Option

Of course, Visual Studio already provides a robust search feature (CTRL+SHIFT+F) which allows you to search for patterns in files based on extensions and user defined search strings. The functionality of this tool mimics the functionality of the Visual Studio feature, but adds some additional options like the ability to (try to locate and) open the *.sln file associated to the matched code file, and the ability to open the folder containing the matched file. This tool can search and match patterns against file names as well as match patterns in files.

Thanks Drew Stainton for this tip: You can further customize the behavior of the Find in Files Visual Studio function by editing Visual Studio's Registry settings: Customize your Find in Files Results experience!

Usage Outside of Visual Studio

This solution plugs into Visual Studio as an added feature, so you can launch it while working within Visual Studio, but it's not dependent on Visual Studio in any way. It can be run on its own, and can be used for searching any code (or any file readable in Notepad, for that matter). I've used it quite a bit to search in VBScript, batch, and Perl scripts as well as for searching file server log files.

Project Goals

The ultimate goal of the project is to significantly reduce the effort required to look through code files. The solution needs to search the file system as fast as possible and present a simple output that the user can interact with.

In summary:

  • Find file names that match the specified extensions and search pattern
  • Find values inside the files that match the specified search pattern and the specified extension
  • Search needs to be as fast as possible
  • Solution needs to persist the user defined data between sessions

Matches need to allow these behaviors:

  • Open file
  • Open file in Notepad
  • Open containing folder
  • Open code file's owning solution file (*.sln)

To Index or Not To Index

This is a question I have toiled over for quite some time. If I index all the files in a location, then the search results would be instant. This would make users very happy, but it would add a lot of overhead to the solution. Ultimately, I decided, for this utility, every search would be real time, and all actions would be conducted at the time of the search instead of indexing prior to execution.

Search Methodology

There are a lot of ways we can perform a search. In the past, benchmarks seemed to indicate that using...

C#
string[] myFiles = Directory.GetFiles
			(USER_PATH, FILEMASK, SearchOption.AllDirectories);

... was considerably slower than using the command line function:

c:\>dir /S /B USER_PATH\*.FILEMASK > ApplicationAppDataDirectory\cache.txt

However, I'm not so certain of this anymore. I have run a series of completely unscientific benchmarks on these two scenarios.

Scenario A: Command.com dir Function

  1. Run the dir command with Process.Start
  2. Use a StreamReader to match a RegEx pattern in each line of the output file
  3. Build a SearchResult object, List<T>
  4. Populate a TreeView control with the results
PathFileMasksPatternSTypeFoundSearchedTime
c:\.log[Ll]aunchingIn Files58152534 sec
VS2005\Projects.cs .vbDirectoryEntryIn Files70308113 sec
VS2005\Projects.csDirectoryInfoIn Files5730377
\\baileyfs01\Music.mp3MegadethFile Name22514,9038

Scenario B : Directory.GetFiles

  1. Generate an output file with StreamWriter based on the Directory.GetFiles results
  2. Use a StreamReader to match a RegEx pattern in each line of the output file
  3. Build a SearchResult object List<T>
  4. Populate a TreeView control with the results
PathFileMasksPatternSTypeFoundSearchedTime
c:\.log[Ll]aunchingIn Files58152557 sec
VS2005\Projects.cs .vbDirectoryEntryIn Files70308114 sec
VS2005\Projects.csDirectoryInfoIn Files5730377
\\baileyfs01\Music.mp3MegadethFile Name22514,90313

These benchmarks were run on a Vista machine with 3GB of RAM and a Q6600 Quad-Core Processor. As you can see from the statistics above, the results slightly favor running the DIR command over Directory.GetFiles().

Image 2

However, in all fairness, it doesn't really make sense to have Directory.GetFiles write the results to a file since realistically, the results are already in an Array.

As it stands right now, I have depreciated the DIR search functionality while I'm testing all the ways to use GetFiles(). The version included in this article's deliverables isn't using DIR, though the classes are present to turn it back on for further benchmarks. Update: I have noticed from daily use that the first time the application is run each day takes a really, really long time to execute the first search but it has not been enough of a irritant to make me go in another direction.

The CommandLine engine is housed in a class called CommandLine.cs, and minus all the abstraction members (fields, properties, constructors, etc.), it looks like this:

C#
private void DoShellExecute()
{
    ProcessStartInfo psi =
      new ProcessStartInfo(m_ApplicationPath, m_ApplicationParameters);
    psi.UseShellExecute = true;
    psi.RedirectStandardOutput = false;
    psi.CreateNoWindow = true;
    psi.WindowStyle = ProcessWindowStyle.Normal;

    if (ExecuteType == ExecutionType.SHELL_EXECUTE_HIDDEN)
        psi.WindowStyle = ProcessWindowStyle.Hidden;

    Process p_exec = new Process();

    try
    {
        p_exec = Process.Start(psi);

        if (m_TimeOutThreshold != null || m_TimeOutThreshold == 0)
        {
            p_exec.WaitForExit((int)m_TimeOutThreshold);
        }
        else
        {
            p_exec.WaitForExit();
        }
    }
    catch (SystemException e)
    {
        m_Error = e;
    }
    finally
    {
        p_exec.Close();
    }
}

On the other hand, the GetFiles() route doesn't need its own type, so we use it directly in the IOWorker class.

C#
/// <summary>
/// Creates the index of the user defined root path
/// </summary>
public void CreateIdx()
{
    foreach (string f in m_uso.filters)
    {
        try
        {
            m_idx.AddRange(Directory.GetFiles(
                m_uso.path, "*" + f,
                SearchOption.AllDirectories));
        }
        catch (UnauthorizedAccessException e)
        {
            System.Diagnostics.Debug.WriteLine(e);
        }
    }
    //m_idx.Reverse();
    m_SearchedCount = m_idx.Count;
    SearchResults(m_uso.pattern);
}

Adding to Visual Studio 2005/2008

Included with this article is the source code as well as the binary application files. If you wish to add this application to the Tools menu in Visual Studio 2008, extract all the files in the VS2008 add-in zip file into the Addins folder in your Visual Studio user path (c:\Users\You\Visual Studio 2008\Addins).

Image 3

As noted earlier, there is no dependency on Visual Studio, so you can just run the *.exe to run the program on its own.

~~ Update 03-2009 ~~

Regular Expressions

Recently I had a requirement where I needed to remove a code block from every page in a legacy web site. The web site was entirely composed of HTM, HTML, and PHP files and in total there were about 3000 of them! The code block looked like this:

JavaScript
<script language="javascript">
  var breakThis = "http://customerdomain/cfaq/browse.php"; //break frames
  if (window != parent) parent.location.href = breakThis;
  //-->
</script>

The previous build of the tool executed searches against each line in each file. In order to match strings that span multiple lines, I added an option to treat the entire FileStream as a single string. The new option Search all lines together, allows you to search entire files at once to match patterns through entire files. In addition, there is now an Ignore Case flag so searches are no longer case sensitive.

Replace In Files

Per the requirement above, I had to find the script block in all 3000+ web files and remove them. A new Replace feature is now available which allows you to replace a string matched in the RegEx search with a new string (or in my case, empty string).

Undo Replace In Files

When you initiate a Replace action, the original files (the ones that were matched in the search action) are backed up and given a *.orig extension. If you find that you shouldn't have allowed an automated Regex.Replace action against 3000 files, you can undo the replace action. The undo feature effectively deletes the modified files and restores the *.orig files to their previous state as the original files.

Reporting/Export

A new report feature is now available which allows you to export the results of each search.

Conclusion

To this point, I'm extremely pleased with the results of this project. I find myself using this search tool constantly throughout the day, and it makes my day to day tasks much easier. Having used it for several days now, I'm looking into indexing to a server, and the possibilities that may open up for finding code blocks.

History

  • 10/08/2008 - Article submitted
  • 10/08/2008 - Corrected incorrect statements regarding existing VS functionality
  • 03/09/2009 - Released v 1.0

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Software Developer
United States United States
I'm a professional .NET software developer and proud military veteran. I've been in the software business for 20+ years now and if there's one lesson I have learned over the years, its that in this industry you have to be prepared to be humbled from time to time and never stop learning!

Comments and Discussions

 
GeneralExcellent! but.... Pin
Doncp19-Mar-09 13:40
Doncp19-Mar-09 13:40 
GeneralRe: Excellent! but.... Pin
thund3rstruck19-Mar-09 14:04
thund3rstruck19-Mar-09 14:04 
Generalexcellent Pin
rodelizius11-Jan-09 18:36
rodelizius11-Jan-09 18:36 
Generalgreat dev util2 Pin
rodelizius11-Jan-09 18:34
rodelizius11-Jan-09 18:34 
GeneralRe: great dev util2 Pin
thund3rstruck11-Jan-09 19:15
thund3rstruck11-Jan-09 19:15 
GeneralRe: great dev util2 [modified] Pin
thund3rstruck9-Mar-09 18:40
thund3rstruck9-Mar-09 18:40 
General[Message Deleted] Pin
rodelizius11-Jan-09 18:01
rodelizius11-Jan-09 18:01 
GeneralSimply fantastic, what a great idea. Pin
mikehatlak16-Oct-08 6:37
professionalmikehatlak16-Oct-08 6:37 
GeneralWhere was this a year ago? [modified] Pin
Jason Barry16-Oct-08 2:24
professionalJason Barry16-Oct-08 2:24 
GeneralRe: Where was this a year ago? Pin
thund3rstruck9-Mar-09 18:38
thund3rstruck9-Mar-09 18:38 
Generalnice Pin
Huisheng Chen9-Oct-08 17:03
Huisheng Chen9-Oct-08 17:03 
GeneralVS Find In Files Pin
Marc Clifton8-Oct-08 9:45
mvaMarc Clifton8-Oct-08 9:45 
GeneralRe: VS Find In Files Pin
thund3rstruck8-Oct-08 10:21
thund3rstruck8-Oct-08 10:21 
GeneralRe: VS Find In Files Pin
Drew Stainton8-Oct-08 13:41
Drew Stainton8-Oct-08 13:41 
GeneralRe: VS Find In Files Pin
thund3rstruck8-Oct-08 15:19
thund3rstruck8-Oct-08 15:19 
GeneralRe: VS Find In Files Pin
Drew Stainton8-Oct-08 16:25
Drew Stainton8-Oct-08 16:25 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.