Click here to Skip to main content
15,868,074 members
Articles / Web Development / HTML

Installer Testing and Verification

Rate me:
Please Sign up or sign in to vote.
4.36/5 (6 votes)
22 Jun 2009CPOL4 min read 40.6K   290   29   14
Installer testing verifier - Useful for installer testing and verification.

Introduction

The main purpose of iCheckEngine is to aid testers in installer testing. In general, during installer testing, a tester matches the test scenario provided in the Install Specification.

This includes test against:

  1. File(s) deployment - Test hundreds of files/folders if they are deployed, and for their file sizes and file content.
  2. Registry(s) - Test registry entries for their creation and value.
  3. Executable version - Test if the correct version of the executable is deployed.
  4. Database - Test pertaining to databases, i.e., Tables/Procedures/Triggers are created, and connectivity and password are correct.
  5. XML configuration - XML values in configuration are properly done.
  6. Settings for environment variables

Note: Above is an unexhausted list, there could be many more tests which need to done for an installer testing, inputs are welcome.

We felt the need for this utility when we had many installers to test. Therefore, I worked to make some part of it automatic.

iCheckEngine provides users with the components mentioned below to test scenarios related to installer testing. At the end of each run, it can be configured to create either a file or HTML based report.

I was able to identify the following components which are required for a simple installation testing (inputs are most welcome).

  • File - Files are properly deployed or not, their size, version, content, etc.
  • Registry - Registry keys are created, values are set.
  • XML - Many applications nowadays store their configuration in XML files, therefore checks related to XML nodes.
  • Database - Checks pertaining to DB, tables, procedures, and triggers.
  • String check - It is useful to test environment variables against some general string match.

Two kinds of report generation are supported - HTML and file based:

  • FileLog - Provides functions useful for file based report generation.
  • HtmlLog - Provides functions useful for HTML based report generation.

Understanding iCheckEngine

The iCheckEngine application can be divided into two parts:

  1. iCheckEngine server and parser: The iCheckEngine server and parser are integrated into the main iCheckEngine.exe. Components consist of a number of DLLs which expose a Check function to be called by iCheckEngine.exe.
  2. iCheckEngine components: Components can be further divided into two parts:
    1. Internal - which are integrated into the main iCheckEngine.exe
    2. External - They expose check functions from the DLL
    3. Note: External components need to be explicitly declared in the file iCheckComponentConfig.xml with tmethods they expose. To keep things simple, these exported methods have a fixed signature as defined by the function pointer ISimpleFunc.

Given below are four important configuration files needed for iCheckEngine:

  1. iCheckConfig.xml - Main configuration files to be provided from command line arguments to iCheckEngine.
  2. Components file - Path for these files are mentioned in iCheckConfig.xml.
  3. Install check file - Path for these files is also mentioned in iCheckConfig.xml.
  4. Most important: It is possible to mention multiple InstallCheck nodes. All files will be processed and reports will be generated based on individual file configurations.

  5. Log file - File path application activity will be logged during parsing/execution of iCheckConfig.

Shown below is the sample content of iCheckConfig.xml:

XML
<Install-Check version = "1.0.0.1">
    <Components FilePath="C:\iCheckEngine\config\iCheckComponentConfig.xml" />
    <LogFilePath FilePath="C:\iCheckEngine\Logs.txt" />    
    <InstallCheck FilePath="C:\iCheckEngine\config\iCheckSampleTest.xml" />
    <InstallCheck FilePath="C:\iCheckEngine\config\iCheckSampleTest_2.xml" />
</Install-Check>

iCheckComponentConfig.xml contains the definition for external components and method they expose. To add new components, these files need to be modified appropriately.

Below is a sample content of the file iCheckComponentConfig.xml:

XML
<?xml version="1.0" encoding="UTF-8"?> 
<Install-Check>

<Components>
    <File Enabled="TRUE" Path="C:\iCheckEngine\iFileComponent.dll">
        <Methods>                
            <FileExistenceCheck />
            <FileNotExistenceCheck />
            <DirExistenceCheck />
            <DirNotExistenceCheck />
            <FileSizeGreater />    
            <FileSizeLesser />
            <CheckFileVersion />
            <MatchFileContent />
        </Methods>
    </File>

    <Registry Enabled="TRUE" Path="C:\iCheckEngine\iRegistryComponent.dll">
        <Methods>
            <KeyExistenceCheck />
            <KeyNotExistenceCheck />
            <KeyValueCheck />
        </Methods>
    </Registry>            
    <FileLog Enabled="FALSE" Path="C:\iCheckEngine\iFileLog.dll">
        <Methods>
            <WriteReportHeader />
            <WriteReportTrailer />
            <WriteMessage /> 
        </Methods>
    </FileLog>
    <HTMLLog  Enabled="TRUE" Path="C:\iCheckEngine\iFileLog.dll">
        <Methods>
            <WriteHTMLReportHeader />
            <WriteHTMLReportTrailer />
            <WriteHTMLMessage /> 
        </Methods>
    </HTMLLog>

        .....Content Stripped.................

Section in iCheckSampleTest.xml:

  • Variable - User defined variables (Note: Environment variables are automatically loaded).
  • ExecuteAtStart - Component method to be called at start of check execution.
  • ExecuteForEachCheck - Component method to be called at call of every check function.
  • ExecuteAtEnd - Component method to be called at end of check execution.
  • Aler<code>ts - Alerts need to be defined here.

iCheckSampleTest.xml holds actual instructions to call the appropriate install component functions for designated tasks. Below is the sample content of a file which contains instructions for an installer check: iCheckSampleTest.xml:

XML
<Install-Check>
    <Variables>    
    <HTMLFilePath value ="C:\iCheckEngine\Reports\TestHtml.html" />
    <ReportFilePath value ="C:\iCheckEngine\Reports\TestReport.txt" />
    <ErrorFilePath value="C:\iCheckEngine\Reports\Data\Errors.txt" />
    <HTMLHeaderTemplate value="C:\iCheckEngine\ReportTemplates\
                    HTMLHeaderTemplate.txt" />
    <HTMLFootorTemplate value="C:\iCheckEngine\ReportTemplates\
                    HTMLFootorTemplate.txt" />
    <HTMLMessageTemplate value="C:\iCheckEngine\ReportTemplates\
                    HTMLMessageTemplate.txt" />

    <HeaderTemplate value=
        "C:\iCheckEngine\ReportTemplates\HeaderTemplate.txt" />
    <FootorTemplate value=
        "C:\iCheckEngine\ReportTemplates\FootorTemplate.txt" />
    <MessageTemplate value=
        "C:\iCheckEngine\ReportTemplates\MessageTemplate.txt" />
    </Variables>

    <ExecuteAtStart>    
    <!--This function will be called before 
            install-verifier start the check.-->
        
    <HTMLLog>
        <WriteHTMLReportHeader>
        <Parameter Filepath="@HTMLFilePath" HTMLHeaderTemplate="
        @HTMLHeaderTemplate" HTMLFootorTemplate="@HTMLFootorTemplate" 
        HTMLMessageTemplate="@HTMLMessageTemplate"/>
        </WriteHTMLReportHeader>        
    </HTMLLog>
    <FileLog>
        <WriteReportHeader>
         <Parameter  Filepath="@ReportFilePath" HeaderTemplate="
         @HeaderTemplate" FootorTemplate="@FootorTemplate" 
         MessageTemplate="@MessageTemplate"/>
        </WriteReportHeader>    
    </FileLog>
    </ExecuteAtStart>

    <ExecuteForEachCheck>
    <!--This function will be called for each checks and 
    every function it executes.-->
    <FileLog>
        <WriteMessage>
        <Parameter  Message="#OutputMessage" Status="#Status" 
         Check="#CheckName" Component="#Component" Method="#Method" />
        </WriteMessage>
    </FileLog>
    <HTMLLog>
        <WriteHTMLMessage>
         <Parameter  Message="#OutputMessage" Status="#Status" 
         Check="#CheckName" Component="#Component" Method="#Method" />
        </WriteHTMLMessage>
    </HTMLLog>
    </ExecuteForEachCheck>

    <Alerts>
    <Check Id="Test">            
        <StringCheck>
        <Equals>
             <Parameter  String1="@USERNAME" String2="
             bd98101" CaseSensitive="true" />
              <Parameter  String1="@USERNAME" String2="
              bd98101" CaseSensitive="false" />
        </Equals>
        <EqualsLike>
            <Parameter  String1="@OS" String2="windows" 
                    CaseSensitive="true"/>
        </EqualsLike>
        </StringCheck>
            
        <File>                 
        <FileExistenceCheck>
            <Parameter Filepath="C:\AUTOEXEC.BAT" />
            <Parameter Filepath = "C:\FileNotExist.txt" />
        </FileExistenceCheck>
        <CheckFileVersion>
            <Parameter Filepath = "C:\WINNT\TASKMAN.EXE" 
            Version = "5.0.2134.1"/>
        </CheckFileVersion>
        <MatchFileContent>
            <Parameter File1 = "C:\iCheckEngine\TempFiles\File1.txt" 
            File2 = "C:\iCheckEngine\TempFiles\File2.txt" 
            CaseSensitive="true"/>
        </MatchFileContent>
        </File>        
    </Check>

    <Check Id="CheckDirectoryExist">
        <File> 
        <DirExistenceCheck>
            <Parameter Dirpath="C:\Winnt"/>
            </DirExistenceCheck> 
            <DirExistenceCheck>
            <Parameter Dirpath="@windir\system32"/>
        </DirExistenceCheck>
        </File>
    </Check>            

    <Check Id="CheckFileSizes">
        <File> 
        <FileSizeGreater>
            <Parameter Filepath="C:\Test.txt" size="10240" />
        </FileSizeGreater>    
        <FileSizeLesser>
            <Parameter Filepath="C:\Test.txt" size="10240" />
        </FileSizeLesser>
        </File>
    </Check> 
        
    <Check Id="CheckRegistry">
        <Registry> 
        <KeyExistenceCheck>
                <Parameter KeyPath="SOFTWARE\Microsoft\
            Internet Explorer\W2kVersion" 
            KeyNode="HKEY_LOCAL_MACHINE"/>
        </KeyExistenceCheck>

        <KeyValueCheck>
            <Parameter ExpectedValue="6.0.2800.1106" 
              ValueName="W2kVersion"
            KeyPath="SOFTWARE\Microsoft\Internet Explorer" 
            KeyNode="HKEY_LOCAL_MACHINE"/>
        </KeyValueCheck> 
        </Registry>
    </Check> 
    </Alerts>
        
    <ExecuteAtEnd>
    <!--This function will be called after install-verifier completes the check.-->
    
        <HTMLLog>            
                <WriteHTMLReportTrailer>
           <Parameter Filepath="@HTMLFilePath" GeneratedDateTime="
           #CurrentDateTime" TotalCheck="#TotalCheck" SuccessCount="
           #SuccessCount" FailedCount="#FailedCount" 
        iVersion = "#iVersion"/>
              </WriteHTMLReportTrailer>
        </HTMLLog>        
        <FileLog>
        <WriteReportTrailer>
           <Parameter Filepath="@ReportFilePath" GeneratedDateTime="
           #CurrentDateTime" TotalCheck="#TotalCheck" SuccessCount="
           #SuccessCount" FailedCount="#FailedCount" iVersion = "#iVersion"/>
             </WriteReportTrailer>
        </FileLog>    
    </ExecuteAtEnd>
</Install-Check>

I believe "A picture speaks 1000 words", and for coding, the output speaks for sabout 1000 lines of code...

Sample HTML Output

Image 1

Sample Text Output

iCheckText_Output.JPG

Thanks

Special thanks to Alex Hazanov for his XML Wrapper that helped in reading XML files.

History

  • Updates 22-June-2009:
    • Source download file:
      • Fixed compiler issue pertaining to Debug and Release versions.
      • Now uses generic paths to allow projects to be extracted in any folder and Run.
    • Article text:
      • Added HTML and TEXT output images to the article.
    s

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Software Developer (Senior)
United States United States
VC++, MFC, COM, Unix Shell Scripts
Linux Kernel Development

Comments and Discussions

 
General+ RemoteInstall Pin
dB.19-Nov-09 2:14
dB.19-Nov-09 2:14 
QuestionRelease version Pin
LOUIS Christian17-Jun-09 4:15
LOUIS Christian17-Jun-09 4:15 
AnswerRe: Release version Pin
Kuldip Rindani17-Jun-09 4:22
Kuldip Rindani17-Jun-09 4:22 
GeneralRe: Release version Pin
LOUIS Christian18-Jun-09 2:23
LOUIS Christian18-Jun-09 2:23 
GeneralRe: Release version Pin
LOUIS Christian18-Jun-09 4:00
LOUIS Christian18-Jun-09 4:00 
GeneralRe: Release version Pin
Kuldip Rindani18-Jun-09 4:44
Kuldip Rindani18-Jun-09 4:44 
GeneralRe: Release version Pin
LOUIS Christian18-Jun-09 20:33
LOUIS Christian18-Jun-09 20:33 
AnswerRe: Release version Pin
Kuldip Rindani17-Jun-09 5:00
Kuldip Rindani17-Jun-09 5:00 
GeneralRe: Release version Pin
LOUIS Christian18-Jun-09 1:02
LOUIS Christian18-Jun-09 1:02 
GeneralCrossplatform Pin
Spatlabor15-Jun-09 17:42
Spatlabor15-Jun-09 17:42 
GeneralRe: Crossplatform Pin
Kuldip Rindani15-Jun-09 22:40
Kuldip Rindani15-Jun-09 22:40 
GeneralCode Download Pin
sam.hill11-Jun-09 16:21
sam.hill11-Jun-09 16:21 
GeneralRe: Code Download Pin
Kuldip Rindani11-Jun-09 18:18
Kuldip Rindani11-Jun-09 18:18 
GeneralRe: Code Download Pin
sam.hill11-Jun-09 19:18
sam.hill11-Jun-09 19:18 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.