Provide User Interfaces for Adhoc Blackbox testing class methods

• Download demo project - 204 Kb

 Contents 

1. Introduction
2. Background
3. First example - keep it simple
3.1 Expose a method
3.2 Create the console application
3.3 Start the application
4. Extend the sample - make it more complex
4.1 Explicitly define Adhoc items sources
4.2 Optimize UI representation
4.3 Start the application
5. Parameter type support 
5.1 Primitive types
5.2 Complex types
5.3 Out parameters
5.4 Ref parameters
6. Build your own TestUIRenderer
6.1 Implement TestUIRenderer
6.2 Interact with TestUIManager
6.3 Expose the renderer to TestUIManager
7. Conclusion

1. Introduction 

Testing methods always is an issue in a professional development-lifecycle and there are a lot of tools you can use to automate method-testing. Probably you don't see any reason why it is important to be able to invoke methods later on - for example in a production environment. But often methods behave in a different way in some situations - especially when external ressources like webservices are involved.

This article addresses everyone who is interested in having a smart user interface with the ability to invoke methods out of an assembly with providing all parameters. The interface will also show the methods output and so gives you a way to monitor application functionality in a very detailled manner.

At the end of this article you will know how to prepare a class and its methods to be part of the Adhoc invokation solution. Furthermore you will be able to use the built-in console-application to start running a concrete test-scenario. For the more curious readers the article also gives a detailled introduction
how to wrap up the solution into an own UI (i.e. Web-Interface or Windows Forms).

2. Background 

When applications start doing strange things in production or stage environment there is nothing more than your logs or the frontend user experience which help you to guess the errors cause. Some of you may already have build a little helper-tool to maintain or monitor your application. So I had, because I wanted to be able to have a blackbox view on different functionalities. My solution was not well planned or structured and I had to extend it whenever I had a new interest on another part of the application. So I got the idea to make a tool, which is able to handle all my interests. It should consider every single part of the application I define. And since applications are usually made of classes and methods these are my points of interest.

3. First example - keep it simple 

Before deep-diving into the code, I will show you how to start with the console UI. It provides a test environment for methods of a prepared sample assembly. 

3.1 Expose a method 

Add a reference to assembly Lerch.TestUI.Core in your Visual Studio project and mark a class as accessible to the TestUI with the AdhocAccessible attribute. Every class having this attribute will be considered. Next choose a method to be invokable by the TestUI and add the AddhocInvokable attribute to it. This already
exposes the method, but we also want to invoke the method with parameters. Finally add an attribute to every single parameter, which should be set by the user via TestUI.

[AdhocAccessible]
public class Calculator
{
    [AdhocInvokable]
    public double Add(
	[AdhocUsable(Caption = "Addend 1")] double dNum1,
	[AdhocUsable(Caption = "Addend 2")] double dNum2))
    {
	return dNum1 + dNum2;
    }
}

The Caption property inside the AdhocUsable attribute gives the parameter a more descriptive appearance in the TestUI. There are a lot of more properties you will learn about later on.

3.2 Create the console application 

Since the TestUI solution is just a small API and no independent application, you have to create it on your own. Make a new console application project in Visual Studio, add assembly-references to Lerch.TestUI.Core and to the assembly containing the exposed method (in my case Lerch.Project.X). Add some code.

static void Main(string[] args)
{
    // load assemblies considered in our test console
    var oAssembly = Assembly.Load(new AssemblyName("Lerch.Project.X"));
    // create the manager
    var oTestUIManager = new TestUIManager(oAssembly, typeof(TestUIRendererConsole)) { Name = "Project Test Console" };
    // start the engine
    oTestUIManager.Start();
}
 

All you have to do is to instanciate the TestUIManager giving it the assembly to consider and a type-reference to a ITestUIRenderer implementation. The renderer does all the work concerning UI presentation. TestUIRendererConsole is built-in for you, but with your own implementation you are able to render the test environment in a web page or Windows form. On the other hand TestUIManager does all the reflection, invokes the method and feeds the Renderer with results. If you keep it simple, that's all.

3.3 Start the application  

It will look like as follows:  

Menu with all classes having the AdhocAccessible attribute.  

Menu with all methods in selected class having the AdhocInvokable attribute. 

Input option for all parameters from selected method having the AdhocUsable attribute.

After parameter input method is invoked. Result is shown to user.  

4. Extend the sample - make it more complex

4.1 Explicitly define Adhoc-items sources  

TestUIManager also accepts more than one assembly to search for AdhocInvokables. This sample adds a second assembly called Lerch.Project.Y. Another option is to limit AddhocInvokable consideration on specific namespaces. The sample will do so and also tells TestUIManager to search in every descendant namespace (third parameter of constructor).  

static void Main(string[] args)
{
    // load assemblies considered in our test console
    var lstAssembies = new List<Assembly>()
    {
        Assembly.Load(new AssemblyName("Lerch.Project.X")),
        Assembly.Load(new AssemblyName("Lerch.Project.Y"))
    };
 
    var lstNamespaces = new List<string>()
    {
        "Lerch.Project.X.Folder.Entities",
        "Lerch.Project.Y"
    };
 
    // create the manager
    var oTestUIManager = new TestUIManager(lstAssembies, lstNamespaces, true, typeof(TestUIRendererConsole)) { Name = "Project Test Console" };
    // start the engine
    oTestUIManager.Start();
} 

4.2 Optimize UI representation 

Since TestUI accesses productive classes and methods it is often not easy for the user to understand the method signature and to provide a valid parameter input.

Attribute properties not only enable you to prepare a more suitable appearance in the console but also have the benefit to control the parameter input.  

4.2.1 AdhocAccessible attribute is a class atribute and has the following properties:

Caption: Give a descriptive name to the class.

Comment: Give a short comment on the class.

Display: Controls the string representation of the class. (Options: Caption, NameWithCaption, FullNameWithCaption, FullName, Name)

[AdhocAccessible(Caption="Calculate something.", Comment="Main operations", Display=TestUIManager.EntityDisplayOption.Caption)]
public class Calculator
{
	// ...
}
 

4.2.2 AdhocInvokable attribute is a method attribute and has the following properties:

Caption: Give a descriptive name to the method.

Comment: Give a short comment on the method.

Display: Controls the string representation of the method. (Options: see above)

[AdhocInvokable(Caption = "Add two numbers.", Comment = "Or make it on your mind.", Display = TestUIManager.EntityDisplayOption.Caption)]
public double Add(...)
{
    // ...
}
 

4.2.3 AdhocUsable attribute is a parameter attribute and has the following properties:

Caption: Give a descriptive name to the parameter.

Comment: Give a short comment on the parameter.

Display: Controls the string representation of the parameter. (Options: see above)

Disabled: Controls, if the parameter gets its value from user input or not.

DefaultValue: If Disabled is true, this property provides a value. If not set, value will be default type value or null.

MinInclusive: If parameter type is numeric, set the minimum.

MaxInclusive: If parameter type is numeric, set the maximum.

[AdhocInvokable]
public double Add(
    [AdhocUsable(Caption="Addend 1", Comment="Greater than 0", MinInclusive=1)] double dNum1,
    [AdhocUsable(Disabled=true, DefaultValue=1)] double dNum2)
{   // ...
}

Restart the application and see the difference. With suitable names, helpful comments and default values the user interface can change in a way letting the user forget that he's actually working on classes and methods. 

5. Parameter type support

Because user input is always of a string there are some limits on the parameter's type.

5.1 Primitive types

All supported even if they are nullable. These types are bool, byte, char, DateTime, decimal, double, Int16, Int32, Int64, int, sbyte, single, string, UInt16, UInt32, UInt64. TestUIManager uses the TryParse()-method of each type to convert from the user input to the appropriate type.

5.2 Complex types 

Usually not supported. TestUIManager will tell you, if it fails to convert the user input to the complex type. If you have complex type parameter in your method do not mark it as AdhocUsable and TestUIManager will provide the type's default value or null to the parameter.

Anyway there are some options to make a complex type usable:

5.2.1 Disable and provide a default value

You learned about that already. Unfortunately only static values can be assigned to the property.

5.2.2 Add a static TryParse-implementation to the type

If the parameter type is a custom type you can implement a TryParse-method. Think of a type Person with two members Name and GivenName. Set up the AddhocUsable parameter as follows:

[AdhocUsable(Caption="Person 1", Comment="Format: Givenname, Name")] Person oPerson
[AdhocUsable(Disabled=true, DefaultValue="Kay,Lerch")] ref Person oPerson2

First one expects a user input with comma seperated values. The second parameter gets a default value - also in the expected format. Now implement the logic to convert the string input to a Person. Secondary provide the logic to convert a Person to a string, since the second parameter is a reference and will have an output to the console after method-invokation.

public class Person
{
    // ...
    public static bool TryParse(string strIn, out Person oPerson)
    {
	oPerson = new Person();
	var arrStrings = strIn.Split(new char[] { ',' });
		
	if (arrStrings.Length > 1)
	{
	     oPerson.GivenName = arrStrings[0].Trim();
	     oPerson.Name = arrStrings[1].Trim();
	}
        return (arrStrings.Length > 1);
}

    public override string ToString()
    {
        return string.Format("{0} {1}", GivenName, Name);
    }
}

5.2.3 Convert user input in renderer

This is what you can do in your own renderer. Because you know the parameter type while rendering the input dialogue, feel free to react with different input-controls on different types. For an array of strings you will probably render a textarea in a web page and handle each line as one array-item. TestUIManager expects object-values, so whatever you create in your renderer, TestUIManager will pass it through directly to the method.

5.3 Out parameters 

Out-parameters are supported. Even if you declare those parameters as AdhocUsable there will be no input option on TestUI. Instead you will be provided with the value after method invokation in the TestUI's result view. There's nothing wrong with marking an out-parameter as AdhocUsable anyway, since you can give it a descriptive name over the Caption property.

5.4 Ref parameters 

Ref-parameters are supported. Like out-parameters their value will appear in the result view. If a ref-parameter is AdhocUsable you can give it a value, because TestUI will give the user an input option before method invokation. The following picture shows you the result view of a method with an out-parameter of type Person and a ref-parameter of type string. You can also see the input option for the ref-parameter.

6. Build your own TestUIRenderer

Console applications are not really easy on the eye. If you want your own user interface this solution gives you all you need. There are three important things you have to know:

- How to implement the ITestUIRenderer interface
- How to interact with TestUIManager
- How to expose your UI to the TestUIManager 

The architecture looks as follows:

 

6.1 Implement TestUIRenderer

You find the interface in Lerch.TestUI.Core library. Implement you own TestUI whereever you want. Think of a Windows forms application you may place your implementation within the forms project. First of all you need to add a public member, which holds a reference to TestUIManager. This is easy, just have a look at the code from the TestUIRendererConsole sample:

public class TestUIRendererConsole : ITestUIRenderer
{
    public TestUIManager TestUIManager
    {
        get;
        set;
    }
    //...
}

The interface has four methods you must implement. The following should render the list of AdhocAccessible classes and should also provide an input option for the user. This method will be called from TestUIManager as soon as you Start() it. The enumeration that comes with the call contains items, which represent AdhocAccessible classes with all reflected properties and those properties set up in the AdhocAccessible attribute.

public void RenderTestUIClassPicker(IEnumerable<AdhocAccessibleClassItem> lstClassItems)
{
    // render input dialogue 
    // ...
    // go on with loading the method picker
    TestUIManager.LoadMethodPicker(lstClassItems.FirstOrDefault());
}
  

The next method should render the list of AdhocInvokable methods from a selected class and should also provide an input option for the user. This method will be called from TestUIManager as soon as you LoadMethodPicker(). The enumeration that comes with the call contains items, which represent AdhocInvokable methods with all reflected properties and those properties set up in the AdhocInvokable attribute. Additionally the currently selected AdhocAccessible class item is provided.

public void RenderTestUIMethodPicker(AdhocAccessibleClassItem oClass, IEnumerable<AdhocInvokableMethodItem> lstMethodItems)
{
    // render input dialogue 
    // ...
    // go on with loading the method picker
    TestUIManager.LoadParameterValuePicker(lstMethodItems.FirstOrDefault());
} 

Continue with the implementation of the method that should render the list of AdhocUsable parameters from a selected method. It should also provide the user an input option for every parameter. This method will be called from TestUIManager as soon as you LoadParameterValuePicker(). The enumeration that comes with the call contains items, which represent AdhocUsable parameters with all reflected properties and those properties set up in the AdhocUsable attribute. You have to consider, that also the Disabled AdhocUsables are in this enumeration. Additionally the currently selected AdhocInvokable method item is provided. 

public void RenderTestUIParameterPicker(AdhocInvokableMethodItem oMethod, IEnumerable<AdhocUsableParameterItem> lstParameterItems)
{
    var dictParamterWithValues = new Dictionary<AdhocUsableParameterItem>();
    // render input dialogue 
    // ...
    foreach (var oTestParameter in lstParameterItems.Where(x => x.Disabled == false))
    {
	// save input values
        dictParamterWithValues.Add(oTestParameter, "some value");
    }
 
    try
    {
	// go on with preparing method invokation
        TestUIManager.Invoke(oMethod, dictParamterWithValues);
    }
    catch (TestUIInvokationException ex)
    {
        // react on invokation exceptions ...
    }
} 

This is where you would add custom conversions on complex types. Instead of "some value" you can give any object to the dictionary.

Finally there is a method to render the result view. This method will be called from TestUIManager as soon as you Invoke().  The enumeration that comes with the call contains items, which represent method outcomes. The invoked AdhocInvokable method item is also provided. 

Ipublic void RenderTestUIInvokationResult(AdhocInvokableMethodItem oMethod, IEnumerable<AdhocInvokationReturnItem> lstReturnItems, long lRuntime)
{
    foreach (var oReturn in lstReturnItems)
    {
        switch (oReturn.ItemType)
        {
            case AdhocInvokationReturnItem.ReturnItemType.Out: /* render output */ ; break;
            case AdhocInvokationReturnItem.ReturnItemType.Ref: /* render output */ ; break;
        }
	// render output
    }
}
 

6.2 Interact with TestUIManager 

As you might already seen, we interacted with TestUIManager in the renderer. This is not required, you can also call TestUIManager from a controller if you're following the MVC pattern. But in some way you have to provide the user input to the TestUIManager and this is done by method calls. Take a look at the methods inside the core library- they all have a brief description.

Since TestUIManager also calls the renderer there is a circular dependency and a high risk of infinite recursions. For instance the TestUIManager does not allow a renderer to call LoadMethodPicker() within RenderTestUIMethodPicker(). The TestUIManager then throws an exception instead of letting
the infinite recursion happen.  

6.3 Expose the renderer to TestUIManager

At last the renderer needs to be exposed to the TestUIManager. Everything else is done behind the scenes. The best place to do it is somewhere in the startup code of your UI project. If your renderer is called TestUIRendererForms the code should look as follows:

// load assemblies considered in our test console
var lstAssembies = new List<Assembly>()
{
    Assembly.Load(new AssemblyName("Lerch.Project.X"))
};
 
// create the manager
var oTestUIManager = new TestUIManager(lstAssembies, null, false, typeof(TestUIRendererForms)) { Name = "Project Test Console" };
// start the engine
oTestUIManager.Start(); 

7. Conclusion 

TestUI core library contains everything you need to wrap up your classes and methods in a smart user interface. It enables adminstrators to invoke your programs functionalities parameterized and to monitor their behaviours. TestUI can also be used to provide an environment for manual blackbox tests without writing test-methods. This is great for Adhoc testing.

In another scenario it is now possible to use TestUI to release a simple class library (or a whole collection of those) as an indepenent UI application. It is on your own to enhance the user experience with your own TestUIRenderer.  

The demo project demonstrates the use of TestUI with a console application. You can refer to its implementation when you want to provide your own user interface with TestUIManager. Feel free to use this solution or even enhance it but leave a comment.