SWAT - A simple Web-based Anomalies Tracker - Part 7

• Download source files - 10.98 KB

Swat Part 7

This is the seventh article in a series describing the development of an application I devised as a learning project. The purpose of the project was to gain experience developing in the .NET environment. The goal I had given myself was to define a web-based application and then develop the application using ASP.NET. The articles describe my implementation solution for the application. The application being developed is a full-featured bug tracking application. The first six articles describe and implement the main functionality of the application, that is the editing features and notification capabilities of the application. In the remaining articles the final two features described in the first article will be implemented, namely analysis and reporting. This articles takes a little detour to cover a missed feature. Then continues by presenting the benefits of analysis and concludes by describing two controls to be utilized in the next article.

• Swat Part 1
• Swat Part 2
• Swat Part 3
• Swat Part 4
• Swat Part 5
• Swat Part 6

Database Concurrency

So I'm working on SWAT part7 and a reader sends in a comment asking how the application protects against concurrent upates to the database. Doh!

What makes it even worse is that I had just finished reading an article on ASP.NET concurrency protection schemes and it didn’t occur to me that I had overlooked that 'feature' in SWAT! By the way the article was ‘Implement Optimistic Concurrency’ by Bob Beauchemin.

OK, so the problem is that two users could be viewing the same record from the database. If both try to update the record, only the last one to make the update will have the changes reflected in the database. The reason is that the data returned to the client is returned as disconnected data, there are no locks on the record(s) retrieved by the client. Therefore any intervening changes to the database are not reflected on the client and the last update will overwrite any intervening updates.

In SWAT’s case we’re lucky in that only one record is being updated at a time. In other circumstances which might allow the user to edit multiple rows at a time then things get a little more complicated because you have to check which row(s) have caused a conflict. The DataAdapter’s ‘RowUpdated’ event, which is called each time a row is updated, can be used in those cases.

There are two approaches to circumvent the concurrency problem. The first is considered a 'pessimistic' approach which assumes that multiple users will always be making concurrent updates and therefore 'locks' the record so no one else can update the record until the first user to fetch the data releases the record. This is not a very good approach since the record could stay locked indefinitely (the user might croak in the middle of an editing session). And as the number of users increases so will the probability for latency due to accessing a locked record.

The second approach is the 'optimistic' approach and assumes that no changes will have occurred to the database while the data is being viewed. Of course chances are that changes will have been made. So this approach checks at the time of update to see if any changes have taken place since the record was retrieved. At that point the decision is made if the update can be proceed.

The solution then, requires that we provide some means of checking if any intervening changes have been made to the record since it was retrieved. The easiest way to do this is simply to have an additional field in the record that will contain a unique ‘update identifier’. This field will be read (and persisted) by the client and then passed back to the database during an update. The ‘update identifier’ is then checked during the update. If it has changed, then there has been an intervening update to the record and the update fails. Otherwise the update succeeds and the identifier is updated with a new unique value that represents the current update.

The ‘update identifier’ can be anything, a timestamp field, a date, anything that’s unique and updateable. For SWAT I simply added an integer field which is incremented during each successful update. Here’s the revised SWATUpdateBug stored procedure so you can see what’s going on.

CREATE PROCEDURE dbo.SWATUpdateBug
(
   @itemname nvarchar(50),
   @description nvarchar(1024),
   @module int,
   @revision nvarchar(10),
   @assignedto int,
   @severity tinyint,
   @priority tinyint,
   @id int,   @updatesequence int
)
AS
DECLARE @sequence int
SET NOCOUNT ON;

SELECT @sequence=bugs.updatesequence FROM bugs WHERE bugs.id=@id

IF (@sequence <> @updatesequence)
BEGIN
    return 0
END
ELSE
BEGIN
SET NOCOUNT OFF;
UPDATE bugs SET itemname=@itemname, description=@description, 
  revision=@revision, severity=@severity, priority=@priority, 
  module=@module, assignedto=@assignedto,
  updatesequence=(@sequence+1)
WHERE id=@id

END

This also takes care of the condition where the bug may have been deleted while another user is editing the record. There are probably many other approaches to resolving the concurrency problem, this is just one solution.

So for SWAT, BindBugData() was revised to persist the new field to a hidden control on the page. Likewise updateBug() was revised to pass the persisted value to the stored procedure. Notifying the user was the only tricky part. What is appropriate? I think each situation requires some analysis as to what may be the best way advise the user. For SWAT I wanted more than just an error message. But how do you return back two sets of data? Remember we're returning the edit page. After some thought I chose a middle of the road solution. Which you can expand if you wish. The solution I implemented was to create a script that would pop-up a message on the client when a concurrency error was detected. The window will display the bug title and the description of the attempted update. Since the editing page will be automatically refreshed with the current database record(s), the user will have on the screen both sets of data and can see the differences. A small change could also be made that will indicate to the user if the failure was due to the record being deleted. But as it is, it should be pretty evident. The updated SWAT code will be available for download in the next article where the complete analysis feature is implemented.

private void updateBug(System.Web.UI.WebControls.DataListCommandEventArgs e)
{
  ...
  cmd.Parameters.Add("@updatesequence", SqlDbType.Int).Value =
     System.Convert.ToInt32(lblUpdateSequence.Text);
  int n=cmd.ExecuteNonQuery(); 
  if (n!=1)
  {
    StringBuilder strMess = new StringBuilder();
    strMess.Append("Concurrency Error!\\n");
    strMess.Append("BUGTITLE:");
    strMess.Append(((TextBox)e.Item.FindControl("txtBugTitle")).Text);
    strMess.Append("\\nDESCRIPTION:");
    strMess.Append(txtDescription.Text);
    StringBuilder strScript = new StringBuilder();
    strScript.Append("<SCRIPT language="'\""JScript\"' event=onload for=window>\n");
        strScript.Append("alert(\"");
        strScript.Append(strMess.ToString());
        strScript.Append("\")\n");
        strScript.Append("</SCRIPT>");
    Page.RegisterClientScriptBlock("ClientScript",strScript.ToString());
  }
  ...
}

The case for analysis

Why do we need to analyze the data? What benefits can be derived from the analysis? Isn’t it enough that we’re testing and using a bug tracking system? The last question is rhetorical, please don’t answer it.

Running the analysis on the data is like an act of introspection. It can indicate to us where we are, where we’re going, and the quality of our code. The analysis won’t provide any absolute answers but will provide plenty of indicators.

Consider it like a thermometer without any markings. You can see the level rising and falling but without the markings it may not seem to be very useful. Well, even without any markings you can still glean some information from the thermometer. We know that thermometer measures temperature. By looking at the level we can tell that it’s somewhere between the minimum and the maximum. That it’s closer to the bottom than to the top. Or it’s pretty close to the middle. With some historical information we can start putting some labels on the thermometer. If it’s an extremely hot day you can put a mark on the thermometer’s level for that day and label it ‘really hot’. Same thing for a ‘really cold day’. Heck, you can even make a marking for the ‘perfect temperature’ (relatively to you, of course).

Same thing applies to the analysis tools. There are no absolutes. No one can tell you that there should be at least 50 bugs found in each module. Or that when you find 50 bugs you’re done. Or that you should test for two weeks. However, as with the thermometer there are some things that you don’t need a gauge to interpret. For example if the number of bugs found continues to rise over time and doesn’t level off and decline....well, there’s a big problem.

This should prompt you to start asking some questions. Are you testing and coding at the same time to try to catch up on a lost case schedule? Does each iteration seem to add more bugs than are being fixed? Are you 'discovering' new things to test with each iteration? Each of these should tell you something. Maybe the original architecture/design was not the correct one. Do changes (fixes) seem very hard to implement. If the application is one monolithic piece of code... There are plenty of possible causes for the analysis result.

The opposite condition should also raise some questions. Suppose the analysis tools show that there are few bugs being discovered. Which is good, that’s what should happen. But make sure that in fact it is the result of excellent design, solid requirements, and super developers. And not due to insufficient or inadequate testing. Remember no testing equals no bugs.

Of course the analysis tools should only be necessary to provide some schedule feedback, to indicate if the project seems to be on schedule or not. Hopefully, the above questions are not asked during testing but rather during the initial design phase. But if they do come up during testing perhaps the good news is that they may provide some insight as to how to do it better next time. Don’t forget the old paraphrase ‘you design-in quality, you don’t test for quality’. Testing should be viewed and used as a confirmation of quality.

You can also use the analysis tools for module comparison within the project. Are there some modules that have a larger number of bugs? Is it because the module is a lot more complicated than the rest? Is the inherent design of the module the cause? Is the developer struggling? Does he need additional training or is the module implementing new unproven technology?

So you see, even without a gauge you can step back and take a look at the project using the analysis tools and get a feel as to how the project is doing. Without analyzing the data you may not know that there is a problem until it’s way too late. The analysis is another tool that we can use to control the project instead of the project controlling us.

Now let’s take a look at the analysis tools that SWAT provides. I have to say that these comprise a minimum set, there are probably quite a few that can be added to provide more insight into the data. These are just some ticklers for thought.

Module Status

This is a very simple chart which indicates graphically the status of a particular module. I would think the developer can easily glance at it and get an idea of where s/he stands. A quick easy check on the amount of work still left to do.

Progress

This chart is a little more instructive. If the above were for a real project and the date span was larger, I think it would be time to call in the troops. The graph indicates that the number of open bugs has been increasing since testing started. Normally you would expect the number of open bugs should start decreasing after some time and the number of closed bugs would increase accordingly. A real plot will also have some cycles in it. However, you would expect that perhaps halfway into your test cycle that the number of open bugs would start to decrease. Otherwise no progress is being made.

The graph also allows you to plot by severity level. So, for example, if you wanted to see...hmmm, here's another variation. I just thought of another useful graph. How about a graph that would plot the bugs excluding one of the severity levels? I'm guessing that it would be interesting to see the plot without, for example, the enhancement bugs. Perhaps the application will be released with enhancements and level4 bugs open. In that case the picture above might look completely different. Well, maybe on a future release.

Activity

This chart represents the testing 'activity' level over time. Typically you would expect to see levels that are representative of your testing cycles. In a typical test phase there may be any number of test cycles. I'm referring to a test cycle as the point when the tester(s) receive a new release of the code. Which should be a controlled process and not as a result of the nightly build. Each test cycle then, may also include some amount of regression testing.

The graph should show quite a bit of activity initially and over time the number of bugs found, fixed, or closed should approach zero. Just a note on the above. I don't mean to imply that development (fixes) and testing cannot take place simultaneously. It's just that the release of code to testing is what needs to be controlled. Otherwise it's just like a dog chasing it's tail. And the results will be the same.

Project Status

Finally, here's a chart that plots the overall status of the project by showing all the modules. In one glance you should be able to get a pretty good idea of where the project stands. It can also indicate any modules where additional effort might be needed or where too many problems have been found. Obviously the size and complexity of the module will have an impact on what is displayed on this graph. However, you'l be able to tell if something doesn't 'look' right. Does a very small or simple module have twice as many bugs as another that is much larger?

SwatChartLib

There are two custom WEB controls that we’ll utilize in the next article. One is a custom DropDownList control that maintains a master/detail relationship and performs all it’s functionality on the client. This was the topic of another article (Master/Detail DDL) so I won’t describe it here. The article describes the control and the downloadable source includes a test app so you can see how to use it. The reason I developed the control was that I didn't want to make a trip back to the server just to fill the modules DropDownList when the selected project changed.

The other control is a graphing control which I will describe below and for which the downloadable code can be found at the beginning of this article. I will only describe the controls interface in this article. The source contains sufficient comments to indicate what is going on. But basically it’s just a lot of GDI coding to render a graph. I’ll mention that there are quite a few graphing controls out there so this is not a very sophisticated one. It simply does what I wanted it to do and I have control over the source which means I can change it and/or expand on it’s functionality if needed.

I chose to implement the charting functionality as a control for two reasons. First it gave me another opportunity at developing a user control. Second it left the door open to replace it with a more sophisticated control in the future. Besides the above, there are a few other reasons for trying to implement any functionality as a control. First, it facilitates it's re-use in other projects. Second, it conceptionalizes the development. It also encapsulates the code so that any changes are constrained within the control. And, it can be tested and developed independently of the rest of the application. Oh, and developing the control also gave me some experience with the new ‘GDI’. How many ways can you define a Pen object?

The SwatChartLib defines a virtual base Chart class and also includes three derived implementations of the Chart class. The Chart class defines one method and only two properties. So you see that it is really not intended as a general purpose charting control. There are lot's of other properties that you can expose by simply adding the access methods. I only exposed the ones I was using. Again, this was not really intended as a general purpose charting control.

...
#region public properties
public string strXaxisLabel
{
    get{return m_strXaxisLabel;}
    set{m_strXaxisLabel = value;}
}
public string strYaxisLabel
{
    get{return m_strYaxisLabel;}
    set{m_strYaxisLabel = value;}
}
#endregion
#region public methods
public Chart()
{
   ...
}
virtual public void DrawChart(Array arraySeriesInfo, Array arrayGraphData,
                              float fYScale, float fYTickStep, 
                              float fXTickStep)
{
}
#endregion
...

Each of the derived classes implements the DrawChart() to render the appropriate graph. There are five parameters passed to the method and I'll describe each. The first parameter, arraySeriesInfo, is just that, information relating to the series to be plotted. It's a two dimensional array where each row represents a series to be plotted. The first element in the row contains a string with the label for the series. The second element contains the color to be used in rendering the series. Aren't these arrays great! You can stuff anything in them. No more being limited to arrays containing only one homogeneous type.

The second parameter is an array that contains the data to be plotted. The first element is reserved for a label for the data point 'set', if any. The data values for each of the series follows the label and completes each row.

The other three parameters represent the max y-value for the graph, the frequency to use in plotting the x-axis tick marks, and the frequency to use in plotting the y-axis tick marks.

Homework

So you’ll need to download the source for both controls and build them. You can give them a strong name and install them in the GAC if needed. Otherwise just add a reference in the SWAT project to each of the controls. You may also want to add the MasterDetailDDL to the Toolbox (Tools...Customize Toolbox) so you can drag/drop from the Toolbox.

OK, I think we're ready. Next time we'll jump right into the code to add the analysis feature.