A custom control for image annotations and image processing

• Download source files - 61.3 Kb

Introduction

This is a major update of my previous article on annotations. I preferred to leave the original in place and focus more on the evolution of the code here.

What was wrong with the previous versions?

At first, I implemented an annotation class for 'border points', i.e., annotations you can modify by settings points that define the border of some region (either programmatically or interactively). The interior and border regions would then automatically be generated using GDI+, and these could be used for hit testing, to compute image properties, etc.

However, most image processing algorithms do not work with border points, but rather with the points that make up the interior of the region that is defined by these border points. Conversion from a set of border points to a region was trivial, but the reverse was not. Consequently, I wrote another class especially for such 'region based' annotations. In this class, the user can set and manipulate the interior region itself, and not the border points. Both the point- and region based annotations descended from an abstract annotation class which implemented the common properties and methods. There was also a class for annotations that did not define an interior region. Generally, when things get that complicated, there is something wrong with the design, and so soon several problems surfaced. Most importantly, drawing the border points of a region-based contour proved tricky and very slow, and did not work properly when zooming.

Why GDI+ regions are REALLY bad for image processing

In the region-based annotation, the user could set the region, and the class would draw it on screen. This involves determining the border region, e.g., by a process called morphological erosion, or by simple checking the 'connectedness' of points in the region. So, I started writing some routines for working with GDI+ regions. However, soon it was apparent that although regions may be an efficient way to store irregularly shaped areas, they are horribly inefficient for image processing! Indeed, the most intensively used method of a GDI+ region is IsVisible, and it is very slow (I suppose location information is not directly available due to the encoding of regions). Also, in image processing, we would constantly add and remove very small, pixel-sized rectangles to regions, something which again proves quite slow. Zooming in or out with complex regions consisting of many small rectangles proved again very slow. On top of that, there are some annoying quirks in GDI+, e.g.: you need a Graphics object to determine a bounding box, but not to determine if a point is in a region ... (can anybody explain the logic of this to me?). As annotations are independent of an image and only obtain a Graphics object when they need to redraw themselves on a surface, this proved troublesome.

The solution to this problem was to use what is generally termed 'mask' images. These are bitmaps (literally, 1-bit images here) that contain information about pixels being in a region or not. Clearly, determining if a pixel is in a region is easy here (and fast with unsafe C# code). Rewriting my image processing routines to use masks instead of regions gave an enormous speed improvement (10 x or more!). I then only needed to write routines that could transform a mask to a region and vice versa, and set the region in a region-based annotation to show it on screen.

A mask image

However, this didn't solve the problem of determining the border points of a region in the region-based annotation class, and now most processing time was spent in the routines doing just that. Also, for some unknown reasons, I kept getting Stack overflows in drawing.dll, when determining the border points of very complex regions.

A lean model for annotation

So, there was no way around it: ditch the region based annotations, and write a routine that could transform a mask to a set (!) of point-based annotations. The internals of the point-based annotations would then efficiently determine the border and internal region using a graphicsPath. These would zoom perfectly, and have the added bonus of providing all the information of a mask: border points, interior and sub regions. On top of that, the annotations class becomes much, much simpler.

Class Annotations

Class that contains all code for the annotations, and all its collections.

• Class AnnotationContainer

  Class used to contain annotations.

• Class AnnotationContainerCollection

  Class defining a strongly typed collection of Containers.

• Class Annotation
• Class AnnotationCollection

  Class defining a strongly typed collection of annotations.

Compared to the previous model, this is quite an improvement!

The routine for determining a set of closed contours from a mask is based on an article called 'A linear-time component-labeling algorithm using contour tracing technique' by Fu Chang et al. (Institute of Information Science, Taipei, Taiwan). It is not implemented completely, but works nonetheless quite well and is very fast.

Image processing

Although I will dedicate a whole article to my image processing routines in the near future, I have included the C# source in the download. It is by no means finished, and not all routines have been thoroughly tested, but it includes some routines for working with regions and mask, as well as arithmetic image operations, image extrema, medianfilter, convolution, resizing/resampling, binary morphology (erosion, dilation, open, close), etc. ...

Conclusion

1. Simple is best: it took 6 months of constant evolution to end up with the most efficient and simple class hierarchy.
2. GDI+ regions are not the way to go for image processing, use mask images instead.
3. To get any sort of speed out of .NET, you find yourself working with so much unsafe code that it's C all over again: allocate memory, deallocate, check leaks, etc... Also, a lot of GDI+ objects are not managed and you need to be really careful about releasing them!