Image processing as fast as possible in C#

• Download source code - 2.7 MB
• Download exe - 1.3 MB

Introduction

Most image processing applications use hardware acceleration for heavy image filters. So why not this bring to C#? That's what this library does.

Effects in this library:

• Blur
• Directional Blur
• Zoom Blur
• Invert
• Tone Level
• Gray scale
• Old Image
• MonoChrome
• Pixelate
• Ripple
• Wave Warper
• Swirl

Custom effects are supported to extend the library.

Using the Effects:

EffectHelper class is the actual worker so you must load it first.

EH = new EffectHelper();
Size s = EffectHelper.GetMaxSize();
EH.Load(s.Width, s.Height); 

EffectHelper class has two methods (ApplyEffect, ApplyMultipleEffects) that you will use to apply an effect. They:

• take the input (imagedata(Frame), effectdata)
• copy imagedata to video memory
• process (one effect or multiple effects)
• Copy imagedata to system memory
• return the output data.

The difference between them is that ApplyMultipleEffects doesn't return the processed image and take it as input again. it process all given effects and then return.

To apply an effect to an image:

BitmapData bd;
Bitmap b = (Bitmap)CurrentBitmap.Clone();
Frame frm = Frame.OpenBitmap(b, out bd);

frm = EH.ApplyEffect(frm, data);

Frame.CloseBitmap(b, bd); 

To apply multiple effects:

BitmapData bd;
Bitmap b = (Bitmap)CurrentBitmap.Clone();
Frame frm = Frame.OpenBitmap(b, out bd);
EffectData[] dataseries = new EffectData[3];
dataseries[0] = SwirlEffect.GetData(20f, 1f, new PointF(.5f, .5f));
dataseries[1] = ZoomBlurEffect.GetData(0.5f, new PointF(0.5f, 0.5f));
dataseries[2] = GrayScaleEffect.GetData();
frm = EH.ApplyMultipleEffects(frm, dataseries);
Frame.CloseBitmap(b, bd);    

To register a new effect:

scaleEff = new ScaleExtended(EH.Device);
EH.Manager.Register("ScaleExtended", scaleEff); 

Effects

Before you read the following code you must know that:

1. You must have a background about how DirectX shader effects work to well understand how these effects work.
2. All effects are rendered perpixel and written by HLSL shader language (it is similar in syntax to C#).

So these effects are not written by c# code but it uses directX to comile the effect code (found in resources folder) to be run on video processor. this takes the advantage of performance but has some limits.

3. You can create your own effects easly by Shazzam tool, but when you use it here you must add this part of hlsl code

technique Tech
{
    pass Pss
    {
        pixelshader = compile ps_2_0 main();
    }
}

The code you write on shazzam tool is simply a small program running on graphics card and the above lines tells the directx hlsl compiler that you will use the method 'main()' as the start of this small program.

Let's start.

1. Blur Effect

float Amount : register(C1);
sampler2D  Texture1Sampler : register(S0);

float4 main(float2 uv : TEXCOORD) : COLOR
{
    float4 c = 0;
    float r360 =  0.0174533f;
    float rad ;
    float xOffset;
    float yOffset;

    c += tex2D(Texture1Sampler, uv);
    float2 v; 
 
    for(int i=0; i<8; i++)
    {
        rad= i*45 * r360;
        xOffset = cos(rad);
        yOffset = sin(rad);
        v.x = uv.x -( Amount * xOffset);
        v.y = uv.y -( Amount * yOffset);
        c += tex2D(Texture1Sampler, v);
    }
   
    c /= 9;
    
    return c;
} 

2. Directional Blur

 float Angle : register(C0);
float Amount : register(C1);
sampler2D  Texture1Sampler : register(S0);
float4 main(float2 uv : TEXCOORD) : COLOR
{
    float4 c = 0;
    float rad = Angle * 0.0174533f;
    float xOffset = cos(rad);
    float yOffset = sin(rad);
    for(int i=0; i<16; i++)
    {
        uv.x = uv.x - Amount/16 * xOffset;
        uv.y = uv.y - Amount/16 * yOffset;
        c += tex2D(Texture1Sampler, uv);
    }
    c /= 16;
    
    return c;
} 

3. Zoom Blur

sampler2D  input : register(S0);

float progress  : register(c0);
float2 center : register(c1);

float4 main(float2 uv : TEXCOORD) : COLOR
{

float BlurAmount =- progress * .4;
 float4 c = 0;    
 uv -= center ;

 for (int i = 0; i < 14; i++)  {
  float scale = 1.0 + BlurAmount * (i / 13.0);
  c += tex2D(input , uv * scale + center );
 }
   
 c /= 14;
  
return c;
} 

4. Grayscale

sampler2D input : register(s0);

float4 main(float2 uv : TEXCOORD) : COLOR 
{ 
        
    float4 color; 
    color= tex2D( input , uv.xy); 
        
    color.r = (color.r +color.g + color.b)/3.0f;
    color.g = color.r;
    color.b = color.r;
    return color; 
}

5. Invert

sampler2D Texture1Sampler : register(S0);

float4 main(float2 uv : TEXCOORD) : COLOR
{
   float4 color = tex2D( Texture1Sampler, uv );
   float4 invertedColor = float4(color.a - color.rgb, color.a);
   return invertedColor;
} 

6. Monochrome

float4 FilterColor : register(C0);
sampler2D  Texture1Sampler : register(S0);

float4 main(float2 uv : TEXCOORD) : COLOR
{
   float4 srcColor = tex2D(Texture1Sampler, uv);
   float3 rgb = srcColor.rgb;
   float3 luminance = dot(rgb, float3(0.30, 0.59, 0.11));
   return float4(luminance * FilterColor.rgb, srcColor.a);
} 

7. Old

sampler2D TexSampler : register(S0);

float4 main(float2 uv : TEXCOORD) : COLOR
{
    float4 color = tex2D(TexSampler, uv);
    
    float gray = dot(color, float4(0.3, 0.59, 0.11, 0)); 
    color = float4(gray * float3(0.9, 0.8, 0.4) , color.a); 
   
    return color;
}  

8. Ripple

float2 Center : register(C0);
float Amplitude : register(C1);
float Frequency: register(C2);
float Phase: register(C3);
float AspectRatio : register(C4);
sampler2D Texture1Sampler : register(S0);

float4 main(float2 uv : TEXCOORD) : COLOR
{
 
    float2 dir = uv - Center; // vector from center to pixel
    dir.y /= AspectRatio;
    float dist = length(dir);
    dir /= dist;
    dir.y *= AspectRatio;

    float2 wave;
    sincos(Frequency * dist + Phase, wave.x, wave.y);
        
    float falloff = saturate(1 - dist);
    falloff *= falloff;
        
    dist += Amplitude * wave.x * falloff;
    float2 samplePoint = Center + dist * dir;
    float4 color = tex2D(Texture1Sampler, samplePoint);

    float lighting = 1 - Amplitude * 0.2 * (1 - saturate(wave.y * falloff));
    color.rgb *= lighting;
    return color;
}  

9. Swirl

float2 Center : register(C0);
float SpiralStrength : register(C1);
float AspectRatio : register(C2);
sampler2D input : register(S0);

float4 main(float2 uv : TEXCOORD) : COLOR
{
    float2 dir = uv - Center;
    dir.y /= AspectRatio;
    float dist = length(dir);
    float angle = atan2(dir.y, dir.x);

    float newAngle = angle + SpiralStrength * dist;
    float2 newDir;
    sincos(newAngle, newDir.y, newDir.x);
    newDir.y *= AspectRatio;
    
    float2 samplePoint = Center + newDir * dist;
    bool isValid = all(samplePoint >= 0 && samplePoint <= 1);
    return isValid ? tex2D(input, samplePoint) : float4(0, 0, 0, 0);
} 

10. Wave wrapper

sampler2D input : register(s0);
float Time : register(C0);
float WaveSize: register(C1);

float dist(float a, float b, float c, float d){
    return sqrt((a - c) * (a - c) + (b - d) * (b - d));
}

float4 main(float2 uv : TEXCOORD) : COLOR 
{ 
    float4 Color = 0;
    float f = sin(dist(uv.x + Time, uv.y, 0.128, 0.128)*WaveSize)
                  + sin(dist(uv.x, uv.y, 0.64, 0.64)*WaveSize)
                  + sin(dist(uv.x, uv.y + Time / 7, 0.192, 0.64)*WaveSize);
    uv.xy = uv.xy+((f/WaveSize));
    Color= tex2D( input , uv.xy);
    return Color; 
} 

11. Pixelate

float2 PixelCounts : register(C0);
float BrickOffset : register(C1);
sampler2D Texture1Sampler : register(S0);

float4 main(float2 uv : TEXCOORD) : COLOR
{
   float2 brickSize = 1.0 / PixelCounts;

   // Offset every other row of bricks
   float2 offsetuv = uv;
   bool oddRow = floor(offsetuv.y / brickSize.y) % 2.0 >= 1.0;
   if (oddRow)
   {
       offsetuv.x += BrickOffset * brickSize.x / 2.0;
   }
   
   float2 brickNum = floor(offsetuv / brickSize);
   float2 centerOfBrick = brickNum * brickSize + brickSize / 2;
   float4 color = tex2D(Texture1Sampler, centerOfBrick);

   return color;
} 

12. Tone level

float Levels : register(C0);
sampler2D Texture1Sampler : register(S0);

float4 main(float2 uv : TEXCOORD) : COLOR
{
    float4 color = tex2D( Texture1Sampler, uv );
    color.rgb /= color.a;

    int levels = floor(Levels);
    color.rgb *= levels;
    color.rgb = floor(color.rgb);
    color.rgb /= levels;
    color.rgb *= color.a;
    return color;
}  

ScaleExtended (Example of effect extension)

sampler2D input : register(s0);

float scalex : register(C0);
float scaley : register(c1);

float centerx : register(C2);
float centery : register(c3);
float4 main(float2 uv : TEXCOORD) : COLOR 
{ 
    float4 color; 
    
    uv.x -= centerx;
    uv.y -= centery;
    
    uv.x /= scalex;
    uv.y /= scaley;
    
    uv.x += centerx;
    uv.y += centery;
    
    if (uv.x < 0 || uv.y < 0 || uv.x > 1 || uv.y > 1 )
    color = 0 ;
    else
    color= tex2D( input , uv.xy); 


    return color; 
}

Multiple effects in one shot

Tthis is an example on applying multiple effects at a time

BitmapData bd;
Bitmap b = (Bitmap)CurrentBitmap.Clone();

Frame frm = Frame.OpenBitmap(b, out bd);

EffectData[] dataseries = new EffectData[3];

dataseries[0] = SwirlEffect.GetData(20f, 1f, new PointF(.5f, .5f));
dataseries[1] = ZoomBlurEffect.GetData(0.5f, new PointF(0.5f, 0.5f));
dataseries[2] = GrayScaleEffect.GetData();
frm = EH.ApplyMultipleEffects(frm, dataseries);

Frame.CloseBitmap(b, bd); 

This code will do the following

SwirlEffect then >> ZoomBlurEffect then >> GrayScaleEffect.

There is an option in the test application "Set As Original Image" to put the result as input:

Advantages

• Take the maximum hardware acceleration of your GPU.
• extend your custom effects.
• can draw a full scene (3D objects, particles, and blend modes) with lights and 3d shadows in your custom effect (full DirectX features).
• every input is handled separately by the ShaderEffect class hence you can pass texture, color, matrix, point, vector, or any other type as input.

Limits

• Number of effects is small but you can use Shazzam Shader Editor to get more and more effects and extend the library .
• Max size of image is hardware dependent. But most Graphics cards support 2048 X 2048 pixels.
• Library uses software vertex processing so it renders only the ps_2_0 effects but you can change the device initialization at EffectHelper to hardware vertex processing to allow ps_3_0 effects( this will run only on Graphics card that support ps_3_0 and if not it will throw exception).
• Effects are running on GPU so any running game with fullscreen mode will pause the effect till returning to window mode or closing the game.