How can i delete a assembly(*.dll) when it is used in a application ?

See more:

C#3.5

I have created a DLL (introp layer (C++)) which responsible for compiling un-managed code (in c).
I am adding this Dll into a C# windows form application.

Now my requirements is

I need to modify the intorp layer DLL as per some checked list and same DLL should be replaced in C# windows form application where it get referenced.

In short i want remove a Dll which is Being used in a Application.

What I have tried:

[DllImport ("kernel32.dll" , SetLastError = true , CharSet = CharSet .Unicode)]
static extern bool MoveFileEx(string lpExistingFileName, string lpNewFileName,
MoveFileFlags dwFlags);
static void Main(string [] args)
{
string fileName = @"e:\1.wav" ;
MoveFileEx(fileName, null , MoveFileFlags .MOVEFILE_DELAY_UNTIL_REBOOT);
}

--------------------------------
File.Delete(filename);
--------------------------------
GC.Collect();
GC.WaitForPendingFinalizers();
FileInfo f = new FileInfo(@"D:\New folder\Test0001.wav");
f.Delete();
--------------------------------
Copy to new file and delete
--------------------------------
rename to new file and delete