Ah ok, there are two instances to ID=2.
The reason you get all 5 is that ID=2 does not exist in the temp table before the insert statement (and remember you only have a single insert statement, not 4 or 5 individual ones).
Try running the SELECT on it's own:-
SELECT a.Id, a.NAME FROM TEST_DEL A
WHERE NOT EXISTS(SELECT 1 FROM #TEMP B WHERE B.Id = A.Id)
Id NAME
----------- ----------
1 A
2 B
2 C
3 D
4 E
(5 row(s) affected)
You can think of the insert as a two-stage operation:-
1. SELECT all the rows in TEST_DEL that don't exist in the temp table.
2. INSERT those rows into the temp table.