The EXE file contains the IL Code (and some other support gubbins)
The framework is called to run your app when the EXE is executed (via the support gubbins) and the JIT compiler converts it into code that is directly executable by the real processor, as the program is running.
The EXE file is a type of PE file, as is a .NET DLL - they are both executable and share a similar structure, so yes, the PE file contains IL code.
But you will not find a file called "myApp.PE" at any time!
There is a good explanation of the execution process here:
http://www.mastercsharp.com/article/90/exploring-your-first-c-application-hello-world[
^] - it's about a third of the way down the page.