Monday, May 4, 2015

How 32-bit applications make system calls in 64-bit Windows

First, I'll describe how 32-bit programs make system calls in 32-bit Windows, then I'll describe how it runs in the WOW64 environment and make system calls in 64-bit Windows.

In 32-bit windows:

Executable loads kernel32.dll and ntdll.dll 
Calling the win32 API, CreateFile, ends up calling kernel32.dll!CreateFileW


kernel32.dll!CreateFileW ends up calling ntdll.dll!ZwCreateFile
ntdll.dll!ZwCreateFile moves the system call number (0x42 for ZwCreateFile) into EAX, and...

...executes the "SYSENTER" instruction, which makes the transition into the Windows kernel.

For 32-bit programs on 64-bit Windows:

Upon starting the program:
CommandLine: "C:\Users\admin\Documents\Visual Studio 2013\Projects\test\Release\test.exe"

************* Symbol Path validation summary **************
Response                         Time (ms)     Location
Deferred                                       SRV*c:\symbols*http://msdl.microsoft.com/download/symbols
Symbol search path is: SRV*c:\symbols*http://msdl.microsoft.com/download/symbols
Executable search path is:
ModLoad: 00000000`00200000 00000000`00206000   test.exe

ModLoad: 00000000`775f0000 00000000`77798000   ntdll.dll
ModLoad: 00000000`777d0000 00000000`77950000   ntdll32.dll
ModLoad: 00000000`738c0000 00000000`738ff000   C:\Windows\SYSTEM32\wow64.dll
ModLoad: 00000000`73860000 00000000`738bc000   C:\Windows\SYSTEM32\wow64win.dll
ModLoad: 00000000`73850000 00000000`73858000   C:\Windows\SYSTEM32\wow64cpu.dll
ModLoad: 00000000`773d0000 00000000`774ef000   WOW64_IMAGE_SECTION
ModLoad: 00000000`75840000 00000000`75950000   WOW64_IMAGE_SECTION
ModLoad: 00000000`773d0000 00000000`774ef000   NOT_AN_IMAGE
ModLoad: 00000000`774f0000 00000000`775ea000   NOT_AN_IMAGE
ModLoad: 00000000`75840000 00000000`75950000   C:\Windows\syswow64\kernel32.dll
ModLoad: 00000000`77330000 00000000`77377000   C:\Windows\syswow64\KERNELBASE.dll
ModLoad: 00000000`74b70000 00000000`74c5e000   C:\Windows\SysWOW64\MSVCR120.dll

Note that there are 2 "ntdll" present in the loaded modules. "ntdll32.dll" is the 32-bit ntdll from c:\windows\syswow64\, and "ntdll.dll" is the 64-bit ntdll from c:\windows\system32.

Making a win32 api call, CreateFile, in WOW64 follows this path:

- API calls kernel32!CreateFileW

- kernel32!CreateFileW calls kernelbase.dll!CreateFileW:

0:000> bp kernel32!CreateFileW
0:000> g
Breakpoint 0 hit
kernel32!CreateFileW:
7585167f ff25d4098575    jmp     dword ptr [kernel32!_imp__CreateFileW (758509d4)] ds:002b:758509d4={KERNELBASE!CreateFileW (7734c299)}

- KERNELBASE!CreateFileW calls ntdll32.dll!ZwCreateFile:


KERNELBASE!CreateFileW+0x330:
7734c5c9 8b35ac103377    mov     esi,dword ptr [KERNELBASE!_imp__NtCreateFile (773310ac)] ds:002b:773310ac={ntdll32!ZwCreateFile (777f00b0)}

- ntdll32!ZwCreateFile sets the API number (0x52 for ZwCreateFile) into EAX, and insteading of executing the "SYSENTER" instruction, it calls wow64cpu.dll!X86SwitchTo64BitMode:

ntdll32!ZwCreateFile:
777f00b0 b852000000      mov     eax,52h
0:000:x86> t
ntdll32!ZwCreateFile+0x5:
777f00b5 33c9            xor     ecx,ecx
0:000:x86> t
ntdll32!ZwCreateFile+0x7:
777f00b7 8d542404        lea     edx,[esp+4]
0:000:x86> t
ntdll32!ZwCreateFile+0xb:
777f00bb 64ff15c0000000  call    dword ptr fs:[0C0h]  fs:0053:000000c0=00000000
0:000:x86> t
wow64cpu!X86SwitchTo64BitMode:
73852320 ea1e2785733300  jmp     0033:7385271E

wow64cpu!X86SwitchTo64BitMode makes a inter-segment jump into the the segment with selector 0x30 (the 3 at the end indicates that it is a ring 3 segment, a.k.a. user-land) Before this jmp, the code is running on segment 23. This segment is running in 32-bit mode (long mode set to 0):

0: kd> dt _KGDTENTRY64 0xfffff800`00b93000+20 /b
nt!_KGDTENTRY64
   +0x000 LimitLow         : 0xffff
   +0x002 BaseLow          : 0
   +0x004 Bytes            :
      +0x000 BaseMiddle       : 0 ''
      +0x001 Flags1           : 0xfb ''
      +0x002 Flags2           : 0xcf ''
      +0x003 BaseHigh         : 0 ''
   +0x004 Bits             :
      +0x000 BaseMiddle       : 0y00000000 (0)
      +0x000 Type             : 0y11011 (0x1b)
      +0x000 Dpl              : 0y11
      +0x000 Present          : 0y1
      +0x000 LimitHigh        : 0y1111
      +0x000 System           : 0y0
      +0x000 LongMode         : 0y0
      +0x000 DefaultBig       : 0y1
      +0x000 Granularity      : 0y1
      +0x000 BaseHigh         : 0y00000000 (0)
   +0x008 BaseUpper        : 0xffff
   +0x00c MustBeZero       : 0xcff300
   +0x000 Alignment        : 0x00cffb00`0000ffff

This jmp will jump to the segment 33, which is set to long-mode, a.k.a. 64 bit mode:

0: kd> dt _KGDTENTRY64 0xfffff800`00b93000+30 /b
nt!_KGDTENTRY64
   +0x000 LimitLow         : 0
   +0x002 BaseLow          : 0
   +0x004 Bytes            :
      +0x000 BaseMiddle       : 0 ''
      +0x001 Flags1           : 0xfb ''
      +0x002 Flags2           : 0x20 ' '
      +0x003 BaseHigh         : 0 ''
   +0x004 Bits             :
      +0x000 BaseMiddle       : 0y00000000 (0)
      +0x000 Type             : 0y11011 (0x1b)
      +0x000 Dpl              : 0y11
      +0x000 Present          : 0y1
      +0x000 LimitHigh        : 0y0000
      +0x000 System           : 0y0
      +0x000 LongMode         : 0y1
      +0x000 DefaultBig       : 0y0
      +0x000 Granularity      : 0y0
      +0x000 BaseHigh         : 0y00000000 (0)
   +0x008 BaseUpper        : 0
   +0x00c MustBeZero       : 0
   +0x000 Alignment        : 0x0020fb00`00000000

This inter-segment jump is the actual transition from 32-bit mode to 64-bit mode.

After this transition, the cpu lands at address 7385271E, which is wow64cpu!CpupReturnFromSimulatedCode. It calls wow64cpu!ServiceNoTurbo:

wow64cpu!CpupReturnFromSimulatedCode:
00000000`7385271e 67448b0424      mov     r8d,dword ptr [esp] ds:00000000`0014fc9c=777f00c2
00000000`73852723 458985bc000000  mov     dword ptr [r13+0BCh],r8d ds:00000000`001bfddc=777eface
00000000`73852723 458985bc000000  mov     dword ptr [r13+0BCh],r8d ds:00000000`001bfddc=777eface
00000000`7385272a 4189a5c8000000  mov     dword ptr [r13+0C8h],esp ds:00000000`001bfde8=0014f98c
00000000`73852731 498ba42480140000 mov     rsp,qword ptr [r12+1480h] ds:00000000`7efdc480=00000000001be840
00000000`73852739 4983a4248014000000 and   qword ptr [r12+1480h],0 ds:00000000`7efdc480=00000000001be840
00000000`73852742 448bda          mov     r11d,edx
wow64cpu!TurboDispatchJumpAddressStart:
00000000`73852745 41ff24cf        jmp     qword ptr [r15+rcx*8] ds:00000000`73852450={wow64cpu!ServiceNoTurbo (00000000`73852749)}

Which calls wow64.dll!Wow64SystemServiceEx:

wow64cpu!ServiceNoTurbo+0x27:
00000000`73852770 ff150ae9ffff    call    qword ptr [wow64cpu!_imp_Wow64SystemServiceEx (00000000`73851080)] ds:00000000`73851080={wow64!Wow64SystemServiceEx (00000000`738cd0b8)}

Which calls whNtCreateFile in wow64.dll:

wow64!Wow64SystemServiceEx+0xd4:
00000000`73bad18c 41ffd4          call    r12 {wow64!whNtCreateFile (00000000`73bbc0f0)}

Which calls NtCreateFile in the 64-bit ntdll.dll:

wow64!whNtCreateFile+0x109:
00000000`73bbc1f9 ff154958feff    call    qword ptr [wow64!_imp_NtCreateFile (00000000`73ba1a48)] ds:00000000`73ba1a48={ntdll!NtCreateFile (00000000`7791e120)}

Which finally calls the "SYSCALL" instruction to transition to the kernel.

ntdll!NtCreateFile:
00000000`7791e120 4c8bd1          mov     r10,rcx
00000000`7791e123 b852000000      mov     eax,52h
00000000`7791e128 0f05            syscall
00000000`7791e12a c3              ret