Reversing Engenering

This material has only educational purpose!
Target: OllyDbg
Difficulty: n/a
Tools: Api Reference, OllyDbg, mASM and Brain.

Introduction
The year is 2004.The ring-3 debuggers are used often and often.Since they offer Windows GUI they are more handy instead of the ring-0 debuggers (like SoftIce).In this essay i will talk (write) about the detection of one of the best ring-3 debuggers - OllyDbg.Many have heard of the IsDebbugerPresent and of the fs:[20] detecting tricks, but what about some other new ones? Here I will present you some of my own detecting tricks.I will give you the general explanation so you would be able to use your fantasy to improve it yourself.

Method I: FindWindow

This method is based on the FindWindows function.As all dialogs the OllyDbg’s main dialog (window ?) has its caption and Class name.Using this Api we will try to find if the OllyDbg’s main window is opened.Microsoft wrote:

The FindWindow function retrieves the handle of the top-level window whose class name and window name match the specified strings. This function does not search child windows

HWND FindWindow(

LPCTSTR lpClassName, // address of class name
LPCTSTR lpWindowName // address of window name
);

Parameters
lpClassName

Points to a null-terminated string that specifies the class name or is an atom that identifies the class-name string. If this parameter is an atom, it must be a global atom created by a previous call to the GlobalAddAtom function. The atom, a 16-bit value, must be placed in the low-order word of lpClassName; the high-order word must be zero.
lpWindowName

Points to a null-terminated string that specifies the window name (the window’s title). If this parameter is NULL, all window names match.

Return Value
If the function succeeds, the return value is the handle of the window that has the specified class name and window name.
If the function fails, the return value is NULL. To get extended error information, call GetLastError
And my code snippet is:

.data
strOllyClsName db "OLLYDBG",0

.code
invoke FindWindow, ADDR strOllyClsName, NULL
cmp eax, 00000000h
jne Olly_Detected

Method II: CreateToolhelp32Snapshot, Process32First/Next

This is an interesting method.It is based on 4 apis (CreateToolhelp32Snapshot, Process32First, Process32Next, GetCurrentProcessId) and 1 structure ( PROCESSENTRY32 ).I will tell you what MSDN says first:

CreateToolhelp32Snapshot

The CreateToolhelp32Snapshot function takes a snapshot of the specified processes in the system, as well as the heaps, modules, and threads used by these processes.

HANDLE WINAPI CreateToolhelp32Snapshot(
DWORD dwFlags,
DWORD th32ProcessID
);

Parameters

dwFlags
[in] Portions of the system to include in the snapshot. This parameter can be one of the following values.

TH32CS_INHERIT - Indicates that the snapshot handle is to be inheritable.
TH32CS_SNAPALL - Includes all processes and threads in the system, plus the heaps and modules of the process specified in th32ProcessID.
TH32CS_SNAPHEAPLIST - Includes all heaps of the process specified in th32ProcessID in the snapshot. To enumerate the heaps, see Heap32ListFirst.
TH32CS_SNAPMODULE - Includes all modules of the process specified in th32ProcessID in the snapshot. To enumerate the modules, see Module32First.
TH32CS_SNAPPROCESS - Includes all processes in the system in the snapshot. To enumerate the processes, see Process32First.
TH32CS_SNAPTHREAD - Includes all threads in the system in the snapshot. To enumerate the threads, see Thread32First.

th32ProcessID

[in] Process identifier of the process to be included in the snapshot. This parameter can be zero to indicate the current process. This parameter is used when the TH32CS_SNAPHEAPLIST, TH32CS_SNAPMODULE, or TH32CS_SNAPALL value is specified. Otherwise, it is ignored and all processes are included in the snapshot.

Return Values
If the function succeeds, it returns an open handle to the specified snapshot.
If the function fails, it returns INVALID_HANDLE_VALUE. To get extended error information, call GetLastError.

Remarks
The snapshot taken by this function is examined by the other tool help functions to provide their results. Access to the snapshot is read only. The snapshot handle acts like an object handle and is subject to the same rules regarding which processes and threads it is valid in.

To enumerate the heap or module states for all processes, specify TH32CS_SNAPALL and set th32ProcessID to zero. Then, for each additional process in the snapshot, call CreateToolhelp32Snapshot again, specifying its process identifier and the TH32CS_SNAPHEAPLIST or TH32_SNAPMODULE value.

To destroy the snapshot, use the CloseHandle function.

Process32First
Process32First retrieves information about the first process encountered in a system snapshot.

BOOL WINAPI Process32First(
HANDLE hSnapshot,
LPPROCESSENTRY32 lppe
);

Parameters

hSnapshot
[in] Handle to the snapshot returned from a previous call to the CreateToolhelp32Snapshot function.

lppe
[in, out] Pointer to a PROCESSENTRY32 structure.

Return Values

Returns TRUE if the first entry of the process list has been copied to the buffer or FALSE otherwise. The ERROR_NO_MORE_FILES error value is returned by the GetLastError function if no processes exist or the snapshot does not contain process information.

Remarks

The calling application must set the dwSize member of PROCESSENTRY32 to the size, in bytes, of the structure. Process32First changes dwSize to the number of bytes written to the structure. This will never be greater than the initial value of dwSize, but it may be smaller. If the value is smaller, do not rely on the values of any members whose offsets are greater than this value.

To retrieve information about other processes recorded in the same snapshot, use the Process32Next function.

Process32Next

Process32Next Retrieves information about the next process recorded in a system snapshot.

BOOL WINAPI Process32Next(
HANDLE hSnapshot,
LPPROCESSENTRY32 lppe
);

Parameters

hSnapshot
[in] Handle to the snapshot returned from a previous call to the CreateToolhelp32Snapshot function.

lppe
[out] Pointer to a PROCESSENTRY32 structure.

Return Values

Returns TRUE if the next entry of the process list has been copied to the buffer or FALSE otherwise. The ERROR_NO_MORE_FILES error value is returned by the GetLastError function if no processes exist or the snapshot does not contain process information.

Remarks
To retrieve information about the first process recorded in a snapshot, use the Process32First function.

PROCESSENTRY32

PROCESSENTRY32 describes an entry from a list that enumerates the processes residing in the system address space when a snapshot was taken.

typedef struct tagPROCESSENTRY32 {
DWORD dwSize;
DWORD cntUsage;
DWORD th32ProcessID;
ULONG_PTR th32DefaultHeapID;
DWORD th32ModuleID;
DWORD cntThreads;
DWORD th32ParentProcessID;
LONG pcPriClassBase;
DWORD dwFlags;
TCHAR szExeFile[MAX_PATH]; 

} PROCESSENTRY32, *PPROCESSENTRY32;

Members

dwSize
Size of the structure, in bytes. Before calling the Process32First function, set this member to sizeof (PROCESSENTRY32). If you do not initialize dwSize, Process32First will fail.

cntUsage
Number of references to the process. A process exists as long as its usage count is nonzero. As soon as its usage count becomes zero, a process terminates.

th32ProcessID
Identifier of the process.

th32DefaultHeapID
Identifier of the default heap for the process. The contents of this member has meaning only to the tool help functions. It is not a handle, nor is it usable by functions other than the ToolHelp functions.

th32ModuleID
Module identifier of the process. The contents of this member has meaning only to the tool help functions. It is not a handle, nor is it usable by functions other than the ToolHelp functions.

cntThreads
Number of execution threads started by the process.

th32ParentProcessID
Identifier of the process that created the process being examined.

pcPriClassBase
Base priority of any threads created by this process
.
dwFlags
Reserved; do not use.

szExeFile
Pointer to a null-terminated string that specifies the name of the executable file for the process.

Windows Me/98/95: The file name includes the path.

GetCurrentProcessId

The GetCurrentProcessId function returns the process identifier of the calling process.

DWORD GetCurrentProcessId(VOID)

Parameters

This function has no parameters.

Return Value

The return value is the process identifier of the calling process.

Remarks

Until the process terminates, the process identifier uniquely identifies the process throughout the system.

Here our target is the parent procces of our program.We check if the parent process is OllyDbg.

The plan:

1.) Get the ProcessId of our program using GetCurrentProcessId .

2.) Begin a loop of CreateToolhelp32Snapshot and Process32First/Next and compare every th32ProcessID member of the PROCESSENTRY32 structure with our ProcessId from GetCurrentProcessId till we find it.

3.) Get the value of the th32ParentProcessID member of the PROCESSENTRY32.

4.) Begin a new CreateToolhelp32Snapshot and Process32First/Next loop, but this time compare the th32ProcessID member with the th32ParentProcessID, which we got in 3.) till we find it.

5.) Go to the szExeFile member of the PROCESSENTRY32 and scan the name string for “Ollydbg.exe”

6.) If we find it we will know that our program is running under OllyDbg.

And here is my example:

Method III: SetUnhandledExceptionFilter
This is a very interesting method of detecting OllyDbg.It is based on Microsoft’s SetUnhandledExceptionFilter function:

The SetUnhandledExceptionFilter function lets an application supersede the top-level exception handler that Win32 places at the top of each thread and process.

After calling this function, if an exception occurs in a process that is not being debugged, and the exception makes it to the Win32 unhandled exception filter, that filter will call the exception filter function specified by the lpTopLevelExceptionFilter parameter.

LPTOP_LEVEL_EXCEPTION_FILTER SetUnhandledExceptionFilter(
LPTOP_LEVEL_EXCEPTION_FILTER lpTopLevelExceptionFilter // exception filter function
);

Parameters

lpTopLevelExceptionFilter

Supplies the address of a top-level exception filter function that will be called whenever the UnhandledExceptionFilter function gets control, and the process is not being debugged. A value of NULL for this parameter specifies default handling within UnhandledExceptionFilter.
The filter function has syntax congruent to that of UnhandledExceptionFilter: It takes a single parameter of type LPEXCEPTION_POINTERS, and returns a value of type LONG. The filter function should return one of the following values:

Value Meaning

EXCEPTION_EXECUTE_HANDLER

Return from UnhandledExceptionFilter and execute the associated exception handler. This usually results in process termination.
EXCEPTION_CONTINUE_EXECUTION

Return from UnhandledExceptionFilter and continue execution from the point of the exception. Note that the filter function is free to modify the continuation state by modifying the exception information supplied through its LPEXCEPTION_POINTERS parameter.
EXCEPTION_CONTINUE_SEARCH

Proceed with normal execution of UnhandledExceptionFilter . That means obeying the SetErrorMod flags, or invoking the Application Error pop-up message box.

Return Value

The SetUnhandledExceptionFilter function returns the address of the previous exception filter established with the function. A NULL return value means that there is no current top-level exception handler.

Remarks

Issuing SetUnhandledExceptionFilter replaces the existing top-level exception filter for all existing and all future threads in the calling process.
The exception handler specified by lpTopLevelExceptionFilter is executed in the context of the thread that caused the fault. This can affect the exception handler’s ability to recover from certain exceptions, such as an invalid stack.

So.The main idea is that we set an exception filter with this api function.After that we cause a small and shy exception.If you have read the blue paragraph you will understand that if the program is debugged the exception will not be passed to the exception handler.We will use exactly this! The debugger will not know what to do and the program will crash! If the program is running standalone it will work normally.

What do i understand under normally? It will enter the Exception Handler, in there it will change the EIP value from the context structure with the address of the next safe command, which will be used to continue the program from there.Before I give you my example I will need to explain you the basic of Exception Handling.When the program enters the Exception Handler the stack looks like this:

[ Return EiP ]

[ Pointer to EXCEPTION_POINTERS structure ]

[ ... ]

[ ... ]

And the EXCEPTION_POINTERS structure is defined like this:

EXCEPTION_POINTERS STRUCT
pExceptionRecord DWORD ?
ContextRecord DWORD ?
EXCEPTION_POINTERS ENDS

What we need from it is the ContextRecord pointer.Which points to the CONTEXT structure:

CONTEXT STRUCT
ContextFlags DWORD ?
iDr0 DWORD ?
iDr1 DWORD ?
iDr2 DWORD ?
iDr3 DWORD ?
iDr6 DWORD ?
iDr7 DWORD ?
FloatSave FLOATING_SAVE_AREA <>
regGs DWORD ?
regFs DWORD ?
regEs DWORD ?
regDs DWORD ?
regEdi DWORD ?
regEsi DWORD ?
regEbx DWORD ?
regEdx DWORD ?
regEcx DWORD ?
regEax DWORD ?
regEbp DWORD ?
regEip DWORD ?
regCs DWORD ?
regFlag DWORD ?
regEsp DWORD ?
regSs DWORD ?
ExtendedRegisters db MAXIMUM_SUPPORTED_EXTENSION dup(?)
CONTEXT ENDS

At the end of the handler EAX must contain the value which tells Windows what to do next.For us this will be:

EXCEPTION_CONTINUE_EXECUTION equ -1

So here is the plan of the protection:

1.) Install the Exception Handler using the SetUnhandledExceptionFilter Api

2.) Cause exception and go to the Exception Handler only if there is not a debugger.

3.) In the handler - get from stack the pointer to the EXCEPTION_POINTERS structure

4.) Get from the EXCEPTION_POINTERS structure the pointer to the CONTEXT structure

5.) Change the regEip member of the CONTEXT to the safe address, from where the program will continue

6.) Set the EXCEPTION_CONTINUE_EXECUTION flag in EAX

7.) Normalize stack and continue to safe address

And here is the example:

Method IV: APi Redirection

One of my own tricks.It is based on the way OllyDbg handles called api functions.Oleh Yuschuk (the author of OllyDbg) has decided to use total api redirection in his debugger, when the debugged program is calling an api function.Here is an example:

00401000 >PUSH ASD.00403033 ; /FileName = “kernel32.dll”
00401005 CALL ; \LoadLibraryA
|
|
‘–> 0040105C JMP DWORD PTR DS:[402004] ; JMP DWORD PTR DS:[<&KERNEL32.LoadLibraryA>]
|
|
‘–> 87FF4120 PUSH BFF776D0 ; PUSH KERNEL32.LoadLibraryA
87FF4125 JMP KERNEL32.BFF957CA

So it seems that he manually loads the Import Table and manually fills the IAT, where all api addresses are redirected to an allocated buffer.In this buffer he calls the api functions in a strange way.But there is one more thing, more important for us.He also emulates (?) the GetProcAddress function so it returnes redirected addresses.For example the real address of IsDebuggerPresent in memory is BFF946F6, but the function returns 87FF4110, which is the redirected address.How are we going to use this as OllyDbg detection trick? Well in many ways.I will give you the shortest.

The plan:

1.) Load the kernel32.dll Library, which will return its Image Base (the location where it is loaded in memory)

2.) Call GetProcAddress for function ExitProcess

3.) Compare the return value (address of api function) from GetProcAddress with the Image Base of kernel32.dll

4.) If it is greater than the Image Base we will be sure that the api function is called directly from kernel32.dll.Else we will be sure that there is api redirection.

The example:

Final Words

That’s all folks!

To follow the path:
look to the master,
follow the master,
walk with the master,
see through the master,
become the master.

*** End ***