Wednesday, January 6, 2010

Heading to CES 2010 in Las Vegas – In-flight Thoughts

Heading to CES 2010 in Las Vegas – In-flight Thoughts

Heading to CES 2010 in Las Vegas – In-flight Thoughts

Posted: 06 Jan 2010 11:03 AM PST

As I write this post I'm sitting on a plane heading to Las Vegas. This is my second year in a row going to CES after taking a break for a number of years. Last year the mood heading into CES was grim, and the bad news just kept coming in daily. Now a...( read more )...(read more)

Help me decide what to focus on for thenext 6 months.

Posted: 06 Jan 2010 07:23 AM PST

Please take this poll ! Read More......(read more)

iMeta Agility & Silverlight

Posted: 06 Jan 2010 02:33 AM PST

Just a little plug for a video I made just before Xmas with the guys from iMeta , a UK company that's built a scrum tool called iMeta Agility with Silverlight. The picture's hyperlinked to their video on Channel 9 – click to take a look and, also, try...( read more )...(read more)

Silverlight HVP Proof Of Concept Released!

Posted: 05 Jan 2010 09:13 PM PST

  I'm incredibly proud to say that we were able to release the Silverlight HVP proof of concept alpha release on time, at 12:05 am Wednesday January 6, 2010 (GMT-5).  As noted in our release schedule this is the first of a series of releases...( read more )...(read more)

Street Fighter Style UI Compositing With Prism

Posted: 05 Jan 2010 07:29 PM PST

If you're into Silverlight or WPF, then you've undoubtedly heard the nonstop clamoring over Composite Application Guidance for WPF and Silverlight - aka Prism. I have a list of things in Evernote that I'd love to write about Prism, but I figured I would...( read more )...(read more)

Angled headers for DataGrid columns…not just angled text…

Posted: 05 Jan 2010 04:04 PM PST

My last post demonstrated one way that you could angle the text in a DataGrid column header. I received a comment asking for how to angle the entire headers, along the lines of what you can do very easily with Excel. Here's an example from Excel: My friend...( read more )...(read more)

Proposed WCF RIA Services Sessions for MIX10

Posted: 05 Jan 2010 07:26 PM PST

Earlier today Microsoft opened voting for the MIX10 Open Call entries. I am happy to see that at least eleven of the submissions mention WCF RIA Services. Even if you are not attending MIX10 it is still important that everyone posts as the sessions will Read More......(read more)

DIA based Stack Walking

Posted: 05 Jan 2010 12:48 PM PST

Hello, I am Manish Vasani, an SDET on the VC++ compiler backend team. In this series of posts, I will talk about call stacks and how to use the DIA SDK for implementing a stack walking client that builds a call stack. If you are interested in knowing how debuggers build the call stack or want to write an application that dumps the call stack for an executing process, you may want to continue reading further.

In this post I'll start with a brief introduction on stack walking, discuss the DIA APIs for implementing it, and then walkthrough a code sample for a DIA based stack walker. Those who are familiar with stack walking would know that apart from the DIA SDK, there are two other commonly used stack walking APIs: StackWalk64 (found in DbgHelp) and RtlVirtualUnwind (found in kernel32.lib). I'll touch upon these two APIs at the end of the post.

 

Introduction:

When the Operating system executes a process, it calls into the program's entry point (or the starting function) and allocates a set of resources to the process. Some of these resources such as stack memory and register context are allocated per thread basis. Every new function call in the thread is allocated a part of this stack memory and has a new register context. A stack frame or an activation record is essentially this runtime state information for an active function call. Typical attributes associated with a stack frame are:

1)      Function associated with the frame

2)      Module (binary) associated with the frame

3)      Return address

4)      Amount of stack memory allocated for arguments and locals for the function

5)      Base pointer (BP) for referencing the arguments passed to the function call

6)      Base pointer (BP) for referencing the local variables in the function (most of the time this is same as 5)

7)      Program counter or the Instruction Pointer for currently executing instruction (IP)

8)      Stack Pointer (SP)

9)      Other Register context

A sequence of active stack frames in an executing thread is generally referred to as a call stack. Call stacks help developers to debug runtime program errors and also denote the program state at a particular execution point. The process of building this call stack is called stack walking. Stack walking clients (such as debuggers) use the stack frame attributes to retrieve the values of function arguments, locals, etc.

 

DIA APIs for Stack walking:

The primary functionality of the DIA SDK is to provide access to the debug information in PDB files. Apart from that, the DIA SDK also provides a set of interfaces  that can be used to implement a platform independent stack walker. These are:

1)      IDiaStackWalker: This is the primary initialization interface for the DIA stack walker.

2)      IDiaEnumStackFrames: This is the stack frame enumerator interface that exposes the primary API to perform a stack walk to the next frame.

3)      IDiaStackFrame: This interface represents a stack frame and exposes attributes of the stack frame.

4)      IDiaStackWalkHelper: This is a helper interface for the DIA stack walker. To understand its primary purpose, let us briefly mention the two primary types of inputs required by the stack walker:

                              i.      Run-time entities: This includes the runtime aspects such as thread's register context, properties of loaded image files such as LOADED_IMAGE, executable binary (or module) corresponding to a given virtual address, etc.

                             ii.      Compile-time generated entities: This includes the PDB records that provide static attributes for frames such as memory allocated for locals/arguments, whether the function corresponding to the frame contains SEH/C++ EH, the unwind program to execute to walk to the next frame, etc.

DIA stack walker uses other DIA interfaces to read the compile time generated entities, i.e. PDB records. It delegates fetching the runtime entities to the DIA clients. The primary reason for this being that DIA doesn't maintain the runtime state information for the process. For example, a given virtual address in process' address space might belong to any one of the modules (executable binaries) loaded in the process. Each module would have a different pdb file and hence a different DIA session to read the pdb. DIA isn't aware about the loaded modules for the process and hence expects its clients to provide this mapping. Most DIA stack walker clients, such as debuggers, already need to fetch and store this runtime information. Hence, it is logical to expect the clients to provide this information to DIA through the implementation of this helper interface.

 

Code Sample Walkthrough:

I will first give the class design for the DIA based stack walker sample and then walk you through its execution flow.

Class design:

1)      MyDiaStackWalker: This is the primary DIA stack walker class. It maintains a handle to the initialization interface (IDiaStackWalker) and the stack frame enumerator (IDiaEnumStackFrames). It also contains few helper methods for other DIA operations.

2)      MyDiaStackWalkHelper: This class implements the IDiaStackWalkHelper interface. I will discuss each of its method in detail later on.

3)      Debugger: This is a Debugger class with a simple debugging engine. It launches the specified executable with debugging enabled and dumps the call stack at every debug break event in the executable. (Debug break events could be trigged by use of __debugbreak() intrinsic in your source code).

4)      ContextManager: This class manages the register context associated with the current stack frame of the current debuggee thread (debuggee is the process being debugged)

5)      ModuleManager: This class maintains the list of modules (executable binaries) loaded in the process along with their attributes.

Execution Flow:

1)      Initialization:

The first step in the application is to initialize the DIA stack walker. This is done through the IDiaStackWalker interface. IDiaStackWalker provides two platform specific APIs for initializing the DIA client: getEnumFrames (for x86) and getEnumFrames2 (for x64 and ia64 platforms). Each of these APIs take a pointer to the class that implements IDiaStackWalkHelper (MyDiaStackWalkHelper) and returns the stack frame enumerator (IDiaEnumStackFrames). You may refer to the method "MyDiaStackWalker::Initialize" in MyDiaStackWalker.cpp to see the initialization logic.

 

2)      Walking the stack:

After the initialization, we launch the specified executable under the debugger. On hitting a debug break event, we initialize the register context and loaded modules for the debuggee. Than we call "MyDiaStackWalker::WalkCallStack" method to walk the call stack and retrieve attributes such as module name, function name, Instruction pointer (IP), etc. for each stack frame:

// method to walk all the stack frames in the call stack

void MyDiaStackWalker::WalkCallStack()

{ 

  wprintf(L"\nWalking call stack...\n");

  IDiaSymbol *pFuncSym = NULL;   

  DWORD count = 0;

  BSTR szFunctionName;

  ULONGLONG ip = 0;

 

  do {

    // Get IP for current stack frame

    pContextManager->get_currentIP(&ip);

    if (!ip) {

      break;

    }

    // Get module for stack frame

    Module *pModule = pModuleManager->FindModuleByVA(ip);

    // Get DIA function symbol for stack frame

    pMyDiaStackWalkHelper->symbolForVA(ip, &pFuncSym);

    // Display stack frame attributes

    if (pFuncSym == NULL) {

      // function symbol could be null if symbols for the module were not loaded.

      // just dump the IP for the stack frame

      wprintf(L"Stack Frame %d: '%ws!%#x()'\n", ++count, pModule->name, ip);

      continue;

    }

    if (pFuncSym->get_name(&szFunctionName) == S_OK) {

      wprintf(L"Stack Frame %d: '%ws!%ws'\n", ++count, pModule->name, szFunctionName);

      SysFreeString(szFunctionName);

    }

    pFuncSym->Release();

    pFuncSym = NULL;

    // Walk to next stack frame

  } while (WalkStackFrame() == S_OK);

}

 

"WalkCallStack" method calls into "WalkStackFrame" to walk each stack frame. WalkStackFrame uses the stack frame enumerator API (IDiaEnumStackFrames::Next) to do the stack walk to next frame. Following snippet shows this:

// MyDiaStackWalker.cpp

// method to walk to the next stack frame

HRESULT MyDiaStackWalker::WalkStackFrame(IDiaStackFrame **ppStackFrame)

{

  IFVERBOSE {

    wprintf(L"MyDiaStackWalker::WalkStackFrame\n");

  }

 

  ULONG celtFetched = 0; 

  IDiaStackFrame *pStackFrame = NULL;

 

  // walk to the next stack frame

  HRESULT hr = pEnumFrames->Next(1, &pStackFrame, &celtFetched);

 

  if (celtFetched == 1 && hr == S_OK) {

    if (ppStackFrame != NULL) {

      *ppStackFrame = pStackFrame;

    }

    else {

      pStackFrame->Release();

    }

  }

  return hr;

}

 

3)      DIA callbacks into MyDiaStackWalkHelper during stack walk:

When we walk the stack frame using the IDiaEnumStackFrames::Next, DIA makes a few callbacks into MyDiaStackWalkHelper. Let me discuss each of the methods of this class to explain its purpose:

1.        Method to read process memory from debuggee's address space (MyDiaStackWalkHelper::readMemory): DIA can't read into debuggee's address space, hence it expects the client to do implement this API. I call into the ReadProcessMemory Windows API for its implementation.

 

2.       Methods to get/set debuggee register context (MyDiaStackWalkHelper::get_registerValue and put_registerValue):

                                                      i.      get_registerValue: This callback API is used by DIA to fetch the register values for the current stack frame. Prior to the WalkCallStack invocation, I use the GetThreadContext Windows API, for fetching the current thread's register context (see "ContextManager::InitializeRegisterContext" in ContextManager.cpp). Register IDs used by DIA are defined in cvconst.h that is available in the DIA SDK.

 

                                                    ii.      put_registerValue: This callback API is used by DIA to update the register values. DIA makes a few callbacks into put_registerValue at the end of every stack walk invocation (i.e. IDiaEnumStackFrames::Next) to update the register context to the next stack frame. Note that the IDiaStackFrame object returned by IDiaEnumStackFrames::Next corresponds to the stack frame that we walked past (i.e. it corresponds to the top stack frame on the first invocation of next, while the callbacks into IDiaStackWalkHelper::put_registerValue correspond to the second stack frame)

Also, note that a call to IDiaEnumStackFrames::Reset to reset the stackwalker resets only the internal DIA stack frame enumerator. Clients should reinitialize the register context (using GetThreadContext) between a call to IDiaEnumStackFrames::Reset and subsequent IDiaEnumStackFrames::Next.

 

3.       Methods to map a virtual address (VA) to some entity:

                                                         i.            Methods to map a given VA to unwind data (MyDiaStackWalkHelper::frameForVA and  MyDiaStackWalkHelper::pDataForVA): Before getting into the specifics of these APIs, let me provide a brief description about the format of unwind data for different platforms.

x86 and x64 platforms have different ABI conventions which lead to the stack unwind information lying in different formats in different files for these architectures. x64 ABI requires the functions to setup their prolog and epilog in a standard way, so that a stack unwind can be performed from any given point in the function by just looking at function's unwind data (pdata/xdata) from the image file. You may refer to articles about Exception Handling (x64) and x64 Unwind information for more details.

x86 platform doesn't have this requirement of setting up the prolog/epilog in a standard manner. Hence, in order to perform a successful stack unwind from any given IP, the compiler tracks and emits additional unwind information in the PDB. This information is recorded inside the Frame Data PDB records. DIA exposes these records through the IDiaFrameData interface. We will discuss the details about IDiaFrameData and how to use it along with IDiaStackWalkFrame to walk the Frame Data records in a follow up post.

As you might have guessed by now, pDataForVA method is specific to x64 and ia64 platforms. This method parses the pdata section in the image file to find the matching pdata entry for the given VA. Structure of the mapped pdata entry that is expected by DIA is quite identical to the _IMAGE_RUNTIME_FUNCTION_ENTRY structure, except that addresses in _IMAGE_RUNTIME_FUNCTION_ENTRY structure are relative to the base virtual address of the image file, while the addresses in our custom defined IMAGE_RUNTIME_FUNCTION_ENTRY_VA structure are absolute virtual addresses (i.e. with base address relocations applied):

// ModuleManager.h

struct IMAGE_RUNTIME_FUNCTION_ENTRY_VA

{

                ULONGLONG BeginAddress;

                ULONGLONG EndAddress;

                ULONGLONG UnwindInfoAddress; 

};

 

frameForVA is an x86 specific method to fetch the Frame Data record for a given virtual address. MyDiaStackWalkHelper::frameForVA does the following mapping: given VA -> module -> pdb -> DIA session for the pdb -> frameByVA.

 

                                                       ii.            Method to map a given VA to the base virtual address of the image (exe or dll) file (MyDiaStackWalkHelper::imageForVA): I use the GetModuleInformation API to obtain the MODULEINFO structure which contains the base address for the module (see "ModuleManager::InitializeModules" in ModuleManager.cpp)

 

                                                      iii.            Method to map a given VA to DIA symbol ("MyDiaStackWalkHelper::symbolForVA"): I first map the VA to an appropriate DIA session and use IDiaEnumSymbolsByAddr::symbolByVA API to fetch the matching DIA symbol.

 

4.       Methods to search for return address on the stack (MyDiaStackWalkHelper::searchForReturnAddress and MyDiaStackWalkHelper::searchForReturnAddressStart): DIA will make these callbacks when walking an x86 FPO stack frame. I return E_NOTIMPL as I do not wish to override the default FPO stack search done by DIA.

Code Sample:

You can download the stack walking sample from the Visual C++ Sample page. Usage of the sample:

Ø  DIA_StackWalk_Sample.exe DIA_StackWalk_Testcase.exe

You may compile the sample with /DVERBOSE to get more descriptive logging. A sample test is also provided.

 

Alternative Stack walking APIs:

Let me talk about the two alternative stack walk APIs that I mentioned at the beginning of the post:

StackWalk64 (DbgHelp API): Similar to the DIA stack walk APIs, StackWalk64 provides a platform independent mechanism for stack walking. If you closely go through all the parameters for this API, you can map them to the corresponding IDiaStackWalkHelper methods. Given that StackWalk64 is identical to DIA stack walk APIs, how do we decide which one to pick over other? I find the following guidelines useful:

1)      If you are already using DbgHelp or are willing to add a dependency to it, than StackWalk64 should be the ideal choice for stack walking. It operates at an abstraction layer above DIA and hence provides a more high level and easy to implement interface for the client.

2)      If you are implementing your own debugger and only want to add the stack walk functionality, than using the DIA stack walker would be a good choice.

3)      If you want to tweak the low level functionality, such as override how default return address search is performed for FPO stack frames, than DIA stack walker would be a good choice.

RtlVirtualUnwind is also a similar stack walk API with identical parameters; except that unlike the above APIs:

1)      It has to be invoked in the same process and

2)      Is designed only for x64 platforms.

 

Conclusion:

Hopefully this post provided you a good overview of how to implement a stack walker client using the DIA SDK.  As always, we welcome your feedback on DIA, and look for part 2 of this series on walking the x86 Frame data records using IDiaFrameData/IDiaStackWalkFrame interfaces.

Please Vote For My Sessions at Mix :)

Posted: 05 Jan 2010 05:13 PM PST

It's shameless, I know. But, please vote for my sessions at Mix :) The voting tool can be accessed here: http://visitmix.com/opencallvote/ Below are titles, descriptions and links to where you can vote for each of my submissions. Votes must be submitted...( read more )...(read more)

Vote for My Sessions at MIX '10

Posted: 05 Jan 2010 11:59 AM PST

URL : http://visitmix.com/opencallvote/?query=Shawn%2... If you would like to hear me blather on at MIX about how to create Silverlight applications, feel free to go visit the MIX site and vote for my sessions. I submitted seven talks so feel free to...( read more )...(read more)

MIX 2010: Voting for sessions has begun (I got two)

Posted: 05 Jan 2010 04:12 PM PST

This year Microsoft decided to have an open call for sessions for the MIX 2010 in Las Vegas. This conference, in case you don't know it yet, is a great 3 days about modern client technologies, such as ASP.NET, Windows Presentation Foundation and of course...( read more )...(read more)

RIA Services and relational data

Posted: 05 Jan 2010 04:02 PM PST

As we've all been guilty, when you see demonstrations of technologies most of the time the data samples show single table solutions. When was the last time you've developed a single-table system? :-) Thought so. In RIA Services demonstrations, most of...( read more )...(read more)

RIA Services and relational data

Posted: 05 Jan 2010 04:02 PM PST

As we've all been guilty, when you see demonstrations of technologies most of the time the data samples show single table solutions. When was the last time you've developed a single-table system? :-) Thought so. In RIA Services demonstrations, most of Read More......(read more)

My Mix10 Sessions – I hope

Posted: 05 Jan 2010 03:15 PM PST

Once again it's almost time for Microsoft's premier conference for anyone involved in the web: Mix. Building upon the success of last year, I am hoping to present again this year. This time though, things are a bit different. I can't just schmooze a bunch...( read more )...(read more)

Silverlight 4 Find: No More Whitelisting Fonts

Posted: 05 Jan 2010 09:36 AM PST

I just found out that in Silverlight 4, the fonts are no longer whitelisted to the ten fonts out of the box. Silverlight will now use any font that it can find on a machine without embedding. For example, in my RIA Services sample, I changed the font...( read more )...(read more)
Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)