August 2008

Christopher Tacke

OpenNETCF Consulting, LLC

Download the Source
  Download the PDF

Introduction

Windows CE, and especially Windows Mobile devices have a UI paradigm that’s quite a bit different than the desktop. As a general rule, only one application, and indeed only one Form is typically ever in front of all other applications because there usually isn’t enough real estate to show multiple forms and MDI layout isn’t supported in CE.

One of the things that this paradigm pushes us toward is knowing either when the visible Form changes or when the user changes applications completely. We can use this for tracking state or even implementing an application authentication scheme.

Unfortunately the Windows CE operating system simply doesn’t provide us any kind of event to tell us when a Form has changed or when the process has changed. Sure, within our application we can watch things like the Deactivate event on a Form, but when our Form gets deactivated, how do we determine what’s on top now? Is it another Form in our application, or another application altogether?

In this article we’ll take a look at a simple utility class called WindowWatcher than can provide us feedback for these state changes through a very simple managed event interface.

Window Handles and Process IDs

The key to our solution is the fact that all Forms are Window and all Windows have a unique identifier called a Handle, or hWnd, and that all processed have a unique identifier called a process identifier, or PID. There are Win32 APIs to retrieve these identifiers, and while there are no corresponding managed-code methods that call them, a little P/Invoke gets us there easily.

The Win32 APIs we’re going to be interested in are:

With just these three APIs we can determine when the foreground Window has changed because the return value from GetForegroundWindow will change, and we can determine if the new foreground Window is in a new process because the PID retrieved from GetWindowThreadProcessId will change.

A Simple Solution

So armed with these APIs, and a general idea of how we can determine the state changes we want, we can create a simple class that encapsulates our logic.

First we need to define a couple delegates for our events:

An event of this type will be raised any time the foreground Window changes, and if the owner of the new Window is a new process, processChanged will be true. In either case inMyProcess will tell us if the new foreground Window is in the process that created our WindowWatcher class.

Now we implement the logic. The core of the logic will be a background thread that periodically checks the current foreground window. To be nice, we’ll make the interval adjustable.

Next we define some variables that we’ll maintain at class scope to keep track of state between checks and initialize them in the class constructor plus define the event of our delegate type:

You’ll notice here that to get the current process ID I’m not just calling a P/Invoke, but instead calling into the Smart Device Framework (SDF). The reason for this is that GetCurrentProcessID isn’t a true API, but is instead a macro that’s defined in the CE headers, and the value it returns depends on the processor architecture of the device. Rather than lay out all of the code needed to determine this it’s a lot easier to just call a function in the SDF. Keeps the code smaller and less confusing.

While we’re thinking about P/Invokes, we might as well define the other two we do need:

To start and stop our watcher, we’ll expose a simple Enabled property. It’s job will be to start or stop our background worker thread.

The only thing remaining is to implement the actual logic engine of this class that does all of the work, which is all in a single method running as the background thread:

You can see that it runs on the period determined by the Interval property. Ever Interval, it checks the foreground Window. If it has changed, it then determines if the new foreground Window is in the same process as the last. Next it determines if the process that owns the foreground Window is the same process that created the watcher class itself and finally it fires off an event of there are any subscribers.

That’s all there is to it. Using the new class is very, very simple. It looks like this:

Conclusion

As Compact Framework developers, and really even as general mobile and embedded device developers, we have to get used to the fact that Microsoft can’t reasonably think of every possible use-case or business challenge we might face. What’s important is that they’ve generally provided us the tools to implement solutions to just about any problem we run into.

In this case we needed to know information about not just application state, but overall device state, and in less than 100 lines of code we came up with a reasonable solution that didn’t require any onerous coding or framework that we’d have to inject into all of our Forms. As with many problems like this, understanding the Win32 API and thinking outside the managed-code box can usually get you a fairly simple solution.

Mark Arteaga

OpenNETCF Consulting

June 2008

May 2008

Chris Tacke

OpenNETCF Consulting, LLC

Download the Source
  Download the PDF

Introduction

I recently saw a post in the public Compact Framework newsgroup[1] that is fairly typical. The poster stated that if you want speed and performance for graphics, you should use native (meaning C or C++) code. It’s a general sentiment that I often read or hear – native code is faster than managed code for XYZ operation, so don’t use managed code. In most cases these claims are being made by developers with little experience in managed development and they rarely have any sort of empirical data to back up their statement.

I knew that I already had some code for measuring GDI performance in native code that I wrote several years ago for testing and tuning a Windows CE display driver, so I decided I’d dig it up and then create a managed application that did the same operations and then compare the results of the two with real evidence and put to rest this silly myth once again.

Native Code Device Test

Before diving into managed code to see how it would perform, I needed to get the baseline test running in native code. The code I had was a pretty simple and straightforward test. It creates an off-screen square bitmap and device context (DC), fills it with a varying shade of grey, and then draws two intersecting green ellipses centered in that square (see Figure 1). The test then repeats the fill and ellipse drawing 10,000 times, each time changing the fill color and the ellipse boundaries to prevent the driver from being able to cache anything and to provide a nice psychedelic pattern to watch as the test runs. The general code loop can be seen in Listing 1 (the full code accompanies this article).

In the tests I ran, the loop executed for 10,000 iterations. On the Windows Mobile 5.0 emulator that ships with Visual Studio 2008 running on my desktop PC this test executes at about 490 iterations per second (one iteration being a fill, two ellipse draws and a blit to the screen). On a Windows Mobile 5.0 Dell Axim x51 (PXA270 processor) it achieves about 250 iterations per second.

Compact Framework Device Test

The next step was to “port” the native code to managed code. The code is straightforward and most of the concepts in the native application have direct parallels in managed code. Listing 2 shows the essence of the managed version of the loop code (again, the full source accompanies this article).

When I ran this test against Compact Framework 3.5 is was rather surprised to see that on the Windows Mobile 5.0 emulator it only achieved 425 iterations per second (15% slower than the native code) and on the Axim it only achieved 210 iterations per second (19% slower than native code).

Testing on the Desktop

To say the least, the results from the device tests surprised me. In my gut I had expected the managed code performance to be within 5% of the native code – 10% at the outside – but to see 15-20% was amazing. I then wondered if it was maybe something specific to the Compact Framework and that maybe we’d not see such a large disparity on the desktop. If I were asked to guess at the desktop results before seeing the device results I would have almost certainly said that the managed code results would be within 5% of the native, but after seeing the results from the device I had to actually run it and see.

I modified both the device and desktop code slightly to get single code bases for both languages that would compile for either target and then re-ran the tests. The managed code test achieved 825 iterations per second on the same PC that the emulator was running on – so roughly twice as fast. The native code, however, surprised me again and this time it was more “shock and awe” than mere surprise. The native desktop version of the test achieved 6200 iterations per second. Yes, 625% faster than the managed code on the same target.

Updating the Managed code

The managed code performed substantially worse than I expected (especially on the desktop). The question that still remains is “why?” What would account for this performance degradation? I would suspect that something like a call to Graphics.FillRectangle would just be a thin wrapper around a P/Invoke to the Win32 FillRect call so does that mean the performance problems we see are purely an artifact of the performance penalty of crossing the P/Invoke boundary[2]? If so we should be able to remove the calls to the framework Drawing classes, directly P/Invoke the GDI functions and get the same, or at least similar results.

I went back and reworked the managed code to be able to run using only framework-provided drawing operations within the test loop or to P/Invoke all of them. I then re-ran the test on the Windows Mobile 5.0 Emulator and the Axim and got 455 and 225 iterations per second respectively. That’s a 7% speed improvement over using purely the framework-provided GDI calls, and generally cut the difference between the native and managed tests in half.

This means that the penalty for crossing the P/Invoke boundary is definitely part of the performance difference between native and managed code in our tests, but it only accounts for about half of it. I can’t readily account for the other half, but if I were to make an educated guess it’s likely because the framework is doing parameter and bounds checking on all calls, which my P/Invokes aren’t doing, and it may be doing something different for data marshaling to the native side as well.

To be complete I then updated the code again so it would compile and run on the desktop as well as the device (the P/Invoke declarations are different between the two platforms). What really surprised me here was that using the P/Invokes on the desktop resulted in the managed test achieving 6150 iterations per second, or within 1% of the native test.

Obviously there’s something I don’t fully understand about GDI under the full framework, but the updated tests show that managed code certainly can perform just as fast as native code. Since we at OpenNETCF tend to focus on Windows CE and Windows Mobile, I decided to leave investigating the huge disparity on the desktop to a later time.

Conclusion

So I set out to gather some hard data and debunk a myth about managed code being slow but I ended up confirming that managed code is indeed slower, sometime a lot slower, than native code - at least for basic GDI operations. Below is a summary table of my results as well as results from another tech reviewer that ran tests on his devices for me (thanks to Peter Nowak).

* = Results from testing 1,000 iterations instead of 10,000

We can see that across the board managed code is always slower, but that with a little extra effort and code we can squeeze a bit more performance out of the managed code. If you really need that extra speed it’s probably worth the effort. Making the changes on the desktop yields an enormous improvement, but again there could be something fundamentally wrong with the desktop test, so do some more investigation before just running wild with the numbers I came up with.

So does this mean that native code is “superior” to managed code and that you shouldn’t still consider using managed code for your development projects? I’m still going to adamantly answer “no.” Managed code is not the answer to all development problems, but neither is native code. Each has their place, their strengths and their weaknesses.

Consider the rapid development nature of managed code and the nice variety of test and continuous integration tools available for managed code developers. It’s very difficult to make a business case that something like a typical enterprise data collection and reporting application should be developed using anything but managed code. It will get you to market faster, with lower cost, more features and less support headache. What’s not to like?

Similarly you’d be hard pressed to make a reasonable case that something like an Ethernet driver should be written in anything but native code. The more difficult decisions are in the grey area between. In these cases you have to weigh the application requirements against your schedule, budget and available resources. Sure, the simple tests I ran here show that for when it comes to raw throughput for simple GDI operations, native code is faster but the human eye doesn’t really need anything faster than 30 frames per second to see it as smooth – so is it really a problem? What else is your application going to do?

If you’re making a flight simulator, then your app is all about drawing and sure – native code would be the choice, but what about a card game? How about a drawing program or a text reader? The point is that only you can decide which is better for what you need to get done. Hopefully this article, instead of providing opinion and conjecture, provides you another piece of real information with hard data that can help you make that decision.

January 2008

Christopher Tacke

OpenNETCF Consulting, LLC

Download the Source
  Download the PDF

Introduction

A few months ago a colleague of mine and I were invited to visit a potential customer to give them our thoughts and a expertise in the design of a new control system. The client makes machines that folds and fills cartons – think of cereal boxes and the like – at amazingly high speeds.

They had been using a proprietary system that they has designed with no OS and they were reaching the limits of just how fast they could run and how many sensors and the like that they could put on the Sercos bus so they figured they could leverage Windows CE (readily available for the controllers they already had) to bring out a new generation of machines with better throughput, faster reset times and all of the sorts of things their customers would love.

After a full day of walking through wonderful UML diagrams that detailed every last bit of how the process for their new system would work a few things were very clear.

1.       They definitely knew how to build boxes. Seriously – it’s far more complex than I imagined, and if I ever need a machine that folds cartons, these guys would have my business.

2.       They spent a large amount of time thinking through a whole lot of the details of how this new system architecture would work. And when I say a large amount of time, I mean probably more time than I’ve spent developing entire products. They had done serious due diligence here.

3.       They had no idea how managed code worked and a large majority of what they had done was going to be unusable.

True story. They had designed a large, very flexible, pluggable, extensible, deeply-abstracted architecture for a machine that should be able to fold several hundred or more cartons a minute and it had all been designed around a managed code foundation. These machines have motors, servos, actuators, conveyors and all sorts of things that have to run in a very finely tuned and orchestrated dance at a very high speed, and if any one item is in the wrong place at the wrong time then the best case is that something gets jammed or broken and worst case is a machine operator gets seriously injured.

The problem with their design was rooted in two fundamental issues; the first being that managed code is not deterministic. That means that when any action happens (like a sensor sees a moving box) then the time for a reaction to occur (an arm moves to make a fold) is unbounded. If you’re trying to make boxes that are all the same size and shape at any reasonable rate of speed, you can imagine that having a fold in the same spot every time might be fairly important.

Often you might hear the term “real-time” but the problem with that term is the definition depends on the system you’re looking at. A system is real-time if the maximum latency (or jitter) of a reaction is less than what you need for your system to operate. If you’re calculating trajectories for an air-to-air missile your definition of real-time is a whole lot different than if you’re trying to measure the movement of a tectonic plate in “real-time.”

At any rate, we know that any environment with a garbage collector, whether it’s C#, VB.NET running on Windows or Java running on Linux, is not deterministic by its very nature, so there’s no point in beating on this point too heavily. If you want to know about the deterministic behaviors of either Windows CE or managed code, there are good resources on the web to get you well acquainted (and you’ll be seeing more from us in the coming months as well).

The other problem with their design using managed code, however, is not so obvious (unless of course you read the title of this paper). If C# were somehow made deterministic, would it then be a good choice for them? We know that a large amount of what they need to do is going to require Platform Invokes (or P/Invokes) and I instinctively felt that there was going to be a problem with performance because of that. Of course I had no data to back that up and I knew of no specific papers or articles that specifically looked at what the performance impacts of P/Invoke calls were, but I knew in my gut that they were going to be a problem, so we went back to the hotel, had dinner and a couple drinks, and I proceeded to put together some test code to prove out what I knew deep-down just had to be true.

Test Theory

My original test code was far more complex that what we’ll be looking at for this article. In it I tried to do more realistic simulations of what they would be doing in practice, including a lot of thread interactions and driver calls. For the most part a lot of those tests would just add complexity and confusion to these tests so for this article I distilled the problem down to just a few test points:

1. How long does it take for a call originating in managed code to arrive in the called native library?
2. How long does it take for the call to return back to the managed caller?
3. Does a callback perform any better than a returned value or parameter?

To measure the first I decided that I would get the current value of the performance counter right before the P/Invoke call and then in the native library I would grab the value again as soon as the method was entered. To make the second and third tests easier, I would simply return the performance counter value collected in the native method back to the managed caller either directly as a return value or through a callback. The managed caller could then do some basic math and arrive at answers for all three questions.

Of course the raw answers themselves are pretty useless. If you want better performance, all you need is a faster processor. To really make any use of these numbers, we need to do the same test in pure native code doing the exact same steps and then do a comparison of the native and managed results. What is important is not the hard numbers, but the ratios or percentages of a managed test to an analogous native test.

The Native Callee

Step one was to create the native library that would get called by both the native and the managed clients. Using the same library for both tests eliminates it as a source of variability, and in this case is was really simple. I simply exported two functions – one that would return the time it received a call through an output parameter and the other through a callback. The code is really simple:

Collecting Control Data

Step two was to gather some data to use as the “control” for the experiment by writing a native client EXE that calls the native DLL and receives back a value through both a return value and a callback. TO make sure I had a decent amount of data I ran 10,000 iterations of the test and output the results to a CSV file so I could plot and analyze the data in Excel. Again, the code is very simple. I opted to load the library dynamically to replicate as close as possible what managed code would be doing:

Collecting Compact Framework Data

The code for collecting the data from the managed application is obviously the same basic idea as the native application. Each of the native APIs are called and the managed application calculates the times and logs them to a file. The app runs this set 10,000 times to get a good sampling. The meat of the logic is below – the full source is in the download for this article.

Since managed code also has the huge variable of the GC, I also decided to rerun the test with a background thread continually allocating and throwing away data.

Now if you’re paying close attention you’ll notice that our timing code here is not identical to that in the managed code. QueryPerformanceCounter itself is a P/Invoke, so we’re inherently adding the time for the return from that P/Invoke to our results. Eliminating this effect would take a bit of work that would make this test setup a lot more convoluted, and we’re trying to keep this simple, but keep it in mind when looking at the results later.

Results

Once I’d run all the tests, I imported the data files into an Excel spreadsheet for analysis and plotting. The following plots provide comparative views of the times for each “segment” of the test (i.e. each of our three initial questions. Each plot has three data sets – the control (native) data and then managed data run with and then without a background thread generating garbage (and therefore garbage collection). The plots show performance counter ticks (elapsed time) on the Y axis and the sample number of the X axis.

Outgoing Direct Call

In Figure 1 we see the times required to make a direct call into the DirectCall method. We can see that a vast majority of the times for all three data sets appear to be about the same and very small. We’ll look at the actual mean values later, but this plot shows some important points:

1. The managed calls occasionally have gigantic outliers. We have a couple calls that took over 15,000 ticks (over 12 milliseconds on the test device).
2. Even the native code has some jitter (we see a maximum of 1024 ticks, or about 860 microseconds) but the jitter, or outlier variation from the mean, is much less pronounced than the managed code.
3. We see the same outlier frequency whether or not we have GC occurring.

So from this plot we see that managed code appears to inherently increase the overall jitter of the system by over an order of magnitude. When you’re designing a system that must perform a task within a certain time boundary, you must look at the worst-case scenarios for your maximum speed, so just on this small sample of raw data we would say that we could safely run native code over 15 times faster than managed (15955 / 1024). Of course this is assuming that the P/Invoke call is part of the time-critical logic.

Figure 1

Figure 2 is shows the same data but with the Y-axis rescaled to show just the low-end mean data. The interesting point that we see here is that the average speed to make the call from managed code is about half the speed of the native call.

Figure 2

Direct Call Return

In Figure 3 we look at the time required to return from a method. Again, it looks like the mean for the native and the managed tests are about the same and very small, but we still see some critical points:

1. Both the managed and native code have occasional outliers
2. The managed outliers are an order of magnitude larger than those from the native code.

Figure 3

Figure 4 again is the same data as Figure 3, but with the Y-Axis rescaled. You can see that the managed code appears to be slightly faster on average (it looks like the “blue” area extends about 1 tick below the green) but all three data sets seem to be about the same.

Figure 4

Returning Through a Callback

Figures 5 and 6 show the results for the callback times and are the most interesting of the group. Figure 5 is simply the same data as Figure 3 but with the Y-Axis scaled to show just data with times below 600 ticks.

Right away we see the same trend as the earlier plots – that the managed code produces outliers much greater in magnitude than the native code. What’s different here, though, is the frequency of those outliers. Instread of just a couple over the 10,000 sample data set, we see a dozen very large outliers (with the GC data set) and still a lot more low-end variation (all the spikes above the average but below around 3000 ticks).

Figure 5

If we zoom in on the low-end data, Figure 4 show some interesting points as well.

1. The average for the managed callbacks is substantially higher (turns out to be over 10 times slower as we’ll see later) than native callbacks.
2. Managed callbacks are never as fast as a native callback
3. GC appears to have a larger impact on callbacks than anything else we’ve looked at

Figure 6

Summary Data

Below are tables that summarize the collected data. The upper pair show statistics for the direct call and return. The lower pair show the callback (call and return statistics). You can see that the number solidify the perceptions we got from looking at the plots. Making a P/Invoke call does indeed take almost twice as long as the comparable native call. And the return is even 20-30% slower.

Analyzing the Data

So now we have a whole lot of data. Data is fine, but what we really need is information. Looking at these results raises more than a couple questions that we can’t readily answer without access to the CLR itself. I don’t have access to that kind of information, but I do have access to people who do, so I sent off some questions to the developer who owns the marshalling code for the Compact Framework to see if we could get some answers.

To his credit he did a lot of footwork on this, injecting stubs into the CLR and looking at outputs from the JIT compiler. He sent me several excellent explanations for several question complete with assembly code. As a managed developer (and even as a native developer) looking at assembly is like looking at a gorgon and we really only need to know the general answers, so I’ll save you the petrification and summarize in somewhat less technical terms.

Why is the P/Invoke call half the speed of the native call?

The managed call doesn't directly call the native method. Instead it calls into a JITted stub method that must perform some overhead routines such as calls to determine GC Preemption status (to determine if a GC is pending and we need to wait). It is also possible that some marshalling code will get JITted into the stub as well. This all takes time. Also keep in mind (as pointed out back in the “Collecting Compact Framework Data” section earlier) that the managed test itself is actually timing the return from the QueryPerformanceCounter (QPC) P/Invoke as well.

Why is the direct return 20-30% slower in managed?

Just like for the outbound call, the return in managed code also includes a JITted stub that does some GC housekeeping to see if a GC is pending plus we still have our QPC return time which slows the managed test.

Why are callbacks so crazy expensive in managed code?

The deep answer to this is a complex answer complete with a lot of assembly code that no reasonable developer really wants to read. What it boils down to, though, is this:

The IntPtr returned by GetFunctionPointerForDelegate is not really the pointer for the JITted callback. It's actually a pointer to a stub that does some additional marshaling. That stub in turn calls several different functions down in mscoree, so the route is not straight and your call stack for the callback itself is fairly deep.

These calls check to see if we're marshaling managed types, packages and loads any arguments into buffers, makes the call and then packages and marshals any return value. Throughout this process the CLR is also doing bookkeeping on the thread states, checking and adjusting GC preemption status, looking for events and exceptions and potentially sending out debugger and/or profiler information. During all of this locks are getting acquired and released and memory is getting allocated, so a GC can potentially be triggered during the process as well.

So a whole lot more is going on that simply jumping execution to the callback function address as happens in native code.

Why is the directcall in 2x slower than native, but the call in for the callback is about the same as native?

This one is baffling and without some really deep investigation with a native profiler it difficult to say exactly what is happing. We would have expected the results to be the same. There may be some issues with code alignment, memory controller caching or something else that isn’t obvious that is affecting this result. For now it remains a mystery but it’s likely to be an artifact of the environment, not the code itself.

What accounts for the diff between the two managed callbacks?

We see that individually they have a pretty solid mean, yet they differ by 40%. Why? And what’s with the “steps” in the data? The start the same – is the “higher” level due to GC activity. Does that make sense that the “with GC” line would hop up after a short period and stay, since we’re creating garbage as fast as we can? If that’s so, then why does the “no GC” hop up to about the same level and then stay there for a period before dropping back down?

These are all reasonable questions and unfortunately without a ton of work with both native and managed profilers the only thing we can do is make educated guesses. My guess is that the difference is that the additional thread doing lots of allocation is contending for the same memory allocation locks that the callback "goo" is using, and it's the contention for these locks that's causing the callback to take longer. We might have some thread context switching that is being done in one case and not the other. Still your guess is probably as good as mine, and it’s not likely that we’ll have a definitive answer soon (if ever), but I don’t think we really need one. The lesson we’re really interested is that the managed callbacks are an order of magnitude slower than native callbacks. A small jitter on exactly how much isn’t terribly important in the overall scheme of things.

Conclusion

So what have we learned here? I’d say we can sum it up in a few points:

1.       Managed callbacks, while extremely useful in some cases, are an order of magnitude slower than native callbacks.
It’s probably advisable to not rely heavily on them or purposely architect a solution to use callbacks if it can be avoided.

2.       Managed P/Invoke calls are slower than native method calls.
While the exact amount can vary, and the difference is small, the fact remains that a P/Invoke is slower. If you’re doing work where time is critical it’s important to keep this in mind.

3.       Managed code has a lot of jitter that must be considered in any solution.
The nature of managed code means that we have a garbage collector doing stuff for us in the background, and when it does it’s work, the performance of other threads is impacted and impacted severely. This isn’t just true for C# or VB.NET. Any runtime with a collection mechanism (which includes Java) is going to have this problem. This means that if you have deterministic constraints in your solution, they either need to have a lot of room for sway or you need to be using native code.

I specifically not using the term “real-time” here because almost all code needs some level of determinism. When a user clicks a button in your app, if it takes 30 seconds to respond that’s still unacceptable, so you do have some deterministic requirement. If you’re measuring the real-time speed of continental drift, a few seconds probably isn’t going to matter much. However if you’re trying to run a precision machine tool in a factory, a hundred milliseconds can easily mean a ruined part or broken tool.

The point is that you need to understand your solutions requirements and understand the limitations of the code you’re writing. Don’t dismiss managed code out-of-hand because of some general (often incorrect) perception that it’s slow and likewise don’t architect it into a solution with tight time requirements just because it’s the environment you know. Both good and bad code can be written in any language and the difference is usually knowledge and experience. The hope is that now you have a little more of that and we can all try not to add to the large pool of bad code out there.

An Introduction to WCF for Device Developers

Chris Tacke

OpenNETCF Consulting, LLC

November, 2007

Download the Source Code
  Download the PDF

Introduction

With the release of Visual Studio 2008 to MSDN subscribers last week, I decided that it was probably safe enough to install the tool and start using it, plus I’m working on a large project where we’re soon going to need to be able to communicate between a PC and a CE device.

Conceptually (and at only a very high level) I had an idea what Windows Communication Foundation (WCF) is and my guess was that it could be used to help solve the problem at hand, which is to generate some form of remoting capability.

Of course I’d never actually created anything that used WCF, nor even looked at WCF code except in some conference sessions, so the first step was to head to the bookstore. I picked up Inside Windows Communication Foundation from Microsoft Press and tried to give it a read. As is typical for me, I found most of it analogous to listening to a white noise machine. It did little to further my understanding of the subject, but left me annoyed and tired.

In all fairness I can’t say it’s a bad book, I just learn far better from concrete examples and hand-on work than I do from reading about doing something. Chapters 6 and 7 which cover Channels actually was interesting and useful and I actually read them rather than skimming. Chapter 6 gave me enough to say that the $40 I spent on it was worth the investment.

So once I was done with the book and a couple beers, I decided it was time to roll up my sleeved and start searching the web for some concrete examples so I could walk through some code in a debugger and learn by breaking stuff.

Little did I know that while there’s a lot of talk about WCF, finding a simple example that comes complete with source code for both ends is not a trivial task. This is especially true when it comes to WCF for devices – there simply is no full sample (that I could find anyway) that explained how to go from nothing but Studio to two devices talking to one another, step by step, with all the code and directions needed.

So after a day and a half of stumbling around and cussing, I finally ended up with something worked and that probably should have taken only an hour to develop. This white paper is a collection of the knowledge I gained, and that single source of a WCF sample that some other developer is now, or will be looking for in the near future.

Requirements

Before you get started, make sure that you have everything tool-wise that you’re going to need.

1.       Visual Studio 2008 (used to be called Orcas) RTM. I downloaded mine from MSDN. If you got yours elsewhere, then your mileage may vary through this article.

2.       Power Toys for .NET Compact Framework 3.5 CTP (September 2007). This one wasn’t obvious until I’d done a fair bit of searching. You *will* need it, so download and install it now. The current link is http://www.microsoft.com/downloads/details.aspx?FamilyID=C8174C14-A27D-4148-BF01-86C2E0953EAB&displaylang=en but if you’ve been browsing Microsoft’s site for years like I have, you know that that link may be dead next week, so search for title text if it fails.

3.       A Windows Mobile 5.0 device. In theory you should be able to use any device that is supported by CF 3.5 (Windows CE 5.0 or 6.0 or WM 5.0 or 6.0), but I used an Axim x51. I can say that I tried for some time to get them WM 5.0 emulator working and failed, so you should at least start with a physical device to remove that variable.

4.       Windows XP development environment. While this isn’t necessarily a requirement, I’m running XP, not Vista, so if there are any UAC steps that are needed to get this working you’re not going to see them in this article. If you’re using Vista, I wish you luck. Please post any necessary deviations as comments.

The PC-side WCFServer application

To make things fairly simple we’re going to use a transport shipped with WCF for devices and set up the server application to run on the PC (sure I need the reverse in my target solution, but as Bill Murray said in What About Bob – baby steps).

So the first thing we need is to create a service that does something on the desktop. We’re not going to go overboard and make it complex, we’re going to have a single function that takes in two integers and returns the sum of them. My thinking is that once you see that working, getting more complex stuff working is simply additional baby steps on top of that.

So first, create a desktop Console application (if you were in a rush and already created a WinForms app before you got this far in the article, delete the Form and change the project type in the Project Properties just like I ended up doing when I was creating this app).

Add the following reference to the project:

System.ServiceModel

Now add a new Class (well it will be an interface, but using the Add Class menu item and then changing the code is less clicking) document named ICalculator and replace its code with the code below. This is the interface that defines the contract our service exposes. It should all be self explanatory, so I won’t bore you with a walk-through.

using System;

using System.ServiceModel;

namespace OpenNETCF.WCF.Sample

{

    [ServiceContract(Namespace = "http://opennetcf.wcf.sample")]

    public interface ICalculator

    {

        [OperationContract]

        int Add(int a, int b);

    }

}

Now we need to create an implementation of that interface to expose as our actual service. Add another Class to the project called CalculatorService and overwrite the generated code in the document with the code below. Again, it’s not rocket science, so we’re not going to cover what it does.

using System;

namespace OpenNETCF.WCF.Sample

{

    public class CalculatorService : ICalculator

    {

        public int Add(int a, int b)

        {

            Console.WriteLine(string.Format(

                "Received 'Add({0}, {1})' returning {2}", a, b, a + b));

            return a + b;

        }

    }

}

The last remaining piece of our service app is the logic that actually creates an instance of our CalculatorService implementation and exposes it. This code is a little more complex (though not a whole lot), so we will look at what it’s doing, but first, overwrite all the code in your existing Program.cs file with the following code:

using System;

using System.ServiceModel;

using System.ServiceModel.Description;

using System.Net;

namespace OpenNETCF.WCF.Sample

{

    class Server

    {

        static void Main(string[] args)

        {

            string hostIP = Dns.GetHostEntry(

            Dns.GetHostName()).AddressList[0].ToString();

#if CLIENT_DISCOVERY_BUILD

            Uri address = new Uri(string.Format(

                "http://localhost:8000/calculator", hostIP));

#else

            Uri address = new Uri(

                string.Format("http://{0}:8000/calculator", hostIP));

#endif

            ServiceHost serviceHost = new ServiceHost(

                typeof(CalculatorService), address);

            try

            {

                // Add a service endpoint

                serviceHost.AddServiceEndpoint(

                    typeof(ICalculator),

                    new BasicHttpBinding(),

                    "Calculator");

#if CLIENT_DISCOVERY_BUILD

                // Enable metadata exchange

                // this is needed for NetCfSvcUtil to discover us

                ServiceMetadataBehavior smb = new ServiceMetadataBehavior();

                smb.HttpGetEnabled = true;

                serviceHost.Description.Behaviors.Add(smb);

#endif

                serviceHost.Open();

                Console.WriteLine(

                    "CalculatorService is running at " + address.ToString());

                Console.WriteLine("Press <ENTER> to terminate");

                Console.ReadLine();

                // Close the ServiceHostBase to shutdown the service.

                serviceHost.Close();

            }

            catch (CommunicationException ce)

            {

                Console.WriteLine("An exception occured: {0}", ce.Message);

                serviceHost.Abort();

            }

        }

    }

}

The first thing you’ll probably notice is my use of a precompiler directive. The reason for this is that we need to run this service once and then have a tool (that we’ll discuss shortly) connect to it and generate some code for us. Once that’s been done we want to remove and change some stuff, but if you extend this example you’re going to have to go back and have that tool do its work again, so it’s handy to keep the old code around. What I did (and you can see this in the code download) is add another configuration based on the Debug configuration to the solution called “Discovery” that sets CLIENT_DISCOVERY_BUILD, so turning this on or off is a simply matter of changing the target configuration.

So what does this code do exactly?

First it gets the local PC’s IP address (which we then use later on. Next it creates an address URI for our service. You’ll see that I use localhost for the discovery run. The reason for this is that the tool (that we’ll discuss shortly – be patient) actually hard-codes this into some other code. We’re going to use string substitution in our client and replacing “localhost” is way easier than searching through a string for some IP that may change between tool runs.

Next we create the ServiceHost and add an endpoint. The important pieces to note here are that we’re using BasicHttpBinding, which is one of only 2 bindings that the CF version of WCF supports (the other is email through Exchange ‘07), and that we give a name to the service, which gets used later, but through generated code.

Next we add the ability to exchange metadata only if we’re in the Discovery configuration. The reason for this is that the tool (wait for it!) can only connect to our service and generate code if this is turned on. I turn it off otherwise simply because it seems like a potential security hole if you leave something like that on in the wild. Maybe not, but we’ll play it safe until experience (or someone who knows more about WCF) tells us that we should do otherwise.

Finally, we open the service and output some stuff on the Console, waiting for the user to hit Enter to end the app.

That’s all there is to it, so make sure that CLIENT_DISCOVERY_BUILD is set and run your app, and you’ll get something exciting like the screen shot below.

Now let’s do a quick sanity check to make sure it’s actually exposing itself as a service. Open a browser on your PC and browse to the address that the service reported (http://localhost:8000/calculator). You should see something like this:

Tell everyone around you to stand in awe – you have created the mythical WCF Service.

The NetCFSvcUtil.exe Tool

Yes, now we’re going to actually talk about the tool that we talked about. The Power Toys for .NET Compact Framework 3.5 ship with a tool called NetCFSvcUtil.exe which is a lot like its desktop counterpart (that you won’t be using here – believe me, I tried). This tool will connect to your service using the metadata exchange that you enabled in your code and output some code for your CF client application to use.

Now I’m not big on code generators – having seen them evolve for many years, one thing that has typically been a truism is that they produce less-than-optimal code, but a quick glance at what this one generates looks reasonable and besides, we’re here to create a WCF app, not criticize code optimization.

Open a command windows and browse to the folder that contains NetCFSvcUtil.exe. The default installation of the Power Toys will put it at C:\Program Files\Microsoft.NET\SDK\CompactFramework\v3.5\bin. Next, run the following command (again, the service must be running when you do this)

netcfSvcUtil.exe /language:cs http://localhost:8000/calculator

You should get an output that looks like this:

And 2 files will be generated in the same folder as NetCFSvcUtil.exe: CalculatorService.cs and CFClientBase.cs. You’ll be using these files in the next step.

The CF Client Application

Now create a Smart Device Windows application named WCFClient and add the following references (failure to add these references will lead to lots of colorful language when the code previously generated by NetCFSvcUtil.exe fails to compile):

-          System.ServiceModel

-          System.Runtime.Serialization

Next add the two files created by NetCFSvcUtil.exe to the project. It’s probably best to copy them to the project folder first to prevent cluttering the SDK bin directory with all of your generated source.

Next update Form1 to look something like the figure below. The names of the controls are on the right and if you want to just copy and paste code, you’ll need your control names to match these.

NOTE

I was never able to get the emulator to connect to service. I was able to ping the emulator from the PC and it was able to browse to a web site, but it wouldn’t connect to the service. I then tried it on a real device and it worked, so I never bothered to go back and try to figure out why the emulator didn’t work. I highly suggest that you do run this sample on a real device.

Now let’s look at implementing the client. The first step I did was to display the device’s local IP address in the Form constructor. I did this when I was debugging because the emulator would not connect to the service and I was wondering if it had a valid IP address.

Update your Form’s constructor code to look like this:

public Form1()

{

    InitializeComponent();

    try

    {

        localIP.Text = Dns.GetHostEntry(

            Dns.GetHostName()).AddressList[0].ToString();

    }

    catch (Exception ex)

    {

        MessageBox.Show("No NIC found?");

    }

}

Now we add all of the service “work” in a Click event handler for the calculate button. Add a handler to the button and add the following code to the handler:

private void calculate_Click(object sender, EventArgs e)

{

    int a = 0;

    int b = 0;

    try

    {

        a = int.Parse(first.Text);

        b = int.Parse(second.Text);

    }

    catch

    {

        // parsing failed, just bail out

        return;

    }

    SMC.Binding binding = CalculatorClient.CreateDefaultBinding();

    string remoteAddress = CalculatorClient.EndpointAddress.Uri.ToString();

    remoteAddress = remoteAddress.Replace("localhost", serviceAddress.Text);

    EndpointAddress endpoint = new EndpointAddress(remoteAddress);

    CalculatorClient client = new CalculatorClient(binding, endpoint);

    try

    {

        answer.Text = client.Add(a, b).ToString();

    }

    catch (Exception ex)

    {

        MessageBox.Show(ex.Message);

    }

}

The important point in this code are that we don’t use the default constructor for the CalculatorClient, because that would end up using a remoteAddress with “localhost” in it, because that’s what the code generator hard-codes it to be. Instead, we extract that value (it’s a static field of the generated CalculatorClient) and then use string substitution to replace “localhost” with the IP address of the PC running the service.

Once we’ve created the CalculatorClient instance, calling the service is trivially simple.

Conclusion

So there you have it - a complete, working, end-to-end example of how to create a WCF service running on a PC and how to consume it using a Compact Framework 3.5 application. As you’ve seen, there isn’t a lot of code to it, and it’s not overly complex to get set up – it’s just not very well documented in any one place.

Now where might you use something like this? One thing that comes to mind is the ability to put functionality on a machine somewhere within your network that can then be called by multiple devices without having to go through all of the configuration and setup required to get an XML Web Service under IIS running. You can then update or fix the service code in one place and all devices using the function immediately get the benefit without having to redeploy or roll out the fix. You could also do computationally-heavy work on a desktop machine with more power, but still do the collection and display on a lightweight mobile device.

At any rate, I hope that this white paper saves someone else a bit of the headache I went through in trying to get a WCF application running. Some additional links that I found at least somewhat useful in researching this are below. Enjoy.

CF WCF Intro materials:

http://blogs.msdn.com/markprenticems/archive/2007/03/27/introduction-to-windows-communication-foundation-for-the-net-compact-framework-messaging-stack.aspx

The Lunch Launcher

http://blogs.msdn.com/davidklinems/archive/2007/11/12/the-journey-of-the-lunch-launcher-part-8-what-did-i-learn.aspx

Using NetCFSvcUtil.exe

http://blogs.msdn.com/danhorbatt/archive/2007/09/12/using-compact-svcutil-to-interact-with-servicecontracts.aspx

Calling WCF Service from CF 3.5

http://blogs.msdn.com/andrewarnottms/archive/2007/09/13/calling-wcf-services-from-netcf-3-5-using-compact-wcf-and-netcfsvcutil-exe.aspx

The CF 3.5 subset of WCF

http://blogs.msdn.com/andrewarnottms/archive/2007/08/21/the-wcf-subset-supported-by-netcf.aspx

Remote Logging WCF

http://blogs.msdn.com/danhorbatt/archive/2007/11/01/remote-logging-wcf-on-net-compact-framework.aspx

Mark Arteaga
OpenNETCF Consulting

September 2007

Download the Source Code
 Download the PDF

Contents

Introducation
Ink Serialized Format Data
Sharing Ink With A Custom Application
Sharing Ink with One Note 2007
What’s Next
Conclusion

Introducation

In the previous article we discussed technical information on the Mobile Ink Library and WISP Lite.  In this article we will discuss how the Ink Serialized Format (ISF) data can be shared on both a Windows Mobile 6 application and an application running on the desktop.

Ink Serialized Format Data

Ink Serialized Format (or ISF) is the binary format in which the ink data is persisted as.  This format directly compatible to the Tablet PC ink format and can be interchanged on both the Table PC and a Windows Mobile 6 device.  Using ISF data that was saved on a Tablet PC on a Windows Mobile device, Windows Mobile will ignore any properties in the ISF stream that are not supported but still preserve the properties when saved again. 

As developer using the Mobile Ink Library you have the option to persist the ink data as either as:

1. Ink Serialized Format - The ink data using ISF.  This is the most compact representation of ink data.
2. Base64 Ink Serialized Format - The ink data using ISF and Base64 encoded.
3. Gif - The ink data in Graphics Interchange Format (Gif)
4. Base64 Gif - The ink data Graphics Interchange Format (Gif) and Base64 encoded

In the next sections we’ll look at sharing ISF data in a custom desktop application and a Windows Mobile application and also importing ISF data created on a Windows Mobile device into One Note 2007.

Sharing Ink With A Custom Application

Since ISF data produced on the desktop is binary compatible on a Windows Mobile 6 Device and vice-versa I decided to take the common scenario of signature capture on a mobile device and display the data on a custom application written using the .NET Framework running on the desktop.

The Mobile Ink Library includes a sample called InkSerializationCF which captures a users signature and embeds an ExtendedProperty within the ISF stream.  We will use this sample to show how the ISF data can be shared with the desktop.

Note, if you want the try this sample on the desktop you will need to install a few SDKs.  Here is a chart summarizing what is need on different operating systems.

Changes Since First Release

Since the Mobile Ink Library was first released, I have gone ahead and made some changes to align the classes more with the Microsoft.Ink classes available. 

The first change was to the OpenNETCF.WindowsMobile.Ink.InkOverlay class.  This class has been renamed to OpenNETCF.WindowsMobile.Ink.Ink class.  The InkOverlay class has been marked obsolete and is still available.

In the above diagram you will notice in the OpenNETCF.WindowsMobile.Ink.Ink class we also added a new property called ExtendedProperties.  These are the custom properties you can add to an ISF stream.  In the case of the signature sample application we add the date the signature was saved, the first name and last name of the person signing.

Next is the InkControlBase class.  We have added a new property called Ink which exposes the Ink object discussed above.  Since we are using the OpenNETCF.WindowsMobile.Ink.InkPicture control as a base class for the OpenNETCF.WindowsMobile.Ink.InkSignature control and since InkPicture inherits from InkControlBase, this will help us keep the same object model as the Microsoft.Ink.InkPicture control.  Here is a class diagram of the controls in the Mobile Ink Library:

Having a similar object model allows us to use the same InkSignature control on both the Windows Mobile application and the Desktop application.

Sharing the InkSignature Control

Now that some appropriate changes have been made to the Mobile Ink Library classes to align with the Microsoft.Ink classes, we can now look at how we can share ISF data generated from a Windows Mobile 6 device within a desktop application.

The InkSignature control is a custom control that will capture signature data.  It derives from InkPicture and since we change the classes to align more with the desktop we just have to change the internal implementation of InkSignature to be able to share the control on both the desktop and the device.  (For more information on sharing code between .NET Compact Framework and .NET Framework see this article in MSDN Magazine.)  By using conditional compile statements we are able to have code in the same source file for both the device and the desktop.

To implement the code sharing we will be working with two separate solutions.  The first is InkSerializationCF which is a .NET Compact Framework project for Windows Mobile 6 device.  The second is InkSerializationFx which is the .NET Framework application that runs on the desktop.  The InkSerializationFx application will only be used to view any signatures saved via the InkSerializationCF application.

The main change to accomplish the code sharing is to the InSignature.Open(string) method.  Here we have used conditional compiles to allow us to use on both the desktop and device.

#if !Fx

public override void Open(string file)

#else

public void Open(string file)

#endif

{

#if !Fx

    base.Open(file);

#else

    this.InkEnabled = false;

    StreamReader sr = new StreamReader(file);

    byte[] data = Encoding.ASCII.GetBytes(sr.ReadToEnd());

    sr.Close();

    this.Ink = new Microsoft.Ink.Ink();

    this.Ink.Load(data);

    this.Refresh();

#endif

    //See if the ISF file opened is valid

    object ret = this.Ink.ExtendedProperties[new Guid(m_sigValidationGuid)].Data;

    if (ret == null)

    {

#if !Fx

        this.Clear();

#else

        this.InkEnabled = false;

        this.Ink = new Microsoft.Ink.Ink();

        this.InkEnabled = true;

#endif

        this.m_firstname = "";

        this.m_lastname = "";

        this.m_dateSigned = DateTime.MinValue;

        throw new Exception("Invalid signature file!");

    }

    //Retreive the firstname extended properties

    ret = this.Ink.ExtendedProperties[new Guid(m_firstNameGuid)].Data;

    this.m_firstname = ret.ToString();

    //Retreive the lastName extended properties

    ret = this.Ink.ExtendedProperties[new Guid(m_lastNameGuid)].Data;

    this.m_lastname = ret.ToString();

    //Retreive the date the sign was signed

    ret = this.Ink.ExtendedProperties[new Guid(m_dateSignedGuid)].Data;

    this.m_dateSigned = DateTime.FromFileTime(long.Parse(ret.ToString()));

}

The Open() method will extract the signature information, and all the ExtendedProperties that contains the date signed, and first and last name.  From the above code you will notice not many changes are needed to be able to use the same source file on the desktop and on the device. 

In the InkSerializationFx project, we need to add a reference to Microsoft.Ink (since OpenNETCF.WindowsMobile.Ink is not supported on the desktop).

Then we also have to add a using statement with a conditional compile so InkSerializationCF does not use it:

#if Fx

using Microsoft.Ink;

using System.IO;
#endif

A conditional compile statement was also added to the Save() method since the InkSerializationFx does not allow any modification of the signature data.

#if !Fx

public void Save(string filename, string firstName, string lastName, InkPersistenceFormat format)
{

...

}

#endif

The InkSerializationCF sample was also modified to save the ISF data in different formats.  One thing to note is when sharing the ISF data with the desktop, you can only use the ISFBase64 format as the regular ISF stream will not load.

After all this talk about code let’s see the results.  Here are the results of a signature on a Windows Mobile 6 device.

And the results on the desktop.

For the desktop signature application, the signature is read-only as on the desktop you don’t want the signature to be modified.  You can add more ink to the ISF file and if you try to load it on the device, you will see those changes.  I will leave it up to the reader to explore that scenario.

Now that you have knowledge on sharing ink data between a desktop application and a Windows Mobile device, next we will look at sharing the ISF data with One Note 2007.

Sharing Ink with One Note 2007

Since ISF data is compatible on the desktop and a Windows Mobile 6 device,  I decided to explore one of the comments in the ‘Readme.txt’ file in the Windows Mobile 6 SDK which said:

“These ink notes can also be imported to OneNote on the desktop.”

Well, not being a regular user of One Note but seeing the value this can possibly have in an application I decided to explore this comment a bit. 

Integration with OneNote

When InkNotesCF was first ported from the native sample it allowed the user to save the note in a certain Xml format as follows:

<?xml version="1.0"?>

<Import xmlns="http://schemas.microsoft.com/office/onenote/2004/import">

  <EnsurePage path="Pocket Notes\InkFiles.one" guid="3855d272-28de-4a3f-b157-4eca51c5d15a" title="\Storage Card\note2003.ink" />

  <PlaceObjects pagePath="Pocket Notes\InkFiles.one" pageGuid="476d389f-a48b-4891-b665-148a6ed7d9d1">

    <Object guid="">

      <Position x="0" y="0" />

      <Ink>

        <Data></Data>

      </Ink>

    </Object>

  </PlaceObjects>
</Import>

As you can see from the Xml namespace the version of OneNote that was used when the native sample was created was OneNote 2003.  This Xml format was also ported to the managed InkNoteCF sample. 

With the release of OneNote 2007, the COM API has changed which also resulted in a new Xml schema to import ink data into OneNote.  We will not go through the changes but for more information on OneNote 2007 from a developers perspective see ‘What’s New for Developers in OneNote 2007 (Part 1)’ and ‘What’s New for Develoeprs in OneNote 2007 (Part 2)’

The new Xml structure for importing data into one is as follows:

<one:Page xmlns:one="http://schemas.microsoft.com/office/onenote/2007/onenote" ID="{0}">

  <one:Title>

    <one:OE author="Mark Arteaga" lastModifiedBy="Mark Arteaga">

      <one:T><![CDATA[\Storage Card\note2003.ink]]></one:T>

    </one:OE>

  </one:Title>

  <one:Outline>

    <one:OEChildren>

      <one:OE>

        <one:InkParagraph>

          <one:InkWord>

         <one:Data></one:Data>

          </one:InkWord>

        </one:InkParagraph>

      </one:OE>

    </one:OEChildren>

  </one:Outline>
</one:Page>

As you can see, the new Xml format (or schema) has changed from OneNote 2003.  If you would like to explore the schemas you can download the ‘2007 Office System: XML Schema Reference’.  I’ll leave it up to the reader to explore the schemas and other technical information.

To import the ink data into OneNote you will need to use the OneNote COM API.   In the sample program that imports the data we use a C# application and it automatically generates a interop assembly called Microsoft.Office.Interop.OneNote.  (for more information on this see this section in the ‘What’s New for Developers in OneNote 2007 (Part 1) article)

The basic flow to create a new note in OneNote is this:

Within the import application, the ink data generated by InkNoteCF is saved to the ‘Unfiled Notes’ section of OneNote.  The actual code looks like this:

Microsoft.Office.Interop.OneNote.ApplicationClass onApp = new ApplicationClass();

try

{

    //Get the path for the unfiled notes

    string unfiledNotesPath;

    onApp.GetSpecialLocation(SpecialLocation.slUnfiledNotesSection, out unfiledNotesPath);

    //Get the ID of the unfiled notes

    string UnfiledNotesID;

    onApp.OpenHierarchy(unfiledNotesPath, "", out UnfiledNotesID, CreateFileType.cftNone);

    //Create a new page

    string pageID;

    onApp.CreateNewPage(UnfiledNotesID, out pageID, Microsoft.Office.Interop.OneNote.NewPageStyle.npsBlankPageWithTitle);

    //Read the file data

    StreamReader sr = new StreamReader(txtInkNote.Text);

    string data = sr.ReadToEnd();

    data = string.Format(data, pageID);

    //Add the content to the one note page

    onApp.UpdatePageContent(data, DateTime.MinValue);

}

catch (Exception ex)

{

    MessageBox.Show(ex.ToString());

}

As you can see from the code, it’s not very difficult to import data into OneNote 2007.

Changes To InkNoteCF

When the InkNoteCF sample was first released, it only supported saving the data in OneNote 2003 format.  Since I did not have OneNote 2003 and only had 2007, I refactored the code to allow the user to save in either the OneNote 2003 or OneNote 2007 format.  In this section we will go over some of the changes done. (Note, if you are interested in importing the ink to OneNote 2003 see this article on MSDN)

The first thing that was modified was the Note class.  The original one only had support for OneNote 2003 so a new NoteBase class was created to accommodate the two different versions.  The new class hierarchy is show in the following diagram.

In the main user interface, a new menu item called ‘Format’ was added to allow the user to save the ink note as either OneNote 2003 or OneNote 2007 format.

I’ll leave it up to the user to delve into the code and to see how the saving code works.  In a nutshell, it just calls the XmlDocument.Save() method.

Running the Code

After all this background information you are probably thinking, what is the result?  Well, here are the screen shots of the application on the device and in OneNote 2007.

What’s Next

From here the Mobile Ink Library is in a good state to be used in managed applications.  We are hoping to improve it some more by wrapping some of the WISP Lite interfaces and align the object model more closely with the desktop.  At this point we have no plans to do that but invite the community to contribute to the project.  You can download the source code for Mobile Ink Library and the Samples at www.opennetcf.com/opensource/mobileink.ocf.

Conclusion

The new WISP Lite API available in Windows Mobile 6 gives the native developer the ability to ‘ink enable’ a Windows Mobile application.  Using the Mobile Ink Library from OpenNETCF, .NET Compact Framework developers can now also ‘ink enable’ their applications.  In this article we discussed how using the Mobile Ink Library, you can easily share ink data with a custom application or with OneNote 2007.  Have fun Inking!

Download the Source Code
Download the PDF

Contents

Introduction
Technical Overview
Mobile Ink Library Custom Controls
Samples Available with the Mobile Ink Library
Conclusion

Introduction

With the recent release of the Mobile Ink Library for Windows Mobile 6, some people are probably thinking 'well what can I use that for'.  In this article we will be going over some technical information and some possible uses for inking on the Windows Mobile 6 platform.

Technical Overview

Windows Mobile 6 Inking

In the latest release of Windows Mobile, there is a new API called Windows Mobile Ink.  What is Windows Mobile Ink?  Where does it come from? Why do you need it? In a nutshell, Windows Mobile Ink is a new Inking API only available for the Windows Mobile 6 platform called WISP Lite.  It is based on Windows Inking Services Platform (WISP) for the Tablet PC. 

Desktop developers have managed classes available under the Microsoft.Ink namespace to allow them to develop managed applications to take advantage of 'inking' features available on Tablet PC.  This article does not deal with the desktop so I'll leave it up to the user to search the many articles available on inking on the Tablet PC.

The Windows Mobile 6 SDK contains some examples of using WISP Lite with native code.  Unfortunately if you are a managed developer, there are no managed classes available for your .NET Compact Framework applications.  The Mobile Ink Library addresses this issue and provides a library of classes which wrap the COM interfaces and the controls that are available on the Windows Mobile 6 platform.

Differences from the Desktop and Device

The Mobile Ink Library and WISP Lite allows a developer to add inking functionality to their Windows Mobile application.  WISP Lite (as the name implies) is a 'lighter' version or a subset of WISP available on the desktop.  For the managed desktop developer, there is the Microsoft.Ink namespace which provides inking capabilities in your .NET Framework application.  For the native developer there is also a COM API available to add inking capabilities to a native application. 

Currently for inking capabilities on the Windows Mobile 6 platform there are only COM interfaces and a 'windowed control' named InkCanvas available to native developers.  Managed developers using .NET Compact Framework don't have any managed classes available but can wrap the COM interfaces and the InkCanvas control using version 2 of .NET Compact Framework.  The Mobile Ink Library has already done this and is available under the OpenNETCF.WindowsMobile.Ink namespace.  This gives the managed developer the ability to take advantage of this new APIs available. 

Both the desktop and the Windows Mobile versions of inking APIs allow the developer to save the ink in a format called Ink Serialized Format or ISF.  The ISF data output, which can be saved as Base64 or Binary, is compatible on both the desktop and mobile device.  If you try to open an ISF stream on mobile device that was saved on the desktop, the mobile device will ignore any properties in the stream that are not supported on Windows Mobile but still preserve the properties when saved again.  See this page for more details on the differences between the desktop and device.

COM Interfaces

There are various COM interfaces available for Windows Mobile Ink and for details on using it with native code see 'Using the Automation Library with Windows Mobile Ink'.  The Mobile Ink Library wraps the appropriate interfaces to allow a managed developer to ink enable a .NET Compact Framework application.  Two interfaces that are primarily used and will be covering next are IInkDisp and IInkOverlay.

IInkOverlay

An IInkOverlay object is used to attach to a window (or control) using the IInkOverlay.hWnd property.  When you 'attach' a window to an IInkOverlay object, the window becomes 'ink enabled' and allows the collection of ink data. 

IInkOverlay also allows you to change the size, shape and color of the ink using the IInkOverlay.DefaultDrawingAttributes property.  The following example changes the size of the pen width using a TrackBar.Value property.

IInkDrawingAttributes attrs = overlay.DefaultDrawingAttributes;
attrs.Width = (float)trackBar1.Value;
overlay.DefaultDrawingAttributes = attrs;

An example use of IInkOverlay is attaching it to a PictureBox and adding ink notes to an image.  The following code sample uses the OpenNETCF.WindowsMobile.Ink.Ink class to achieve this:

PictureBox pb = new PictureBox();
//Create a new InkOverlay object
OpenNETCF.WindowsMobile.Ink.Ink ink = new OpenNETCF.WindowsMobile.Ink.Ink();
//We need to set the handle (or control) the inkoverlay object is bound to
ink.IInkOverlay.hWnd = pb.Handle;
ink.IInkOverlay.Enabled = true;

To change the editing mode of the Ink object the EditingMode property can be set with one of the following values:

public enum InkOverlayEditingMode
{
    Ink = 0,
    Delete = 1,
    Select = 2
}

IInkOverlay exposes a property called Ink which gets or sets the IInkDisp object that is associated with the IInkOverlay object.

IInkDisp

IInkDisp is responsible for collecting the ink strokes from a user.  It also has the ability to Save() and Load() the ink strokes collected by an IInkDisp.  To create an instance of IInkDisp, use OpenNETCF.WindowsMobile.Ink.InkDisp class has one static method called CreateInstance().  

Usually you will want to create a new IInkDisp when you want to clear existing ink in an IInkOverlay.Ink.  An example of this is:

IInkDisp inkDisp = InkDisp.CreateInstance();
overlay.Enabled = false;
overlay.Ink = inkDisp;
overlay.Enabled = true;

Note, to set the IInkOverlay.Ink property, you must first set IInkOverlay.Enabled to false and then once you set the IInkOverlay.Ink property set the IInkOverlay.Enabled back to true. 

When saving the ink strokes, the Save() method returns a byte[]  and there are four possible options defined in OpenNETCF.WindowsMobile.Ink.InkPersistenceFormat:

·       InkSerializedFormat - The ink data using ISF.  This is the most compact representation of ink data.

·       Base64InkSerializedFormat - The ink data using ISF and Base64 encoded.

·       Gif - The ink data in Graphics Interchange Format (Gif)

·       Base64Gif - The ink data Graphics Interchange Format (Gif) and Base64 encoded

These options allow the developer to save in various formats depending on their application requirements.  For example, if you are emailing the collected ink, the receiver may not be able to render the ISF format so you will probably want to send it as a Gif image attached to the email.

IInkDisp has a property call ExtendedProperties which allows the you to save custom data within the ink data.  For example, using ExtendedProperties you can 'embedded' some data to know when the ink data was last saved.  One caveat is ExtendedProperties are only persisted when saving as either InkSerializedFormat or Base64InkSerializedFormat so if you save as a Gif or Base64Gif, this data will be lost.

That summarizes IInkOverlay and IInkDisp which are probably the two main classes needed for inking.  I'll leave it up to the reader to explore some of the other methods and properties available in these interfaces and some of the other interfaces available.

InkCanvas Control

The InkCanvas control is a windowed control and allows the developer to avoid using the COM Interfaces for inking in their application.  In addition to the COM Interfaces wrapped by Mobile Ink Library, the library also includes a managed wrapper appropriately called InkCanvas (found under OpenNETCF.WindowsMobile.Ink.InkCanvas). 

InkCanvas simplifies capturing ink within your application.  It uses windows messages to request operations and notify the user of any changes like adding or deleting ink.  If using the IInkDisp COM object is desired, the OpenNETCF.WindowsMobile.Ink.InkCanvas.Ink property can be used.  InkCanvas also has the following properties which may be useful

·       Mode - Gets or sets the editing mode

·       Ink - Gets the IInkDisp object

·       BackColor - Gets or Sets the background color

·       ZoomLevel - Gets or sets the zoom level

·       CanvasSize - Gets or sets the canvas size

·       RecognizedText - Gets the recognized ink as text

·       PenStyle - Gets or sets the style of pen to use for example pen or highlighter

There are also various methods available in the InkCanvas control which is pretty self explanatory so I'll leave it up to the reader to explore. 

Ink Recognition

One of the great benefits of the Windows Mobile Ink API is the ability to convert the ink data to a text string via ink recognition. 

The following code sample will convert ink data to a text string using the COM objects:

private Ink m_ink;

public override string ToString()
{
    //Get the appropriate COM objects required.
    IInkDisp disp = m_ink.InkDisp;
    IInkStrokes strokes = disp.Strokes;
    IInkRecognitionResult results = strokes.RecognitionResult;

    //Get the recognized string.  An alternate way of doing this is strokes.ToString()
    string ret = results == null ? "" : results.TopString;

    //Release the COM objects
    if (results != null)
        Marshal.ReleaseComObject(results);
    Marshal.ReleaseComObject(strokes);
    Marshal.ReleaseComObject(disp);

    //Return the recognized text
    return ret;
}

If you use the InkCanvas control the RecognizedText property can be used.  Internally the InkCanvas control does the following to recognize the ink data:

public string RecognizedText
{
    get
    {
        return Ink.Strokes.RecognitionResult.TopString;
    }
}

public Interfaces.IInkDisp Ink
{
    get
    {
        if (OpenNETCF.Windows.Forms.StaticMethods.IsDesignTime)
            return null;
        IntPtr pInk;
        NativeMethods.SendMessage(childHandle, (int)ICM.GETINK, 0, out pInk);
        if (pInk == IntPtr.Zero)
            return null;
        return (Interfaces.IInkDisp)Marshal.GetObjectForIUnknown(pInk);
    }
}

Essentially both methods would produce the same results for the same ink data.

Mobile Ink Library Custom Controls

The Mobile Ink Library consists of four controls to make it easier for the developer to ink enable a .NET Compact Framework application.  Using the IInkOverlay is pretty straight forward, but to simplify the life of the developer various controls were created to ink enable .NET Compact Framework applications using Windows Mobile 6.0.

InkControlBase

InkControlBase is the abstract base class in which the InkRecognizer, InkPicture and InkSignature inherit from.  The following is the class diagram of the control:

As you can see from the diagram InkControlBase inherits from System.Windows.Forms.Control.  'Inking' the control is enabled by the Ink property which wraps the IInkOverlay interface (for more information on IInkOverlay see the above).

Since IInkOverlay requires a handle to the window (or control) to attach to, we override the OnHandleCreated() in the InkControlBase class to attach the control.  The following code is used to accomplish this:

protected override void OnHandleCreated(EventArgs e)
{
    base.OnHandleCreated(e);
    if (StaticMethods.IsRunTime)
    {
        if (m_inkOverlay == null)
        {
            //Create the IInkOverlay object
            m_inkOverlay = new Ink();

            //Listen for all stroke events  
            m_inkOverlay.IInkOverlay.SetEventInterest(InkCollectorEventInterest.Stroke, true);

            //We need to set the handle (or control) the inkoverlay object is bound to
            m_inkOverlay.IInkOverlay.hWnd = this.HandleInternal;
            m_inkOverlay.IInkOverlay.Enabled = true;
        }
    }
}

You will notice the following line:

m_inkOverlay.IInkOverlay.hWnd = this.HandleInternal;

HandleInternal is an internal virtual property and returns the Handle of the InkControlBase control used to attach to the IInkOverlay. 

/// <summary>
/// Gets the internal handle to associate the IInkOverlay object with
/// </summary>
internal virtual IntPtr HandleInternal
{
    get
    {
        return this.Handle;
    }
}

Classes inheriting from InkControlBase have the option to override this property to return a different handle.  We will visit this again when we discuss the remaining controls.

InkControlBase provides all the basic functionality such as Clear(), Open() and Save() methods.  These methods internally use the Ink property to make the required calls to IInkOverlay. 

InkRecognizer

InkRecognizer class was created to allow the developer to recognize ink inputted by a user into plain text and inherits from InkControlBase. 

From the class diagram, you will see that InkRecognizer does not do much except override the ToString() method.  The ToString() method will return the text string that is recognized by the IInkOverlay object using the following code:

public override string ToString()
{
    if (StaticMethods.IsDesignTime || this.InkOverlay == null)
        return base.ToString();
    else
    {
        //Get the appropriate COM objects required.
        IInkDisp disp = Ink.InkDisp;
        IInkStrokes strokes = disp.Strokes;
        IInkRecognitionResult results = strokes.RecognitionResult;

        //Get the recognized string.  An alternate way of doing this is strokes.ToString()
        string ret = results == null ? "" : results.TopString;

        //Release the COM objects
        if (results != null)
            Marshal.ReleaseComObject(results);
        Marshal.ReleaseComObject(strokes);
        Marshal.ReleaseComObject(disp);

        //Return the recognized text
        return ret;
    }
}

From the code above you will notice that Ink property is used to access the IInkDisp, IInkStrokes and IInkRecognitionResults to return the recognized text.  (For more information on text recognition see the above).  The BasicRecognitionCF sample demonstrates the use of the InkRecognizer control.

InkPicture

InkPicture control allows the developer to add ink to a picture using the standard System.Windows.Forms.PictureBox control.

The InkPicture control is not as straight forward as the InkRecognizer. 

The first difference is InkPicture overrides HandleInternal.  If you recall, HandleInternal provides the handle to which the IInkOverlay object will be attached to.  InkPicture internally has an m_pictureBox field which is the standard PictureBox control used to display an image and we use the m_pictureBox.Handle to attach to the IInkOverlay object.

The InkPicture control exposes the internal PictureBox control as a property to allow the developer to modify any properties for the picture box.  It also exposes an Image property to get or set the image that is displayed in the PictureBox.

InkSignature

Capturing signatures is a very common requirement for Windows Mobile line of business applications and using and using the Windows Mobile Ink API (or WISP Lite) is a perfect solution.  Since the InkPicture control has the main requirements for capturing signatures (which is really just capturing ink data) it can be used as the base class for InkSignature.

Since this is a signature control, we may want to give some sort of indication to the user to sign.  To do this we can add an image to the control using the Image property.  Below is an example of what that could possibly look like:

The InkSignature control also exposes a FirstName, LastName and DateSigned property.  These class properties are saved as ExtendedProperties within the Ink Serialized Format (ISF) file when the signature data is saved. Be aware that these properties are only saved when the ink data is saved using ISF and won't be saved if saved to a GIF file.  The InkSerializationCF Sample demonstrates the use of the InkSignature control.

InkControlBase Workarounds 

During the development of these controls, it was found that IInkOverlay behaved differently in managed code than it did in native code.  Whenever a user started inking on the signature control, the InkSignature control could never lose focus by tapping on any part of the screen (for example the Textbox).  Specifically, the Capture property of the control was set to True whenever a user started inking which caused other controls to not receive any Click events.  It was also noticed that the InkControlBase control would never receive focus when clicked on because of IInkOverlay.

To work around this a workaround was implemented in the OnMouseDown and OnMouseUp of the InkControlBase. 

Focus Issue

First we'll look at OnMouseDown which resolves the 'focus' issue:

protected override void OnMouseDown(MouseEventArgs e)
{
    base.OnMouseDown(e);
    if (this.ClientRectangle.Contains(e.X, e.Y))
        OnMouseDownInternal(e);
}

From the code above you can see that on mouse down we check to see if we are in the bounds of the control, if we are we call OnMouseDownInternal().

internal virtual void OnMouseDownInternal(MouseEventArgs e)
{
    this.BoundingControl.Focus();
}

OnMouseDownInternal() will call the Foucs() method on the BoundingControl.  In the case of InkRecognizer it calls the InkControlBase implementation which returns this.

internal virtual Control BoundingControl
{
    get
    {
        return this;
    }
}

In the case of InkPicture and InkSignature, we override the BoundingControl property and return m_pictureBox.

internal override Control BoundingControl
{
    get
    {
        return this.m_pictureBox;
    }
}

This allows us to set focus to the underlying control and still have the control fire off events like GotFocus.

Capture Issue

The next issue faced was the Capture issue.  It seemed IInkOverlay always set Capture to true for the bounded control.  To work around this issue we had to override OnMouseUp as follows:

protected override void OnMouseUp(MouseEventArgs e)
{
    if (!this.ClientRectangle.Contains(e.X, e.Y))
        OnMouseUpInternal(e); 

    base.OnMouseDown(e);
}

Again from the above code you can see that we call OnMouseUpInternal() where we set the Capture property to false and then send a 'manual tap' so the user does not have to tap the screen twice.

internal virtual void OnMouseUpInternal(MouseEventArgs e)
{
    //HACK When using the IInkOverlay object in managed 
    //code seems that the control Capture property is set to true.  
    //When trying to click outside the bounds of the 
    //control (ie a button) the control would not 
    //receive the mouse click
    this.BoundingControl.Capture = false;
    SendTap(Control.MousePosition.X, Control.MousePosition.Y);
}

The SendTap() method makes a native call to mouse_event to manually send another tap to the screen since the first tap (or click) was consumed by the InkBaseControl.

internal void SendTap(int x, int y)
{
    NativeMethods.mouse_event(
        NativeMethods.MOUSEEVENTF.LEFTDOWN | NativeMethods.MOUSEEVENTF.ABSOLUTE, 
        (int)((65535 / Screen.PrimaryScreen.Bounds.Width) * x), 
        (int)((65535 / Screen.PrimaryScreen.Bounds.Height) * y),
        0,
        0);
    NativeMethods.mouse_event(NativeMethods.MOUSEEVENTF.LEFTUP, 0, 0, 0, 0);
}

InkCanvas

The InkCanvas control was discussed briefly in the technical overview but we will go through it again since it's a custom control part of the Mobile Ink Library.  The InkCanvas control is a managed wrapper to the native InkCanvas control available on Windows Mobile 6.  The InkCanvas control gives you the ability to add inking capabilities to your Compact Framework application on the Windows Mobile 6 platform without the need to use any of the COM interfaces. 

Here is the class diagram of InkCanvas:

From the above class diagram you can see there are three different modes that the InkCanvas can be set to. 

·         INK - Sets the InkCanvas to inking mode and allows the collection of ink data

·         SELECT - Sets the InkCanvas control select ink strokes collected

·         DELETE - Sets the InkCanvas to delete ink strokes as the user taps on the stroke

There is also a PenStyle property which allows you to change the size, color, width and pen type of the stroke being collected.  If you need to interact with the IInkDisp interface, you can also use the Ink property and use the COM interfaces available.  The main method of communication with the native InkCanvas is via windows messages.  Since the InkCanvas control inherits from the OpenNETCF.Windows.Forms.Control2 class, Control2 helps simplify the creation of native control and capture any messages such as StrokeAdded. This allows us to raise relevant events such as StrokeAdded, StrokeDeleted and CharacterAdded.  The InkNotesCF Sample demonstrates the use of InkCanvas control.

Samples Available with the Mobile Ink Library

Also included with the installation are five sample applications demonstrating the use of the library.  The following is a brief description of the samples.

InkControlDemo

InkObjDemo

BasicRecognitionCF

InkSerializationCF

InkNotesCF

Conclusion

In this article we took a look at the Windows Mobile Ink API for Windows Mobile 6 and the OpenNETCF Mobile Ink Library.  The Mobile Ink Library opens the doors to .NET Compact Framework developers to ink enable their Windows Mobile 6 applications.  In the next article we’ll look at using the same Ink Serialized Format (ISF) data on both  a Windows Mobile 6 application and a desktop application.