Skip to main content

Living on the Edge of Emerging Technology

Go Search
Living on the Edge of Emerging Technology
  

Other Blogs
There are no items in this list.

 ‭(Hidden)‬ Admin Links

 Infusion TechEdge on WordPress

Retrieving Data
Adding multi-touch to your Windows 7 applications

This week at the Microsoft PDC, Windows 7 was announced and one of it's core features is to bring Multi-Touch to the standard desktop. This post will explain at a high level the support you can build into your applications to make them touch-aware.

Levels of Multi-Touch Interaction

In Windows 7, there will be varying degrees of touch support, depending on the amount of investment you want to put into your application. This section explains those levels in detail.

Basic

Basic support is offered by the fact that Windows 7 will extend the common controls (scrollbars etc.) to include support for touch gestures. This means that even legacy applications will support a basic level of touch compatibility. One example shown at the PDC was scrolling up and down a command prompt window with touch alone.

Gesture Support

The next level of support that applications can have is to incorporate gesture support. This means utilizing the new touch API's offered in Window's 7 to recognize user gestures.  An example of this is Internet Explorer 8 recognizing forward and back "flick" gestures to move forwards and backwards through the page history. This level of support requires some application changes, but it doesn't fundamentally change the way your application works, so should be fairly easy to incorporate into your application.

Touch Optimized

The highest level of support comes from touch optimized applications. These applications go way beyond incorporating gestures to do simple things - it involves redefining the whole user experience (UX) to make the interactions more focused on touch. In general, this means taking some of the core concepts of Surface development (e.g. making the content the user interface, rather than having a secondary UI for controlling the content). The classic example of this is the photos application, where touch can be used to directly manipulate the photos (zoom, rotate, pan, drag & drop to another folder, paint on top of the photos using ink etc.)

In a subsequent post, I'll explain the API support you can leverage in Windows 7 to support these varying degrees of touch compatibility.

Windows 7 multi-touch API's

Windows 7 will come with new API's for supporting multi-touch interactions from a variety of applications. This post will explain at a high level how these API's work and how you can incorporate touch into applications built on top of different UI frameworks.

UI Framework Support

Upcoming releases of the major UI frameworks (Native, WinForms/Win32, and WPF) will incorporate features for touch support in Windows 7. Here's a quick rundown of support in each of these frameworks.

Native Controls

In Windows 7, the native controls (those contained in comctl.dll such as scrollbars and dialog boxes) will contain support for minimal touch gestures (such as panning a scrollbar). Because these controls are used by almost every native application that exists, these features will come for free on Windows 7.

WinForms / Win32

In the WinForms world, touch and gesture support will be brought through via a set of new WinProc messages (such as WM_TOUCH* and WM_GESTURECOMMAND).

WM_TOUCH* is a set of messages that behave much like mouse messages... there will likely be messages for WM_TOUCH_DOWN, WM_TOUCH_MOVE, and WM_TOUCH_UP (but these may get rolled into a single WM_TOUCH event with a parameter in the future). By handling these messages in your WinForms application, you can start to build more complex touch interactions into your UI.

WM_GESTURECOMMAND is a message that recognizes standard gestures such as 2 finger zoom and rotate. It is designed to offload the processing of the direct touch events and to add a level of compatibility across applications (e.g. to ensure zoom is a standard gesture across applications). Features of WM_GESTURECOMMAND include coalescing of multiple touch contacts (e.g aggregating two fingers moving closer together as a single zoom out command, or a quick touch down and touch up as a tap command). Gesture commands also bubble up the window hierarchy starting at the lowest child level, allowing parent windows to handle multiple gestures for different child windows.

WPF

WPF applications cannot take advantage of the changes to the API's (because WPF windows have a single HWND, instead of a HWND per child). Because of this, the support in WPF had to be re-written. The API for WPF will be similar to (but definitely not the same) as the Microsoft Surface SDK.

At a high level, the WPF UIElement and UIElement3D classes will be extended to have new touch events (which will be very similar to the existing mouse events). The same level of support for gestures (touch coalescing and bubbling of events) can be expected for the WPF api. In addition, event arg classes for touch events will provide a rich set of information including Original Position, Delta, and methods for the calculation of a transformation matrix that can be applied directly to controls.

In addition, Microsoft's roadmaps for WPF and Surface seem to be meeting at a common place with dotnet 4.0 and Surface SDK 2.0... the Surface controls that are touch enabled (SurfaceButton, ScatterView et. al.) will be unified and available across both API's. There will still be some differences (e.g. Surface's support for domino tags), but it's good to see that there will at least be some cross over.

A Lap around Mesh Applications

Arash Ghanaie-Sichanie gave a good talk on building Mesh-enabled Web Applications.  During this talk, Arash demo'd the Blockbuster Mesh App that many of us developing Infusion's PDC demo had heard about.  Basiclaly the blockbuster app lets you choose movies out of a catalog to add to your wish list.  You can also rank the movies and those invited into the application can see your ratings.

A few items worth noting are captured below:

Key tenants of Mesh Enabled Web Applications

  • Extends websites to desktop, desktop to website
  • Client-soide web apps hosted on live mesh services
  • Same app that runs in cloud or runs locally