WPF and Windows Media Player COM control

The WMPLib offers direct control to Windows Media Player to your C# applications. This provides a rich set of events and controls that is really useful if you would like to create a custom media player.

The code I wrote scans the computer (My Music) and builds up a library of artists, albums and songs using the directory structure. Any images found are used to create folder icons. I later found out that it is possible to access the Windows Media Player media collection library to access this information so next time I´ll probably use that) However, this requires the albums to be imported/added to the Windows Media Player.

There is a few neat trick available when using the WMP. One is that it sends events when ever the player state changes (see list of events). This enabled me to initiate a progress bar that displays the current position in the song (code 3) as well as skipping to the next song when a the current playing song was completed (code 8).

To add WMP in you code

Using WMPLib;

WindowsMediaPlayer _wmp;

public MyApp()
{
_wmp = new WindowsMediaPlayer();
// The event below tells WMP to call your own MediaProcess code
// when the WMP player changes state
_wmp.PlayStateChange += new _WMPOCXEvents_PlayStateChangeEventHandler(MediaProcessChange);

}
private void MediaProcessChange(int newState)
{
//The integer "newState" contains the state of the media player, ranging from 1 to 11
// see http://msdn2.microsoft.com/en-us/library/bb249361(VS.85).aspx for mor infomation

switch(newState)
{
// case 1: // Stopped
// case 2 : // Paused

case 3 : // Song starts playing

progress.Maximum = _wmp.currentMedia.duration;
progress.SetValue(ProgressBar.ValueProperty, _wmp.controls.currentPosition);

BackgroundWorker _worker = new BackgroundWorker();
_worker.WorkerReportsProgress = false;
_worker.WorkerSupportsCancellation = true;

_worker.DoWork += delegate(object s, DoWorkEventArgs args)
{
while (_wmp.controls.currentPosition <>
{
// Dispatcher to update the U.I progressbar every two seconds
Dispatcher.BeginInvoke(DispatcherPriority.Background, (SendOrPostCallback)delegate { progress.SetValue(ProgressBar.ValueProperty, _wmp.controls.currentPosition); }, null);
Thread.Sleep(2000);
}
}; // End delegate
_worker.RunWorkerAsync();
break;

// case 4 : // ScanFordward
// case 5 : // ScanBackwards
// case 6 : // Buffering
// case 7 : // Waiting

case 8: // Song ended, play next

PlayNextSong(null,null);
break;

// case 9 : // Transitioning
// case 10 : // Ready
// case 11 : // Reconnecting
}
}

The "progress" object is a XAML added obj of the type ProgressBar. These handy U.I elements can be set to the maximum value (instead of 1 to 100) The maximum value I set to the entire lenght of the song. This way the current position scale to the bar.

There is a good tutorial for making a media player at Pauls Blog. Additionally, another custom media player by Sacha Barber. The MSDN network has extensive API info as always (but perhaps in a really boring format)

Gaze Media Player

The component I´ve been working on is now capable of the basics. Great feeling just looking a song titles and then skipping through the playlist by gaze =) There is room for improvements, would be nice have a component like a slider where one could go to a specific part of the song. (don´t know how many times I´ve been playing guitar to a song while learning it and going back and forth between mouse and guitar. Let´s see what a weekend could do =)

Screenshot of the music player component, layout not finalized.. Updated version has a song progression bar and volume controls.

The play-button houses another ellipse shaped menu with the regular controls for (next, previous, play, stop)

Last week of prototyping

Being back from a short easter holiday the final week of developing the prototype is here. The functionality of the interface is OK but there is major tweaking and bug testing to be performed. The deadline for any form of new features is Monday 31st of March. I intend to use all of April for setting up the evaluation experiments and procedures. As always there is so much more that I would like to incorporate, every day brings new ideas.

The final version will include:

• A configurable dwell button component. This enables drag and drop dwell buttons into Windows projects with individual configuration on dwell-time and icons etc.
  
  
• A novel gaze based menu system that utilized saccade selection (two step dwell) This highly configurable interface component displays itself when activated. It aims at solving the midas touch problem while utilizing the screen real estate in a better way. The two steps in the activation process can be set to specific activation-speeds (dwells) More on this later..
  
  
• A gaze-based memory game using a 36 card layout. By perfoming a dwell the cards "turn" over and shows its symbol (flag). The user the selects another card. If matching then remove them. If different, turn them back over. Got some nice graphical effects.
  
  
• A gaze based picture viewer that zooms into the photos in gaze of the user. More on this later.
  
  
• A gaze based media/music player. This component will scan the computer for artists, albums and songs. These items are then accessible by a gaze driven interface where the user can create play-lists and perform the usual functions (volume+-, pause, stop, next, previous etc.)
  
  
• Perhaps just one little surprise more..
  

Eight weeks so far. Curiosity, dedication and plain ol´ hard work, nothing else to it.

Midas touch, Dwell time & WPF Custom controls

How do you make a distinction between users glancing at objects and fixations with the intention to start a function? This is the well known problem usually referred to as the "Midas touch problem". The interfaces that rely on gaze as the only means input must support this style of interaction and be capable of making distinction between glances when the user is just looking around and fixations with the intention to start a function.

There are some solutions to this. Frequently occurring is the concept of "dwell-time" where you can activate functions simply by a prolonged fixation of an icon/button/image. Usually in the range of 4-500ms or so. This is a common solution when developing gaze interfaces for assistive technology for users suffering from Amyotrophic lateral sclerosis (ALS) or other "paralyzing" conditions where no other modality the gaze input can be used. It does come with some issues, the prolonged fixation means that the interaction is slower since the user has to sit through the dwell-time but mainly it adds stress to the interaction since everywhere you look seems to activate a function.

As part of my intention to develop a set of components for the interface a dwell-based interaction style should be supported. It may not be the best method but I do wish to have it in the drawer just to have the opportunity to evaluate it and perform experiments with it.

The solution I´ve started working on goes something like this; upon a fixation on the object an event is fired. The event launches a new thread which aims at determining if the fixation is within the area long enough for it to be considered to be a dwell (the gaze data is noisy) Half way through it measures if the area have received enough fixations to continue, otherwise aborts the thread. At the end it measures if fixations have resided within the area for more than 70% of the time, in that case, it activates the function.

Working with threads can be somewhat tricky and a some time for tweaking remains. In addition getting the interaction to feel right and suitable feedback is important. I'm investigating means of a) providing feedback on which object is selected b) indications that the dwell process has started and its state. c) animations to help the fixation to remain in the center of the object.

--------------------------------------------------------------
Coding> Windows Presentation Foundation and Custom Controls

Other progress has been made in learning Windows Presentation Foundation (WPF) user interface development. The Microsoft Expression Blend is a tool that enables a graphical design of components. By creating generic objects such as gaze-buttons the overall process will benefit in the longer run. Instead of having all of the objects defined in a single file it is possible to break it into separate projects and later just include the component DLL file and use one line of code to include it in the design.

Additional styling and modification on the objects size can then be performed as if it were a default button. Furthermore, by creating dependency properties in the C# code behind each control/component specific functionality can be easily accessed from the main XAML design layout. It does take a bit longer to develop but will be worth it tenfold later on. More on this further on.

Microsoft Expression Blend. Good companion for WPF and XAML development.

Windows Presentation Foundation (WPF) has proven to be more flexible and useful than it seemed at fist glance. You can really do complex things swiftly with it. The following links have proven to be great resources for learning more about WPF.

Lee Brimelow, experienced developer who took part in the new Yahoo Messenger client. Excellent screencasts that takes you through the development process.
http://www.contentpresenter.com and http://www.thewpfblog.com

>Kirupa Chinnathambi, introduction to WPF, Blend and a nice blog.http://www.kirupa.com/blend_wpf/index.htm and http://blog.kirupa.com/

Microsoft MIX07, 72 hours of talks about the latest tools and tricks.http://sessions.visitmix.com/

Better feedback!

Since having a pointer representing the gaze position on the screen becomes distracting some other form of feedback is needed. As mentioned before having a pointer will cause your eyes to more or less automatically fixate on the moving object. Remember, the eyes are never still and even more the eye tracker does create additional jitter.

What we need is a subtle way of showing the user that the eye tracker has captured the gaze coordinates to the same location as he/she is looking at. It's time for trying to make it look a bit nicer than the previous example where the whole background color of the button would change on a gaze fixation.

The reason for choosing to work with Windows Presentation Foundation is that it provides rich functionality for building modern interfaces. For example you can define a trigger to an event, such as GazeEnter (ie. MouseEnter) on a button and then apply a build in graphical effect on the object. These effects are rendering in real time such as a glowing shadow around the object or a gaussian filter that gives the object an out of focus effect. Very useful for this project. Let's give it a try.

This is the "normal" button. Notice the out-of-focus effect on the globe in the center.

Upon receving a glance the event "Image.IsMouseOver" event is trigged. This starts the built-in rendering function BitmapEffect OuterGlowBitmapEffect which generates a nice red glowing border around the button.

The XAML design code (screenshot) for the button. Click to enlarge.
Notice the GlowColor and GlowSize attributes to manipulate the rendering of the effects.
To apply this to the button we define the element Style="{StaticResource GlowButton}" inside the button tag. Further the globe in the center can be brought back in focus and highlighed with a green glow surrounding it inside the button canvas.

The globe is defined as an image object. Upon focus the triggers will set the gaussian blur effect to zero, which means in focus. The glow effect produces the green circle surrounding the globe.

Putting it all together in a nice looking interface, using the Glass Window style, it looks promising and a real improvement since yesterdays boring interface. Providing a small surrounding glow of giving the image focus upon fixation is much better than changing the whole button color. The examples here are perhaps somewhat less subtle than they should, just to demonstrated the effect.

Screenshots of the second prototype with new U.I effects and events.

The "normal" screen with no gaze input.

OnGazeEnter. Generate a glowing border around the button

The other example with a green glow and the globe in full focus.