Wednesday, November 26, 2014

Kinect for Windows 2 and XNA demo

Over the Thanksgiving break I managed to get my new Kinect for Windows 2 working with XNA. I couldn't find any code samples online using the new Kinect SDK, so hopefully this will help out others who are trying to do something similar.

This demo will show how to display the Kinect's video feed which is provided by its 1080p color camera. This is similar to the Color Basics-WPF C# Sample provided in the Kinect SDK 2 except that it is tailored for XNA.


Prerequisites

You must have the Kinect for Windows 2 correctly installed along with the SDK. There are plenty of online tutorials showing you how to program with the older Kinect; this is for the latest version.

Microsoft is no longer maintaining XNA, but but you can still use it in Visual Studio 2013 as long as you install the following:

  1. Install Visual Studio 2010. Any edition will work.
  2. Install XNA Game Studio 4.0
  3. Install XNA 4.0 Refresh for VS 2013. After you have downloaded the zip file, extract it and run XNA Game Studio 4.0.vsix

If you do not have access to Visual Studio 2010, you might try MonoGame. Just be aware that it does not have a content pipeline converter (software that converts content like sound files into xnb files). I have not tried MonoGame with Kinect, but I don't see any reason why it wouldn't work.


Create an XNA Project

First create an XNA project by selecting FileProject... from the menu. Then select XNA Game Studio 4.0 template under Visual C# and select Windows Game (2.0). Name the project KinectVideoXna.


Add Kinect Reference

Right-click the project in the Solution Explorer and from the context menu select AddReference.... Type kinect in the dialog box's search box, and check the Microsoft.Kinect reference. Then press OK. You should now see Microsoft.Kinect among the project's References in the Solution Explorer.


Adding Code

Add the Kinect namespace:

using Microsoft.Kinect;

Add some class-level variables:

// Texture to draw
Texture2D videoTexture;

// Active Kinect sensor
private KinectSensor kinectSensor;

// Reader for color frames
private ColorFrameReader colorFrameReader;
        
// Intermediate storage for receiving frame data from the sensor
private byte[] colorPixels;

Initialize the sensor and the data structures used for capturing data from the sensors:

protected override void Initialize()
{
	kinectSensor = KinectSensor.GetDefault();

	// Open the reader for the color frames
	colorFrameReader = 
		kinectSensor.ColorFrameSource.OpenReader();

	// Specify a handler for frame arrival
	colorFrameReader.FrameArrived += Reader_ColorFrameArrived;

	// Create the ColorFrameDescription using rgba format
	FrameDescription desc = kinectSensor.ColorFrameSource.
		CreateFrameDescription(ColorImageFormat.Rgba);
	
	// Allocate space to put the pixels to be rendered
	colorPixels = new byte[desc.Width * desc.Height * 
		desc.BytesPerPixel];

	// Open the sensor
	kinectSensor.Open();

	// Create texture large enough to hold the color frame
	videoTexture = new Texture2D(graphics.GraphicsDevice, 
		desc.Width, desc.Height);

	base.Initialize();
}

Create the handler for the color photo sensor where we'll store the captured photo into videoTexture:

private void Reader_ColorFrameArrived(object sender, ColorFrameArrivedEventArgs e)
{          
	// ColorFrame is IDisposable
	using (ColorFrame colorFrame = e.FrameReference.AcquireFrame())
	{
		if (colorFrame != null)
		{
			// Copy color frame into the array
			colorFrame.CopyConvertedFrameDataToArray(
				colorPixels, 
				ColorImageFormat.Rgba);                   

			// Avoid exception when SetData method is used
			GraphicsDevice.Textures[0] = null;

			// Put pixel data into a texture
			videoTexture.SetData(colorPixels);
		}
	}
}

Finally, draw the videoTexture containing the color photo to the screen:

protected override void Draw(GameTime gameTime)
{
	GraphicsDevice.Clear(Color.CornflowerBlue);

	if (videoTexture != null)
	{
		// Draw color video
		spriteBatch.Begin();
		spriteBatch.Draw(videoTexture, new Rectangle(0, 0, 
			graphics.GraphicsDevice.Viewport.Width,
			graphics.GraphicsDevice.Viewport.Height), 
			Color.White);
		spriteBatch.End();
	}

	base.Draw(gameTime);
}

Press Ctrl-F5 to build and run the program. You should see color video of whatever your Kinect is pointed at.


Problems?

When I first tried to build and run my program, I got the following error message:

The primary reference "Microsoft.Kinect, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL" could not be resolved because it was built against the ".NETFramework,Version=v4.5" framework. This is a higher version than the currently targeted framework ".NETFramework,Version=b4.0".
To fix this problem, I closed the project in Visual Studio and opened the project's .csproj file in a text editor and changed the following line:
<TargetFrameworkVersion>v4.0</TargetFrameworkVersion>
to
<TargetFrameworkVersion>v4.5</TargetFrameworkVersion>
Then I re-opened the project in Visual Studio and re-built the application with no problems.

Friday, October 10, 2014

Teaching an upper-level Web Development course

We're half way through the fall semester, and the spring semester will be here before we know it. Every spring I teach an upper-level Web Development 2 elective which builds on the Web Dev 1 course that I teach every semester. The Web Dev 1 course covers the fundamentals: HTTP, HTML, CSS, JavaScript, Ajax, and PHP.

The goal of Web Dev 2 is to give students more breadth and depth in web development, focusing on both client and server-side technologies. I try to choose technologies that are widely used so my students will be more marketable after graduation, and in the past I've taught a variety of technologies: advanced JavaScript, jQuery, Java servlets and JSPs, ASP.NET Web Forms, ASP.NET MVC, and creating web services with ASP.NET and JAX-RS. Of course you can't cover everything in 16 weeks, so I have to put a lot of thought into what can be adequately covered in a limited amount of time.

This summer I did some development work at Flatirons Solutions using AngularJS and web services with Spring. It got me wondering if my Web Dev 2 course should focus more on JavaScript frameworks. One of my developer friends this summer tried to convince me that JavaScript was also taking over the server and that I should be teaching Node.js instead of Java.

It can be a struggle for computing professors to ensure our courses remain relevant when technologies are always changing. Fortunately I've got a lot of Facebook friends who are developers and Harding CS alumni, so I decided to ask their opinion:

If you were in college and could take an advanced Web Development course, what topics would you like it to cover?
This elicited quite a few responses which I'll summarize here:

  1. JavaScript frameworks like AngularJS, Ember, Knockout
  2. ASP.NET MVC and Web API
  3. Node.js
  4. Python and Django
  5. Ruby on Rails
  6. LESS
  7. Web services
  8. Web security
  9. Single Page Applications (SPAs)
  10. Automated web testing, A/B testing, UI testing
  11. Understanding HTTP
  12. Picking the right data store (relational, NoSQL, Map/Reduce, etc.)
  13. Caching and latency issues
  14. Teams that implement the same project on different platforms
  15. Git
  16. Web architecture - focusing less on development and more on architecture

There may have been a few things I left off, but these received the most mentions. (Thank you for your input, guys!) I'm still making up my mind, but this is what I'm leaning toward teaching this spring:

  1. JavaScript - The more JavaScript you are exposed to, the better. I'd like my students to be familiar with a number of advanced JavaScript features and some popular libraries like Underscore.js.
  2. jQuery- Everyone needs to know the most popular JavaScript library in use.
  3. Node.js and Express - Although I have little experience with Node, I think it would be helpful for my students to apply their JavaScript skills on the server as well as the client. I will probably tie in MongoDB which will give them experience with a NoSQL database. We will likely create a web service and consume it with an app written with...
  4. AngularJS - Very popular JavaScript framework with strong job demand. Plus I got some experience with it this summer. We will write unit tests using Jasmine and use Karma for integration testing.

My students will work in pairs on their projects and use GitHub to house their code like they do in my GUI course. We'll use the WebStorm IDE which I came to love this summer. I'll inject other topics like security where they are applicable.

A weakness of this plan is that my students will not get to use Java or ASP.NET which are very popular server-side technologies. However, they will leave this class fluent in JavaScript.

Now... need to find time in my schedule to learn Node.js, prepare all new quizzes, assignments, and projects, and also prepare for my Game Programming course.

Friday, August 01, 2014

Last day at Flatirons

Today is my last day at Flatirons Solutions. I've really enjoyed these eleven weeks and learned a ton. My web development courses, in particular, will definitely benefit from the exposure I received to some new technologies that I'll be integrating into those courses. I'm very thankful to George and Paula for giving me the opportunity to spend my summer here, and I hope that I made some major contributions.

Tuesday, June 24, 2014

Half way through our Colorado summer

We're now entering our sixth week living in Boulder. My family is getting more accustomed to apartment living. It's ironic though how we thought we were escaping the Arkansas heat only to live in an apartment lacking air conditioning! At least the temperature drops each night.

We had a little scare a few weeks ago. Becky had developed a large lump on her thyroid, and after meeting with a surgeon, she was strongly encouraged to have it removed in case it was cancerous. My sister (who is now eligible for sainthood) flew out here from Chicago so she could watch the kids while Becky recovered from surgery. The surgery went well, and praise God the lump was not cancerous! Becky is still healing from the surgery, but she is doing really well. The kids were pretty oblivious to the whole thing since Aunt Sass kept them busy with swimming and museums and making pizza. Have I mentioned how awesome my sister is?

Since the surgery, Becky and the kids have been occupying themselves with all kinds of activities like tennis lessons, swimming lessons, and now karate classes while I toil away at Flatirons each day. wink When I arrive home in the evenings we often eat and go out to a park or discover some new part of Boulder. On the weekends we have visited Red Rocks, Estes Park, and a few other places. In the photo below we hiked around Bear Lake at the Rocky Mountain National Park. As you can tell from our Chacos, we were not expecting to see snow!

We've gotten to see a number of old Colorado friends which has made our time out here very meaningful. Some of our best friends from Searcy also stopped by to visit for a couple of days, and we were able to dine at Casa Bonita and see a Rockies game with them.

Next week my brother and his wife will be visiting from Texas, and some more good friends from Searcy will be coming up July 4th weekend. Becky has some college friends coming up to visit the last week of July. Lots to look forward to!

I'll conclude with a note of thankfulness for the Boulder Valley Church of Christ. They have really taken us in, and we are so thankful for the many ways they supported us during Becky's surgery. One of the ladies watched the kids for us before Aunt Sara arrived, and one of the elders came to the hospital to pray with us immediately before the surgery. These are people who truly love the Lord.

I'm also thankful for the incredible VBS they put on. You would not believe how much effort they expended to entertain and teach about 50 children about God's love for a full week. Ethan and Braden absolutely loved going to VBS each day. I think it's really great that the body of Christ can be found nearly everywhere you go.

Sunday, June 01, 2014

Week 2: Finally making some contributions

Last week at Flatirons was admittedly difficult. I really like to learn new stuff, but if I'm not contributing much and all I'm doing is trying to take in lots of information, time can pass by very slowly.

This week I finally was able to write some code and fix some lingering bugs in our web application. We are using AngularJS which is new to me, and our code base is quite large, so I was a little on the slow side. By the end of the week I felt like I finally had a good idea how the application was designed and where to go to modify the app's functionality. My speed should start improving.

It really felt good to write code again. I really enjoy teaching, but I seldom have time to contribute to software that others are using. It's amazing to me how quickly time goes when I am programming and getting things to work right. Each bug fix makes me say "YES!" inside (and sometimes outside!).

Saturday, May 24, 2014

A Tale of Two Cities: Searcy and Boulder

I've just finished the first week of my summer position as a software developer at Flatirons Solutions in Boulder, Colorado. My family and I drove up from Arkansas on Sunday, and I started Monday morning. Everyone at Flatirons has been very friendly and helpful, but it's been a sharp learning curve getting up-to-speed. I spent the entire week getting familiar with the system I'll be working on, reading system documentation, learning about 10 new technologies or tools, and figuring out how things are done at Flatirons. I have yet to contribute anything, but hopefully next week that will change.

Our living situation has also been a bit of an adjustment. Moving from a house to a small two bedroom apartment has had its challenges. We got spoiled living in Arkansas in our own home where our kids could be loud and play in our back yard. Now we have to be quiet or anger the neighbors below, and there's no yard to speak of. We also have a very busy street just 30 feet away from our front door, so we've had to get used to much more noise and foot traffic.

What has been most notable is the change in culture from Searcy to Boulder. Both cities have that college town feel, but that is about where the similarities end. Searcy is small (population 20K), conservative, largely Christian, and enjoys a low cost of living. Boulder is large (100K), very liberal (some call it "the San Francisco of Colorado"), religiously diverse, and ridiculously expensive (49% higher than Searcy). Searcy is hot and humid, Boulder is a mile above the ocean and very dry.

Boulderites are a very healthy bunch whereas Searcians like their fried Southern cuisine (Colorado has the lowest state obesity rate, and Arkansas has the 7th highest). In Searcy you are lucky to see one person a day riding their bike; you will easily run over a biker in Boulder if you blink. I've never seen so many people walking, running, or biking.

Boulder is also well-known for their environmentalism. You will get a "look" at the grocery store if you show up without your own bags, and many vehicles are gas-sippers. The Searcy grocer will double-bag practically everything, and SUVs and enormous trucks dominate the roads in Searcy.

What I really enjoy about Boulder is the beauty. It lies just east of the Flatirons, a beautiful range of mountains that can be seen from our apartment. We went hiking this morning beginning at Chautauqua Park and soaked-up the beauty that God has created. Searcy has its beautiful places as well, but it's not quite in the same league.

One week down, nine more to go!

Wednesday, May 14, 2014

Off to Boulder

This summer I'll be working at Flatirons Solutions in Boulder, Colorado. I'm very excited by this opportunity to take a break from doing research and spend some time developing software and learning new tools and techniques that I can incorporate back into the classroom. I grew up in Denver, so I'll have the opportunity to see lots of old friends; my kids will get to see where their dad grew up.

I hope to blog some about my experiences in Boulder, so stay tuned.