This demo will show you how to write a simple XNA application that reads hand motion from the Kinect v2. The Kinect sensor can detect motion for your entire body, but here I'll focus on just detecting hand motion and whether the hand is open (all fingers out) or closed (in a fist) as shown in the screenshot below.
Prerequisites
You must have the Kinect for Windows 2 correctly installed along with the SDK. There are plenty of online tutorials showing you how to program with the older Kinect; this is for the latest version.
See the Prerequisites section from my previous post on installing the necessary software to code this demo using Visual Studio 2013.
Create an XNA Project
First create an XNA project by selecting File → Project... from the menu. Then select XNA Game Studio 4.0 template under Visual C# and select Windows Game (2.0). Name the project KinectMotionDemo.
Add Kinect Reference
Right-click the project in the Solution Explorer and from the context menu select Add → Reference.... Type kinect in the dialog box's search box, and check the Microsoft.Kinect reference. Then press OK. You should now see Microsoft.Kinect among the project's References in the Solution Explorer.
Inside the KinectMotionDemo namespace and immediately after the Game1 class, create a class that represents a Hand. We will keep track of whether the hand is left or right, open or closed, and its location to be displayed on the screen.
Note that a Microsoft.Kinect.Body represents a person's body, and Kinect can track up to six people at the same time.
Add some code in the Initialize method to initialize the sensor, create arrays large enough to track up to six people, and add an event listener so we'll know when body sensor data is available.
Now write the event listener for the body frame reader that will obtain the sensor data for the left and right hands of all the bodies that are being tracked. This method will call UpdateHandInfo, a method that will determine if the hand is open or closed and determine where it is located in depth space so it can be accurately mapped to an (x,y) location on the screen.
Find four different images that you would like to display for your left and right hands when they are open or closed. I used images that look a lot like hands, but you can be more creative. Make sure that you use only letters, numbers, and underscores in your filenames because these will be converted into variable names!
Add the images to the KinectMotionDemoContent project in the Solution Explorer by right-clicking on the KinectMotionDemoContent project and selecting Add → Existing Item.... An open dialog box will appear. Select the four PNG images you want to use and press OK. You should now see all four images in your KinectMotionDemoContent project as pictured below.
Now load the PNG images in the LoadContent method.
Press Ctrl-F5 to build and run the program. Stand in front of your Kinect, and you should see the PNG images move as you move your hands. Try opening and closing your hands to see the open/close images being displayed. If you have a friend nearby, ask them to join you so you can see four hands moving about the screen.
Problems?
When I first tried to build and run my program, I got the following error message:
The primary reference "Microsoft.Kinect, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL" could not be resolved because it was built against the ".NETFramework,Version=v4.5" framework. This is a higher version than the currently targeted framework ".NETFramework,Version=b4.0".
To fix this problem, I closed the project in Visual Studio and opened the project's .csproj file in a text editor and changed the following line:
Eight students from our Computer Science department met with about 50 fifth graders at Kensett Elementary on Friday, Dec 5, to participate in the Hour of Code. As you can see from the pictures, the kids were having a blast! Most of them ran through the Angry Birds tutorial, and a few tried the Frozen and Flappy Bird tutorials. When the hour was over, many of them got really excited when I told them they could keep working on the tutorials when they got home... just open a web browser and go to code.org!
Over the Thanksgiving break I managed to get my new Kinect for Windows 2 working with XNA. I couldn't find any code samples online using XNA with the new Kinect SDK, so hopefully this will help out others who are trying to do something similar.
This demo will show how to display the Kinect's video feed which is provided by its 1080p color camera. This is similar to the Color Basics-WPF C# Sample provided in the Kinect SDK 2 except that it is tailored for XNA.
Prerequisites
You must have the Kinect for Windows 2 correctly installed along with the SDK. There are plenty of online tutorials showing you how to program with the older Kinect; this is for the latest version.
Microsoft is no longer maintaining XNA, but but you can still use it in Visual Studio 2013. There are two different ways to get XNA working with VS2013:
Quick Way
Install XNA 4.0 Refresh for VS 2013. The zip file contains four components that you will need to install in succession.
Long Way
If you have VS 2010 laying around, this method will also work although it takes much longer. I found this method necessary when installing XNA in a lab setting. Using the Quick Way would not work for users who were not administrators.
Install Visual Studio 2010. Any edition will work.
Install XNA 4.0 Refresh for VS 2013.
After you have downloaded the zip file, extract it and run XNA Game Studio 4.0.vsix
If you want to skip both these options, you might try MonoGame.
Just be aware that it does not have a content pipeline converter (software that converts content like sound files into xnb files). I have not tried MonoGame with Kinect, but I don't see any reason why it wouldn't work.
Create an XNA Project
First create an XNA project by selecting File → Project... from the menu. Then select XNA Game Studio 4.0 template under Visual C# and select Windows Game (2.0). Name the project KinectVideoXna.
Add Kinect Reference
Right-click the project in the Solution Explorer and from the context menu select Add → Reference.... Type kinect in the dialog box's search box, and check the Microsoft.Kinect reference. Then press OK. You should now see Microsoft.Kinect among the project's References in the Solution Explorer.
Press Ctrl-F5 to build and run the program. You should see color video of whatever your Kinect is pointed at.
Problems?
When I first tried to build and run my program, I got the following error message:
The primary reference "Microsoft.Kinect, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL" could not be resolved because it was built against the ".NETFramework,Version=v4.5" framework. This is a higher version than the currently targeted framework ".NETFramework,Version=b4.0".
To fix this problem, I closed the project in Visual Studio and opened the project's .csproj file in a text editor and changed the following line:
We're half way through the fall semester, and the spring semester will be here before we know it. Every spring I teach an upper-level Web Development 2 elective which builds on the Web Dev 1 course that I teach every semester. The Web Dev 1 course covers the fundamentals: HTTP, HTML, CSS, JavaScript, Ajax, and PHP.
The goal of Web Dev 2 is to give students more breadth and depth in web development, focusing on both client and server-side technologies. I try to choose technologies that are widely used so my students will be more marketable after graduation, and in the past I've taught a variety of technologies: advanced JavaScript, jQuery, Java servlets and JSPs, ASP.NET Web Forms, ASP.NET MVC, and creating web services with ASP.NET and JAX-RS. Of course you can't cover everything in 16 weeks, so I have to put a lot of thought into what can be adequately covered in a limited amount of time.
This summer I did some development work at Flatirons Solutions using AngularJS and web services with Spring. It got me wondering if my Web Dev 2 course should focus more on JavaScript frameworks. One of my developer friends this summer tried to convince me that JavaScript was also taking over the server and that I should be teaching Node.js instead of Java.
It can be a struggle for computing professors to ensure our courses remain relevant when technologies are always changing. Fortunately I've got a lot of Facebook friends who are developers and Harding CS alumni, so I decided to ask their opinion:
If you were in college and could take an advanced Web Development course, what topics would you like it to cover?
This elicited quite a few responses which I'll summarize here:
JavaScript frameworks like AngularJS, Ember, Knockout
ASP.NET MVC and Web API
Node.js
Python and Django
Ruby on Rails
LESS
Web services
Web security
Single Page Applications (SPAs)
Automated web testing, A/B testing, UI testing
Understanding HTTP
Picking the right data store (relational, NoSQL, Map/Reduce, etc.)
Caching and latency issues
Teams that implement the same project on different platforms
Git
Web architecture - focusing less on development and more on architecture
There may have been a few things I left off, but these received the most mentions. (Thank you for your input, guys!)
I'm still making up my mind, but this is what I'm leaning toward teaching this spring:
JavaScript - The more JavaScript you are exposed to, the better. I'd like my students to be familiar with a number of advanced JavaScript features and some popular libraries like Underscore.js.
Node.js and Express - Although I have little experience with Node, I think it would be helpful for my students to apply their JavaScript skills on the server as well as the client. I will probably tie in MongoDB which will give them experience with a NoSQL database. We will likely create a web service and consume it with an app written with...
AngularJS - Very popular JavaScript framework with strong job demand. Plus I got some experience with it this summer. We will write unit tests using Jasmine and use Karma for integration testing.
My students will work in pairs on their projects and use GitHub to house their code like they do in my GUI course. We'll use the WebStorm IDE which I came to love this summer. I'll inject other topics like security where they are applicable.
A weakness of this plan is that my students will not get to use Java or ASP.NET which are very popular server-side technologies. However, they will leave this class fluent in JavaScript.
Today is my last day at Flatirons Solutions. I've really enjoyed these eleven weeks and learned a ton. My web development courses, in particular, will definitely benefit from the exposure I received to some new technologies that I'll be integrating into those courses. I'm very thankful to George and Paula for giving me the opportunity to spend my summer here, and I hope that I made some major contributions.
We're now entering our sixth week living in Boulder. My family is getting more accustomed to apartment living. It's ironic though how we thought we were escaping the Arkansas heat only to live in an apartment lacking air conditioning! At least the temperature drops each night.
We had a little scare a few weeks ago. Becky had developed a large lump on her thyroid, and after meeting with a surgeon, she was strongly encouraged to have it removed in case it was cancerous. My sister (who is now eligible for sainthood) flew out here from Chicago so she could watch the kids while Becky recovered from surgery. The surgery went well, and praise God the lump was not cancerous! Becky is still healing from the surgery, but she is doing really well. The kids were pretty oblivious to the whole thing since Aunt Sass kept them busy with swimming and museums and making pizza. Have I mentioned how awesome my sister is?
Since the surgery, Becky and the kids have been occupying themselves with all kinds of activities like tennis lessons, swimming lessons, and now karate classes while I toil away at Flatirons each day. When I arrive home in the evenings we often eat and go out to a park or discover some new part of Boulder. On the weekends we have visited Red Rocks, Estes Park, and a few other places. In the photo below we hiked around Bear Lake at the Rocky Mountain National Park. As you can tell from our Chacos, we were not expecting to see snow!
We've gotten to see a number of old Colorado friends which has made our time out here very meaningful. Some of our best friends from Searcy also stopped by to visit for a couple of days, and we were able to dine at Casa Bonita and see a Rockies game with them.
Next week my brother and his wife will be visiting from Texas, and some more good friends from Searcy will be coming up July 4th weekend. Becky has some college friends coming up to visit the last week of July. Lots to look forward to!
I'll conclude with a note of thankfulness for the Boulder Valley Church of Christ. They have really taken us in, and we are so thankful for the many ways they supported us during Becky's surgery. One of the ladies watched the kids for us before Aunt Sara arrived, and one of the elders came to the hospital to pray with us immediately before the surgery. These are people who truly love the Lord.
I'm also thankful for the incredible VBS they put on. You would not believe how much effort they expended to entertain and teach about 50 children about God's love for a full week. Ethan and Braden absolutely loved going to VBS each day. I think it's really great that the body of Christ can be found nearly everywhere you go.
Last week at Flatirons was admittedly difficult. I really like to learn new stuff, but if I'm not contributing much and all I'm doing is trying to take in lots of information, time can pass by very slowly.
This week I finally was able to write some code and fix some lingering bugs in our web application. We are using AngularJS which is new to me, and our code base is quite large, so I was a little on the slow side. By the end of the week I felt like I finally had a good idea how the application was designed and where to go to modify the app's functionality. My speed should start improving.
It really felt good to write code again. I really enjoy teaching, but I seldom have time to contribute to software that others are using. It's amazing to me how quickly time goes when I am programming and getting things to work right. Each bug fix makes me say "YES!" inside (and sometimes outside!).
I've just finished the first week of my summer position as a software developer at Flatirons Solutions in Boulder, Colorado. My family and I drove up from Arkansas on Sunday, and I started Monday morning. Everyone at Flatirons has been very friendly and helpful, but it's been a sharp learning curve getting up-to-speed. I spent the entire week getting familiar with the system I'll be working on, reading system documentation, learning about 10 new technologies or tools, and figuring out how things are done at Flatirons. I have yet to contribute anything, but hopefully next week that will change.
Our living situation has also been a bit of an adjustment. Moving from a house to a small two bedroom apartment has had its challenges. We got spoiled living in Arkansas in our own home where our kids could be loud and play in our back yard. Now we have to be quiet or anger the neighbors below, and there's no yard to speak of. We also have a very busy street just 30 feet away from our front door, so we've had to get used to much more noise and foot traffic.
What has been most notable is the change in culture from Searcy to Boulder. Both cities have that college town feel, but that is about where the similarities end. Searcy is small (population 20K), conservative, largely Christian, and enjoys a low cost of living. Boulder is large (100K), very liberal (some call it "the San Francisco of Colorado"), religiously diverse, and ridiculously expensive (49% higher than Searcy). Searcy is hot and humid, Boulder is a mile above the ocean and very dry.
Boulderites are a very healthy bunch whereas Searcians like their fried Southern cuisine (Colorado has the lowest state obesity rate, and Arkansas has the 7th highest). In Searcy you are lucky to see one person a day riding their bike; you will easily run over a biker in Boulder if you blink. I've never seen so many people walking, running, or biking.
Boulder is also well-known for their environmentalism. You will get a "look" at the grocery store if you show up without your own bags, and many vehicles are gas-sippers. The Searcy grocer will double-bag practically everything, and SUVs and enormous trucks dominate the roads in Searcy.
What I really enjoy about Boulder is the beauty. It lies just east of the Flatirons, a beautiful range of mountains that can be seen from our apartment. We went hiking this morning beginning at Chautauqua Park and soaked-up the beauty that God has created. Searcy has its beautiful places as well, but it's not quite in the same league.
This summer I'll be working at Flatirons Solutions in Boulder, Colorado. I'm very excited by this opportunity to take a break from doing research and spend some time developing software and learning new tools and techniques that I can incorporate back into the classroom. I grew up in Denver, so I'll have the opportunity to see lots of old friends; my kids will get to see where their dad grew up.
I hope to blog some about my experiences in Boulder, so stay tuned.
As I stated in an earlier post, I taught an Android course this semester, and one team created an app called HU Pal that gives students access to their class schedule and chapel attendance information. This info is normally locked in an online system called Pipeline which requires users to login with a username and password. Once you have given HU Pal your username and password, it logs into Pipeline automatically and scrapes the information from a couple of web pages.
Brent Ward, one of the developers of the HU Pal app, wrote a document detailing how they created their app to login to Pipeline and screen-scrape personal information from web pages using jsoup. You can download the document here. Although the information in the document is specific to Pipeline, the techniques they describe can be generalized to any online system that requires user authentication.
This semester I taught an Android App Development course for upper-level computer science students. This is the second time I've taught this class. Nineteen students were placed into six teams. Each team came up with their own idea for an Android app and created a final beta that was evaluated by their peers:
CrashPad: Rent an apartment or check your apartments that are being rented.
EatSmart: Track your dietary intake in the Harding Cafeteria.
HevaHavoc: Unique paddle game that uses the accelerometer to control the paddles.
HU Pal: See your class schedule, check your chapel attendance, and show a campus map.
Puzzle 15: Customize the background of a 15 puzzle game with any photo.
Pentago: Play the game of Pentago against the computer or another human.
None of these apps are currently available to the public, but a few students said they were going to work on them some more and eventually make them available on the Android Play store.
Last night we had an Android Showcase in the Rhodes Field House along with the CS and Engineering Capstone courses. Thane, David, and Brent (pictured below) were given the award for the app receiving the best peer evaluations (HU Pal).
Overall I am quite pleased with how the projects turned out. I think most of my students would say they learned a lot about developing a successful app, even if their app fell short of what they hoped it would be. Most of the teams worked well together although there was a little friction which is to be expected. This is the only class where I force my students to work in teams, but I think the experience will prove valuable to them as they are forced to work in teams in their future professions.
I attended SIGCSE 2014 in Atlanta last week. It's the fifth Symposium in the past six years I've been able to attend. There were over 1200 attendees (mainly computer science educators) at the Symposium this year.
Getting females engaged in computing remained a huge focus as usual. Interestingly enough, I would gauge that close to 50% of the conference participants were females (education is generally not short on females). There was also a big emphasis on getting computing into K-12. Hadi Partovi of code.org (you know, the Hour of Code guys) gave a keynote one morning on this topic which really pumped up the audience.
Big Data and Data Science seemed to be two important and related topics that received quite a lot of attention. Scanning the conference program, there were two papers on Data Science, and two posters and two papers about Big Data. I'm surprised no one offered a workshop on either topic.
One topic that seemed less important was mobile programming. There was one paper on using Android projects in CS1, and a workshop on App Inventor, but this topic is not nearly as hot as it was a few years ago when I offered an Android workshop to a packed room of CS educators in Dallas. I think it's because mobile programming is so ubiquitous today.
On Friday evening I manned a poster entitled Resources for Teaching Web Science to Computer Science Undergraduates by Michael Nelson and myself. (You can find the resources here.) I was somewhat surprised that only two of the twenty or so individuals I spoke to that evening knew anything about web science. One individual had been teaching a similar course but didn't know it could be called "web science". It was nice to see a lot of interest in the topic.
The next morning was my favorite session: Nifty Assignments. But this time I was one of the presenters! I introduced Schelling's Model of Segregation using some history of the Little Rock Nine as back-drop. The other presentations were pretty fantastic.
One thing I love about SIGCSE is being immersed into new ideas that I will use to make my teaching better. The thing I don't like is the feeling that I'm not teaching as well as some of my peers! Equally good motivation to keep on improving.
This is a huge weekend in the NFL. The AFC Championship features the Denver Broncos (my favorite team) and the New England Patriots, headed by Peyton Manning and Tom Brady, respectively.
Manning is an incredible QB. But he's also very talented when it comes to comedy. I would argue he is one of the best comedic commercial actors in the NFL. Almost everyone has seen his MasterCard commercials. He's also made satirical movie commercials, appeared on SNL, and has some skits with his brother Eli that have gone viral like Football on Your Phone and Football Cops. It's hard not to like a guy that is so talented and hilarious at the same time.
Brady is of course also an incredible QB with three Super Bowl wins to back it up. But where Manning gets all the love, Brady is the most disliked QB in the NFL. I've never been a Brady fan either; I've always thought he was full of himself.
These past few months, however, I've come to like Brady just a little bit more. OK, he was my fantasy team QB which helps some. But Brady also stared in a couple of hilarious commercials for Under Armour over the past couple of years that I just saw recently. The first is Tom Brady's Wicked Accent which features an angry Brady smacking a stand-up cutout of himself. The follow-up is Tom Brady's Best Friend where he apologizes for his behavior in the first commercial and then loses his temper once again. He's also pretty hilarious when searching for someone to high-five.
Manning is still on top when it comes to comedy. But Brady should have at least a little of our comedic respect.
DISCLAIMER: If the Broncos lose this weekend, I may delete this blog post out of disgust for Brady.
The Spring 2014 semester has begun. The cold and lifeless campus is now full of energy and life. I'm teaching four courses: Programming 2, Internet Development 1, Internet Development 2, and Android Application Development. Most every course in our department is over capacity which reminds of me of how it was in the late 1990s. Hopefully we won't end up crashing like we did in the early 2000s.
As we begin a new year, there is unfortunately some sadness. Just a few days ago, one of our undergraduates was killed in an auto accident while heading back to school. Kailey Massey was just 20, and she apparently ran into the back of an 18-wheeler on I-30. There will be a memorial service for her on Wednesday in chapel.
This comes on the heals of another student's death. Harding grad student Lauren Bump, age 24, was
murdered in a San Antonio park near her home over the winter break. There was a memorial service for her on Sunday.
President McLarty did a great job addressing these two tragic events in chapel yesterday morning. Both of these girls had a genuine love for God and let it show in their lives (see Lauren's last blog post entitled Success). They will truly be missed.