Inteview Ideas for a Hands on Programmer Test

posted Thursday, August 26, 2004 3:11 PM

As I stated recently we started doing a hands on programming test as part of our interview process.  We've going through a handful of candidates and haven't seen anyone do as well as we'd like nor anticipated.  We're trying to come up with a practical exercise that someone can finish in 1-1.5 hours.  Rather then present what we're currently doing I'd like to open it up to my loyal readers (you didn't know you were a loyal reader did you). 

The Challenge
Construct a coding exercise for an interview candidate that can be completed in 1 to 1.5 hours.  The basic things we're trying to make sure of during our test is that the candidate:

  • Can use VS.NET comfortably
  • Can access a database (SELECT data using a stored procedure)
  • Can develop a web form for collecting data with simple validation
  • Can insert data into a database (INSERT stored procedure)
  • Can display data using a DataGrid, DataList, or Repeater

Are these fair expectations for .NET interview candidates?  Are there other things you'd look for?

23 Comments

# re: Inteview Ideas for a Hands on Programmer Test

Thursday, August 26, 2004 5:40 AM by Darrell    
Well, if I had to do that in 1.5 hours, I probably would use a DataSet, to make sure that I could focus time on making sure it all worked in the end. :)

If I finished early, I might begin exploring other designs.

# re: Inteview Ideas for a Hands on Programmer Test

Thursday, August 26, 2004 5:42 AM by Darrell    
Need to add:

My deliverable would look decent (I'm not great with GUIs) and would be as robust as possible, regardless of whether I used a DataSet or custom business entities. If you want the candidate to show off other attributes (such as design patterns, etc.) then you might want to make the goal clear to the person up-front.

Let us know how it goes!

# re: Inteview Ideas for a Hands on Programmer Test

Thursday, August 26, 2004 5:47 AM by Ivaylo    
Steve,
Would you consider a candidate that has no experience with DataGrid, DataList, or Repeater, but have enough experience to build his own versions of those (read: reusable components with the same level of complexity) if he has to.

# re: Inteview Ideas for a Hands on Programmer Test

Thursday, August 26, 2004 5:50 AM by Jake    
These would work for an ASP.Net developer ;)

There are .Net developers who don't use web forms, data grids, and even databases...

# re: Inteview Ideas for a Hands on Programmer Test

Thursday, August 26, 2004 6:17 AM by Steve Hebert    
If I had to do it, I’d look at each item as a directive. I’d complete each item in the shortest amount of time using datasets. If you want business objects (as you blogged earlier), ask for them. The test by itself implies (right or wrong) that you are testing for resourcefulness, the ability to follow directions and code quality.

My thought is that time limits on tests are designed to make slow people feel better about themselves. If I can’t finish in ½ the time, I’m disappointed in myself – the Microsoft cert tests are a good example of that.

# re: Inteview Ideas for a Hands on Programmer Test

Thursday, August 26, 2004 6:20 AM by Jimmy Nilsson    
Hi Steve,

Very interesting blog post!

I might be old-fashioned, but I would go for more conceptual tests. For example, learning to use VS.NET I think is pretty simple for someone that understands, again for example, OO. I'm not so sure about the opposite.

Regarding relational databases, I would check if they understand set-based processing. Regarding UIs... Perhaps check that they understand event-based programming. And so on.
:-)

Assume a very skilled and experienced programmer, but from another platform such as Java, tries your test. That person will probably fail, but in just a few weeks that person could be just great.
:-)

Best Regards,
Jimmy
www.jnsk.se/weblog/
###

# re: Inteview Ideas for a Hands on Programmer Test

Thursday, August 26, 2004 6:26 AM by Steve    
I've switched my thinking regarding business objects / datasets, I don't care what the person uses I just want to see that they know what they're doing. Jimmy, I agree 100% regarding your thoughts. If we had a Java guy come in we wouldn't give him our current test. We only give it to people who claim to have good ASP.NET experience. If we were looking to fill different positions I'd spend more time on conceptual things like OO, patterns, and etc, however we currently need people who can come in and hit the ground running on a couple ASP.NET projects. We aren't finding a lot of people...

# re: Inteview Ideas for a Hands on Programmer Test

Thursday, August 26, 2004 6:30 AM by Mark Jordan    
Just a thought... if I went into a job interview and was expected to spend 1.5 hours to code a solution, I would probably think twice about working for that company. First of all, you're using a foreign computer. Secondly, as we all should know, time constraints will, by default, render a less-than-quality product. Granted, it is only a test to see if abilities and competancies of the individual. Nevertheless, I wouldn't want to do that test just to get a job.

Now, however, at the risk of contradicting what I just said above... if I were asked to produce such a solution on my own time (ie. in my home, with my resources, etc.), I would consider it a definite opportunity. Some might argue that if I were able to take it home, then I would be able have free reign to the internet and other resource to complete it. Isn't that the point, though? Aren't these resource that you would have at your disposal if you were actually hired for that position?

Of course, that would be difficult if the individual did not have Visual Studio.NET, at the very least, at home (yes... there are some that don't do computing at home, even though they do at work... I'm not one of those as my wife can attest 8^). So, maybe, give them a choice. I would appreciate the choice if I was considering that job.

# re: Inteview Ideas for a Hands on Programmer Test

Thursday, August 26, 2004 6:31 AM by Jimmy Nilsson    
Ah, sorry Steve. I misunderstood you.
:-)

Best Regards,
Jimmy
www.jnsk.se/weblog/
###

# re: Inteview Ideas for a Hands on Programmer Test

Thursday, August 26, 2004 6:34 AM by Eric Wise    
Hey Steve, I actually have similar plans for this. My test is a little different though:

1. Load data into a datagrid using given SELECT stored proc.

2. Insert Data into a database using given INSERT stored proc with parameters

3. Analyze Data from a given SELECT stored proc, if you find a certain value x, throw a custom exception called "MySelectException". Place the primary key of the record with value x into a property of the exception called "ErrorRowPK"

1 and 2 just make sure they can work with data.

3 is fun because inheriting the exception class to generate a custom exception type is very easy (finding code sample online is easy as well, I do give them internet access), plus it lets you know that they kind of understand inheritance. Adding the property shows that they understand how to populate a property and how you can extend an inherited object.

These tasks should be completable in an hour, especially since the sql is already written and the connection string is configured.

# re: Inteview Ideas for a Hands on Programmer Test

Thursday, August 26, 2004 6:39 AM by Steve    
Mark, We do allow the candidates to use MSDN while they're working on the "exercise", so they're not totally in the dark. The problem I have with having someone doing it at home is you can't really tell if they know they're stuff or not. They could have the .NET guru next store work on it with them, or they might take a week to complete it. I suppose if we told them a day or two before that could work.

# re: Inteview Ideas for a Hands on Programmer Test

Thursday, August 26, 2004 8:06 AM by Mark Jordan    
Set a time limit to a number of days. Then review the solution with them... you'll know in a heartbeat if they did it themselves or with another person. Have them explain their choices and design, any patterns they've used, why they "did it this way vs. that way", etc. As far as I'm concerned, even if they did get help from someone, if they can answer those questions in a legible, concise, logical, and confident manner, then they obviously are knowledgable about the subject matter questioned, and hence would be qualified.

The major problem, I guess, that I have with the 1.5 hour thing is the lack of design, architecture, and coding standards utilized. Given 1.5 hours to pump out a solution, I would just hack it together... and my response to my interviewer as to "why such the poor quality?" "Sir/Madam... quality can only be provided given the approriate amount of time and monetary budget." I would rather see a well thought out, designed, and well-coded solution as an interviewer... that shows the REAL intellect of the candidate.

# re: Inteview Ideas for a Hands on Programmer Test

Thursday, August 26, 2004 8:10 AM by Steve    
Good points! I think you might be winning me over to the take home test idea....

# Re: Inteview Ideas for a Hands on Programmer Test

Thursday, August 26, 2004 8:50 AM by mats.helander@matssoft.com    
Hi Steve!

My suggestion for an additional point in the test would be: Write a couple of NUnit tests for the business logic.

If they don't already use NUnit, allow them 10 minutes extra and show them the ropes. That way, even if you don't hire them you will have changed their lives for the better!

Oh, and another implication is that in order to test with NUnit, they would actually have to separate out their business logic, which I'm guessing is one of the things you'd be keeping an eye out for...

/Mats

# re: Inteview Ideas for a Hands on Programmer Test

Thursday, August 26, 2004 10:24 AM by Ron Green    
Hey, I think I might be able to pass this interview. ;-)
Now, if you were only in Phoenix, where I really would like to live, I'd be bugging you for an interview.

# re: Inteview Ideas for a Hands on Programmer Test

Thursday, August 26, 2004 10:52 AM by Steve    
I actually thought about putting NUnit on the desktop to see if anyone used it. I'd love to add it to the test but given the fact we're not finding many people who are doing too great on what we have thus far I think I'd just be setting myself up for disappointment.

Ron, if you change your mind about where you'd really like to live drop me a line ;-)

# re: Inteview Ideas for a Hands on Programmer Test

Thursday, August 26, 2004 2:04 PM by Steve Hebert    
The more I read the thoughts on code testing an applicant, the more I like the idea of having them walk in with code they've written. Preferably just send in code ahead of time. I think I'd get more out of grilling the applicant on every angle to their approach and seeing how much "deep thinking" goes into their coding. The applicant is sending in a piece of code they are willing to say proudly "I wrote this" and stand behind it. It's too easy to say "I cut corners here or there" on a code test.

"Deep Code Thoughts" - sounds like a geeky SNL skit.

btw, do you have the job posted on the net?

# Friday Post: Deep Coding Thoughts

Thursday, August 26, 2004 2:29 PM by Steve Hebert's Development Blog    
Friday Post: Deep Coding Thoughts

# re: Inteview Ideas for a Hands on Programmer Test

Friday, August 27, 2004 12:36 AM by Michael Knox    
Years ago I used to give a test in a job interview that constisted of "here is a tablet, write a <insert program type here>." I always told the person that I did not care about syntax and that they were free to ask me questions, since they would in real life as well. The ones who started putting DataControls on a Windows form didn't get the job, the ones who talked about higher level things and more abstract ideas did. The test never failed to find _programmers_, as opposed to just IT majors.

More recently I have started asking more non-technical questions about how a person writes a program or even more vague questions that give me a sense of a person's experience, and usually a good indicator of someone's ability to move ahead with new technology as it comes out. It has only failed me once and I ended up giving that guy glowing recommendations for Classic ASP positions.

Anyone can remember terminology and write basic programs, but not all of those people can really program. I myself usually keep a reference book or the help files handy.

My two cents.

# re: Inteview Ideas for a Hands on Programmer Test

Friday, August 27, 2004 10:20 AM by John Rusk    
Steve,

I think it could be useful if, at the end of the test, you asked the candidate, "What would you have done differently if you'd had more time?". I think that addresses some of the concerns about candiates feeling obliged to use "quick and dirty" solutions just to meet the time limit. If a candidate felt that way, that should become apparent in the answer to this questions.

With any luck, the question will also flush out candiates who, given enough time, would have built an _over_ complicated solution.

Some other thoughts:

- Using the Northwind database might be a good idea, since many candidates will already be familiar with it. (Unless you want to test their ability to understand a data model that they _haven't_ seen before.)
- I agree with you that the test should be at your premises, not the candidates home. I'd give them access to MSDN but not the internet.
- In a perfect world, you might have several different coding exercises, and allow the candidate to pick one. Since, as you've said, your proposed test will only be useful for those with ASP.NET experience.

By the way, we tried another approach at one of my former jobs. I constructed some code, about 1 page worth, which had a number of errors. Some were genuine bugs, while others were "style" issues (e.g. the code needed refactoring to remove duplication, there were no comments, there was a hard-coded string that tied the code to a particular version of Windows, etc). The open-ended question was simply, "How might this code be different if you had written it?". I think it worked reasonably well, in terms of prompting the candiates to think about, and talk about, code.


# re: Inteview Ideas for a Hands on Programmer Test

Tuesday, August 31, 2004 4:34 AM by Sean Chase    
I like it Steve. I'd probably add one question that seems to dictate added depth (IMHO). Have them pull out a value from a control (or some similar operation) in the ItemDataBound event. I'd probably want to discuss the Page lifecycle and state issues as well.

# re: Inteview Ideas for a Hands on Programmer Test

Wednesday, September 01, 2004 2:56 AM by Jeff Gonzalez    
Internet access is a must I would think. I know I find more information via google and google groups than anywhere else. MSDN documentation is good, but its not often the best.

The test should be done in office though, just to keep people honest. The other thing that we are doing now is having people come back a day or 2 later to do a peer review. In this scenario a couple of people will go over the test with the candidate and ask them why they did x or why they did y a particular way.

Ill keep you posted on how that turns out.

# re: Inteview Ideas for a Hands on Programmer Test

Thursday, September 02, 2004 1:03 AM by Darran Hurd    
I still would like to think that people may have an ability to research solutions to a logical end, or know where to go to find something better than what was done before.

.NET is evolving constantly; people get into Libraries/new coding methods and expand on what they can do with them creatively. I am not a fan of canned interview tasks like this, because it may be turned into a test of "are you like us" over what new can you offer to the company.

Coding "teams" need to reference each other and new tasks are bound by the creativity of the management. Should the management style be ever searching for "cool ways of doing things" or "best practices", then you are not able to remain static.

From a developer's creativity standpoint, we bust our humps to learn all sorts of extra technology and technical skills to advance our career. Why else would we basically train ourselves in server management and advanced DBA skills just to return a Data Grid/Data List? One answer is that .NET affords us a new set of skills and a wealth of power not seen before. When before was it possible to address threads, memory leaks, garbage collection, database integration, transformation, and interoperability in a few lines of code and on the same page?

If I am not thinking big, and improving internally at the same time as externally, then why would you want someone to be there? Check the definition of “intrinsic motivation“ and see where that leads to as a means of overcoming present shortcomings in technical knowledge or previous training/education.

Post a Comment

 
 
Prove you're not a spammer: 
6 + 6 =