June 2003 - Posts
Today I got a call from a client regarding a bug in a system I had
developed a couple months ago. Unfortunetely the system was developed
prior to when I began using Test Driven Development (TDD) on my projects.
I quickly cracked open the browser and verified the bug on my local
system. After finding where the bug was occuring I began to try a couple
of "Obvious Implementations" with no luck. I quickly realized that my
method for testing on whether I had fixed the problem was ridiculous. I
was testing it in the browser and removing records after each test from the
database (so I could repeat the test again). It was obvious that I could
have done this for hours without making a whole lot of progress. Instead
of continuing this process I decided to create a test project and start some TDD
on the functions. I quickly implemented some logic to verify that my
function was doing what I expected. A big fat red bar verified that I
had not done a very good job of removing the errors in my
logic. I quickly altered some of the logic and basked in the glory of the
GREEN. I realized there were a couple of others things that I hadn't
accounted for in the logic so I continued in a test/red/green/refactor cycle
until I was confident I had uncovered all the potential pitfalls of the
method.
Although the original development of the project didn't utlize TDD I was able
to quickly bring it into my project and feel immediate benefits from it.
The more TDD I do the more I can't imagine doing things any other way. I'm
sure as I continue my journey I'll find ways of improving the way I do TDD and
maybe I'll find something new to take its place. For the time being I'll
be happy admiring my green bars!
Over the past couple of months I've been working on a pretty good size .NET project that has used some agile development practices. I've been doing Test Driven Development using NUnit, we have a continuos integration environment (NAnt), and we've been following some of the patterns and practices recommended by many in the "Agile Movement".
Overall my experience has been very good. I've run into some issues with doing TDD with a database but that is mostly due to our process which I hope to change on our next project. This has been the first project that I've really followed the TDD methodology. I have worked on several other projects prior to this one that I used NUnit on but it was really in a "when time permits" scenario.
I have gone through a bit of a transformation over the course of my current project. In the beginning of the project I was writing my tests only after I had written my objects, controls, and UI. I found the writing of the unit tests to be very burdensome, and quite frankly a total pain in the ass to write when I was following this method. After completing a couple tasks and getting that "yuck" feeling about writing test classes I decided to change my ways. I had heard so much about the test first methodology that I had to at least give it a try.
I no longer suffer from the "what a pain in the ass" feeling while writing my tests. I've found that writing my tests first forces me to think through the problem more then just going off and writing the code for my objects. I also find I'm better able to identify areas in which my objects may break when writing tests first. I've begun to advocate the test first methodology pretty strongly within my team and will continue to advocate it on future projects. To help give myself a better overall grasp of TDD I'm about to start on Test Driven Development: By Example by Kent Beck. What I learn from Kent's book as well what I've just recently learned from reading Agile Software Development by Robert Martin will provide me with a solid foundation of information to use in the fight for changing how we do things around here. Wish me luck!
I recently was discussing some strategies for performing searches against database tables where a set of optional parameters can be supplied. The scenario I'm talking about is in regards to doing searches on a table based on a set of ID's within the table, not free text searches. The individual that I was discussing this with was not looking forward to writing the proc to perform the search becuase there was approximately 10 optional criteria. I proded a little to see what the problem was and found that he was planning on implementing the optional criteria by having a bunch of if/else if statements checking each of the criteria. Rather then going that route, I suggested an alternative technique.
Assume we have a table called Employee with the following fields:
EmployeeID int
DepartmentID int
ManagerID int
LocationID int
Now assume you want users of your application to be able to find an Employee by specifying any of the criteria they so desire. User1 may just want to find all employees who Joe Manager [ManagerID: 5] manages, User2 may want to find all employees who Joe Manager manages, but only if they are in the Technology Department [DepartmentID: 49]. Using the technique I shared with my co-worker would result in a proc that looks like this:
CREATE PROCEDURE ap_SearchEmployee
(
@EmployeeID int = null,
@DepartmentID int = null,
@ManagerID int = null,
@LocationID int = null
)
as
SET NOCOUNT ON
SELECT * FROM Employee e
WHERE
(@EmployeeID IS NULL OR e.EmployeeID = @EmployeeID)
AND (@DepartmentID IS NULL OR e.DepartmentID = @DepartmentID)
AND (@ManagerID IS NULL OR e.ManagerID = @ManagerID)
AND (@LocationID IS NULL OR e.LocationID = @LocationID)
The where clause checks each parameter to see if it is NULL which means we don't want to filter our results by that parameter. If the parameter is not null it then checks the fields value with that of the parameter. By using this technique my co-worker was able to remove the duplicate logic and nasty if/else blocks within his stored procedure which provided him with a much cleaner solution!
A couple of months ago I was part of a team that performed an analysis of a large VB6 client server application. Since that time I've thought a lot about the various options the client has for future development on the application. The application needs to go under a major re-architecture that will require a significant amount of the application to be seriously altered. Based on this I've thought a lot about how the VB6 app could begin the migration to .NET.
The business logic and data access within the application is currently interspersed throughout the entire application which make the app a maintenance nightmare. To help with its current issues the biz logic and data access will be one of the first things that is rewritten during the re-architecture of the application. As I see it their are a couple of different options for the re-write/re-architecture:
- Keep everything in VB6 :-(
- Rewrite the business logic and data access logic in .NET, expose the interfaces of the new components via web services, and consume the web services in the VB6 app.
- Rewrite the business logic and data access logic in .NET, create a .NET Adapter component (COM Callable Wrapper) that uses Remoting to call methods on the new biz/data access components.
I know there are some other options that have ran through my mind as well but they've run off into a corner of my brain that is locked so I'll stay with the above three for the time being. Also keep in mind these are more or less random thoughts and not detailed plans so I'm not even sure of their feasibility.
There are many advantages and disadvantages to each of the options above. By staying with an all VB6 environment the client can avoid the potential risks with .NET interoperability, as well as leverage the existing skills of their developers. The downsides of sticking with the all VB6 solution is its VB6 :-).
By going with one of the options that includes writing the business logic and data access logic in .NET the client can take advantage of all the benefits that .NET provides. Having the ability to work in an object oriented language is itself enough to make me think beginning the migration to .NET would be worthwhile. Additional benefits would be the ability to attract top talent for future development on the application, the benefits of future enhancement to the .NET Framework (don't think VB6 is getting too many), and the ability to move the app away from a technology that isn't particularly well suited for the requirements of the application. Overall I think the benefits of going the .NET route far outweigh the benefits of sticking with an all VB6 application. Some additional risks will arise if the .NET migration is begun but considering the high risks already associated with the application this seems minor.
Over the next couple of weeks I hope to play out in code some of the thoughts I have about where the application could move. Those working on the application have quite a challenge ahead of them and I'm hoping they begin to move in the .NET direction in case I join them!
Follow up question: What experiences are others having in a .NET / VB6 interoperability scenario? How are you handling the interop? Web Services, CCW, ...? Darrell, since you seem to be the only one how reads my blog entries do you have any thoughts?
I feel myself slowly moving more and more towards an Agile development methodology. I'm currently reading Robert C. Martin's Agile Software Development which is proving to be most insightful. The biggest concern I have with the agile approach is in regards to applying it to fixed bid projects. A very large portion of the work that I do is fixed bid. The agile approach is very iterative with constant interaction with the client and continous feedback. This means lots of changes (theoretically) and seems to me that it would make it almost impossible to estimate a project accurately. I haven't come across any material as of yet that talks in detail about how the agile approach effects estimating and bidding of projects. Is an agile methodology only appropriate for non-fixed bid projects? If not how is an accurate bid placed? Has anyone had a positive experience using an agile approach on a fixed bid project?
Update:
Ok, Perhaps I'm thinking about this in the wrong way. This article, as well as Agile Software Development talk about doing 3 week iterations and at the end of each week delivering a working peice of software. What I'm still not 100% clear on is how the overall budget is figured out for the project. I guess I'll need to comb my way through some more of the articles over at agilealiance.com until I find my answer...
Today I was doing a little upload form that I figured would take about an hour. After having my form working (in about a 1/2 hour) I moved on to adding some validation. I dropped a couple RequiredFieldValidators on my form and loaded the page up in the browser for some final testing. The first problem I had was that the .IsValid property of the page was always coming back as false. Hm, what could this be from. After a little investigation I discovered that it was a RequiredFieldValidator that I had on a HtmlInputFile control that was causing the IsValid property to always come back false. Since the HtmlInputFile control doesn't maintain state between post-backs the server side validation was always failing. So, I quickly added a CustomValidator control to my form to verify a file was actually uploaded, did some quick coding to actually ensure the HtmlInputFile.PostedFile.ContentLength was greater then 0 (since that would tell us a file was uploaded), and finally hit F5 to finish testing. Well it turns out that since its a HtmlInputFile the CustomValidator's ServerValidate event never gets fired. I'm guessing that this is again due to the fact that the control doesn't maintain state. The confusing thing is that the documentation for the ControlToValidate property of Validators says that an HtmlInputFile control is a valid control to validate via the validation controls. Am I missing something or is this "support" a little lame?
<WORKAROUND hack="True">
if(FileUploadControl.Visible && FileUploadControl.PostedFile.ContentLength == 0)
{
SetMessage("Please select a valid document to be uploaded.");
return;
}
</WORKAROUND>
[Listening to: Crash Burn - Blues Traveler - Four (05:14)]
Over the past couple of months I've been working on a project which is using a test first methodology along with a continous integration build process. Overall the experience has been very good and I will definitely be sticking with the development practices we're using on the project, however, there is one minor annoyance that has been bugging me throughout our development.
Each of our tests interact with the database in some way. Our objects are dependent on having data available so the Setup of our test classes insert the proper data to be used within our tests. To enable all the developers on the project to be able to run tests simultaneously without stepping on each other we've created a DTS package that transfers the latest version of the database to the developer's machine. When a test class is completed it is our policy that the latest version of the code should be retrieved, built, and run. This ensures that everyone's tests successfully run with the latest version of code, and most importantly ensures we only see GREEN BARS! Overall the process works nicely, we haven't stepped on each others virtual toes too much and we've only had a couple stonings for those who have checked in broken code.
Now onto my annoyance. While I'm writing my tests I take a very iterative approach. So I write a bit of test code and then go and write the method to make it pass. This often involves adding parameters to stored procedures, or in some cases changing the structure of the database. Each test class might require two, three, or more changes to the database. Since I take an iterative approach this requires me to do one of two things:
- Run the DTS package on each iteration (uggggggggg, talk about interupting the flow!)
- Edit the object in both my local dev database as well as in the master databases (uggggggggg, talk about interupting the flow!)
Um...ok, something just occured to me that I can't believe didn't occur to me before writing this post. Hey there stupid Steve, why not change the process you're performing instead of forcing yourself into the nasty cycle your in.
Ok, now moving from my annoyance onto the solution that for some stupid reason has never occured to me before writing this post. Instead of making changes on the master database, make changes to your development environment. Instead of editing the object directly, script out all the database objects and place them in VSS through a Database project (which we've been meaning to do anyhow). Then setup two targets in the database project, one to the local dev database and one to the master database. While writing the tests make the changes to the scripts for the database objects (either directly alter the create scripts or create alter scripts) and run them against the local database. When the tests are complete and result in all green, run the change scripts against the master database! This will result in merging all changes at once rather then during each iteration. I do see the possibility for some overhead in managing all the scripts and tracking that everything is run against the master database when things are complete but it will certainly work out a lot better then the current setup. There is also the issue of merging other developers database changes with the local database but that could be resolved somewhat by having people refresh their local database before starting on new tasks. Well, I've gone from having an annoyance with our current process to realizing a major oversight on my part (hey at least everyone else on the team has overlooked it as well), so now its time to go and convince the other team members that we need to change the way we're doing things to make the process more streamlined....wish me luck!
Well after following the handy dandy
instructions from
Daniel Bright I've got w.bloggar going. I'm hoping this will make it easier for me to post becuase those 4 or 5 clicks I need to make now to get to the admin for my blog is just killing me ;-)
You are Morpheus, from "The Matrix." You
have strong faith in yourself and those around
you. A true leader, you are relentless in your
persuit.
What Matrix Persona Are You?
brought to you by Quizilla
I dropped into Ingo Rammer's How to Develop Distributed Applications using .NET Remoting WebCast this afternoon. Ingo is the man when it comes to .NET Remoting and his webcast certainly didn't disappoint. Also of interest is the Ingo .NET Rocks spot. Finally, to finish my plug you might as well sign up for his Distributed .NET Newsletter as well.
Recently one of my co-workers dropped me an IM asking how he could find what stored procedures depended on a certain table. After a little digging we came up with the following query which will return a listing of stored procedures that depend on a particular table (T_TableName).
select distinct so.name from syscomments sc
inner join sysobjects so on sc.id = so.id
where xtype = 'P' AND charindex('T_TableName', text) > 0
It would be cool if within SqlBuddy, VS.NET, SQL Enterprise Manager, (name your favorite DB tool) you could right click on a table and view stored procedure dependencies!
Update: For those of you interested in the real way to do this use sp_depends, thanks Darrell!
While writing my post about code generators I remembered a link to an article that I found on Martin Fowler's website about Reflection vs. Code generation. The article more or less recommends against using reflection at runtime as seen by the last line in the article:
So, in the future, if you feel you need to conquer a complicated problem using reflection, just remember one rule: don't do it at runtime
While I certainly understand the points the author makes I can't help but disagree on some level. I've seen some tremendous benefit from using reflection at runtime to create dynamic assemblies. It provides a flexible, extendible, and performance friendly solution to ensuring code isn't duplicated in every class for common tasks (such as persistence).
What thoughts do you have on code generation vs. runtime reflection in a .NET environment?
Johnathan Goodyear's latest article on whether .NET Code Generators are worth it hit the nail right on the head as far as I'm concerned. I've spoken with a fair number of people from both sides of the argument and I've come to pretty much the same conclusion as the angryCoder. When used correctly code generators can provide a time savings, but when used incorrectly they can turn out to be more hassle then their worth.
The project that I'm currently working on is using the Entity Object Framework that I spoke about briefly in my previous post. As part of the development of the EOF I also created a simplistic code generator.The code generator works off a database schema and creates a set of Entity Objects for the tables within the schema. The generator also decorates the properties of the generated objects with Persistent attributes so the objects can be persisted to the database. The generator is meant to be used at the start of a project to provide some initial time savings in the development of the Entity Objects.
Throughout the development of the system the schema for the backend database will undoubtedly change, as well as the interfaces that we want to expose on our entity objects. For these reasons we only rely on the generator to get us off to a running start, not to do all our work (although we have joked about whether we could make this a reality). So far the initial use of the code generator has provided us with a nice time savings. Now that we're in the middle of heavy duty development the generator will sit off in the corner until a new projects comes along that can see some value in its use.
Yesterday I came across DonXml's post on Attribute based programming. In the referenced article he talks about how he placed attributes on enum's to create a data access layer that provided him with a great deal of flexibility. Anyway, his post has prompted this little ramble on how I've used attribute's with the "Entity Object Framework" (EOF) that I've developed over the last couple of months.
The basic concept behind the EOF is that as a developer we should be able to rely on a set of base objects to handle persisting values to our database. The process and steps required to map properties of objects to parameters within a stored procedure call or within a SQL statement is tedious, monotonous, and frankly just not interesting. In order to remove the burden of writing methods on each object for persisting thier values to the database I decided to lean on attribute based programming.
An Entity Object within the Entity Object Framework decorates the properties which should be persisted to the database using a Persistent attribute. The Persistent attribute has a set of properties on it that define specifics about how the properties should be persisted. The constructor for the Persistent attribute accepts a parameter name, size, and type, as well as flags for if it should be sent as null if certain conditions apply.
Each entity object inherits from a BaseEntityObject class that has a Save() method. When the Save() method is called on the entity object it checks to see if it has a data manager assembly that knows how to persist it to the database. If the data manager assembly doesn't exist, the object is passed to a factory method that creates a dynamic assembly for persisting the object. The dynamic assembly is created using the System.Reflection.Emit class. Each property of the object is reflected upon, and the properties that have the Persistent attribute defined are included in the dynamically generated assembly that is used to save the object.
By leveraging attribute based programming as well as the Reflection.Emit class I've been able to reduce the amount of code necessary for persisting objects to the database and kept the performance of the methods that persist the object in line with the performance I would receive by writing the methods by hand!
One of the cooler .NET features I've come across recently is the ability to override settings in a configuration file's appSettings section by using the file attribute. If you specifiy the file attribute on the appSettings tag in your .config file you can override a settings, or provide additional settings. Below is the blurb from the MSDN online documentation regarding the file attribute. (I attempted to provide a sample in this blog entry but have given up for the time being since I can't get this editor to not screw up the xml I enter, anyone have any advice?)
Specifies a relative path to an external file containing custom application configuration settings. The specified file contains the same kind of settings that are specified in the , , and elements and uses the same key/value pair format as those elements.
The path specified is relative to the main configuration file. For a Windows Forms application, this would be the binary folder (such as /bin/debug), not the location of the application configuration file. For Web Forms applications, the path is relative to the application root, where the web.config file is located.
Note that the runtime ignores the attribute if the specified file can not be found.