I Am Not Myself


Lego Indiana Jones and the Mystery of the Failing Build

Over the course of the last couple months I have been working diligently to migrate the application I inherited from the wild wild west of unmanaged corner cutting contractor land to the happy kingdom of semi-intelligent source control.

I have successfully migrated what we have come to know as “what we think production is built with” source in to Team Foundation Server source control, cleaning up along the way. Why exactly this particular batch of contractors felt it was necessary to include references to both the 2.0 and 3.0 Enterprise Application Blocks, I may never know. I suspect they felt it was 2x more “Enterprise-y”.

Next up on my path to Agile Nirvana, I whipped up a set of build scripts that produce a consistent build every time. Imagine that! My server support people are so happy with me.

The system I am working on is somewhat interesting. The application went live last summer and was officially handed over to a group of maintenance developers. A new project was immediately started to add new functionality while the maintenance guys fixed up bugs. I currently work with the New Development group.

The maintenance guys decided they wanted in on all this awesome TFS goodness. So I spun them off a branch and let them go on their merry way. That was until they decided they were ready to start rolling forward into production.

That is when I realized that all of my build scripts no longer worked. I would fire up a build using Team Builds in Team Explorer, it would do it’s thing pulling down source code and firing off MSBuild to compile and then throw errors.

I checked on the build server in the location where source is downloaded and Team Build was no longer pulling down the source for my Main branch. The only thing be retrieved from Source Control was my Branches directory. Obviously the dirty maintenance branch hosed by lovely build process.

I checked my build definitions and mapping files. My source tree looks something like this:

The node is a branch of $/PROJECT/Main/APPLICATION. My workspace mapping pretty much looks like so:

<InternalMapping ServerItem=”$/APPLICATION” LocalItem=”flibbityjibits” Type=”Map” />

I have a few cloaked paths as well all related to $/PROJECT/Main/Something. When I execute my Team Builds Branches and TeamBuildTypes are retrieved from source control and Main is ignored.

My buildlog shows that CoreGet is retrieving the latest:

Get Version=”T” Recursive=True Force=True Workspace=”NinjaWarrior”

There was something I was missing here. I shot off an email to the Seattle ALT.NET mailing list, more in a effort to articulate the issue well than expecting them to solve my issues or anything. Kinda just getting it all out as I understand the issue for the eventual call to tech support.

I went home that night utterly devastated that my painstakingly lovingly crafted system had failed so spectacularly.

At home I was playing some Lego Indy with the wife on the Xbox and letting the subconscious work on it. And finally the light bulb went off.

I have other branches in the Branches folder that caused no problem when they were created. I thought to myself, “Self, it’s not the branch that did it.. It must be something that you did at the same time you created the branch.”

My assumption about the branch breaking my build came directly from my Jump To Conclusions Mat.
I created this branch for an external group who is adding functionality. In an effort to keep my main pristine, I locked it down so that only members of the group “Tech Leads” could check in to Main.

This had to be the cause.. when I realized it last night I almost went back in to work… but come on its Lego Indy!

So I get to work this morning and checked the membership of “Tech Members” and surprise, surprise… TFSService is a member… The build service uses this account to access TFS…

So I quickly added it to the leads group and verified that my builds worked again and went to repair the dent I put in the wall from beating my head against it.

So Lessons Learned here:
1. Respect the build user, he can make you a very sad panda.
2. I’ve automated the build, its time to get a nightly build going so these changes are fresh in my mind when stuff breaks.
3. While I am at it might as well look into setting up CI to the Dev environment with the nightly.
4. Indy really can accomplish anything.

So in the end, I created my own problem. Thanks to all the ALT.NET mailing list readers for silently laughing at me while I verbalized (textualized?) the issue on the mailing list and worked through it.

I’ll be here all week.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: