The art and magic of manual deployment

Well... I promised myself that I would write a post on this blog at least once per week. I actually tried but always halted on the one question: "Uhm - what to write about - I've only got some 16 hours remaining..."

What is the purpose of such a goal if you have to dig deep to find something remotely interesting to write about? I therefore announce that I will only be writing whenever I feel like it. Heck, nobody of you guys (all three: You, my mom and my parent's dog - I haven't got a clue how many people are actually reading this) even knew that I made a promise to begin with but it sounds pretty good, right? ;o)

I have went from a complete newbie in my new job to actually being able to deliver some kind of work. Slowly I have begun moving tasks from the "Sprint backlog" to the "In progress" and actually some of them have moved to the "Finished" part of the whiteboard without being moved back a few hours later. It feels good to participate in getting a job done. We're releasing March 25 and have loads to do - I haven't got the time to write this but the second option was benchmarking on various Danish newspapers for news I couldn't care less about anyway.

The reason for choosing Option #1 was that yesterday I suddenly realized how much time we spend deploying stuff to our staging enviroment. Everything is done manually so if you have changes in 4 files and you added a new Usercontrol and included a new Javascript file to the solution you will have to _manually_ (yes, manually) find all files and upload them to the staging server. That is as crappy as it can possibly be. Where do we get these files? From the developer's machine, what else? Yes, my boys, everything is left to the man behind the machine with the largest possible margin for human errors to occur. And of course we have no backlog of uploaded files on the staging server so if you accidently overwrite the wrong file with one that will break the stagingserver's build you leave your users f***ed until you correct the error (that is: restore a backup from your own machine or call the server maintenance guy to restore the file for you from yesterdays backup because you haven't got a working backup locally. Yeeeehaaaaaaa!!!).

I have read a bit about how we can use our Continuous Integration enviroment to deliver a clean build for every checkin to be able to make revised deployments. My thoughts are to build the scripts neccessary to let the buildserver create a fully deployable package for every checkin which will enable us to synchronize i.e. revision 1447 of the codebase (stored in it's own folder) with the content on the staging server using robocopy. I have no experience on creating scripts apart from reading on various blogs about related issues but I promise to write a few follow-ups on the matter as we get along. Until then I will be stuck with issues such as deploying files from my machine which I acciddently forgot to check in.... There's nothing but work to do here, I suppose ;o)

The Decision Trap

I have been developing software for about 6 years now - give or take. Throughout that time I have had numerous encounters with users when I try to explain to them why they have to make a certain decision to enable me to figure out how I am supposed to approach a specific problem. Maybe I am not the best to communicate (being married I know that for a fact) so I have had a few meetings which went pretty ugly because I was unable to tell the users why they had to have an opinion on matters such as a particular detail in the flow diagram or in a certain submit form. The answer sometimes was "I don't care - I just want to be able to f***** submit an order! Why can't you just make it work like I have told you a million times before in our previous meetings?". I do not remember the exact words so maybe the F-word was left out in the actual conversation but the other parts of the conversation was close to the one described.

I have always taken this as a sign of users really not being interested in doing their job properly. They didn't want to spend the time to investigate on their responsibilities when they should describe a certain story or task for me - in the end that was something which the computer guy would probably do better. Somewhat simplified and with a few exceptions I must admit that I have thought this about my product owners more than just a few times. Just until this weekend actually when I was put in a user's situation without even knowing it. The story goes:

At home we decided a few months ago that we needed an outer house with a second bathroom. We had one but it was not heated, the rooms were badly placed, not wired properly and the entire building was not isolated at all. We have a kid and expect a few more to follow in the coming years so we need a place beside the main house where you can get rid of dirty clothes, dust of the contents of the sandbox in our yard and change shoes before opening the door to the kitchen entering the main house. So we went to the bank and lend a lot of money and last weekend we tore the outer house apart completely. Down to the bones - nothing left but 4 walls and a roof. What a mess..... Afterwards we dug out 40 centimeters down to be able to get water pipes, sewer lines, electriciy and yadi-yadi into the floor before isolating and sealing off everything with concrete for the rest of our life. I have my dad helping me out because he recently retired and I felt a strong obligation to make sure that he doesn't sit on is behind all day and disintegrate. The fact that he has been a farmer for 20 years and a craftsman for 40 in various enterprises and knows everything about building a house there is - that's nothing but a nice sideeffect ;o)

After digging and moving dirt all last weekend he spent the last week building infrastructure such as sewers, electricity, cold and warm water etc. Last Saturday I helped him with the tubes for the floor heating system we decided upon and everything went well. We went in for a cup of coffee because he had a few questions regarding the placement of various items such as toilet, shower, washing machine etc. so he shoot at us and we answered the best we could. I hadn't given much of it any thought because I am not very detail oriented as a person - luckily my wife had thought a bit more about things than me so we quickly gained a shared understanding on what we wanted to do with Outerhouse version 2.0 whenever the project was finished in late 2008.

My dad at some point caught me off guard with a question on how much isolation we wanted to put into the walls. I replied with a tired face: "What do you suggest?" and he replied somewhat similar to "Well there are a few options" and started explaining that if you went for option A you didn't have to put as much into the walls as option B and C because - but if option B was the chosen one and we didn't consider D and E we could get into trouble later on if we decided to i.e. change placement of the tumble dryer.... I felt an urge to cry out loud: "I just want to sit down and take a dump! That's my primary success criteria for building a restroom - I want to empty my back reading my favourite Gary Larson comic! If it is cold outside I want to be able to turn up the heat and pay a hideous heating bill 6 months later. Can you build me a toilet and some floor heating, please? That's all I ask for".

Needless to say I didn't say anything but went for one of the options of his choice being tired of having to take all kinds of matters unknown to me into account and deciding for myself. I realized later on that I was just being a user caught in the Decision Trap - knowing that your decisions today will have great impact on all kinds of things totally unknown to you and at a great cost in time and money if you make the wrong call today.

I learned an important lesson that day - and the next time I get into a meeting with users being tired of stupid developers asking stupid questions about stupid issues I will have a little first hand experience which hopefully will enable me to put myself in their situation before ruling them out as impatient and irresponsible product owners ;o)

Minimize Outlook to System tray

I found this small registry hack today because I have always been tired of having Outlook consume space in my Windows Taskbar. Enter this registry key and whenever you minimize Outlook it will only appear in the System tray. Sweeeeet....

Disclaimer (primarily because 90% of the readers of this blog are from the States ;o) ): Do NOT in any way try to alter settings in the Registry if you do not know what you are doing. I will not be held responsible blablabla.... if you mess up your own machine.

Hack is for Outlook 2003 - the Key path will be different if your Outlook is not the 2003 version.


Hive: HKEY_CURRENT_USER
Key: Software\Microsoft\Office\11.0\Outlook\Preferences
Name: MinToTray
Type: REG_DWORD
Value: 1 = System Tray, 0 = Taskbar


Reference: Click here

How to share knowledge amongst teammembers

I had an idea today - it was triggered by the countless number of Emails flooding my (and my teammember's) Inbox. The problem as I see it is not that people send Emails - that is perfectly fine. The problem is that they often send me valuable information which I have a hard time backtracking when I need the information 3 months after I recieved the Email....

I tried figuring out how to avoid this scenario and came up with a few stories:

S1: A user should be able to save valuable information received by Email
S2: A user should be able to search through stored information
S3: No anonymous access to the information should be possible
S4: The cost in time of storing information should be very close to zero to assure that the shared container of information will be updated frequently.


I think we actually achieved all goals in a matter of 2 hours work... How did we do it? Well, Uncle Google to the rescue:

I had an idea that saving everything in a private blog could be the silver bullet for us and ensure that story S1 and S2 was easily solved so I created a GMail account which could serve as a shared login to ressources hidden behind the login. I quickly set up a blog and clicked around a bit to ensure the privacy of the blog (no search engine should index the content, no pingback to tracking sites etc.). Then I changed the blog from being open for anybody to only being open to the people I invited. Everything went fine - until I discovered that RSS is not possible for a private blogs... Oooops. That wasn't part of my master plan at all because I always had the idea that invited people should be able to subscribe to the RSS-feed of the blog.

I tried fiddling around Google to see if I could establish a workaround but I discovered that due to the nature of RSS it is not possible to validate the viewers... Bugger, bugger, bugger, bugger, bugger... I knew for sure that my idea required that viewers should be notified somehow whenever new content was added otherwise the Email-hell would continue.

I actually thought that my idea was going down the tubes until I discovered that Blogspot has this nice feature called Mail-To-Blogger. When you want to add something to the blog you could simply do it by Email to make it magically appear on the blog! Voila, problems solved...

I set everything up and now whenever somebody in our team has something they want to share with us ("I updated the server X with patches Y and Z - it shouldn't have any effect on process B but be aware anyway. Read about the patches here" (where the link is the valuable information nobody can find 3 months later) they send an Email to each of us as usual - but they also send a CC to a special Email-address to save the Email in our private blog. If I recieve something such as a new password to a service we use I send it to this Email to ensure that when I go on vacation nobody is stopped in their work because the Email is located in my Inbox only. Sweeeet.......

Security? Well I have made a lot of effort to turn off RSS, to explicitly tell Google to NOT index anything on the blog etc. The content of the blog is viewable only to the people I have sent invitations to - the Gmail-account which was used to create the blog has been shared between all members so everybody are able to administer the blog. Does anybody have any experience with private blogs and how private they really are? Please let me know if you do.

We will evaluate the blog in a few weeks time to see if we have had any value out of it. The Email-hell is still there but Emails are not bad just because they are Emails. They only become a living hell when you as a user are unable to find that single Email which contains information you need and that's the problem we are trying to solve right now. I will get back to you with an update when we have more experience on the subject.

Definition of quality

I just started in my new job - I am about to work with 3 highly intelligent and competent software developers plus whatever consultants and external ressources we add to our pool of ressources.

I have a few personal goals with this new job. One of them is to try to ask the questions which are impossible to answer correctly but nevertheless make people think. So I took the initiative and gathered our small crew to a little session which was supposed to be a wrap-up of all the things I had worked on in my previous job. What had I been doing, what had I learnt, what kind of personal insight had I gained and so on... Primarily I wanted to focus on the process and not so much on tech-stuff and implementation of projects.

I started out and soon I had wound up all the failures and fuckups of my previous projects... Yes, I have made lots of weird, unmaintainable sicko-code which however had made the business happy and had earned loads of money to the company. I started thinking: "What is quality?". Ooooooohhhhhmmmmmm........ A sudden insight appeared before my eyes and I felt it coming: "Define quality!". I asked the other guys when we came around the subject in our talk and asked them to define quality. They went quite blank and I knew I hit something. When I asked them who should define quality one of them said without blinking: "Well... in this firm we do..."

I have thought a lot about what quality is after we had our talk. What is quality? As a software engineer I tend to focus on issues such as code readability, ability to extend, logging, choice of technology and so on. For a guy working in a sales departement I think quality can be summed up to: "Can I earn a shitload on money on this feature?" He couldn't care less that I use LINQ or NHibernate to fetch data from the database. He only cares that when somebody clicks "Confirm order" the order is stored in a way which makes it possible to invoice the customer.

It makes a lot of sense to me to define quality as the least possible amount of effort spent to keep the sales departement happy. Why should we measure things differently? If they are happy it means they are able to sell - and if they are able to sell you know you will also be payed for your work at the end of this month. I regret that I haven't found any better definition because it actually promotes cowboy-coding to be an legal approach to new software in about 80% or 90% of all new business cases!! Oh my God...

Somebody save me - how can I go to work tomorrow knowing that I am obliged to be writing sloppy, unmainainable code in order to have a short Time To Market which will enable Sales departement to earn a shitload of money because we were first movers on the subject?

The day before the day

This is it - I'm leaving my office for the last time tomorrow. Yup - I have gotten myself a new gig at DGI (Danish site only) which is a non-profit sportsorganisation. They want to build up services and websites for their members which they can use to i.e. administer members, create tournament plans for competitions and other services which I haven't really got a clue about yet.

I'm leaving my current project with a sense of akwardness and sadness. We have been late for a long time delivering what we were supposed to deliver mainly because the scope of the project has changed and we have suffered from numerous of confrontations between team members, between team members and the house and between the house and the house... I think we have been like boiled frogs - we somehow grew accustomed to living in chaos all the time with changing directions and team members being frequently added ore replaced. We have lived in it for so long that we as a team didn't or at the end couldn't even manage to describe why we failed on deadline after deadline. Phew... The last thing I have heard on my way out of the door is that the ASP.Classic guys have estimated a solution which is based on the current ASP.Classic platform. The same platform that my project was intended to replace with a .NET solution. Do not get me wrong - I totally understand that the business HAS to take some sort of action and it is not plainly wrong to go with what you know when everything else have failed you. I am just not overly excited by having worked on a project for about three quarters of a year just to see everything being buried three days before I leave for another job. It sucks - big time.

I have learned a number of things which will be a big, red warning sign for me in future projects:

- If developing in an agile enviroment and the team fail to deliver running code after a sprint
- When the technical infrastructure for various reasons breaks down often enough to inflict progress
- No technical backlog at the beginning of the project
- No established estimation techniques of backlog stories from project kickoff
- No infrastructure established at kickoff (testing servers, QA servers)
- Lack of communication between projects
- Lack of communication within the project
- Lack of written documentation on completed issues

...plus a number of other issues I probably will be painfully aware of in the weeks to come.

I have a strong feeling that I can't save the world on my own by doing all these things the right way from day one in my new job so my main focus when I start on Friday at 9am is this: I will NOT accept that we fail to deliver running code at the end of future sprints. If - (I know - if and if and if) - if we had delivered something which could be deployed and tested by the business I am sure we would never have been in this situation. We might be somewhere else but 90% of all the issues we have discovered have been a combination of lacking communication on all levels and the fact that we never delivered anything. The business thought they knew what we were about to give them but they hadn't actually seen anything on the screen. We blew it totally on this one - and it has cost my to-be ex-employer quite a lot of money having us coding away on something which probably will never be used anyway.

"Success is something built on the top of all your failures" - damn, I am gonna be good at this some day ;o)

Regards K.

Test-Driven Development and the quality issue

There has been quite a lot of fuzz lately regarding Test Driven Development (TDD) - I have seen a number of posts where poeple are asking if the maturity of the tools on the market make old-school TDD obsolete. I have also seen posts where Ben Hughes asks if TDD really ensures quality.

That made me think about the term "quality" - what is software quality seen from a customer's point of view and how can we convince them that the software we produce are better than the one of our competitors?

I don't really think you can measure very much out of TDD in a sense of software quality seen from a customer's point of view. I mean - do you really care how Microsoft develops Windows XP and Windows Vista? I couldn't care less - neither do our customers as long as our sh** works every time. Even when our customers have a negative experience because they are unable to fulfill whatever task they were set to do because we as software engineers have failed to deliver - do they start to ask questions? Not really - they start looking for the products developed by our strategic competitors on the market to look for a replacement of the crap we have convinced them to install on their harddrive.

The principles and paradigms on which a software product has been built upon can and should never be a quality metric for our customers. They will never care anyway. TDD is just one (very useful) tool out of many when we develop software. In terms of quality it should only be a metric inhouse whether or not our software has a certain degree of code coverage for instance... We must not be mislead to believe as developers that anyone else but us really care what tools are in our toolbox!