Testing for what you can't see

So - we delivered a product by the end of last sprint. October 1. was the deadline and we made it fine. Or to be precise: The code that we delivered worked fine. Nothing crashed, for instance. Well - was the release succesful? Not exactly...

It turned out that when we put code up to our staging environment we didn't have much of a plan for saying: "What does it mean to put something onto our staging environment?" We have always focused on being able to automate our releases so a release to our staging server never takes more than a few minutes and can be performed by anyone who has the password to our buildserver. In other words: Our Continuous Integration setup rocks - but what is it all worth if nontechnical errors are not caught until somebody is actually starting to USE our code? When is a feature Done - how do we test and assert that Done is really Done?

I will for the rest of my life never let nontechnical requirements to a feature be part of what IT can be blamed for if something gets released and doesn't work the way it was intended. That's just not what we are hired to do as software engineers - we build software. We are not supposed to have the deep insight in a business domain because we are hired to express the business needs towards their domain in programs and websites which provide the business with solutions to their problems. However - that's a dreamworld scenario because in a lot of companies IT has become the knowledge base of your business domain because of the insight you get into a domain when it is mapped to code. You have become bloated with too much information. How many of you have tried this - hands up everyone. I have both mine up right now ;o)

What usually happens is that the person implementing software also begins to take excessive responsibility for the business domain and while being responsible is good, being overly responsible is bad because people tend to think that "That's not our problem - that's something for the IT guys. They know what to do". When the business starts to react like this is the sound of an emergencyalarm going off.

I have a feeling that this is the usual case in most small companies and in quite a few large ones as well. IT knows what to do and how things are supposed to work. When this happens IT becomes responsible for the business domain and suddenly you have the business coming to the developers to ask for insight into their own domain.

I have a feeling that when we in collaboration with the business were unable to catch that a number of non-functional requirement caught us off-guard a few days before October 1. it is due to a number of reasons

• IT departement is supervising the business domain - but developers can't test their own code. Developers are in my opinion the lousiest testers the world will ever see - and that goes for both technical and nontechnical requirements.
• The IT guys takes the majority of responsibility because they know the business domain best - there can be a number of reasons for this (business culture for instance) but the business should never be excused for not being Master Jedi in their own domain and for not knowing their own domain at least as good as the developers.
  
• We tested what we could see - nobody tested for the things that wasn't visible.
  

What are your experiences in this matter? How do you lay off responsibility to the business domain experts and convince them that they are truly experts in their field and should take that responsibility on them? What are your Def. of Done criterias for completing a story? Lots of question asked, I know - drop a comment if you have something to share with me :o)

Regards K.