I've recently been part of quite a complex integration project. Lots of fun building it - not so fun maintaining it. The project is an ticket ordering solution where people would buy and pay for a ticket to an event in system #1 (a.k.a The TOC) and then be redirected to our site (a.k.a The Checkin Site) where the user should subscribe to various events (running, bicycling, kayaking etc). You should not be able to enter the checkin-site without having paid for a ticket in The TOC. One of the non-technical requirements were that a user should really not notice that the two systems operated on different domains so we put quite a lot of effort in data flow and encryption to ensure that data and login information would silently flow from The TOC to The Checkin Site so the shift from The TOC to The Checkin Site would be almost transparent to the user.

A number of painpoints were discovered along the way. This is a list of discoveries which I hope someone (such as myself in a distant future when I've forgotten all about the pains and headaches during the last 6 months.........) could find useful in the future when building integration stuff. Here goes: 

  1. Thou should persist external data as-is 
The solution I implemented parsed a customer's order to a DTO and persisted it alongside other informations about the order (timestamps etc). It proved to be a wrong decision not to persist external data in the format we received it before parsing occurred. We have everything stored relationally and being able to query a customer's data in The TOC using SQL rather than relying on a mix of webservice calls and console applications is - well... You can imagine the outbursts and occasional teeth-grinding...

  2. Thou should agree on required fields
Is a person's last name required in your system in order to create a person? What about Email? Gender? It proved to be a less-than-Apple-like userexperience once a user entered our checkin because our two domains (ticket ordering / payment versus event reservation) had different perceptions on data validation. Had we only sat down and talked for a while about the input fields in the two systems we would probably have discovered that a person's birthdate is essential data on a person in our system (due to age validation and other stuff) but birthdate is less-than-important when ordering and paying for a ticket. Even though the user had entered their personal data in the TOC and those data were transferred to us the user would have to fill in the blanks such as Age and Gender as well once they entered the Checkin site. It would have been a much smoother experience from the user's point of view to be able to enter all information in the same workflow and just be presented with the data they entered later on.

  3. Thou should agree early on end-to-end integration test phases
When planning we didn't take into account that The TOC was still undergoing heavy development even though we agreed on a testing / bugfixing phase one month prior to release. It resulted in numerous testcases (such as "Order two adult and one child tickets, buy additional product X and Y in a quantity of 3, go to checkinsite, subscribe to a running event. Validate your receipt onscreen) which were outdated before they could be submitted to our testers because the guidelines for ordering tickets in The TOC didn't match what was currently running on The TOC's testing environment. It resulted in basically all tests were coming back without the tester had ever made it to the Checkin Site because the tester weren't able to order and pay for a ticket using the guidelines provided... Which resulted in developers sitting on their hands ready to fix bug when no bugs were reported in. We ended up cancelling the setup and took all quantitative tests and gave them to a dedicated ressource sitting next to our development team to ease the barrier of communication between the tester and our development team when flaws in guidelines were discovered. 

  4. Thou should be able to subscribe to events 
Only a few days after we released we found out that quite a few people didn't realize that there was a second step involved - for whatever reason a number of people never clicked the 200x300 pixel orange "Click here to enter the Check-in site" button on the receipt from the TOC. Because TOC notification services don't exist  the checkin system weren't notified when new tickets were ordered so the only option for us a the Checkin Site was to implement a pull-based console application doing a "Give me all customers who have ordered something during the last 24 hours" and check if all customers who had had updated their ticket order during the last 24 hours were known to us. They might never hit the Checkin button or they could have added an upselling products to their initial order (such as breakfast Thursday) after their initial order and subsequent checkin had taken place. Especially updates to existing orders proved to be a challenge until the synchronization thingie started to take those scenarios into account. We scheduled the job to run every 24 hours - we could have had a smaller timespan between job runs but I would much have preferred a message-based solution where we could have subscribed to notifications instead as the primary source of notification. I doubt we would have had a setup without some sort of pull mechanism in order to do a full sync once in a while but it is a tedious and slow way to make data flow your way if you don't get to know anything about your customers unless you repeatedly ask for it. 

  5. Thou should reconcile data early and often 
We encountered problems along the way with The TOC's payment gateway which caused a lot of customer support because user's transactions timed out. This in turn caused a lot of manual handling of customer's data in The TOC's backend which in turn caused that a customer in some cases would be created twice with tickets attached to both customer records in The Ticket System. The way our synchronization worked was to get all orders from a given customer - we didn't (and don't) have anything to merge two customer's data. This in turn caused some customer's synchronizations to fail because a given ticket type (an adult ticket) is required in order to create an event subscription. This in turn causes problems in validating that we actually have all data in our systems because numbers now do not match......... We should early in the process have planned for data being out of sync and implemented patterns for dealing with customer's records not matching what we expected (or agreed on). One of the pitfalls were that the developers at The TOC's company ensured us that no customer could submit an order without ordering an adult ticket. True - but once the problems with their payment provider kicked in customer's data were handled manually in the backend systems which didn't take our special business rules into account.... Voila, data weren't what we expected even though we were all in good faith and worked well together to win the race. At least we should for our part have had a plan for reconciliation of i.e. all adult tickets ordered versus what we had registered in our own backend. It should have been possible from day 1 to merge two customer's data into one order. The morale is regarding to data from external systems: Trust nobody. Expect the unexpected. When (not if) a sync job fails on a given record - can you gracefully recover? Who gets notified and how? Settle early how you handle ongoing support when the unexpected happens and somebody needs to take a dive into the bowels of the system to figure out what is going on. 

 Well... there's probably more to come but I can't think of more "lessons learned" right now. Until next time...

Regards K.

Announcing T.REST – a testing framework for REST ressources

I’ve finally done it – made my debut in the OSS community. As of today my company I have released a bunch of refactored helper classes from a former project as a Codeplex project. The baby have been named T.REST and is a testing framework suited for regression-testing REST ressources. We I developed it as part of a migration process from homegrown backend systems to CRM 2011 and it has evolved to be interesting enough that people outside my company have been curious about our testing abilities regarding REST so the decision for my company to release our testing framework as a OSS project was really a no-brainer.

The case is simple: Imagine that you depend on a REST ressource serving i.e. locations to your Google map. You really want to be sure that the look and feel of the service does not change because your clientside Javascript consuming the service is extremely hard to write tests against and you know for sure that you probably won’t be reading the newsletter email with the paragraph “By the way, service XYZ will introduce breaking changes in the following release” placed somewhere in the middle of lots of other boring stuff

The project will probably evolve over time but the essence of the framework can be expressed in a few lines of code:

[TestInitialize]
public void TestInitialize()
{
    RessourceFactory.Init(Assert.Fail, Assert.AreEqual, Assert.AreNotEqual, Assert.AreEqual, Assert.AreNotEqual);
}

[TestMethod]
public void JqueryUIDemo()
{
    var expected = new Dictionary<string, Type>
                        {
                            {"latitude", typeof (decimal)},
                            {"longitude", typeof (decimal)},
                            {"title", typeof(string)},
                            {"content", typeof(string)}
                        };

    var res = RessourceFactory.Create(new RestConfiguration
                                            {
                                                Url = "/svn/trunk/demos/json/demo.json",
                                                Host = "jquery-ui-map.googlecode.com",
                                                ExpectedObjectSignature = expected,
                                                ExpectArrayResult = true
                                            });
    res.ValidateSignature();
}

This lines of code will assert the following:

• That the REST ressource returns an object with properties



• That the number of properties matches the number of properties in the expected result



• That no properties were found which were not specified in the expected result



• That the types of the properties returned by the REST ressource matches your expected result

T.REST is released under the MIT license so feel free to use in any way you desire.

There are a number of quirks in the original implementation which I have tried to refactor out in this initial release but if you give it a try and stumble upon something please use the Codeplex Issue tracker. If time allows it I would like to write some more code examples uncovering the small quirks and how-to-do’s using the framework so keep an eye on my blog and on the documentation on the Codeplex project for updates.

Regards K.

Google announces Dart - a programming language for the web

I was at GOTO Aarhus today and Google had announced that they would present a new programming language developed by Google to the open public for the first time. So what did they come up with this time?

Dart is “a programming language for the web”. It has been developed by Lars Bak (the guy who created the Javascript engine V8 in Google Chrome). If we take one step up the ladder it is a paradigm shift which enables developers to write compiled code invented and prepared for being run in a browser. Scripting languages have been ruling the web world for eons but the inherent disadvantages with runtime interpretation and the DOM itself have driven Google towards a decision that we need to take web development to the next level. Tooling and frameworks for i.e. Javascript (such as JQuery) CSS and HTML have evolved around making things easier for the developers but they have been constrained by the nature of the web, i.e. the request/response paradigm of the HTTP protocol etc. With the emerging HTML5 standards the rules of the game will fundamentally change and I believe Google – again – have been quick enough to embrace that fact and ask somebody to drive this development forward in a direction which is pointed out by Google. I can’t remember speed being an key point of interes in any web-browser until Google released a beta of Google Chrome which made Internet Explorer look really bad. Google gave people the impression that they could get much more and a much richer browsing experience just by using another browser and guess what? People just love it when they get more for free.

Dart runs in a Dart VM which can be integrated into the browser – that calls for insanely fast websites / webapplications when combined with i.e. HTML5 offline capabilities. Dart can also be ported to Javascript with a tool called DartC so it can run in browsers which do not support a native Dart VM. It is a commercial decision, not a technical one I’m pretty sure. Lars answered a question regarding the possibility to run Javascript directly from Dart and his response was crystal clear: It was not an option and it wouldn’t become one. “Everything starts falling apart”, he said if you allow developers to hack around shortcomings in a language – the nature of Dart isn’t scripting anyway so the Dart team have made a clean cut there. That is for the good I think and another indicator that Google regards Dart as a programming paradigm which could rule out Javascript as the tool to solve a given problem in a lot of cases. There’s plenty of room for both languages but due to the heavy attention on mobile browsing experience in the community today I would expect mobile browsers to be the first to adapt VMs. The constraints regarding CPU size, memory shortage, network latency etc. on a mobile platform calls for VMs which are able to host and run compiled, not interpreted code. Mobile platforms are all the rage due to the fact that smartphones and tablets in various forms are about to take over from laptops and desktops as the main Internet browsing platform so new tools and languages and a large community will emerge for sure during the next few years.

Dart is still work in progress and Lars emphasized that a lot so we won’t see a large community evolving in the near future I believe - but due to the fact that some 15 developers from Google stood up at the end of the keynote so people could see their faces and eventually catch one of them and ask questions during a break proves that this isn’t just some prototype gadget Google have given birth to. It’ll be exciting to see the reactions from the other browser vendors such as Mozilla and Microsoft. Will they go in another direction and try to market their solution to the same problems identified by Google? I personally believe it won’t be long until Microsoft releases some sort of VM-like prototype to the Microsoft community just like they did with Internet Explorer 8 which had a brand new Javascript engine as a response to the V8 made by Google… On the other hand they might stick with optimizing Javascript performance but no matter what they’ve got to come up with something. It’ll be fun to see what they will come up with and how the community will evolve around Dart.

Ressources:

GOTO Conference

Dart language.org

Google Code blog – Dart announcement

 

Regards K.

Debugging dynamically loaded Javascript files with IE Developer Toolbar

I’m currently stuck using IE Developer Toolbar because my current project involves Microsoft xRM. Neat platform but it is not crossplatform (yet – they’ve got something coming in the next release) so we’re using Internet Explorer for the time being.

The Javascript setup in our solution includes dynamically loading some custom Javascript files but appearently the IE debugger refuses knowledge of your dynamically loaded JS files . That sucks – really, it does because if you don’t know that you can just search for your dynamically loaded Javascript using the Search bar at the top right corner you’re in a world of s***. Then you’re left behind using good ol’ alert-boxes and console logging… I don’t know about you but I’ve been there, done that and it’s not an option for me.

I looked around and the solution is simply searching for your javascript content once the page (and your dynamically loaded files) have been fetched from the server. Search for something in the file you want to debug - in my case I’m trying to find the namespace “NC.Gruppe”:

Now we’re talking… The file isn’t available in the list of loaded Javascript-files but you’re able to set breakpoints anyway. If you know that what’s you’re supposed to do or know your way around Google.

Thanks to Заметки for starting writing in English

How to make System.Data.SQLite.dll work on a 64 bit Windows server

I have a pet project in which I’m using SQLite for persisting day-to-day gasoline prices from various companies. No-one ever thought about that one, right? If this was 2001 I would have at least 20 employees and millions of venture capital already… Luckily this is 2011 and I’m not wasting anybody’s time and money on this one.

Anyway – I’m using SQLite as a persisting mechanism and I had a great deal of trouble making it work. I am for various reasons currently working on a 32 bit Windows 7 laptop and my production server is (luckily) a 64 bit Windows 2008 server. Everything worked fine on my laptop but once I deployed my solution to the server I got various error all evolving around an error message “Unable to load dll SQLite.Interop.dll”.

I thought at first that I just needed to adjust my Visual Studio project settings so all projects in my solution would build as 32 bit. That should work because as they say on MSDN: “If you have 100% type safe managed code then you really can just copy it to the 64-bit platform and run it successfully under the 64-bit CLR”

Short story was: I tried every possible combination of platform targeting, I tried deploying my code with both the 32 bit and 64 bit System.Data.SQLite.dll, I tried just about anything but never made anything work – and I really couldn’t figure out why because it ought to work but didn’t.

After digging for a while I realized that SQLite for .NET is simply a wrapper on top of the original C++ implementation… A few clicks verifying what had to be missing on the server 5 minutes later I had installed the 32bit Visual C++ package and everything started working.

The morale here is: I had a rock-solid idea about SQLite.Net that it wasn’t a wrapper around native code but was SQLite written in pure .NET but never confirmed it by looking it up. I’ve done it before and I’ll probably end up there again in the future but it is always a good idea spending a few minutes learning about the architecture of the tools you’re about to embrace as part of your toolbox. Had I learned from the beginning that there was a C++ assembly hidden somewhere I wouldn’t have spent an entire evening grinding teeth at my computer… Lesson learned this time for sure.

Using Jint to unittest your Javascript in C#

I recently stumbled across Jint and found it interesting to a degree that I have spent a few hours getting to know the product. 99 times out of 100 I don’t have the time or the energy to dig deeper into new products but the timing was well so off I went.

What is Jint? It is an opensource implementation of a Javascript interpreter. The project defines itself in the following terms: “Jint is a script engine based on the Javascript language…Jint aims at providing every JavaScript functionalities to .NET applications”. Does this mean that I can take a piece of Javascript and execute it in a .NET Console application? Yes it does – and it works out to be a much more frictionless experience than you might expect. I tried integrating QUnit with CruiseControl.NET a while back to test Javascript in a managed environment and even though I made it work 95% it really didn’t feel like a comfortable way to go. Let’s see some code (example is from the project’s website)

script= @"
  function square(x) { 
    return x * x; 
  };
  
  return square(number);
  ";
  
var result = new JintEngine()
  .SetParameter("number", 3)
  .Run(script));

Assert.AreEqual(9, result);

Really?  Yes, indeed… I decided to try it out on one of our own internal Javascript API methods and came up with this

[TestMethod]
public void Basic_GetRestHost_ValueReturned()
{
    string expectedValue = "http://restservices.localhost";

    var jint = new JintEngine();
    var returnVal = jint.Run(File.ReadAllText("dgiapi.js") + "return $dgi.getRestHost();");

    Assert.AreEqual(expectedValue, returnVal);
}

Test passes, 4 lines of code, not too much ceremony along the way… It took a while to figure out that the Run-method isn’t chained – I thought I could preload our API in a base class and use a second “Run” method to invoke the call to $dgi.getRestHost() but never made it work. It might not be best practise – it probably isn’t but haven’t dug deeper there yet.

Conclusion: Jint really looks promising. One of the major showstoppers along the way of testing Javascript for me has always been the lack of integration with buildservers and Continuous Integration but Jint seems to close the gap here. I can definately see some usages in our business here where we spend more and more of our time writing Javascript instead of serverside .NET code – especially because we’re currently migrating from a self-breed internal business application to a new, shiny installation of Microsoft xRM in which we will inevitably end up with Javascript to extend the standard user interface (hide buttons, load data into dropdowns etc. etc). It is business critical that these scripts works like expected so it would be nice to be able to unittest at least parts of them in a Continuous Integration environment. I’ll look forward to a PoC of Jint under these circumstances.

The project is work in progress and I submitted a bug yesterday - it was fixed this morning in a dev branch. Thumbs up.

How to qualify an Enterprise-y software solution from a tech-guy’s perspective

At some point in your career you will inevitable come across one of those major projects which will replace what you’ve got with something better, more reliable, more suited for exactly your business needs, easier to maintain and a whole lot more… You know the drill. Let’s for the fun of it call the legacy system bound for replacement OldSystem and the new system for ShinySystem

If you’re really lucky someone at some point in time will ask you to qualify a number of vendor’s ShinySystem towards eachother. Maybe the decision has already been made and you find yourself stuck in an Middle-Eastern-style arranged marriage but at least you should still need to know the new ShinySystem on a higher level. What should you be looking out? I’ve had the pleasure of working with both legacy systems and shiny Sharepoint-solutions during the last three years so I’ve faced these type of questions before. I remember that I used to be terrified answering questions such as “What do you think of this ShinySystem” because frankly I really didn’t know what to look for from a technical point of view. During the last two or three years I’ve summoned up the gazillions of mistakes I’ve made along the way and here it is – the ultimate “What you should ask your vendor once the guys from Sales have left the room”.

Disclaimer: I’ve tried to keep the list technology-agnostic but keep in mind that I’m a .NET guy… Also bear in mind that this list is intended to qualify larger Enterprise-y systems. If you intend to use this list for qualifying a .NET control you intend to buy for 99 dollars you are way off target. I assume that you’re looking at something you will be hosting yourself on servers which are not cloudbased and you have full control over because cloudbased services is a whole other ballpark. Last, but not least – you’re a tech guy so I’ll be focusing on tech stuff and not issues such as licensing and stuff. Enough rambling – here goes:

Security model

I’ve come to the conclusion that in just about any system there is a concept of security. It can be disregarded - i.e. by placing an internal website on the local intranet and telling nobody it exists except the five developers who need access to it – but the concept of security still exists and have to be taken into regard when designing the system. The concept of excluding someone from doing something is what security is all about. The way security is handled by a system tells you if the system is designed with security in mind. If it is not – what else wasn’t taken into account you should ask yourself in your position… I would start bothering developers on ShinySystem with questions like the ones below:

• How are users and roles created and maintained in ShinySystem?
  • Users, roles, permissions and securables are completely different things. If you think “read” and “write” are roles, please stop reading until you’ve learned the distinction.
  • User authentication and user authorization are completely different things. If you don’t know what I’m talking about, please stop reading until you’ve learned the distinction.
• Is there a concept of a user group?
• How are permissions maintained?
• Is it possible to define your own set of permissions?
• Is it possible to break inheritance?
• Please explain how auditing works in ShinySystem

That should quickly lead you towards areas of the system which nobody else in your organization knows exist – but the have to live with the consequenses if ShinySystem doesn’t live up to business expectations regarding security. If you have a business case regarding security which OldSystem doesn’t solve according to business needs you should analyze why it doesn’t work in OldSystem, solve it on a whiteboard using ShinySystem and maybe make a Proof of Concept / prototype if you’re still uncertain whether or not it will work. I regard a feasible security architecture as a key factor when qualifying software products – simply because an inefficient architecture affects about 100% of the users who will be using the system and users expect things like security to “just work”. They will blame you and not the vendor if security issues comes to cause friction in their daily work so watch out on this one.

Custom code

You’re looking at any given ShinySystem and have a business case ready which should be possible to solve in ShinySystem. At some point even the sales guys are having a hard time keeping up their appearences because it isn’t possible to model your entire business using Drag&Drop in that shiny visual modelling tool in ShinySystems main window. This is where you probably will have to code something on your own. Most major vendors design their core applications with extensiability in mind - Firefox extensions have been around for eons by now – so you should expect that any Enterprise-y ShinySystem candidate provide some way of letting you write your own code in terms of “custom workflow”, “plugin”, “webpart”, “custom control” and so on. Given that you CAN write custom code and extend functionality – start wondering:

• Which programming languages can I use?
• What is the development cycle when writing custom code on your development machine
  • One answer could be “Write code, build, deploy to test website, reset webserver, test change”
• It is very likely that there is an API somewhere to interact with ShinySystem
• How does the ShinySystem API look like (there is likely to be one)
• Are there any undocumented features in the API?
• Which programming languages can you use?
• How does an API error message look like – can I tell what’s going on just by looking at it?
• How do I upload my changes to production?
• If my custom code fails how will it affect the stability of ShinySystem?
• How do I debug an error in a custom component?
  • During development?
  • Once an error shows up in production?

Storage

Applications come and go. Data lives forever. There are basicly two distinct types of replacing OldSystem with ShinySystem – those where you move the OldSystem data (or parts of it) to another, shinier location and those where you don’t. And then again: If you decide or is being asked to move parts of your data storage to another location there are two types of migration projects: Those where you are able to buy tooling and expertise and those where you’re on your own. If you are migrating from SQL 2005 to Oracle you have migration tooling avaliable to you. But what if you are migrating from i.e. Sitecore CMS written in .NET to Drupal which is written in PHP which is completely different using another programming language and another databasetechnology? You are on your own on this one. Maybe you’re lucky to find tools and guidelines which can help you but it’s up to you to find and use them. In either case you will end up in a migration project writing mapping code which extracts data from OldSystem and inserts them into ShinySystem. This discussion quickly descends into lowlevel technical details but remember that you want to know where your data is in ShinySystem and how it looks like – especially now when cloudbased services is all the rage and data flows around. Do you know on which servers your Google Emails are being stored? Are those servers located in Europe or United States? Do you care? If those emails are crucial to your business in regard to storage management policies you HAVE to care – that’s my point.

Side-note: Developers are usually in full control of an entire system but keep in mind that some Enterprise-y systems don’t allow you to mingle with the physical data storage. A dedicated Microsoft Support SWAT team will hunt you down and bleed you to death with a blunt knife if you as much as add a new column to a table in the database behind Microsoft xRM. In xRM you’re allowed access through webservices but the underlying data storage could be flat text files for all you know. You do of course but Microsoft early on decided to take full control of the database behind Sharepoint and xRM.

Keep in mind that vendors should have an answer ready on these topics:

• Where are my data once I click “Save” on a new user / a new item etc in ShinySystem?
  • The answer here is of great interest if ShinySystems is located in the cloud
• How do I perform a backup / restore of my new datastore?
• In regard to tooling
• How do I monitor, profile and optimize the current use of data?
• How do I upgrade the database scheme?
• How do I deploy database sceme changes to production?

Scalability

Scalability is a very loaded word – what is scalability after all? Does your system scale well if ShinySystem responds in a timely fashion? Wikipedia offers one feasible conclusion: “Scalability is the ability of a system, network, or process, to handle growing amounts of work in a graceful manner or its ability to be enlarged to accommodate that growth”.  The article also suggests possible scalability dimensions such as administrative, functional, geographic and load scalability. One might add that a system should be able to scale down as well – if you’re doomed to maintain 10 servers because that’s the way you installed it but you figure after 6 months that you only need 2 servers but your are unable to reconfigure your installation because “thats the way it is” – your system doesn’t scale very well in my opinion.

I’m from a webbased world and the bottleneck in just about any case I’ve experienced with users complaining about long response times in the end came down to problems extracting data from the database. The database might be flooded with requests – SQL statements were poorly written or there might be an infinite loop on a webpage with a database call in it which effectively exhausted the entire database from responding to other requests in a timely fashion. If ShinySystem is a webbased solution you want to dig into how ShinySystem communicates with the datastore that’s for sure.

• How would you suggest to scale up ShinySystem if the number of users exceed what we’re expecting?
• Can parts of ShinySystem be run on multiple servers?
  • Large websites might run on N webservers with one single database server but ShinySystem might consists of both the webserver AND the underlying database.
• How would you scale down ShinySystem?
• Is caching a baked-in feature of ShinySystem?
• How does it work?
• How do you administer and tweak caching features?
• How does ShinySystem communicate with it’s datastore – synchronous or asynchronous?
• How would you throttle or prioritize data requests from various parts of ShinySystem?

There is of course litterature available on the subject – I’m currently reading “Scalable Internet Architectures” by Theo Schlossnagle.

Upgrading

You’ve built your ShinySystem, the users are happy, you’re overall happy and have learned your way around the things and limitations within ShinySystem 4.0 which is the version you’re running. Then a new version 4.5 is ready for sale and management obviously wants to upgrade because reporting and statistics are better in version 4.5. Now you’ve got a headache because management of course expects everything to be running as usual with all the new stuff made available to them. Who can blame them on such an assumption?

How you and your vendor have prepared yourself for such a situation (and upgrades will be part of your life from time to time once you start playing with standard Enterprise-y software) is of great interest. Remember: Every time you decide to extend standard functionality you take sole responsibility for it to work after an upgrade of the underlying system. It can’t be emphasized enough. Often vendors will make a great effort to make it easy for developers to extend their standard software – by providing templates, base classes, fully documented APIs wrapped in a multitude of languages and so on. It looks nice but remember that whatever you do – if you extend standard functionality it’s you and not some ShinySystem key account manager who will be called upon once you perform an upgrade and things start to fall apart.

You should really be looking out for systems that are hard to model towards your business needs without writing custom code. If you’re facing problems where you’re trying to configure ShinySystem to fit business needs 100% and 9 out of 10 times it turns out you have to write custom plugins to make things happen you’re bound for trouble. The size of an upgrade project will grow exponentially to the number of plugins you need to test on a new version in my experience.

It is sometimes also a question of mind-fiddling with your managers and project leaders. If you give them a choice of delivering 80% of any given task within the next day using standard functionality ready on your testing environment – and delivering 100% of business requirements in about three weeks give or take a week because those last 20% requires you to write custom code, test it, test everything else and still risk breaking all the things you didn’t take into account because of obscure dependencies and you-know-the-drill… most (or some at least) sane project managers will go with the 80%. Sometimes the core business value is hidden in the last 20% and you have to take your time writing code and testing it but you will do it knowing that it is vital to the business that you do it and spend money on you giving a custom extension your very best shot.

Developer community

“Can I find solutions to my problem in ShinySystem on Google?” It really comes down to that. I make a Google search no less than 20 times a day every day at work when I’m doing programming chores. If you decide to go with any Enterprise-y system you really, really want to go with one that has a decent developer community – which means people who use the system you’re about to get married to and provide developer solutions to developer problems. It can’t be emphasized enough – solutions to your problems will take longer to develop, will lack quality and probably won’t follow best practises if you have to educate yourself in ShinySystem because noone outthere can help you. Developer communities embrace both opensource products and products like Microsoft Sharepoint – there are surprisingly many people out there who love Microsoft Sharepoint to a degree where they are building webparts every day providing them to you for free. Dedicated bloggers (also non-MVP's) are consistently adding new posts about problems and “how-to-avoid-strange-messages-like-XYZ-if-you-want-to-ZXY” experiences – things you are likely to experience for yourself once you make a final decision to go with ShinySystem. I don’t value the developer support you might get from ShinySystem as high as the one you get from real-world users with no relation to the company who built and sell ShinySystem solutions – simply because when it comes down to it ShinySystem’s own developers will never advise you to use another product which they may know of and which might solve your problems in a much better fashion than ShinySystem is able to.

• Have people before me used Shinysystem with success?
• If I write “Shinysystem blog” and “ShinySystem best practise” in Google what do I find?
• Are there any companies which offers developer and user education in ShinySystem?
• To what extend do people with wifes and families love ShinySystem so much that they can’t resist develop ShinySystem plugins in their spare time for free?

If you decide to go with any ShinySystem, even a major one with an active community surrounding it – spend a few hours writing blogposts about your own discoveries along the way. All those posts available to you have been written by someone for you to use for free, right? If you like what others have written and use their work to get things working at your current gig you should at least consider sharing your experiences with ShinySystem on a blog or similar – anything goes as long as Google indexes it once in a while.

Epilogue

There are a ton of questions regarding debugging, logging, development environments, licensing issues, testing abilities, how to automate builds and deploys, surveillance and monitoring of production systems and so on which I won’t cover simply because I’ve been writing for hours now and need to stop at some point and get some work done…

My final statement for now will be this one: There are always tradeoffs – it is impossible to have your cake and eat it too so you need to focus on what’s important for you as a developer.

• Is it an extensive range of tooling?
• Is it the programming language available to you?
• Will you be running screaming away if you’re told you can’t add indexes to the database because database optimization is being handled for you behind the scenes by ShinySystem?
• Do you need an extensive developer community you can turn to or are you best off by inspecting code and figuring things out for yourself?
• …

If you’ve come all the way to the bottom of this post (that would be around here, I guess) I guess you have a few ideas about other areas of concern or bulletpoints missing in the areas covered by this post. Do write them in the comments section and I’d be thrilled. Until next time…