Hacking or engineering? what, when and why

I am working on a research project whilst at the same time reading Refactoring by Martin Fowler and Growing Object-Oriented Software Guided By Tests by Steve Freeman and Nat Pryce. I think both books go really nicely hand in hand with the general philosophy of test driven development with both supporting each other very well.

However I have a dilemma – I want experimental results quickly and will learn a lot on the way, however following a rigours test driven approach with refactoring seems to mean I am getting better software but I am also getting further from my results.

In theory (and I do believe the theory) I will end up with much better designed software through using the techniques described in the books. Coupling should be lower making the code more flexible and reusable – important in an ever changing research environment. Also I can make bigger changes with the confidence that I can do them incrementally testing that contracts are kept all along. This is all great, but I am still learning and as such the code base could do with much refactoring simply to enable more through unit testing. As it happens this will also benefit the architecture, but it will cost time.

Also as this progresses I can see that one ends up with a larger structure of code made of smaller bits, so on one hand you are making the software more modular and flexible and generally simplifying and making more readable blocks of code, but is this at the expense of a more complex overall structure?

I can only guess for now, and for now I will still be doing a degree of ugly hacking but it is feeling more and more wrong.

Speaking to other researchers some have told me of there regret for not starting out with unit testing and well engineered code, whilst others have completed plenty of high quality published research on the back of throw-away, proof of concept hacking, and have no real experience of a more engineering oriented approach.

I suspect there isn’t a black and white answer and it depends on the purpose of your code, like most things it is probably generally a nuanced and slightly subjective call based on the details of your requirements.

I will try and update this post with more reflections on this problem as I make progress.