For the past year I've worked in an environment that has enabled me to study the effects of bad performance on developer culture. Every small thing was hard to do, like wading through mud. I've learned a lot, but I was also ready to give up more than a few times. In the process I've learned something about the effects of different cycles of development on the success of a product. It all depends on feedback. From my recent experience I have determined the mechanics that apply to slow environments. In a year or so I hope I can write an article confirming the other side of the spectrum (where everything is fast and fun). It could also be that the principles described here are not applicable at high speed.

Understanding feedback cycles is very important to the main point I'm making. I'll define four cycles that will come in handy. Then I will explain how I think slowing down each of the defined cycles increases the risk of failure and how speeding them up helps increase effectiveness.

Read the rest of this post »

This post could just as easily be called Why you should not use Subversion if you want to do continuous deployment, but I don't like to start a flamewar. A lot has been written about the pros and cons of git vs. Subversion, using git with svn and what not. Central to most arguments for dvcs are speed, offline support and merge support. I'm not going into the details of a raw comparison, I'm going to look at a use case and show what advantages a move to git has in this particular example.

The use case here is continuous deployment. The definition is: the technical process of automatically turning each commit into a deployment on production. It has been done before, there are tools available and is well documented. My argument is that doing this is easier with git than with Subversion.

Read the rest of this post »

I've stayed away from Scala for the past two years, mainly to protect myself from the early adopter syndrome. Now that most my colleagues are getting so well versed in it that I don't understand what they're saying anymore I decided that it was time to leave my comfort zone and not further improve my Java skills. My next project will not be in Java. Scala seems like a decent choice from what I've heard so far, but there are also already people hating the language. What's up with all that? Is Scala really too complicated for an involuntary Java Joe like me?

Read the rest of this post »

At a client I was confronted with a development environment “in the cloud” that consists of a virtual workstation that you can work on through a client from a local setup. This is really nice from a maintenance point of view if you regularly have to replace installations or roll out new machines for users. It sucks for development purposes, because the performance is awful. Testing this and arguing for its improvement I ran into an argument that is interestingly wrong about poorly performing workstations.

The argument goes something like this: Say the time it takes to get a job done is 5 minutes, 4 are spent by you pounding keys and thinking, 1 is spent by the computer running the sludge you produce through its CPU. If the computer takes 5 times as long, you only loose about half your productivity, but you can win most of that back by doing two things at the same time. Because, obviously, if you can make sure the computer is running sludge A while you already pound the keys for B, the computer doesn’t have to wait for you to finish your pounding.

In this blog I aim to counter this particular argument.

Model of the our creative mind

Creating something elegant requires focus, skill and motivation. If any of these three elements isn’t present creation doesn’t happen. During our daily work, skill doesn’t change much, motivation is high when we start on something. Focus is something we have to work for and if we cannot get it motivation suffers. I don’t want to focus on the effects of skill in this blog, let’s assume it is a constant. Motivation and focus are the interesting variables here, so see how they relate to the performance of the development environment.

During development work, focus is about asking “What if ..” questions and answering them by your actions. “What if I set this switch to true, would that cause the light to blink?” –> set the switch to true, rebuild, run, look, see –> “No it doesn’t, what if I try it on an older version of the code?”. Some questions are quick to answer, some are slow. If the question is slow to answer we use the remaining time to come up with other questions. This increases the number of unanswered questions in our mind. Once our backlog of questions is too big our focus is gone.

Let’s make a start at using a simple model. Let’s say G is the rate at which your brain generates “What if”’s when idling and A is the rate at which your actions can answer them. It is clear that if A is smaller than G the number of questions runs up.

When A/G is high….

We’re constantly low on questions to answer. If there is one it is immediately answered or discarded as irrelevant, no need to prioritize. In fact we can walk away from our desk and keep on generating questions in our head when we’re back the backlog is quickly processed. This results in better focus. It gives us more control over what we will do next.

Why multitasking doesn’t work

If our tools increase the cycle time, A goes down. We have less answers to our questions per minute. At the same time as we have more wait time, G goes up. This increases the number of unanswered questions in the back of our mind.

This wouldn’t be so bad if we magically had a prioritized backlog of those questions and we would only work on the most valuable ones, but unfortunately, like in real life backlogs don’t magically prioritize themselves. This takes time and focus. It even gets so bad that if you’re working on something and the number of unanswered questions grows out of control it becomes ridiculously hard to focus on answering the question at hand. In essence it becomes more likely to get side tracked. The result is an even more skewed A/P. When this happens people say they’re swamped or overloaded or something along those lines. Productivity is down, that’s for sure.

The other nasty effect of multitasking is that it delays the reward you can give yourself for doing something right. You’re constantly coaching yourself by positive feedback if you do something right and negative feedback if you do something wrong. There is a negative effect of delaying feedback on learning. In particular unlearning bad behavior works very poorly when feedback is delayed.

I’m intrigued to see that in organizations where slow environments are common or even the standard, I also see the highest resistance to change. It’s a leap, but it could very well be that the resistance to change is in fact partly caused by the slow environments delaying feedback and not the other way around.

Given the long title you could think that this is going to be a blog about a ridiculous edge case. It isn't, honestly! It is a blog about an edge case, but not ridiculous and not as far fetched as you think. Let's say that you have a class logging on DEBUG and TRACE. The debug messages contain (frequent) checks, but lack the full information needed to analyze the problem. This information (say a full xml message) is logged on TRACE level. Now your friendly system administrator decides that he needs this trace logging to go into a different file from the debug logging. He has good arguments and you don't want to pick a fight. Logback provides a lot of configuration options that you can do lots of neat things with. But it can't fill this requirement. Really? Well, there is a hack.

Read the rest of this post »

I like splitting wood. It’s a simple physical activity and I’m good at it. I can tell you exactly why I like splitting wood; it is very similar to why I like doing some other things, like building software for example.

There has been a lot of discussion in software development about this thing some people call craftsmanship. I don’t like the discussion part of it that much, so I’ll talk about splitting wood and baseball a bit before I come back to what I do like about software development. I hope that will make you understand that I value being a good software developer, maybe a craftsman an artist or engineer. I don’t value thinking about which one of those words I can use safely today and have you get what I mean.

When you split a log (assuming you have a decent tool) things are quite simple. There are a lot of physics involved, like the direction of the grains, the cohesion between them. The moisture is important too, the momentum of the axe, the angle… But when I swing I don’t think of those. I just look at the log, know where I want to hit it and then it happens the way I saw it in my head. It doesn’t always go right, but it does a lot of times. When it goes right I know it before I register it. Right at the moment the axe has its maximum momentum and the direction is set I know there is nothing to correct and the scenario can play out the way it should. And then there is the moment when the axe hits the wood and sinks in effortlessly. There is the well articulated “chk” immediately followed by a “pang” as the tension makes the log snap apart. If you hit as hard as you can you will send two parts flying away, but that doesn’t give me most satisfaction. I like it when I hit just hard enough and two parts end up standing next to each other. I can immediately go on and take another swing without stooping over and straightening the logs. That feeling is the most addictive thing I’ve found in life. I can continue splitting wood far beyond comfort, just to get another fix of that feeling. I am not alone in this I believe.

A long time friend of mine is a baseball player. He told me a similar story about hitting a ball just right. The pitcher throws, not too slow, but you catch the ball with your eye. You swing at it and as the bat cuts through the air, you know it’s going to connect right. Then the ball hits the bat so that you hardly feel it. You just year the “ping” and you know it’s a home run. I’m not a baseball player, but apparently there is nothing like that feeling. I believe it is the same feeling that I get from splitting logs.

A related thing has been described before by Csíkszentmihályi Mihály. It is referred to as “Flow” or “The Zone”. If you haven’t seen Csíkszentmihályi’s work you definitely should have a look (feel free to copy paste now to avoid spelling errors). I believe this immediate feeling of joy as you do something right lies at the basis of flow.

With some effort I can get this feeling when developing software too. It is even possible to share this feeling with a whole team of developers. When that happens I love my job so much I’d do it for free until I starve. With some more effort it is possible to do this on a project that actually delivers business value to a customer. This means I can do this great thing without starving if I play my cards right. I need a way to get the people that stop me from doing this to get a clue and let me do my thing. You probably know the types I’m talking about: it’s the manager up high that makes me work on a virtual desktop that runs vista; it’s the architect that forces me to use a five year old version of a bad framework and makes me deploy to Weblogic 5; it’s the customer that waves three years old specs at me when I ask him a question about how he wants you to implement a feature. It is also the colleague that refuses to talk to me about my real problem if it is not related to cool new obscure technology X that somebody blogged about this morning.

If I can get these people to grow up by talking to them about Agile, XP, or Craftsmanship, or Lean, or Oompa Loompa’s; I will do it. I’m a junky remember? I want to get my daily flow fix!

When Robert Martin said we were tired of writing crap, it didn’t ring true with me. I’m not tired of it at all. I’m prepared to write a lot of crap to get it right just once. If we do have a nice bit of marketing on the word craftsmanship I’m fine to surf that wave for now. Next year we’ll call it Real Engineering, the year after Real Development. Why the hell not. As long as some of us get what it is about and they are in my team I’m totally cool with it.

Read the rest of this post »

Read the rest of this post »

I recently got confronted with some interesting issues caused by cyclic dependencies and I found that the theory behind them is not as widely known as it should be. So here is a light introduction to refer to it in the future. To most java developers, cyclic dependencies are an academic metric that you should care about only in job interviews. There is more to them though and if you check for them and fix them in your project you might come to some interesting conclusions. In this blog I will show you some techniques and tools to find and resolve cyclic dependencies. More importantly, I will focus on why it matters to avoid cyclic dependencies.

What is a cyclic dependency?

Just a quick reminder, there is plenty of literature out there that I don’t care to reproduce. A cyclic dependency exists when a package depends on itself indirectly (for example A –> B –> A). A package depending on itself directly is not a cyclic dependency. The bigger the cycle the harder it is to find. That’s all? Indeed, consider yourself up to speed!

There is a little more to cyclic dependencies than their definition though, let’s get to the interesting stuff.

Why are cyclic dependencies a bad thing?

A famous design principle states that modules in software should be loosely coupled and highly coherent. With a cyclic dependency this principle is violated. The packages that have a cyclic dependency become incoherent (they rely on their collaborators to fulfill their contracts). These packages also become tightly coupled (a change in one package requires a change in the other package too). High coupling and low coherence are bad because they increase the cost of changes by

• increasing the effort needed for analysis throughout the incoherent packages,
• increasing the impact of the change on the system,
• increasing the risk of error during the change.

A cyclic dependency is a tell tale sign that a connection exists that shouldn’t be there. There are other design problems that can indirectly cause a cyclic dependency though, so never rely on tooling blindly!

How can you find them?

There are a couple of tools around to find out cyclic dependencies. To integrate in your build the first candidate would be jdepend. It’s free and it does the job. For more detailed manual analysis jdepend isn’t suitable though, for this type of work it is worth it to buy a tool like Structure101 (I’ve never had the need to look beyond for better). This takes a little bit of practice, but it’s easy enough to get yourself flying in an hour or two. Having the ability to click through and dive into problems is a big plus over skimming through logfiles.

How can you fix them?

Once you’ve found a cyclic dependency or a tangle as Structure101 calls the packages that are related through a cycle, you need to break the link. This is where you need to think back about the design, no tool can tell you what direction the dependencies should be in. In a healthy system, usually the weakest link (least amount of dependencies) is a mistake that you must correct, but in certain cases the whole package level design is wrong. If you find a tangled mess and you don’t know how to move forward, usually the easiest thing is:

• put all of the mess in the same package,
• look for interfaces that you want to extract and group in their own packages,
• untangle the implementations.

The upside of this approach is that you won’t be introducing unnecessary interfaces for things that should have been in the same package. If you overly decouple a system in that way it will become harder to maintain because the cohesion is low.

What’s in it for me?

As a reviewer I’ve found many cyclic dependencies by accident while figuring out how stuff was supposed to work. IDE’s typically support a click-through interface to follow an implementation of a method that is being invoked. If – like me – your following the trail without knowing where it leads a cyclic dependency quickly leads to an annoying infinite loop that your brain is stuck in. Of course my awesome intelligence saves me from going through the loop more than twice (three times maybe), but it’s annoying none the less. This is an important advise: don’t annoy your reviewer.

Apart from making code that is hard to follow for an outsider, cyclic dependencies also can make changing your code harder. Fixing cyclic dependencies teaches you a lot about the structure of your code and that makes it a useful exercise already. The cleaner design that you get from it can be very refreshing, try for yourself sometime!