An Ode To “It Can’t Be Done”

Yesterday, I blogged on its 25th birthday about the Manifesto for Agile Software Development.

Whenever I mention Agile (as she’s known to her friends and to her enemies), inevitably there’ll be at least one comment along the lines of “It can’t be done” or “Nobody’s ever done it”.

They may qualify that by citing organisational or technical obstacles to applying the values and principles laid out in the document, like entrenched management or unskilled developers or a risk-averse, command-and-control culture.

For sure, these are all obstacles I’ve faced many times. Fair comment.

Having said that, these are also obstacles that I’ve overcome in 90% of instances.

It’s possible to manage upwards, winning the support needed for – at the very least – your customer to actively engage with an agile development process.

You don’t need to change the whole organisation to create an island of self-organisation and rapid feedback. This is why I focus on teams and individuals, and don’t get involved with organisation-level “agile transformations”. From what I’ve seen in the last 25 years, They. Just. Don’t. Work. Not on that scale. But that’s a whole other post.

As for developer skills, well there’s a reason I necessarily ended up becoming a trainer and mentor in skills like usage-driven analysis & design, TDD, refactoring and continuous integration. To some extent, I’ve had to train most of the teams I’ve worked with going back to the late 1990s.

My response when friends complain that the team they’re stuck with “don’t know what they’re doing” is to ask “And what are you going to do about that?”

Command-and-control cultures are often a product of lack of trust. And in many cases, that lack of trust has been earned by past performance. Many disappointments, many broken promises. So organisations micromanage, and if anything’s guaranteed to end an experiment in software agility, it’s that.

To earn trust, teams need to deliver rapidly, reliably and sustainably. To gain the autonomy needed to deliver rapidly, reliably and sustainably, teams need to be trusted. Catch 22.

Here’s the thing, though. Nothing’s stopping you from having a conversation about that. There’s no law that says you can’t spell out the impasse to the managers you’ll be needing that trust from.

There’s a window of opportunity here for someone coming in to the organisation with fresh eyes to break the cycle. And that’s been my specialty for most of my career.

This is where it can help to have some courage, as well as a healthy disrespect for hierarchical authority.

Does it ruffle feathers? You betcha! You’ll struggle to find a project manager who has a good word to say about me.

Did we actually deliver? Almost always, yes.

Often begrudgingly, organisations have to acknowledge that the medicine is working after the team’s delivered and delivered and delivered, and users are making positive noises about the quality of it.

The fact that it’s hardly ever your code that wakes the head of engineering up at 2 am definitely sends a message.

The bargain with the Devil, of course, is that you have to deliver, and deliver, and deliver. Results are a much easier sell than promises. You really need to know what you’re doing.

While the rest of the engineering department may still be stuck in micromanagement hell, my experience has been that a reliable track record – suitably trumpeted to make sure the right people notice (dev teams need good PR) – will usually encourage management to back off.

Not always, of course. Sometimes it really is just about status and control, and I’ve used more ruthless tactics in those situations.

To borrow from comedian Stewart Lee, I can do office politics. I just choose not to.

I’d much rather have an honest conversation, but in leadership roles, my job is to remove the obstacles standing in the way of the team (and the customer, and sometimes the customer is the obstacle, so some tough love may be required).

If the choice is between creating value, and preserving the status quo, you can probably guess which door I normally go through.

Some teams are working in industries that are heavily regulated, where development processes are highly prescribed. There are a lot of boxes that need to be ticked before software can be released.

Here’s a little secret that I’ll let you have for free: you can fit a more iterative process inside a less iterative process without the high-ups and the box-tickers realising. As John Daniels and Alan Cameron Wills once said to me when we hired their company to do some specialised custom work for my client, “Yeah, we can make it look like that.”

The manual demands “Big Design Up-Front”? We stick a UML reverse-engineering step into our build pipeline. And for once, they get an architecture model that accurately reflects the actual design! We don’t need to mention that we already wrote the code.

(I would normally do something like that anyway, because an up-to-date-ish bigger picture helps us steer the architecture more effectively.)

More generally, when teams I’ve worked with have faced obstacles to rapid, reliable and sustainable development, we’ve almost always found ways to remove them or go around them or dig a tunnel under them.

9 times out of 10, the solution is very simply not to ask for permission. Just f**king do it.

Do we need the boss’s approval to write the tests first? No. If we’re delivering, what business is that of theirs?

Do we need C-suite buy-in to check-in changes in smaller batches? Again. Who’s gonna know?

It’s a happy accident that some of the changes we could make that have the highest leverage – the biggest impact for the smallest investment – are also the least visible outside the team. We don’t need to get budget to start writing unit tests. We don’t need the CTO’s signature to run a linter.

It’s usually only when the change you want to make will require people outside the team to change that buy-in is really required. This is where cohesive cross-functional teams shine best. The less we need from other teams or from management, the more autonomous we can choose to be.

And becoming agile under the radar reduces the risk of attracting the attention of organisational antibodies. We can be delivering very visible results, and the mechanisms involved can be largely invisible.

I’ve worked in teams where we’ve done everything from setting up our own token-ring network because the client wouldn’t allow us on theirs (obviously, don’t do this if you work for MI5), to inventing a team member so we could use their PC as a build server after they refused to give us a machine.

If it’s in the business’s best interest, we can usually find a way. Just takes a bit of creative thinking, a pinch of courage, and a little dash of charm. “Wah, wah, wah, we want a build machine!” is less persuasive than you think. At the very least, be ready to explain why it’s in their interest. A lot of the time, “It can’t be done” really means “I couldn’t persuade them”.

I’d guesstimate that more than half the successes I’ve been involved with doing software development in an agile way have been despite the hand we were dealt.

And never underestimate the bargaining power of a united team. Those 10% of times when greater agility stayed firmly, stubbornly out of reach were when I was the lone voice in the wilderness. In those situations, I cut my losses. And I’ve grown much better at recognising the signs before I engage.

By 2000, I was usually in a position to change the makeup of my team, too. Extreme Programmers of a feather and so on.

The poker metaphor is very appropriate here, since the real benefit of agile software development is minimising risk. We don’t let uncertainty pile up into big batches and big releases, and managers usually realise that their picture of actual progress is far more realistic, enabling them to make more informed decisions grounded in the reality of user feedback about working software.

So not only can it be done, it has been done, and successfully, many times. I’ve seen it, and I’ve lived it.

You might as well tell me that my house “can’t be built”.

Psst. Did you know that, as well as delivering high-quality training and coaching in agile development skills like TDD, refactoring and CI, I’m also available as a fractional principal engineer. I’ve worked with clients on an ongoing basis helping dev teams to turn “It can’t be done” into “We did it!” Let me have those difficult conversations with management and absorb the heat coming from the project office.

Tech Leaders: Low-Performing Teams Are A Gift, Not A Curse

It’s an age-old story. You’re parachuted in to lead a software development organisation that’s experiencing – let’s be diplomatic – challenges.

Releases are far apart and full of drama. Lead times are long, and unless it’s super, super necessary – and they’re prepared to dig deep into their pockets and wait – the business has just stopped asking for changes. It saves times. They know the answer will probably be “too expensive”.

The only respite comes with code freezes and release embargos – never on a Friday, never over the holidays. But is a break really a break when you know what you’ll be coming back to?

Morale is low, the best people keep leaving, and they can almost smell the chaos on you when you’re trying to hire them.

The finger-pointing is never-ending. And soon enough, those fingers will be pointing at you. Honeymoon periods don’t last long, and the clock is ticking to effect noticeable positive change. They lit the fuse the moment you said “yes”.

I know, right! Nightmare!

Except it isn’t. For a leader new in the chair, it’s actually a golden ticket.

The mistake most tech leaders make is to go strategic – bring in the big guns, make the big changes. It’s a big song and dance number, often with the words “Agile” and/or “transformation” above the door.

This creates highly-visible activity at management and process level. But not meaningful change. Agility Theatre is first and foremost still theatre. You’re focusing on trying to speed up the outer feedback loops of software and systems development – at high cost – without recognising where the real leverage is.

Where I work as a teacher and advisor is at the level of small, almost invisible changes to the way software developers work day-to-day, hour-to-hour, minute-to-minute.

The biggest leverage in a system of feedback loops within feedback loops is in the innermost loops.

Think about it in programming terms: you have nested loops in a function that’s really slow, and you need to speed it up. Which loop do you focus your attention on – the outer loop, or the inner loop? Which one would optimising give you the most leverage for the least change?

I worked with a FinTech company in London who called me in because they’d been going round in circles trying to stabilise a release. Every time they deployed, the system exploded on contact with users – costing a lot of money in fines and compensation, and doing much damage to their reputation.

“Agile coaches” had been engaging with management and, to a lesser extent, teams to try to “fix the process”.

I took one look, saw their total reliance on manual testing, and recognised the “doom loop” immediately. Testing the complete system took a big old QA team several weeks, during which time the developers were busy fixing the bugs reported from the last test cycle, while – of course – introducing all-new bugs (and reintroducing a few old favourites).

They’d been stuck in this loop for 9 months, at a cost running into millions of pounds – plus whatever they’d been paying for the “Agile coaching” that had made no discernible difference. You can’t fix this problem with “stand-ups”, “cadences” and “epics”.

I took the developers into a meeting room and taught them to write NUnit tests. Prioritising the most unstable features, the problem was gone within 6 months, never to return.

And something else happened – something quite magical. Not only did the software become much more reliable, but release cycles got shorter. Better software, sooner.

That’s quite the notch on the belt for a new CTO or head of engineering, dontcha think?

After decades of helping organisations build software development capability, I know from wide experience that taking a team from bad to okay is way easier than taking them from okay to good, which is way easier than from good to excellent.

Having helped teams climb that ladder all the way to the top, I can attest that the first rung is a real gift. Your CEO or investors probably can’t distinguish excellent from good, but the difference between bad and okay is very noticeable to businesses.

Unit testing is table stakes in that journey, but the difference it makes is often huge. Low-visibility, but very high-leverage.

Here’s the 101: the inner loops of software development – coding, testing, reviewing, refactoring, merging – are where the highest-leverage changes can be made at surprisingly low cost.

90% of the time, nobody but the developers themselves need get involved. No extra budget’s required (you might even save money). No management permission need be sought. No buy-in from other parts of the organisation is necessary.

High-leverage changes that can slip under the radar, but that can also have quite profound impact.

And results are much easier to sell than promises. Imagine, when your honeymoon period’s up, being able to point to a graph showing how lead times have shrunk, and how fires in production have become rare.

I’ve seen that buy tech leaders the higher trust and autonomy they need to make those strategic changes that the infamous organisational antibodies may well have attacked before. They were immunised by results.

Now for the really good news. In all those cases I’ve been involved in – many over the years – tech leaders only did one thing.

I am not an “Agile coach” or a “transformation lead”. I do not sit in meetings with management. I do not address strategic matters or matters of high-level process.

I work directly with developers, helping them to make those high-leverage, mostly invisible changes to their innermost workflows. Your CEO will likely never meet me, or even hear my name.

The first rule of Code Craft is that we do not talk about Code Craft.

Right now, you may be thinking, “But Jason, we’ve got AI. Developers can produce 2x the code in the same time.”

Do you think 2x the code hitting that FinTech company’s testing bottleneck would have made things better? Or would that just have been more bugs hitting QA in bigger batches?

Attaching a code-generating firehose to your dev process is very likely to overwhelm bottlenecks like testing, code review and merging – making the system as a whole slower and leakier. I call it “LGTM-speed”.

Tightening up those inner loops is even more essential when we’re using “AI” code generators. If we don’t, lead times get longer, releases get less stable, and the cost of change goes up. Worse software, later.

Good luck presenting that graph to the C-suite!

If you need help tightening up the inner feedback loops in your software development processes – with or without “AI” – that’s right up my street.

Visit https://codemanship.co.uk/ for details of training and coaching in the technical practices that enable rapid, reliable and sustainable evolution of software to meet changing needs.