Codemanship’s Code Craft Road Map

If you check out the suggested reading list at the back of the Codemanship TDD course book, you’ll find what is essentially my road map for mastering code craft. There are 8 books that make up what I have found to be the essential code craft skills, namely:

  • Unit test design
  • Test-Driven Development
  • Software Design Principles
  • Refactoring
  • Specification By Example
  • Changing legacy code
  • Continuous Integration
  • Continuous Delivery

These are the core skills needed to rapidly iterate working software so we can learn our way to a solution that’s fit for purpose and that will be easy to change as the problem changes, so we can keep it fit for purpose for as long as the customer needs.

They are foundational skills that – if it were up to me – we’d instill in all developers.

You cannot learn these skills in a 3-day training workshop with maybe 1.5 days’ hands-on practical experience. Realistically, it’s going to take a smart person 2-3 years to read these books and learn to effectively apply the ideas. My TDD course is your road map for that journey.

But, I’m sad to say, it’s a journey that maybe only 20-25% of developers who come on the course actually take.

I’m painfully aware that, for the majority, a code craft training course is like spending 3 days being shown how to cook healthy meals, and then being sent back to your day job flipping burgers. Some will be inspired to try the recipes themselves, but have to do it in their own kitchens and in their own time. The usual outcome – after they’ve mastered the techniques – is that these developers outgrow their organisations and move on, taking the skills with them.

As an outside observer who stays in touch with hundreds of people who came on courses, I can see this happening. Developers who came on a course, got inspired, and invested their own time, rarely stay put. Their current organisation begins to frustrate them – “We make burgers! That’s just how it’s done here!” – and they’ve made themselves much more bankable in the meantime. On average, developers with good code craft skills earn 27% more.

With very few exceptions, the organisations where developers got inspired and stayed have been the ones who gave them the time and the support to continue learning. Now they mentor junior developers in those same skills – some are even trainers themselves now – and that initial investment in them pays dividends for their employers.

Of course, you might be reading this and thinking “We better not train our developers. They’ll just leave.” But here’s the thing: you need these skills. They’re foundational, remember. If you want reliable software that does what the business needs, when it needs it – and is easy to change as those needs change – then there are no shortcuts. Employers are falling over themselves to hire people with these skills, but where are developers learning them in the first place?

It’s not the training that’s the issue here. Our TDD course just kick-starts the learning process. By giving teams a practical hands-on introduction to the key ideas, and a road map to take with them on their journey, you can save a good deal of time and money on wrong turns and dead ends. (It took me well over a decade, a tonne of trial-and-error, and a library full of books to get a handle on these skills and distill it down.)

But they still have to go on the journey. And if you don’t invest in that, they’ll take the journey by themselves and will very likely be delivering the benefits to some other employer soon enough.

 

Do Our Tools Need A Rethink?

In electronics design – a sector I spent a bit of time in during the 90s – tool developers recognised the need for their software to integrate with other software involved in the design and manufacturing process. Thanks to industry data standards, a PCB design created in one tool could be used to generate a bill of parts in a management tool, to simulate thermal and electromagnetic emissions in another tool, and drive pick-and-place equipment on an assembly line.

I marveled at how seamlessly the tools in this engineering ecosystem worked together, saving businesses eye-boggling amounts of money every year. Software can work wonders.

So it’s been disappointing to see just how disconnected and clunky our own design and development systems have turned out to be in the software industry itself. (Never live in a builder’s house!) Our ecosystem is largely made up of Heath Robinson point solutions – a thing that runs unit tests, a thing that tracks file versions, a thing that builds software from source files, a thing that executes customer tests captured in text files – all held together with twigs and string. There are no industry data interchange standards for these tools. Unit test results come in whatever shape the specific unit test tool developers decided. Customer tests come in whatever shape the specific customer testing tool developers decided. Build scripts come in whatever shape the build tool developers decided. And so on.

When you run the numbers, taking into account just how many different tools there are and therefore how any potential combinations of tools might be used in a team’s delivery pipeline, it’s brain-warping.

I see this writ large in the amount of time and effort it takes teams to get their pipeline up and running, and in the vastly larger investment needed to connect that pipeline to visible outputs like project dashboards and build monitors.

It occurs to me that if the glue between the tools was limited to a handful of industry standards, a lot of that work wouldn’t be necessary. It would be far easier, say, to have burn-down charts automatically refreshed after customer tests have been run in a build.

For this to happen, we’d need to rethink our tools in the context of wider workflows – something we’re notoriously bad at. The bigger picture.

Perhaps this is a classic illustration of what you end up with when you have an exclusively feature/solution or product focus, yes? Unit tests, customer tests, automated builds, static analysis results, commits, deployments – these are all actors in a bigger drama. The current situation is indicative of actors who only read their parts, though.

Code Craft Bootstrapped

I’ll be blogging about this soon, but just wanted to share some initial thoughts on a phenomenon I’ve observed in very many development teams. A lot of teams confuse their tools with associated practices.

“We do TDD” often really means “We’re using JUnit”. “We refactor” often means “We use Resharper”. “We do CI” often means “We’re using Jenkins”. And so on.

As two current polls I’m running strongly suggest, a lot of teams who think they’re doing Continuous Integration appear to develop on long-lived branches (e.g., “feature branches”). But because they’re using the kind of tools we associate with CI, they believe that’s what they’re doing.

This seems to me to be symptomatic of our “solution first” culture in software development. Here’s a solution. Solution to what, exactly? We put the cart before the horse, adopting, say, Jenkins before we think about how often we merge our changes and how we can test those merges frequently to catch conflicts and configuration problems earlier.

Increasingly, I believe that developers should learn the practices first – without the tools. It wasn’t all that long ago when many of these tools didn’t exist, after all. And all the practices predate the tools we know today. You can write automated tests in a main() method, for example, and learn the fundamentals of TDD without a unit testing framework. (Indeed, as you refactor the test code, you may end up discovering a unit testing framework hiding inside the duplication.)

Talking of refactoring, once upon a time we had no automated refactoring tools beyond cut, copy and paste and maybe Find/Replace. Maybe developers will grok refactoring better if they start learning to do refactorings the old-school way?

And for many years we automated our builds using shell scripts. Worked just fine. We just got into the habit of running the script on the build machine every time we checked in code.

These tools make it easier to apply these practices, and help us scale them up by taking out a lot of donkey work. But I can’t help wondering if starting without them might help us focus on the practices initially, as well as maybe helping us to develop a real appreciation for how much they can help – when used appropriately.

4 Out Of 5 Developers Would Choose To Stay Developers. Is It Time We Let Them?

Following on from yesterday’s post about squaring the circle of learning and mentoring in software development, a little poll I ran on Twitter clearly shows that a large majority of software developers would prefer to stay hands-on if they had the choice.

I’ve seen many developers over my 28-year career reluctantly pushed into management roles, and heard so very many talk about how much they miss making software with their own hands. But in too many organisations, the only way to progress in terms of seniority and pay is to move away from code.

Some choose not to progress, making do with the pay and the authority of a developer and biting their tongues when managers who haven’t touched code in years tell them to do silly things. But then we often find that ageism starts to kick in eventually, making it harder and harder to get hired in those hands-on roles. “Why is she still coding?” There’s an assumption among hirers that to still be in these “less senior” roles at, say, 45 is a failure to launch, and not a success in being exactly where you want to be, doing what you love.

A conversation I had recently with a team highlighted what can go wrong when you promote your best developers into non-development positions. They found themselves have to refer technical decisions up to people who no longer had a practical grasp of the technology, and this created a huge communication overhead that wouldn’t have been necessary had the decision-making authority been given to the people responsible for making those decisions work.

I’ve always believed that authority and responsibility go hand-in-hand. Anyone who is given the responsibility for making something happen should also be given the necessary authority to decide how to make it happen.

Not all developers welcome responsibility, of course. In some large organisations, I’ve seen teams grow comfortable with the top-down bureaucracy. They get used to people making the decisions for them, and become institutionalised in much the same way soldiers or prisoners do. What’s for dinner? Whatever they give us. When do we go to bed? Whenever they say. What unit testing tool should we use? Whichever one they tell us to.

But most developers are grown-ups. In their own lives, they make big decisions all the time. They buy houses. They have kids. They choose schools. They vote. It’s pretty wretched, then, seeing teams not being trusted to even purchase laptops for themselves. When teams are trusted, and given both responsibility and authority for getting things done, they tend to rise to that.

And developers should be trusted with their own careers, too. If they were, then I suspect there’d be a lot more active coders with decades of experience to share with the ever-growing number of new developers coming into the industry.

How A Developer’s Career *Should* Work

Apprenticeships are in the news today here in the UK, with the rather shocking revelation that since the government introduced their new apprenticeship levy scheme – where employers must pay 0.5% of their income into a special government-run “bank” to be used to fund apprenticeships of all kinds – the number of people starting apprenticeships has actually fallen.

I met with the people in charge of the scheme, and was less than hopeful that it would actually work – especially in our industry. Time and again I see decision makers vastly underestimate the time and resources needed to grow a software developer, and their planned software developer apprenticeship looked lacking, too. Later, I heard from multiple employers who’d taken on dev apprentices, and they were really struggling with the lack of practical support. Who will teach these young developers? Who will mentor them? How long is it really going to take before we can leave them to work unsupervised?

The sad fact is that many of these employers just heard “cheap developers” and didn’t stop to think “Ah, but why so cheap?” The brutal answer is: because they’re not developers. Yet. The whole point of the apprenticeship is that you turn them into developers. And training a software developer takes a lot of time and lot of money.

If you saw that one house on a street was half the price of the other houses, would you not stop to ask “why?” In the case of apprentices, it’s because you only bought the land. You have to build a house on it.

The main sticking point here is that somebody who knows what they’re doing has to make themselves available to help the apprentices learn. And they need to be very available, because that requires a big investment in time.

Our industry, though, has structured itself to make this investment unworkable. The most senior developers are either too busy getting shit done, or they’re not active developers any more. With the best will in the world, no amount of transferrable skills are going to get transferred if the person who has all that useful knowledge last programmed in COBOL on an IBM mainframe. It would be like being taught economics by someone who only speaks Anglo-Saxon.

In order to square this circle, our industry needs to be restructured to make sustained, in-depth skills transfer possible.

This is how it could work:

  • At the start of a developer’s career, the most productive thing they can do with their time is learn. Career’s should start with a few months of nothing but learning. All day. Every day. A coding boot camp might be a model to follow here – provided we all acknowledge that the learning doesn’t end with boot cam graduation. It’s just a kick-start.
  • After graduating boot camp, developers become apprentices. They work on real teams doing real work 3-4 days a week, with a the other 1-2 days released for further dedicated, structured learning. This would continue for 2-3 years as they build their skills and their confidence to a point where employers feel happy leaving them to work unsupervised. It might even lead to a degree, to validate their progress.
  • Once they’ve completed their apprenticeship, developers pay their dues and return the investment employers have made in them by delivering working software of real value, while continuing to gain experience and learn more and more. There might be a decade or more of this real-world work. They continue to be mentored by more experienced developers, but in a more hands-off kind of way. A nudge here, a kind word there etc. Enlightened employers will recognise that dedicated learning time is still a wise investment, throughout a developer’s career. They may still devote 10-20% of their time to this, but at this level of achievement, it’s more like doing a PhD. We might expect developers to eventually add their own contributions to the software development landscape in this phase of their career. Maybe write a useful new tool, or invent a new technique. Maybe speak at conferences. Maybe write a book.
  • During this – and I hesitate to use this term – “journeyman” phase, developers may find they’re called upon increasingly more to mentor less experienced developers, and to share their knowledge freely. I believe this is an important part of a developer’s progress. I’ve found that what really tests my understanding of something is trying to explain it to other people. An increasing emphasis on sharing knowledge, on mentoring, and especially on leading by example, would mark the later stages of this phase.
  • Eventually, developers reach a phase in their career where the most productive use of their time is teaching. This is the “profess” in our profession. And this is where we square the circle. Who is going to do the teaching in the boot camps? Who is going to train and mentor the apprentices? Simple answer: we are.

Now, for sure, not every developer will be cut out for that, and not every developer will want to go down that route. Some will become managers, and that’s fine. We need more developers in technology management positions, frankly. But your average corporation doesn’t need 20 CTOs. It may well need 20 active mentors, though – keeping hands-on with the latest tools and technologies so they can offer practical help to 100 less experienced developers.

At present, in the vast majority of organisations, no such career path exists. We are set up to move away from the code face just at the time when we should be working side-by-side with apprentices. I had to invent that role for myself by starting Codemanship. Had such roles existed for someone with 20 years’ experience, there would have been no need. I didn’t start a business to start a business. I started a business so that – as the boss – I could offer myself my ideal job.

And, as the boss, I understand why this job is important. It’s the most useful thing I can offer at this stage in my career. This is why I believe it’s important that more bosses come from a software development background – so they can see the benefits. As it stands, employers – for the most part – just don’t get it. Yet.

There’s more at stake here than pay and perks for developers who might progress beyond the current career ceiling that too many organisations impose on people who still write code. One factor that strongly determines the way a business invests its money is who is holding the purse strings. I sometimes rail at the infantalisation of software professionals, where we must go cap in hand to “mummy and daddy” for the most insignificant of purchases. If I need a new monitor at home, I go out and buy a new monitor. Easy. In the world of corporate tech, not so easy. I recall once having multiple meetings, escalating all the way up to the Director of IT, about buying a £200 whiteboard.

If the budget holders don’t understand the technical side of things – perhaps they never did, or it was so long ago they were directly involved in technology – then it can be hard to persuade them of the benefits of an investment in tools, in books, in training, in furniture, etc. As a business owner, I experience it from the other side, watching in dismay the hoops some teams have to jump through to get things they need like training.

Codemanship training does not appeal to CTOs, on the whole. Most don’t see the benefits. They buy it because the developers tugged at their sleeve and whined and pleaded long enough that the boss realised the only way to make them shut up was to buy a course. In that sense, code craft training’s a bit like the candy they display at supermarket checkouts.

A very few more enlightened companies let their developers make those decisions themselves, giving them budgets they can spend without having to get purchases approved. But they’re in the minority. Many more teams have to crawl over broken glass to book, say, a TDD workshop.

On a larger scale, decisions about what developers’ time gets invested in are usually not in the developers’ hands. If it were up to them, I suspect we’d see more time devoted to learning, to teaching, and to mentoring. But, sadly, it’s not. They have to ask for permission – quite probably from someone who isn’t a a developer, perhaps even someone who thinks writing software is a low-status job that doesn’t warrant that kind of investment in skills.

When that changes, I believe we will finally square the circle.

It’s About The Loops, Stupid!

2 essential models of software development:

1. Queuing – value “flows”, software is “delivered” etc. This is the “incremental” in “iterative & incremental”

2. Learning – teams converge on a working solution by iterating a design

The 1st model is fundamentally wrong & damaging.

The queue is the pipeline that takes the idea in our heads and turns it into working software users can try for real. But over-emphasis on that queue tends to obscure that it’s a loop feeding directly back into itself with better ideas.

And when we examine it more closely, considering the technical practices of software delivery, we see it’s actually lots of smaller interconnected loops (e.g., customer tests, red-green-refactor, CI etc) – wheels within wheels.

But our obsession with the pipeline tends to disguise this truth and frames all discussions in terms of queues instead of loops. Hence most teams mistake “delivered” with “done” and ignore the most important thing – feedback.

Psychologically, the language we use to describe what we do has a sort of queue/flow bias. And hence most “agile” teams just end up working their way incrementally through a thinly-disguised waterfall plan (called a “backlog”, or a “roadmap”).

They’re the workers in a parcel loading bay. They’re just loading “parcels” (features) on to a loading bay (dev) and then into a truck (ops).

Most dev teams have little or no visibility of what happens after those parcels have been “delivered”. What they don’t see at the other end is the customer trying to make something work with what’s in the parcels. Maybe they’re building a rocket, but we keep sending them toasters.

This mismatch in goals/motivations – “deliver all the toasters” vs “build a rocket” – is how the working relationship between dev teams and customers usually breaks down. We should all be trying to build the rocket.

It took humans decades (and hundreds of thousands of people) to learn how to build a rocket, with a lot of science and engineering (guesswork, basically) but mostly a lot of trial and error. We must learn with our customer what really needs be in the parcels we deliver.

The Xmas Train Kata

polar_express_train_christmas_train_little_girl_christmas_polar_winter_holiday-1162035 (2)

It’s been a while since I set a programming challenge with a seasonal theme – about a year, in fact – so here’s a new one which I hope you’ll enjoy, even if you don’t celebrate Xmas.

Imagine we run a railway that Santa and his helpers use to ship presents from their factory in Lapland non-stop to a distribution depot 860 km away just outside Helsinki. (That’s how it works. Don’t argue.)

The elves working at the depot need to know what time to expect the train so they can make sure everything’s ready at their end – reindeer fed, sleigh oiled, Santa’s lunchbox packed, etc. To aid in improving the accuracy of their ETA, they have commissioned us to write a software system.

Positioned at intervals of about 1 km, they’ve placed sensors along the track that send a signal to the depot as the train passes. There are contact points at the front and the rear of the train, so each passing of a sensor triggers 2 data messages – front and rear. The train is 200 m long.

Included in each data message is location information about that particular sensor telling us how far along the track from the factory in Lapland it is in kilometres.

The elves, of course, insist on a JSON format for these data messages, which looks like this:

First Message

{
    'passing': {
        'datetime': '2019-12-24T19:37:46.854Z',
        'contact': 'front',
        'distance': '183.449 km'
    }
}

Second Message

{
    'passing': {
        'datetime': '2019-12-24T19:37:52.229Z',
        'contact': 'rear',
        'distance': '183.449 km'
    }
}

Our software must determine the time that has elapsed between the first and second message, and use that information to calculate the speed of the train as it passed that sensor, and from there calculate an estimated time of arrival at the depot to the nearest minute.

This estimate will be updated with every new pair of messages sent by each sensor along the track to give the elves a live picture of the progress of the train.

Although in real life the train would need to vary its speed depending on conditions, for this exercise assume the train accelerates from rest at a constant 1 m/s/s and decelerates at the same rate, and has a maximum speed of 150 km/h.

For this exercise, you will need to create two programs:

  • One that simulate’s the train’s journey, sending pairs of sensor-passing data messages at intervals approximating every kilometre of the train’s progress
  • Another to receive these messages and update the train’s estimated time of arrival and display it on the screen

The sensors will send one final message when the train has reached the depot to let the receiver know the journey has ended.

Best of luck. The elves are counting on you!