Action->Object vs. Object->Action

One of the factors that I see programmers new to objects struggling with is our natural tendency to separate agency from data. Things do things to other things. The VCR plays the video. The toaster heats the toast. The driver drives the taxi. Etc.

I think it’s possibly linguistic, too, that we – in most natural languages – but the object after the verb: play(video), toast(bread), drive(taxi).

Thing is, though – this isn’t how object oriented programming works. Objects encapsulate agency with the data it works on, producing video.play(), bread.toast() and taxi.drive().

In OOP, the cat kicks its own arse.

You’re absolutely correct if you’re thinking “That isn’t how we’d say or write it in real life”. It isn’t. I suspect this is one of the reasons some programmers find OOP counter-intuitive – it goes against the way we see the world.

Ironically, Object thinking – while not intuitive in that sense – makes discovery of actions much easier. What can I do with a video? What can I do with bread? And so forth. That’s why Object->Action still dominates UI design. Well, good UI design, anyway. Likewise, developers tend to find it easier to discover functions that can be applied to types when they start with the type.

When I wrote code to tell the story of what happens when a member donates a video to a community library, each line started with a function – well, in Java, a static method, which is effectively the same thing. This is not great OOP. Indeed, it’s not OOP. It’s FP.

And that’s fine. Functional Programming works more the way we say things in the real world. Clean the dishes. Set the timer. Kick the cat. I suspect this is one reason why more and more programmers are draw to the functional paradigm – it works more the way we think, and reads more the way we talk. Or, at least, it can if we’re naming things well.

(There’s a separate discussion about encapsulation in FP. The tendency is for functional programmers not to bother with, which leads to inevitable coupling problems. That’s not because you can’t encapsulate data in FP. It’s just that, as a concept, it’s not been paid much attention.)

If you’re doing OOP – and I still do much of the time, because it’s perfectly workable, thank you very much – then it goes Object->Action. Methods like play(video) and kick(cat) hint at responsibilities being in the wrong place, leading to the lack of encapsulation I witness in so much OO code.

It’s like they say; give a C programmer C++, and they’ll write you a C program with it.

 

 

 

Do Your Unit Tests Know *Too* Much?

A common pitfall of extensive unit testing reported by many teams is that, as the number of tests builds up, changing the implementation under test forces them to rewrite many, many tests. In this scenario, the test code becomes a barrier to change instead of its enabler.

Having witnessed this quite a few times first-hand, I’ve got a few observations I want to share.

First of all, it’s actually usually a problem caused by unmanaged dependencies in our implementation code that causes changes to ripple out to large numbers of tests.

Imagine we wrote a unit test for every public method or function in our implementation, and we decide to change the way one of them works. If that breaks a hundred of our unit tests, my first thought might be “Why is this method/function referenced in a hundred tests?” Did we write a hundred distinct tests for that one thing, or is it used in the set-up of a hundred tests? That would imply that there’s no real separation of concerns in that part of the implementation. A lack of modularity creates all kinds of problems that show up in test code – usually in the set-ups.

The second thing I wanted to mention is duplication in test code. There’s a dangerous idea that’s been gaining in popularity in recent years that we shouldn’t refactor any duplication out of our tests. The thinking is that it can make our tests harder to understand, and there’s some merit to this when it’s done with little thought.

But there are ways to compose tests, reuse set-ups, assertions and whole tests that clearly communicate what’s going on (many well described in the xUnit Test Patterns book). Inexperienced programmers often struggle with code that’s composed out of small, simple parts, and it almost always comes down to poor naming.

Composition – like separation of concerns – needs to be a black box affair. If you have to look inside the box to understand what’s going on, then you have a problem. I’m as guilty of sloppy naming as anyone, and that’s something I’m working on to improve at.

There’s one mechanism in particular for removing duplication from test code that I’ve been championing for 20 years – parameterised tests. When we have multiple examples of the same rule or behaviour covered by multiple tests, it’s a quick win to consolidate those examples into a single data-driven test that exercises all of those cases. This can help us in several ways:

  • Removes duplication
  • Offers an opportunity to document the rule instead of the examples (e.g., fourthFibonacciNumberIsTwo(), sixthFibonacciNumberIsFive() can become fibonacciNumberIsSumOfPreviousTwo() )
  • It opens a door to much more exhaustive testing with surprisingly little extra code

Maybe those 100 tests could be a handful of parameterised tests?

The fourth thing I wanted to talk about is over-reliance on mock objects. Mocks can be a great tool for achieving cohesive, loosely-coupled modules in a Tell, Don’t Ask style of design – I’m under the impression that’s why they were originally invented. But as they give with one hand, they can take away with the other. The irony with mocks is that, while they can lead us to better encapsulation, they do so by exposing the internal interactions of our modules. A little mocking can be powerful design fuel, but pour too much of it on your tests and you’ll burn the house down.

And the final thing I wanted to highlight is the granularity of our tests. Do we write a test for every method of every class? Do we have a corresponding test fixture for every module in the implementation? My experience has been that it’s neither necessary nor desirable to have test code that sticks to your internal design like cheese melted into a radio.

At the other extreme, many teams have written tests that do everything from the outside – e.g., from a public API, or at the controller or service level. I call these “keyhole tests”, because when I’ve worked with them it can feel a little like keyhole surgery. My experience with this style of testing is that, for sure, it can decouple your test code from internal complexity, but at a sometimes heavy price when tests fail and we end up in the debugger trying to figure out where in the complex internal – and non-visible – call stack things went wrong. It’s like when the engineer from the gas company detects a leak somewhere in your house by checking for a drop in pressure at the meter outside. Pinpointing the source of the leak may involve ripping up the floors…

The truth, for me, lies somewhere in between melted cheese and keyhole surgery. What I strive for within a body of code is – how can I put this? – internal APIs. These are interfaces within the implementation that encapsulate the inner complexity of a particular behaviour (e.g, the cluster of classes used to calculate mortgage repayments). They decouple that little cluster from the rest of the implementation, just as their dependencies are decoupled from them in the same S.O.L.I.D. way. And their interfaces tend to be stable, because they’re not about the details. Tests written around those interfaces are less likely to need to change, but also more targeted to a part of the design instead of the whole call stack. So when they fail, it’s easier to pinpoint where things went wrong. (Going back to the gas leak example, imagine having multiple test points throughout the house, so we can at least determine what room the leak is in.)

Even for a monolith, I aim to think of good internal architecture as a network of microservices, each with its own tests. By far the biggest cause of brittle tests that I’ve seen is that your code quite probably isn’t like that. It’s a Big Ball of Mud. That can be exacerbated by leaving all the duplication in the test code, and/or by over-reliance on mock objects, plus a tendency to try to write tests for every method or function of every class.

You want tests that run fast and pinpoint failures, but that also leave enough wiggle room to easily refactor what’s happening behind those internal APIs.

Code Craft is More Throws Of The Dice

On the occasions founders ask me about the business case for code craft practices like unit testing, Continuous Integration and refactoring, we get to a crunch moment: will this guarantee success for my business?

Honestly? No. Nobody can guarantee that.

Unit testing can’t guarantee that. Test-Driven Development can’t guarantee that. Refactoring can’t guarantee it. Automated builds can’t guarantee it. Microservices can’t. The Cloud can’t. Event sourcing can’t. NoSQL can’t. Lean can’t. Scrum can’t. Kanban can’t. Agile can’t. Nothing can.

And that is the whole point of code craft. In the crap game of technology, every release of our product or system is almost certainly not a winning throw of the dice. You’re going to need to throw again. And again. And again. And again.

What code craft offers is more throws of the dice. It’s a very simple value proposition. Releasing working software sooner, more often and for longer improves your chances of hitting the jackpot. More so than any other discipline in software development.

Visualising Architecture – London, Sept 4-5

Just wanted to quickly announce the first of a new series of hands-on 2-day workshops on software architecture aimed at Agile teams.

Visualising Architecture teaches diagramming techniques that developers can use to help them better understand and communicate key architectural ideas – essential to help quickly and effectively build a shared understanding among team members.

The public course will be in London W1A on June 4-5. If you’re looking to build your Architect Fu, this is a great investment in your’s or your dev team’s software design skills.

You can find out more and book your place by visiting the Eventbrite page.

Software Architecture (for Architectless Teams)

Since the rise of Agile Software Development post-2001, it’s become more and more common for development teams to work without a dedicated software architect. We, quite rightly, eschewed the evils of Big Design Up-Front (BDUF) after the Big Architecture excesses of the 1990s.

As someone who worked in senior architecture roles, I can attest that much of the work I was asked to do – e.g., writing architecture documents – added very little value to the end product. Teams rarely read them, let alone followed their blueprints. The resulting software turned out the way the developers wrote it, regardless of the architect’s input. Which is why I returned to hands-on development, realising it was the most direct way to have an influence on the actual software design.

But I can’t help feeling we threw the baby out with the bathwater. Too many teams seem to lack a vision of the bigger picture. They don’t see how the pieces fit together, or how the system fits in the real world of businesses or homes or cars or hospitals etc. The design emerges through the uncoordinated actions of individual developers, pairs and teams (and teams of teams) with no real oversight and nobody at the steering wheel.

Big decisions – decisions that are hard to reverse – get made on the fly with little thought given to the long-term consequences. For example, thanks to the rise of build and package management solutions like Maven, NuGet and NPM, developers these days can take on dependencies on third-party code without giving it much thought. We’ve seen where unmanaged dependencies can lead.

Most damaging, though, is the inability of many Agile teams to effectively collaborate on the design, making the bigger decisions together in response to a shared understanding of the architectural requirements. I’m reminded of a team I worked on who went off in their pairs after each planning meeting and came up with their own architectures for their specific user stories. The software quickly grew out of control, with no real conceptual integrity to the design. It became three architectures in one code base. We had three distinct MVC styles in the same software at the same time, and even three customer tables in the database. A little time around a whiteboard – or a bit of mob programming every now and again – could have minimised this divergence.

I firmly believe that software architecture is due for a comeback. I doubt it would return in the Big Architecture form we rejected after the 90s – and I’ll work hard to make sure it doesn’t. I doubt that such things as “software architects” will make a grand return, either. Most Agile teams will still work without a dedicated architect. Which is fine by me.

But teams without architects still need architecture. They need to be discussing, planning, visualising, evaluating and evolving the Big Picture design of their software as a cohesive unit, not as a bunch of people all doing their own thing.

Through Codemanship, I’m going to be doing my bit to bring back architecture through a new training offering aimed at Agile teams who don’t have a dedicated person in that role.

Rolling out later this summer, it will focus on key areas of software architecture:

  • Visualising Architecture (yes, the boxes and arrows are back!)
  • Architectural Requirements (runtime and development time requirements of design)
  • Collaborative Design Processes (how do we go from a user story to a software design, making key decisions as a team?)
  • Contextual Architecture (how does our software fit into the real world?)
  • Strategic Architecture (how does our software solve real business problems?)
  • Evaluating Architecture (how do we know if it’s working?)
  • Evolving Architecture (sure, it works now, but what about tomorrow?)
  • Architectural Principles (of maintainability, scalability, security etc)
  • Architectural Patterns (common styles of software architecture – SOA, Monolith, Event-Driven, Pipes-and-filters, etc)

That’s a lot of ground to cover. certainly a 2-3-day workshop won’t be anywhere near enough time to do it justice, so I’m devising a series of 2-day workshops that will focus on specific areas:

  1. Visualising
  2. Planning
  3. Evolving
  4. Principles & Patterns

All will place the emphasis on how architectural activities can work as part of an Agile development process, and all will assume that the team shares the responsibility for architecture.

Keep your eyes on the @codemanship Twitter account and this blog for news.

 

The Power of Backwards

Last week, I posed a challenge on the @codemanship Twitter account to tell the story of Humpty Dumpty purely through code.

I tried this exercise two different ways: first, I had a go at designing a fluent API that I could use to tell the story, which turned out to be very hard. I abandoned that attempt and just started coding the story in a JUnit test.

This code, of course, won’t compile. If I paste this into a Java file, my IDE has many complaints – lots of red text.

humpty_initial

But, thanks to the design of my editor, I can work backwards from that red text, declaring elements of code – methods, classes, variables, fields, constants etc – and start fleshing out a set of abstractions with which this nursery rhyme can be told.

humpty could be a local variable of type Humpty.

humpty_backwards

dumpty could be a field of Humpty, of type Dumpty.

humpty_field

sat() could be a method of Dumpty, with parameters on, a and wall.

dumpty_method

And so on.

Working backwards from the usage code like this, I was able to much more easily construct the set of abstractions I needed to tell the story. That is to say, I started with the story I wanted to tell, and worked my way back to the language I needed to tell it.

(You can see the source code for my second attempt at https://github.com/jasongorman/HumptyDumpty)

As well as being an interesting exercise in using code to tell a story – something we could probably all use some practice at – this could also be an exercise in working backwards from examples using your particular editor or IDE.

In TDD training and coaching, I encourage developers to write their test assertion first and work backwards to the set-up. This neat little kata has reminded me of why.

but don’t take my word for it. Try this exercise both ways – forwards (design the API then tell the story) and backwards (tell the story and reverse-engineer the API) – and see how you get on.

If I’ve spoiled Humpty Dumpty for you, here are some other nursery rhymes you could use.

 

 

 

 

Design Discovery In TDD Through Refactoring – An Example

There are many ways we can introduce new types of objects in a test-driven approach to design. Commonly, developers reference classes that don’t yet exist in their test, and then declare them so they can continue writing the test.

But if you favour back-loading the bulk of your design decisions until after you’ve passed the test, there’s a simple pattern I often use to help me discover objects through refactoring. The advantage of this see-the-code-then-extract-the-objects approach is that we’re more likely to end up with good abstractions, since the design of our objects is based purely on what’s needed to pass the test.

Let’s take a shopping basket example. If I write a test for adding an item to a shopping basket that references no implementation, with all the code contained inside the test:

This design currently suffers from a code smell called “Primitive Obsession”. There’s just raw, exposed data. And we’d expect this if we hadn’t encapsulated any of the logic inside classes (or closures, if you’re that way inclined.)

The test passes, though. So, as icky as our design might be, it works.

Time to start refactoring. First, let’s isolate the action we’re testing into its own method.

What I’m interested in when I see a stateless method like addItem() is whether there’s an object identity lurking among its parameters – a potential this pointer, if you like. Right now, this method does action->object. To make it OO, we want to flip that around to object.action. I reckon the object in this case – the thing to which an item is being added – is the basket collection. Let’s introduce a parameter object for it.

This method is about adding an item to the shopping basket – as evidenced by the line:

shoppingBasket.getBasket().add(item);

We can eliminate this Feature Envy by moving addItem() to shoppingBasket, making it the target of the invocation.

Now we can see that the code for calculating the total really belongs in the new ShoppingBasket class, putting that work where the data is. First we extract a total() method.

Then we can move total() to the object of its Feature Envy.

Looking inside the ShoppingBasket class, we see we have a bit of cleaning up to do.

Let’s properly encapsulate the list of items.

Our test code is much, much simpler now (a sign of improved encapsulation).

But we’re not done yet. Next, let’s turn our attention to the Long Parameter List code smell in addItem().

We can introduce a parameter object that holds all of the item data.

And now that we have a BasketItem class that holds the item data, we can add that to the list instead of the ugly and not-very-type-safe object array.

Then we clean up the test code a little more, with various inlinings.

Almost there.

The total() method has Feature Envy for data of BasketItem.

Let’s fix that.

This helps us to improve encapsulation in BasketItem.

Two last little notes: firstly, it turns out that for this particular action, product code and product description aren’t needed. I see developers do this often – modeling data based on their understanding of the domain rather than thinking specifically “what data is needed here?” This can lead to redundancy and unnecessary complexity. Until we have a feature that requires it, leave unused data out of the design. Always be led by function, not by data. We might get a requirement later to, say, report how many units of P111 we sold each day. We’d add the product code then.

Let’s fix that.

And finally, I started out with a test fixture for a “Shopping Cart”, but as the design emerged, that concept changed. It’s important to keep your tests – as living documentation for your code – in step with the emerging design, or things will get confusing.

Let’s fix that.

Now, you might have come up with this design up front. But then again, you might not. The benefit of this approach to discovering the design is that we start only with what we need, and end with an equivalent version with the code smells removed and nothing we don’t need.

The price is that you spend more time refactoring. But as the model emerges, we tend to find that this overheard decreases, and the emphasis shifts to reusing the design instead of discovering it every time. In other words, it gets easier as we go.