Door: Thijs Zumbrink
With almost two years since the last post, you’d think that nothing much happens on this blog. To that I’d say you couldn’t be more wrong! Under the hood there have been many changes and I’d like to detail a few of them in this post.
But why would I actively develop a blog for years without writing in it? Because it’s a safe place to experiment with new approaches to development, and if it’s a success, write about it in the same place! And this post is about one of those experiments that has everything to do with maintainable code.
But first; what lead to the need for this change in code? As developers we should always strive to automate things we do repeatedly. One of those things is shipping new changes to production. The holy grail in this department is a fully automated CI/CD pipeline, that takes any change, tests it for correctness and then automatically ships it to the live server. All without requiring interaction. Quite convenient for those small fixes that you can do in a few minutes, but you’re actually too lazy to deploy at that moment.
But if all your new bugs automatically reach production within minutes, your solution is worse than before! This only works well with reliable tests, and reliable tests only work well with… Maintainable code!
So we need to get code into a maintainable state which is friendly to automated testing. In the code of this blog, and in many projects that want to go through this transition, a radical change (read: cleanup) in the software architecture was in order. So what’s the starting point? What did my code look like before?
I’ve made it a personal challenge (but professionally this is wise too!) to only refactor, not rewrite. But the starting point of the application was an absolute mess! It started in the time I grew up as a PHP scripter, not long after inlining MySQL statements right into the HTML. Of course it worked, but it was unmaintainable to say the least, and any hope of a reliable CI/CD system could be thrown right out the window. Nevertheless, with the standpoint of only refactoring, it was still possible to work with this…
Following the roadmap laid out in Modernizing Legacy Applications in PHP, I cleaned up a lot of internals, separating views from logic, separating database access to specific classes, using a router, controllers and domain objects, etc… This already did a lot of good to the code, but the approach (or perhaps just how I applied it) didn’t offer that much in itself to creating the ultimate maintainable and testable code base.
Later it would turn out that this had much to do with dependency injection and true separation of concerns, things I thought I quite understood but never truly grasped in practice. That is to say, I refactored fat models using services, I applied separation of concerns and I used dependency injection, but somehow testing was still a pain. That is… Until I took a deep dive into…
I’ve taken a good deep look at Hexagonal Architecture and The Clean Architecture. The things I liked were dependencies only pointing inwards toward the pure core of your application, and using interfaces to “plug” the outsides into the insides if the dependency wants to go in the other direction. Uncle Bob explains to make your framework an implementation detail of your application. Your application doesn’t depend on the framework, the framework is a plugin to your application. Don’t marry your framework! You may date the framework, maybe switching a few times, but never marry your framework! Just use the good parts of the framework to do things like authentication, database access, logging, requests/responses, you name it.
What does this have to do with testing? If the framework became a pluggable implementation detail of your application, then your business logic doesn’t directly depend on it. This means it’s possible to completely remove the framework during testing. It doesn’t exist! Let than sink in… If you ask a developer about their issues with writing (unit) tests, the likely answer is that their business logic is not testable because they are so dependent on the framework, the database, network, you name it… What Clean Architecture gives us, and what has been implemented in Schalpoen today, is a completely isolated unit test suite than runs dozens of tests in under a second start to end!
Now how do we make this concrete? I’ll write down here a roadmap of sorts that I went through and that I think is applicable to most developers that want to apply this to their applications.
Watch The Clean Architecture talk by Uncle Bob. Also read up on similar architectures like Hexagonal and Onion.
Start with your focus on a small part of your application. You do not have to refactor everything in one go. Take a simple interaction at the beginning! For example one that reads something from the input and writes a record to a table.
Move the existing code for this interaction to an Interactor class.
Identify which data sources (tables, files, network connections, …) are talked to.
For each data source, make a Repository according to the similarly named design pattern.
Treat the Repository as a dumb data store. No application logic in there whatsoever! This is very important! You should have methods for storing and retrieving. If you need more complex methods for this Interactor, start over with something simpler first.
Write the interface to the Repository first. Define the method signatures that you need from the perspective of your application.
Create two implementations of the interface: one that your application actually needs (if your Posts are stored in the database, make a DatabasePostRepository), and one that behaves the same but stores your objects in a simple array (e.g. MemoryPostRepository).
Realize that the memory implementation of your Repository loses all data at the end of its lifetime, but within that lifetime it’s extremely valuable because it behaves the same as its real counterpart, it’s blazingly fast, and it doesn’t depend on anything.
Read up on Dependency Injection. Don’t concern yourself yet with DI containers, but learn the concept of accepting your dependencies (Repository interface in our case) in the constructor of your Interactor class.
Refactor the Interactor code to use this Repository instead of accessing the data in the way it used to.
Write a test for the Interactor, where you simply instantiate it (remember; you can just call the constructor and give it the memory version of the Repository) and run it.
Assert that your Interactor does the right things by controlling beforehand what the contents of its Repositories are, and asserting them afterwards.
Where the code was before you moved it to the Interactor, instantiate the Interactor and its dependencies and run it.
Go back and tackle the next piece of application logic, taking more difficult ones each time and apply what you've learned so far.
When you’re ready, and not before, use a DI container. Many tools are best understood after suffering without them first.
Enjoy your steadily growing, blazingly fast test suite, which enables you to apply aggressive test coverage techniques like Mutation Testing.
Learn from experts before you start
Take small steps
Organize your business logic
Abstract away dependencies
Enjoy writing tests
Learn while doing
Revel in your success
I have to be clear: this isn’t a quick fix, nor is it always painless. Depending on the current state of your code it may actually be extremely challenging to refactor. For example if your domain objects inherit from any framework objects like an ORM then it’s going to be very difficult. You may actually have to re-implement them in new classes or use some tricks to be able to make the transition.
It may also be challenging to apply this architecture to the full 100% of your application, as you may leave the most difficult parts for last. In that case you could end up with a mix of architectures, which isn’t always bad but can still be messy.
I’ve had the "luck" of starting with frameworkless and architecture-light code, which made my experience with The Clean Architecture nothing but pleasurable!