Feature toggles are a useful development technique when there is a need to keep releases flexible and reduce risk of introducing new features to production environments. In our last blog post we covered some of the advantages of utilising feature toggles in your projects. As with any tool, this should be applied with an understanding of the potential pitfalls when putting it to use. This article highlights some of the challenges and concerns that we had to navigate at Flight Centre when converting applications over to the new development process.Because toggles require both the old code and new code to be preserved on the same branch, it is likely that the code base of your application will temporarily become more bloated and complex until toggles are removed. There are a number of ways to separate the old and new logic during development, though the simplest means by far is to use a conditional statement to check if the toggle is active (discussed in the previous article). Depending on the nature of the change being toggled there are other techniques that can be employed, such as dependency injection and dynamic frameworks, but in our experience most small changes are best served by a modest IF statement to toggle. Unfortunately this does have a tendency to also increase the cyclomatic complexity of the code, especially when multiple changes are made and toggled in the same area. Another factor that will contribute to the size of the application are the unit tests that have to be updated to test both the old and new code paths.If you have a number of applications that you manage, then another dilemma will be how to easily configure and monitor the state of available feature toggles without logging into each application individually. This is further compounded by multiple environments (Staging vs Production), different internal customers and different development platforms. When converting a suite of legacy applications, it is important to have a clear goal in mind for how the toggles are to be set and read by the application as well as users and developers. If this is not thought out properly, then the additional level of configuration that is introduced by toggle states will make diagnosing issues significantly harder and quality assurance may become problematic. An ideal scenario is to have a centralised dashboard of the toggle states for applications across different environments so that the customers, developers and testers can all easily identify what the intended behaviour of the application should be. Another point to consider if toggle states are not easily visible, is that identifying what toggles have been active for some time and can be removed becomes more laborious, which in turn can lead to toggles residing in the code base far longer than they have to.A surprisingly common (and often embarrassing) mistake that developers make when first working with feature toggles is actually forgetting to toggle the new code developed in the first place. Once your customers and colleagues expect to be able to toggle a defective feature off when required, it is a large shock and disappointment when a bug cannot be isolated without a patch or rollback. We have seen this happen to the best developers who understand the need to follow an outlined toggle methodology, though sometimes it is also a challenge to convince developers that small changes are worth wrapping in a toggle. Regardless of how small the change is however, there will inevitably come a time when having a toggle available is preferable to the alternative.When developers are faced with applying feature toggles to their work, some changes will prove to be particularly tricky to wrap conditionally. Two examples that we have seen crop up in different applications are code changes that require a corresponding modification to the database and changes to internal configuration files such as XML mappings. In our experience the first can be solved with some care by ensuring that the database changes are backwards compatible with the old logic, so that toggles can be switched on and off without reverting schema changes. The second is a little trickier as the normal solution of wrapping code changes conditionally is not possible in a static configuration file. One option that we have used successfully is to create a new revision of the file and dynamically load the appropriate version based on the state of the toggle. In my experience, most development problems that have been touted as being impossible to toggle only required a little more design consideration to achieve. The end result of course is a far more graceful deployment process.The elephant in the room when considering feature toggles is that implementing the same change will take a developer more time to code and test. For requirements that involve systemic change to an existing application, the cost of feature toggles can rise rapidly due to the number of affected places in the code base. Furthermore, if the application is not developed in a modular fashion and has a lot of repeated code then this can add even more frustration when trying to wrap the required changes in a toggle. Thankfully, this can often be used as an opportunity to modularise the code base, which will make implementing the new toggled logic easier and future maintenance a lot simpler once the toggle is deprecated and duplicate code removed. A common misunderstanding is that all toggle combinations also need to be tested as part of the quality assurance process. A better approach would be to regression test the system with all newly developed toggles deactivated, which is a configuration that is initially deployed to production. After this is complete, individual toggles can be activated and tested in isolation.Introducing feature toggles amongst other methodology changes to Flight Centre was an extreme example where the goal was to homogenise the development culture across a vast multitude of applications and development teams that had, up to that point, very individual development guidelines. The truth is that most implementations of feature toggles will likely follow a more pragmatic approach depending on the needs of the customer and nature of the applications being supported. Many of the disadvantages discussed here can be mitigated with different techniques which we will explore in more detail in future posts.
