Refactoring Legacy Code Safely using Scientist .net

Don’t touch that code, it might break !!!

At Moonpig we have legacy systems which need to be refactored to improve our architecture, and enable a more scalable solution. As with many companies, lots of this code was written such a long time ago, and can be so complex, that no one fully understands the consequences of making changes. Scientist .net helps to de-risk changes and check that refactorings are not changing the behaviour of the code in ways that were not intended. The framework is especially useful in situations where there could be lots of edge cases which are difficult to capture even with lots of up-font testing before a release.

Testing in Production

With Scientist .net you test your code in production with Live data. The new refactored code and existing code are run side by side with the same Live input data, and the results of the two sets of code are compared. If any differences are found, they are logged so that they can be investigated later on. Then just the result from the existing code is actually returned, and result from the refactored code is discarded.

Because the Live data will be fully representative of all the scenarios that the system has to cope with, and at a real level of performance load and timings, you can be confident that it covers all of the required combinations of inputs. (The length of time the experiment needs to run to capture all of these inputs is up to you as it depend on the domain of your change).

[Also be careful of performance issues, running two sets of code side by side may have a performance impact, this is something you will need to think about on a case by case basis!!]

Code Example

The framework was originally written in Ruby, but we use the .net implementation. Here’s an example of how the code looks

Scientist.Science<Product>("search improvement experiment", experiment =>
{
   experiment.Use(() => ProductSearchOld(searchParameters));
   experiment.Try(() => ProductSearchNew(searchParameters));
});

Both the Use and Try methods are called and the results compared. The order that they are called is randomised each time, any exceptions raised from the Try method are swallowed and do not affect the live production code. Also the durations of the methods are recorded and can be logged to check for any performance impacts. There is support to output to a custom logging implementation so that if you have a centralised logging stack then it can be integrated, (we use the ELK stack). It can also run multiple Try methods, and also run the different implementation in parallel with async calls.

We also wrap our experiments inside a feature toggle so that we can run them for a short amount of time to collect some data, and then switch them off and check the results.

Usefulness

Scientist is not necessarily useful for all changes, we don’t use it for every method update. It’s not as easy to use if the method may be changing data in a database which you do not want to be changed twice. It is most useful for refactoring legacy code.

The scientist technique does not necessarily need a framework to use, this style of implementing changes in a staged approach could be setup in any language. We’ve found the technique to be a very useful safety net, and it’s helped us to catch bugs before fully releasing changes for customers to see, and it has increased our confidence that changes are behaving in the way that we intended.

 

For more information check out the github repos here –

.net – https://github.com/github/Scientist.net

Ruby – https://github.com/github/scientist

 

Leave a Reply

Your email address will not be published. Required fields are marked *