We’re now 4 days in to our Moonshots event, a week in which the entire company is dedicated to new innovation.
Friday ended on a real high, with 12 of our 18 teams doing a show ‘n’ tell; what they’d achieved in such a short space of time was both exciting and inspiring. But no matter how impressed we are, no matter how much we love the ideas, what really matters is that our customers love them.
The test and learn cycle is core to the way we work. As much as possible we try to validate assumptions early by getting our ideas in front of customers and getting their feedback. We do this both by regular A/B testing and also by testing face to face with customers. Moonshots is no different, and over the last two days we’ve had 8 customers in to feedback on some of our Moonshots projects.
So what did they think???
By and large the reaction was really positive and as always, we picked up some really helpful hints at how to improve the initial ideas.
Customers loved the two new products we’re planning to launch – particularly when they saw the physical products themselves. They had thought one was too expensive, until they saw the size, so now we know we need to make that clear at the point of sale – size really does matter!
Two features were well-received but customers struggled to use them, so we’ll be refining the UI before they go live to a wider audience.
One new feature fit in so seamlessly, customers weren’t actually sure what they were testing. They also loved the animation with it, and it’s always nice to surprise and delight your customers!
To expect a full house would have been too much, and there was one feature that customers couldn’t see the value of. But every cloud has a silver lining, and they did suggest a different way in which it would be useful, so there’s a good opportunity to pivot.
Our User Experience Researcher, Amy, said “It’s one of the best test sessions I’ve run. Most of the ideas had a really positive reaction, and customers really enjoyed trying out such a wide variety of different things.”
So now that we have some feedback, what happens next?
Over the last few years we’ve heavily invested in continuous deployment in order to facilitate a lean development process and validate ideas early. Feature toggling is a cornerstone of that approach, and once we’ve refined ideas based on the initial feedback we’ve had, we’ll gradually start enabling the new features and testing their success.
Where we’ve introduced new products we’ll be tracking their popularity, and where we’ve launched new features, or feature enhancements, we’ll be running A/B tests to measure their impact.
One of the key Moonshots rules was that the code could be rough and ready. Whilst code quality is highly valued, we know that until our ideas have been validated by customers, time dedicated to making the code scale-able and maintainable is potentially time wasted. The ideas that prove successful with customers are the ones we will invest in.
Nothing developed during the Moonshots will have a long term future in its current form. We’ll look to prove that customers like the ideas, but we’ll also be looking to see how they use them and what improvements we might need to make. That learning might inform further changes to the UX, and potentially to the implementation. With this knowledge we’ll be refactoring and iterating on successful ideas to ensure they meet customer’s needs and meet the standards of production code to which we hold ourselves. Ideas that haven’t proved popular we’ll remove altogether.
So it will be some weeks before we can really evaluate the success of the teams’ efforts, but what are the aspirations for the future of Moonshots? We asked Engineering Director, Steve Beard, and Product Director, Jane Honey, where they’d like to see Moonshots go.