We’re not religous about user testing, and most of our smaller features only go review by one or two people before getting deployed. But larger features can get a special treatment, and here’s how we do it.
It depends on the size of a feature, but before shipping a major feature, we typically work like this:
- First, every developer needs to give their feedback, and we fine tune until we’re happy
- Then we show the feature to a customer success agent, watching their screen while they explore. We tend to show a feature to the most enthusiastic person first, take their feedback on board, show it to the next person, and so on
- Once all SI staff are happy, we pick one or two customers and let them explore a new feature while we watch (screensharing again). This often reveals entirely unexpected problems or shortcomings. A favorite user testing session went like this: We had rewritten a very large admin screen from scratch, made it tons more useful, stable, faster and prettier, and we were really proud of it. Our tester, a long term SI customer, looked at the screen briefly, muttered “yep, looks like always, nothing special here.” (we were silently crying at this point). Then she said “dang, there is still no preview-feature for emails that I trigger, does it have to be this cumbersome?”. This was her one feedback – and we realised that we had missed an elephant in the room. Fortunately, she did like the overall screen (although she didn’t realise just how much worse the original version had been) and after adding said email preview feature, she was happy.
- Once we’ve satisfied two or three key users, we enable a new feature in beta mode for select customers to use permanently. We ask them via mail or phone how things are going, and since each of our application pages has a prominent “give feedback” button, we do hear when there are problems
- In the next phase, we might announce the beta over our newsletter or via in-application notification, and allow customers to enable a new feature if they like. This increases adoption and feedback again.
- In the final step, we switch all customers to the new feature, while leaving a “keep the old version?” button in place for another month or two. Once nobody uses the old feature anymore, we entirely drop it, and all associated code.
Well, it’s no rocket science really, but many applicants have been burnt by much worse development processes, so we wanted to share this!
