Any Product Manager worth the Jira Cards they write on will tell you that listening to customers is the most important part of the job. A common question I get from Junior Product Managers is how to successfully run User Studies in the Business to Business environment. There are numerous books, articles, other medium posts and talks on this topic, but most of them go through B2C approaches, not situations where the company and customer work well together and the product is very specific.
 |
| Watching the push back of a Boeing 737-200 freighter in freezing conditions. Location: Nome, Alaska |
Step 1: Justify the Research
Understanding the why is the first step to User Studies. Anyone can run these for the sake of it, but we need to truly understand and justify doing them, because not all products require it, and our users have a limited capacity to deal with our questions, after all, they’ve got day jobs.
Some Good Reasons:
- We don’t have the industry knowledge in house to tell us how to approach the product, and need a subject matter expert.
— Great! Our collective customer base mostly knows what they’re doing, so doing user studies with them is a great way to glean information.
- Our customers didn’t receive our last release very well, and you think the team is missing something in the user process.
— Your gut instinct is probably right, and we need to sit down with the users and watch them work to understand what we missed.
Some Bad Reasons:
- The PM / UX / Sales team wants you to see the customer.
— Meeting customers is great, but we need a purpose and a problem.
- The customer really wants us to spend time with them and see how their process works.
— This is a great way to build a custom feature for that customer.
Some Alternatives:
- Use existing metrics / KPIs / Business Intelligence.
- Walk over to Customer Success and ask them what the customers want.
- Call up your favourite customer and see what they’re up to.
- Go online and read up on what the industry is doing.
Goal Setting
Now that we know that we want to do the study, we need to establish clear goals and outcomes. For me personally, I tend to work on brand new products and therefore my goal is to understand the entire workflow of a range of customers so that I can build an end-to-end solution that will suit their needs and the wider industry. Of course, for smaller feature requests, this can be as simple as “Identify ways to make x work faster for the user.”
Step 2: Research what the Industry is doing
Before you even approach a customer, you should make sure you do some preliminary industry research on the product or feature you’re going to run a user study about. This is beneficial for a few reasons. Firstly, you will gain a bit of subject knowledge, but more importantly, you will begin to understand the practices used in other companies and by your competitors. This will help formulate what you need to do to gain a competitive advantage, where your product lacks, and what the rest of the industry is doing.
Step 3: Draw up some Prototypes
Adobe XD has become my best friend, because it lets me draws squares, circles, and lines easily that somehow become wireframe designs. Customers love visual ques, and you’ll always get a better reaction with a design over a document. Based on the knowledge you currently have, draw up a few designs, processes, and components.
Many people would argue that the UX team should be doing this — and I agree, they should be involved from Day 1, but as a PM, its also good for you to understand and cement your knowledge by applying what you’ve learnt in an activity like this.
Regardless of who does it, based on Step 5 and 6, these prototypes should be continually refined and grown as you meet more and more customers. Keep copies of them all, and pick and choose the ones you want to show customers, but you will start to organically grow the product designs through this process, and give your UX team a journey as they start planning design.
Step 4: Find a diverse set of Customers
This one is the tricky bit and I suggest walking over to Customer Success and asking them to do the hard work. A diverse range of customers is really important so you don’t end up building a product for a narrow customer base that can’t really grow or scale.
Cultural Difference should be factored for when you pick your customers — remember that some cultures react differently to external guests being with them to others, so its important to work with a range of customers where both sides are comfortable and open.
Step 5: Observe the Customer in their Environment
Observing customers physically and in their environment is crucial, and often gets a lot of pushback by management. Remember, the cost of visiting a customer is far lower than the cost of reworking a product, so justify it, and go do it.
On site with the customer, ask to just observe what they’re doing. Don’t breathe down their necks, and don’t ask a question every second, but try to be their apprentice for the day “learning the ropes”. This is my favourite part of being a Product Manager, it teaches me new skills, lets me experience what our customers go through in their role and how it relates to our product, and sometimes, I get to do cool things (like watching jets push-back on the tarmac!!).
During this process, its important to take detailed notes on their day, processes and outcomes. Also take note of distractions, morale, and everything else that will impact the user on a day to day basis. Often its the external factors that influence your product more than anything else.
Step 6: Discuss and Debate with the Customer
This isn’t a trial by Product Manager and its certainly not a debating club, but now is your chance to flesh out what you know, what the customer knows, and what you want to know. This is a good place to understand where the differences are between what your customer wants vs what the industry needs, and also understand what the specific customer might be doing differently to the other customers you’ll be interviewing.
I am really honest and transparent with my customers in this process, and openly question their methods, suggest improvements, and provide feedback on processes I think they’re doing really well.
This is also an important phase to write down the compromises the customer will agree to. These compromises can play a major role in product backlog priorities downstream, and by understanding what the customer will accept now vs 6 months from now will be a huge help as we focus on execution.
Step 7: Follow up at the end of the study
At the end of your tour of duty visiting customers and understanding how they work, you should have a laundry list of notes, drawings, funny circles and highlights. Take these and spin this into a customer facing whitepaper that you can give back to your customers. Every customer is unique, and by creating a white paper that accounts for all the needs of your customers, you can give your customers a non-promising look into what you learned, finalise and collect last minute feedback, and literally get everyone on the same page.
I want to emphasize that your customers may be current or future competitors, so its very important you don’t disclose specific processes, reveal customer names, or use analogies for a specific customer. Words like “In general, most customers followed x approach”, and “It was a highlight to see many customers using x”, will go down a lot better than “One customer used x strategy”.
Step 8: Present Internally
At this point, you should have a clear understanding of what you’re customers are doing, and hopefully a lot of artefacts that you can use to build your product. Be sure to document these, and start the wheels with UX, Engineering, Sales, Customer Success and whoever else is relevant. Sharing knowledge is key to develop customer empathy internally, and will help the wider company understand what’s needed to build product well.
Some Other Notes
What about our Unconscious biases?
Step 6 should help remove these, but biases will always exist, and the best way to mitigate this is by acknowledging it. Cold hard facts from a good sample will always provide effective business intelligence.
What about UX Researchers?
If you have UX Researchers, that’s great! But remember that they’re focused on user experiences, whereas your role is to understand at a depth below that as well. Customer processes and procedures influence engineering too!
Who else should I have with me?
I like visiting with a Customer Success Manager because customers seem to like them, and they can fill in the blanks or gotchas that the customer may not describe because its obvious to everyone (except you). Also consider taking a more Junior PM as part of upskilling.
Who shouldn’t I have with me?
This is a gut instinct you need to make, but I prefer visits where management isn’t around (on both sides — even if management are lovely), because it helps the user study flow a lot better as both sides see each other as being on the same level and page.