The 7 learnings of highly effective product interviews

I had the good fortune to be in the lead to hire product managers at Shpock for quite a while now. Shpock is one of the leading European mobile marketplaces, with millions of active users. I’ve interviewed dozens and dozens of candidates from across all of Europe to make sure we grow the team with the right people. Most of these were first round interviews after an initial screening of the CV.

It’s no secret that it is particularly difficult to hire a good product manager since the role has such a wide range of responsibilities, requiring a multitude of skills. One of the most important things to do is to figure out what type of product manager you need. Once you’ve done that, you can figure out what type of candidate you’re talking to and if it matches what you offer. This can be hard to see on a CV, as this is often a collection of the right buzzwords, and can typically be exaggerated quite a bit. Potential mismatches exist for roles outside of product management of course, but my intuition tells me that the risk of a mismatch is positively correlated with the complexity of the role you’re hiring for.

I started using a template for note-taking during the interview process and made sure we checked off some of the core skills that are important to us in a product role. Below is my list. It’s not perfectly polished, but I use this with every candidate and make sure to touch the majority of these topics during the first round.

• Communication — Clear and concise English skills
• Creativity — Able to come up with high level creative solutions
• Mobile B2C — Native or at least web consumer platform experience
• Ownership — Can they own a product and team
• Discovery — Design sprints, UX process, user testing, prototypes
• Data — Conversions, A/B testing, tracking
• Strategy & vision — Long-term thinking, USP, how to implement vision
• Team setup — Cross-functional, size, reporting structure, improvements
• Planning — Scrum, OKRs, roadmaps
• Marketplaces — Biggest challenge, experience, who is our customer

This will be different from company to company of course, but I suspect at least the majority of the above are qualities you’ll want to check for when hiring a (senior) product member into your team. It’s also not an exhaustive list by any means. It doesn’t check for technical capabilities at all for example. If you’re hiring a more technical product manager, I imagine this list to be different.

After having interviewed all these people, here are my top 7 learnings for effective product manager interviews.

1. The best interviews are more like conversations

My style of interviewing is typically more conversational to begin with, and I’ve gotten some good feedback from candidates that this was well received (which is important in itself). When the candidate seems to lack certain core product skills from the list (e.g. user empathy), it is important to keep testing that potential weakness with deeper questions, ultimately making the interview a lot less conversational. On the other hand, if you feel the candidate masters all core skills of product management pretty well, you can easily go into a more conversational style. This still enables you to assess their skills, but with the added benefit of them being more relaxed during the time of the interview.

If you felt the interview was more like an interrogation, you were most likely digging deep trying to convince yourself the candidate is worth a shot. If this was the case, the candidate is probably not the correct one for the job. If you’re in doubt, the answer is no. Good interviews feel like smooth conversations, and sometimes you should even expect to learn something yourself.

2. People are a product of their environment

Although it is not strictly necessary for a candidate to be in a company that has a modern product development structure and an agile way of working (cross-functional, autonomous, and outcome driven teams), you can still understand if they at least question if their team and reporting structure is efficient. I strongly believe that product managers need to be good at process management. Efficiency is of course not a goal in itself, but it will for sure help you a lot when delivering features and testing hypotheses that you or the team came up with during your product discovery process. I’ve noticed that many candidates who are in a completely inefficient environment also do not know they are in one. Maybe they haven’t seen what an actual good product organisation looks like, but I cannot help but assess this as a negative point. Books and articles on product management are easy to pick up, and it doesn’t take a genius to reflect this back to how your organisation works on a daily basis. Just reading the classic product management book ‘Inspired’ by Marty Cagan, which I consider required reading for each product manager, can easily make you see that you’re doing it wrong.

Now, I’m the first one to admit that an agile way of working is a broad mindset and product operations don’t necessarily need fixed frameworks like OKRs or even scrum. I’ve seen good organisations succeed without them, but if you need to go through a business developer to talk to your remote development team, there is clearly a problem.

3. Decide to hire generally or specifically

It seems that there are two distinct philosophies to hiring. You can hire for a specific position or hire more generally. When you’re looking for a product manager with, say, search experience, it makes a lot of sense to try to hire someone specifically with that skill. If you’re unable to hire such a person, you’re looking at a longer term investment in an employee as they’ll have to learn concepts like fuzzy matching, synonyms, etc.

The ideal case to me seems to be hiring for both at the same time. Product managers are generalists by definition, but ultimately it makes sense for them to be T shaped, just like good developers are nowadays. I believe this concept should be more ubiquitous throughout the industry. The challenge here is of course to find those people in the first place. The experience I have is that the vast majority of product people are generalists and have some experience everywhere, instead of really specialising in some topic. Just to be clear, I don’t count domain knowledge as specialisation per se, but rather define it as a deeper subset of a domain or a technical aspect of a product (e.g. payments architecture).

4. Prepare to spend a lot of time

In Europe, where I have done all my interviews, there is a massive shortage of good product managers. There is a shortage of knowledge, awareness, training, and just about everything you need to foster a deeper and better product culture (see this article for some statistics to show you I’m not exaggerating). The many candidates from all across the continent that I’ve seen have had big differences in seniority, based on multiple factors.

One such factor is if the person is living in one of the main tech hubs in Europe. London, Berlin, Barcelona, Paris, Stockholm, and Amsterdam are clearly the biggest ones. What happens in those hubs is a cross-pollination of product skills, which is of utmost importance to grow product managers, but also to attract more talent to these types of jobs. London for example has an abundance of good product managers. The trend towards remote teams has many strong advocates, and is stronger than ever in these times, but the importance of cross-pollination in tech hubs is at least one example that speaks against remote work.

In short, if you are not in one of the main tech hubs (Shpock’s HQ is in Vienna), prepare to search for a long time, pay the right price, and strongly consider relocating people from elsewhere.

5. Look for good storytellers

First of all you need to master good communication as a product manager. This is perhaps the most cliché competence required for any job, but it is surprisingly average for many candidates that I’ve seen. A lot of people in Europe don’t have English as their native language, but it is absolutely crucial to have good English skills, both speaking and writing. I often joke that a European who speaks good English can communicate perfectly with an American, but not with many other Europeans that don’t master the language (since the chances of them having different native languages is quite high). Being able to clearly and concisely communicate as a product manager in a universal language is an absolute must. There is just no way around it.

On top of good communication skills you need storytelling skills. Quite a few product managers I’ve seen cannot really build good presentations for example. I respect that they are perhaps more data driven and make boring slides, but that seems to me a bad excuse. I would say with some confidence that less than half of the candidates I meet are capable of really making a great presentation and building a good story. Now, you can argue slides are not important, and that’s fair. But I see this as not just a ‘make some pretty slides’ skill. In my opinion, it’s a core competence of product managers to be great storytellers. Teams should be autonomous, but full autonomy is an illusion, and good storytelling skills can come in more than handy if you need to get something approved. Notice that I intentionally do not call this salesmanship, because the term “selling” is something that is often negatively connoted. It can sound like a trick with short term interest, which is not the case here.

Storytelling as a product manager is ultimately an effective way to get a point across, and it can be used to bring alignment together with constant and proactive communication. A product manager could have a complex feature with many different layers that you want to push forward that needs a lot of explaining, but he or she will still want to make it understandable outside of their own team. In the end, someone’s ability to present a convincing story about something is directly correlated with the clarity of their understanding of that thing.

6. Value conciseness

For each interview I took short notes in real-time, keeping track of positive, negative, or neutral comments in a simple bullet list. The most common negative thing is that candidates talk for too long on a clearly formulated question, sometimes deviating into completely unrelated topics. This can be interpreted as contradictory to the first point of conversations, but a conversational style does not necessarily mean deviating from the topic at hand. The ‘risk’ with product interview questions (as opposed to technical questions in a developer interview) is that they are typically very open ended. Asking about a good product strategy is a very broad question, and it takes skill to answer it precisely and stick to the point.

To try to keep the conversation focused, I’ll sometimes interrupt the candidate so we can move on. But since it is rude to interrupt a candidate all the time, I won’t always do this. In that case you’re losing valuable time that could otherwise be used for more on-topic questions. I cannot overstate how important it is to be honest when you don’t know something. I believe some of the best advice when being interviewed is to be confident about not knowing things. Candidates that are open about their weaknesses, and give an explanation on how they plan to tackle a challenge that would require them to improve on their weakness, get a bonus point.

7. Embrace uncertainty

You have to embrace the fact that product management interviews (and perhaps all interviews), no matter how well executed, will give you limited insights. Even after multiple rounds you will still face uncertainty of what the candidate will be like once you embed them in your company and teams. You can see interviewing as a process to eliminate as much doubt as possible regarding the candidate’s ability to fill the position, without really ever getting to certainty.

Here are three examples of product skills you can barely test for in a conversational style:

1. ‘Creative solution thinking’

This is similar to ‘product spidey-sense’ in Ken Norton’s classic article How to hire a product manager. He believes it can be tested, for example by asking questions about your own product, but I remain skeptical. I’ve discussed this with multiple product colleagues and friends, and nobody has a good way of doing it fast, especially not in a conversation and without the candidate doing a task. Everyone is copying each other in the industry, so how can you know for sure someone can come up with a creative solution themselves, as opposed to just suggesting some improvements that are copied from elsewhere? Now, creativity is not always about coming up with new ideas but also about applying existing solutions and concepts to new environments. Still, my opinion is that true creativity is coming up with ‘new’ solutions. The best technique I’ve heard is to ask a product manager what they would do if technology X would be invented. What product and/or service would they build using that new technology (e.g. autonomous flying cars) to add value for potential users. They can obviously not copy ideas from other products here, as they don’t exist yet.

2. Product strategy

I’d say there are roughly two types of product managers. Those who are guided mostly by user insights (user testing and interviews) and derive a strategy from that, and those who are much more opinionated and push their vision forward. You can assess which of these two types a candidate typically leans towards, and have an interesting conversation about it. But sometimes the conversation doesn’t get further than the interviewee saying all the right words, without you really being able to see if they can come up with a proper product strategy themselves.

3. Product ownership

This is a much needed skill for any product manager, and being proactive is just one aspect of it. You’ll first want to check if they have really owned some part of a complex product in the past by asking the right questions. If you succeed in doing that, it is still hard to assess how good they really were at it. Were all engineers and stakeholders satisfied, was it a true cross-functional setup, etc.

As a concluding thought, I see that many companies put people in hiring positions without them having any training or understanding of how to really do it well. There is a striking similarity here to people being promoted and suddenly finding themselves in a position to lead other people. After all, it is not because you are, say, a good developer, that you are a good leader. Of course, the process of leading people or interviewing is not an exact science. This is perhaps why you are typically expected to learn by doing instead of going through long training workshops. But as with leadership, there is often a poor execution on interviews because the people on the other side are still figuring out what they are doing.

What’s Next?

If you want to get more insights like these on all things product, follow me on twitter here.

Thanks to Boris Libeert, Felicia Nagel, Damir Dizdarević, Mathias Fritsch, Faris Deshmukh, Patrick Jolley, and Daniel Evans for reading drafts of this article and making thoughtful suggestions. Last but not least, thanks to Elise Van Pée for being the true driving force behind many of the successes I’ve had.