Menu Close

Tracking progress on product delivery

On joining Jama Software, I discovered that the product and user experience teams didn’t have deep knowledge about who used the product and what they wanted. Plus, the company shifted focus to customers who build integrated systems or other complex products that combine hardware and software and may be in regulated industries. These industries especially need something like Jama to break their market features into requirements, specifications, tests, and keep all of this information connected and in one place.

As I discussed in another project about improving findability in Jama, I planned and completed field studies with users who make integrated systems to learn about their work ecosystem and the roles or personas that matter most. I wanted product, user experience, and development to make features for a few people they knew well. In my fieldwork, I explored a “day in the life” of people who use–or should use–Jama. In a typical day, how was product information created, shared, connected, managed, and analyzed, either with or without Jama? Who did this? What worked? What didn’t? Why?

What I discovered about our personas is that they were not all equally served through the software. One personal was crucial to bringing Jama into the organization as a better, safer way to make products, but got no personal benefit from the software. I named him Brad the Bridge, a product leader who has one foot in the business side of the company and the other in product development.

User persona Brad the Bridge wants real-time information about product development.
Brad the Bridge is often pulled in two directions by business needs and product development.

Brad needed a very simple, yet difficult and ever-changing piece of information: how is it going? Will we deliver this product on time? With good quality? Brad wanted to keep his focus on what comes next for his product lines, so he often had to trust the people in product development to give him an accurate assessment about progress or problems. He trusted them to own the plan. But he also had to ask frequently for updates so he could inform the business about progress. Brad spent a lot of time and effort answering the same question. Jama had no way to roll up system activity into simple, real-time trends for Brad.

One of the themes from a field study with users is all about Brad and why Jama didn't serve him well.
One of the themes from a field study with users is all about Brad and why Jama didn’t serve him well.

Brad clearly wanted real-time data about product development activities and progress, much of which lived in Jama or in products integrated with Jama.

Another need for Brad is gaining more insight into what is coming next for his products. I participated in a product exploration launched by Jama’s CEO about whether we should give customer-facing roles a way to tell Brad what should be next for his products through Jama. Idea management products do this, for example, letting staff make suggestions or vote on ideas. I suggested a quick round of research to learn whether product leaders like Brad would find value in this and worked with a product manager to schedule and interview product leaders like Brad and their customer-facing colleagues.

We discovered that Brad does not think all voices are equal, as he determines what to deliver next to the market. Brad believes that consultants or others who work with customers after the sale may have a valuable perspective to share. But salespeople often do not, as they are focused on how to make a sale right now, not on the sustainable future of the product. Still, Brad thinks the best data is found in what customers say directly. The problem Brad has is how to analyze that information for compelling patterns, how to see a signal in all the noise. I presented these findings to product leadership.

People like Brad can't easily track the current product story in Jama Brad doesn't want to create and share the product vision in Jama Customer-facing teams want to contribute to product direction

The user experience team generated a range of future-leaning ideas for a company kickoff. I led this effort on the UX team and included a way for Brad to analyze and cluster various sources of information about the marketplace, including customer feedback. Brad can literally see what’s bubbling up from his favored sources of customer and market data.

After neglecting him for years in favor of more technical personas, Jama made a sudden, dramatic pivot and decided to launch a product aimed squarely at Brad’s need for real-time progress metrics for his products, something that could lead to better predictions about delivery times and product quality. We would present trending metrics based on activity in Jama’s core product, as product delivery teams create, connect, and test product requirements.

Time was short, and we needed to quickly help development teams get clarity and create proof-of-concept trend metrics using existing Jama data and third-party analytics products. I led the user experience team in creating concepts for the first few trend metrics we would deliver to help Brad track product development progress. We managed to fit in a few concept reviews with product leaders like Brad, and felt confident about the chosen direction and value of the initial trend metrics.

Design concept for a trend metric that measures requirements volatility
Design concept for a trend metric that measures requirements volatility–added, deleted, or changed product definition

Volatility measures how much your product definition is changing, and it’s fine to have higher volatility earlier in your process, but it should subside. This is a sign that teams are aligned on the product they are making.

Quality coverage and progress trend metrics indicate whether your product requirements are connected to tests and if those tests have been run and passed.

A concept for selecting how you want to analyze data from Jama
A concept for selecting how you want to analyze data from Jama
Concept for quality coverage
A concept for quality coverage–how many of your product requirements have a related test
A concept for quality progress
A concept for quality progress–how many tests related to your product requirements have run and passed?

The vendor review resulted in buying a small analytics company and adapting their product to our needs. In six months time, we released Jama Analyze. We continue to learn what Brad the Bridge wants to know about the product development cycle and how to give him rich, timely answers to his simple question: how is it going?

Trend metrics from Jama Analyze
Actual trend metrics in Jama Analyze, using data from our core product, Jama Connect

Jama’s new direction attracted attention–and investment. Embracing Brad’s need to be a more data-driven product leader helped land Jama $200 million in growth capital from Insight Venture Partners

Jama currently plans to follow with another product offering that would help Brad assess market and customer data for compelling patterns to discover his next opportunity–the same idea I brought to the company through research and as part of a comprehensive product vision for the future.