When regulated industries make products, they not only do the work to define and create these devices, they provide auditable evidence that they did it in a safe manner, complying with all standards and federal guidelines.
Today, Jama Software serves regulated industries and helps them manage product definition. Early on, we weren’t sure how to attract and support product development companies that have such stringent standards to follow. The company’s strategy was to focus on one vertical first, then learn more industries in turn. We started with medical devices.
Initially, we wanted to understand the problem more broadly. What is it like to make a regulated device that might harm someone? I worked with a product manager to understand how compliance activities happen for medical device companies. We visited customers, and I led interviews and task analysis activities with various roles throughout the product development process, learning the flow and interplay of their work. At every stage, I asked about additional regulatory activities: what happened, who was involved, and what was difficult about it.
I boiled down the task analysis and interviews into simple grids that showed the workflow through all stages of their development cycle, what regulatory activities happened in each stage, who did the work, and what they found difficult.
Early indicators showed the most pain around publishing product definition content in the right format for safekeeping or possible FDA audits. But product management chose risk analysis as their focus, instead. They wanted to offer a better way to capture risk analysis content and connect it to product definition, as well as any necessary product changes to lower risk. Most medical device companies follow the ISO 14971 guidelines for risk analysis, something about which we knew nothing. My team had only a month to shift gears and have high-level designs ready.
I led the effort and worked with another designer to quickly find and review spreadsheet examples of ISO 14971 approaches to risk analysis. There is not one right way to do it, just guidelines. We wanted to give some flexibility, yet keep it as simple as possible, averaging out the many examples we reviewed and emphasizing the value of connecting your existing Jama content to a risk analysis, instead of doing it externally. We created a concept based on the simplest approach to rating hazards and harms and allowed for some changes in labels and scales for computing risk levels.
I recruited three medical device customers to review the concepts with us and give feedback about how well it fit into their current process. Our participants each had many years of experience in setting up a risk analysis process at medical device companies.
The results of the concept review changed the priority for this feature and stopped the project. This was the right call. Design thinking methods should help stop mistakes as much as they move good ideas forward. Through the concept reviews, we confirmed our fear that, because the standard only offers guidelines, everyone did this differently and would not change their risk analysis process lightly because that invites scrutiny from the FDA. Giving our users a little flexibility in the feature would not be enough. More configuration options meant more design and development time. We also learned that, given a wish to change anything they wanted about how they define and make medical devices, risk analysis was not the biggest problem to solve. Publishing was, just as we saw in our early research. After that, people wanted to be able to control product versions better in Jama Software or see how all of their requirements trace to test results. But risk analysis ranked lower than all of these.
After improving other aspects of the product, the product team eventually returned to risk analysis for medical device companies. As a manager of user experience, I helped the lead designer absorb past lessons first and gain a strong background in the ISO 14971 standard. I wrote a design plan for the product manager, lead designer, and technical lead on the project that included further task analysis and interviews with medical device customers, giving the team a chance to learn first-hand how they do risk analysis today.
I helped the product manager and designer analyze their field research. Nothing contradicted our earlier research findings, and we discovered some new user themes and product opportunities for risk analysis:
- Risk analysis has clear roles: moderator and reviewers
- Moderators find it hard to keep risk teams focused on the scope of the review
- Tracing a risk analysis to product impact is key: connect the feature and any changes to the analysis
- Publishing risk reports is necessary and painful
I also guided my designer, as he created another risk analysis concept based on his deeper understanding of the ISO standard and what he learned in the field. We both ran concept review sessions to get feedback on direction, internally first, then with customers:
The concept tested well, and the feature went into production. Jama Software released risk management and immediately had a flood of interested customers.
Design thinking is crucial for tackling problems that demand a deep understanding of people and problems entirely unknown to you. I consider stopping a poorly conceived effort almost a bigger success than getting a well-designed product or feature delivered to our users, and I enjoy giving teams the chance and method to learn something new this way. There is a turning point in projects like these where everyone is speaking the same language about the feature and suddenly has the complexity in hand. I like making that moment happen.