When you are manufacturing a product, it makes perfect sense to sit down before the machinery begins to whir and think about everything that might possibly affect the delicate balance of people, equipment, materials, methods, and environment that you have set up to deliver a consistent product day after day. In a previous post we used the analogy of a commercial bakery, so let's stick with that. Imagine you're starting to produce a new type of cookie. You might analyze the risks and think of how to mitigate them. What if your chocolate chip supplier is unavailable? How will your bakers ensure consistency with a new recipe? Once you're identified and mitigated all the risks, you need to repeat the assessment periodically to account for changes that happen over time.
Running a clinical trial, though, is less like manufacturing a cookie and more like running a restaurant. If you were opening a restaurant, you might do a risk assessment before it opens to identify the big risks. What if we're short-staffed? What if the oven breaks? If you're an EXPERIENCED restaurant owner, though, chances are the risks you identify are risks common to all restaurants and you're already controlling for them with back-up staff, a line of credit, and so forth.
It's not a useless exercise. You might identify one or two risks that are unique to your restaurant (e.g., "What if the fad for Greek/Mexican fusion disappears?"). But in a restaurant, how you manage risks on a day-to-day basis is more critical to your success than a one-and-done (or even periodically repeated) risk assessment. Hour by hour, the General Manager is evaluating supplies, staffing, reservations, cash flow, the weather, the economy of the local area, changes in traffic patterns, and the mood of the chef.
The analogy holds for clinical trials. Start-up risk assessments can be useful, but are they three-hours-of-my-time useful? 250-lines-of-a-spreadsheet useful? In a past blog post, we explored the origins of our risk assessment methodology and learned that the three-pronged risk assessment (evaluating risks in terms of probability, impact, and detectability) is not a validated quantitative method and is arguably worse than the "expert intuition" method, where knowledgeable team members get together and ask, "What's likely to go wrong here?"
In a clinical trial, just like in a restaurant, change is constant, so the real work in risk management is done day-to-day, rather than up front with annual reviews. Our recommendation is to conduct an initial risk assessment as efficiently as possible but focus the team's energy on day-to-day risk assessment--otherwise known as good old-fashioned project management--which includes the following:
- Sensitivity to conditions. The critical-to-quality factor method of evaluating risks focuses on the elements of the clinical trial. We find it more useful to be aware of conditions that typically increase the likelihood of negative impacts to critical data and processes, including complexity, novelty, resource constraints, layers, time constraints, and environmental factors. Like the restaurant manager, the clinical trial project manager is constantly assessing how these factors might put deliverables at work.
- Understanding how things should work. The sponsor's functional lead should have a fairly detailed understanding of the roles, responsibilities, and activities within that function--especially if a vendor is following its own procedures. That means reading vendor SOPs and learning study team roles and responsibilities, and but also asking lots of questions about how and why things work and prompting teammates to think through "what-if" scenarios.
- Sampling. If you're managing a restaurant, you don't just sit in your office and have meetings - you walk the floor, tour the kitchen, and sample the food. In a clinical trial, we don't rely on our CRO's status updates; we learn more by reading trip reports, reviewing audit trails, perusing the Trial Master File, and examining help desk tickets.
The challenge is getting "credit" for those continuous risk discussions, which could be accomplished by dedicating a portion of each team meeting to discussions of risk and documenting those discussions in a way that they can be retrieved as evidence of continuous risk review.
