In our previous blog post on data integrity, we showed an example of a partial data flow diagram and walked through a risk assessment, focusing on source reliability, risk of falsification, and risks of transcription. In today’s post, we consider “external data.”

“External data” is the industry term for data that are not typed into the clinical database by the clinical sites. In the old days, central laboratory and pharmacokinetic data were the primary sources of external data, and frequently they were uploaded into the EDC database so they were “all in one place,” and thus easily viewable by the investigator.  Today, specialty labs have proliferated, and external data are also generated by electronic patient reported outcomes assessments, clinical outcomes assessments, wearable devices, central readers, and adjudication committees, among other sources. These data are rarely uploaded into the clinical database; rather, investigators log into various portals or receive reports from the vendors generating the data.

Today, we focus on lab data. When we construct our data flow diagram, one lab data “swim lane” might look something like this: As lab scenarios go, this is pretty simple – sites are shipping safety samples a central lab, and that lab is analyzing samples and also cross-shipping samples to a specialty lab for analysis. Even here we have many opportunities for loss of data integrity. Let’s focus on the risk assessment question, “Where data are being transcribed, transformed, or transferred, how easy would it be to introduce inaccuracies or to lose data?”  Let us count the ways:

• Site staff could place the wrong barcode on a vial.
• A vial could hemolyze or become unstable during shipment or during storage.
• Vials could be lost during shipment.
• A lab could accession a sample incorrectly, entering the wrong subject number or scanning a bar code under an incorrect subject number.
• Sites could miss emails or forget to download results from portal.

These are the risks we can come up with without thinking too hard. To mitigate these risks, we might put standard measures in place – for example,

• Site staff will be trained to have a second person verify the barcode.
• Site staff will paste a second copy of the sample barcode in the subject’s source record, so it can be traced later.
• Site monitors will be trained to verify receipt of shipments in the lab portal during site. monitoring visits and verify that PIs have documented lab report review in a timely manner.
• Data management will verify subject numbers and sample timepoints indicated in the EDC system with those reported by the laboratory.

These seem like prudent measures, but as anyone who has worked with lab data knows, it’s what you don’t know you don’t know that will get you. Here are examples of some surprises that study teams have learned about lab data integrity after the fact:

• The safety lab actually has two accessioning systems – one for most samples, and a second for a subset of analyses that are performed by a part of the lab that was part of a recent acquisition.
• The safety lab’s analyzers capture most results via their LIMS system, but certain analyses are conducted manually and data-entered manually. There is no verification check or double data entry for results that are entered manually.
• The specialty lab configures its reporting system so that potentially unblinding results are not reported to the sites, but neither the sponsor nor CRO has visibility into what kind of testing is performed on this configuration.
• The specialty lab uses Excel as its database, which sometimes expresses long accession numbers in scientific notation.

Each of these issues came to light when something when (in hindsight, predictably) wrong.  These are not good surprises.  To identify these issues during risk assessment, the team has to be willing to dig deep, working with each laboratory to get a detailed walk-through of the process and ensuring that the data flow diagram reflects these nuances so they can be mitigated. Unfortunately, most clinical study teams do not have an in-house lab expert.  When labs are critical to participant safety or study endpoints, they should be qualified and overseen by staff with expertise in Good Clinical Laboratory Practice (GCLP). When labs are subcontracted through a CRO, there's an additional layer between the lab's Quality Assurance department and the sponsor's, so sponsors should insist on the right to audit subcontracted laboratories for cause, if needed.

What if you don't have a lab expert?  You might borrow one from the GLP team or Clinical Pharmacology; hire a consultant; or visit the lab, either during an audit or as an oversight activity, and ask lots of questions. Even a little knowledge helps mitigate risk.