In our last post, we posited that audit trail reviews are useful to verify that the system users are using the data capture system properly. To guide our planning for audit trail reviews, let's first consider how users might misuse the system - either maliciously or unintentionally:
- Intentional misuse
- A site could invent study participants and enter false data in the EDC system for them
- A site monitor could cut a site visit short, then after they return home, set the SDV flags to "reviewed" to give the impression that they had SDVd all required data
- A sponsor medical director could ask the system administrator to give her permission to change data, then change a few key values to make it seem as though the treatment were effective
- Unintentional misuse
- Two site coordinators could accidentally enter data for the same participant twice, under different subject numbers
- A site monitor could SDV all required data at a site, then, after returning home, realize that they forgot to set the SDV flags to "reviewed," and then set them days after the visit.
- An administrator could unintentionally give a user inappropriate permissions - for example, a sponsor user could accidentally be given a role that allows them to change data at a site
- During an unlock process, a user could accidentally change data that wasn't intended to be changed
- In an effort to be helpful, a call center team member could change data for an investigator, even though there was no formal mechanism for capturing the investigator's approval for the data change
- A PI could forget to delegate responsibilities for data entry to a team member but then request EDC access for that team member, who then changes data
- An inexperienced site coordinator could respond to a CRA's queries about a missing visit by adding the visit in EDC even though it did not occur
- The PI could forget to log in to the IRT system throughout the study, then encounter a situation where emergency unblinding is required
If we suspected any of these scenarios, we would naturally go straight to the audit trail to figure out the sequence of events. For example, if the site called to complain that a monitor hadn't spent very much time on site, we would look at the audit trail to see when that monitor's entries were dated. But what sorts of reviews should we be doing routinely? E6 tells us that our audit trail reviews should be "risk-based, adapted to the individual trial and adjusted based on experience during the trial."
Below is an example of an audit trail review plan that a study team might put into place:
| Data Capture System | Frequency or Trigger | Responsible Party | Purpose of Review | Methodology for Review |
| EDC and IRT | Quarterly | Data Manager | Verify that only users who are permitted to enter and change data per the DMP are given roles with data entry/change permissions | User history report filtered for roles with data entry/change permissions that includes all user changes in the reporting period (not just the current roles) |
| EDC | During SDV | Site Monitor | Review audit trail during SDV to verify that only delegated users are entering and changing data and that timing of data entry is consistent with study activity | Visual inspection of audit trail/comparison to delegation log |
| EDC | Quarterly | Clinical Trial Manager | Review a sample of audit trails against monitoring visit schedules to verify that on-site tasks are being performed while the Site Monitor is on-site | Search for key Site Monitor actions in the audit trail and compare them to dates of monitoring visits |
| EDC | Monthly | Data Manager | Verify that site staff are entering data within the required time period per contract | Report of the date of entry vs. visit date for each visit; look for average and outliers per site |
| EDC | For post-lock changes | Data Manager | Verify that the authorized changes and only the authorized changes were made post-unlock | Review all actions in audit trail performed post-lock |
| EDC | In cases of suspected fraud/misconduct | Data Manager | Review timing of data entries at site for | Report of the date/time of entry for each entry; look for unusual patterns (e.g., visits entered before they took place) |
| ePRO | Monthly | Data Manager | Review changes made by help desk against help desk tickets to verify that only changes permitted by DMP and authorized by an investigator in advance were made | Report of data changes made by help desk users |
| IRT | Prior to site activation | Site activation team | Verify the PI activated IRT account before the study started | Search for PI initial log-in |
This is an example, not a recommendation. Your risk may vary. It does, however, illustrate the thought process that should go into planning audit trail reviews. Instead of asking, "What are we going to review?" we should be asking, "What are our risks for intentional and unintentional misuse of the system?"
Note that some of the methodologies listed above require sifting through the audit trail, but others are reports that include audit trail data. Such reports give a different picture of the data than listings that show only the current data. For example, most CROs have a process for quarterly review of current EDC role and permission assignments to verify that current users have been assigned to appropriate roles, but frequently those reports capture only the assignments that are current when the report is being run -- not changes that might have been undone by the time the report was run.
