Skip to content

CRIO Integration

Part 1 - Recruitment Data Ingestion for KPI Analysis

Executive Summary

Clinical Trial Management Systems (CTMS) are used by many sites to manage the recruitment of patients. For example, Next Stage uses the CRIO CTMS. Data contained in these systems, namely the status of patients in a study, can give Trially insight into the recruitment trajectory of a patient that Trially has matched to a study. That data in turn can be used to track a number of Trially’s KPIs that can help Trially communicate and improve on its efficacy, accuracy, and impact; these KPIs can in turn help Trially clients’ financial departments analyze their recruitment methods and revenue from referrals.

In order to obtain these KPIs, we will need to create API integrations with CTMSs. This will involve setting up a new backend that will ingest raw patient recruitment data that will be saved in our data lake. Post-MVP, this raw data will be organized into a new recruitment application data schema (pending review). The new schema will support the current functionality of the recruitment app while also evolving to support the storage and efficient querying of recruitment data on a per study, site, and client basis.

Motivation and Goals

We want to ingest recruitment data from CTMSs in order to track a number of company KPIs. We will begin the ctms-integrations backend with a CRIO specific integration.

Goals

  • Develop a new backend that ingests raw recruitment data from CRIO into Trially’s GCS using CRIO's patient webhook.
  • After part 2 of this project, implement a new relational data schema in the recruitment app that allows for storage and querying of KPI data per study, site, and client where we can store the raw data in modeled form for both the CRIO integration and any future CTMS integrations (pending review)
  • After part 2 of this project, create basic, excel tables of recruitment funnel data per study, site, and client

Non-goals

  • Create internal dashboard UI of KPIs per study, site, and client (non-MVP)
  • A backend that supports onboarding multiple CTMSs. The CTMS Integration backend will be created to support the CRIO integration to start but will evolve to support any CTMS integration when we integrate with a second CTMS. However, the data schema is being built with the flexibility to support other CTMSs.

Proposal

The integration will consist of the following components:

  1. A new CTMS Integrations backend and repo called ctms-integrations that runs on Cloud Run
  2. A webhook that will ingest data about a patient’s recruitment status in a study
  3. A json file containing patient recruitment data that is saved to its tenant specific bucket in GCS
  4. A remodel of the recruitment app’s data schema where the modeled patient recruitment data will be saved (post-MVP, pending review)
  5. An alteration of the webhook so that it parses through the json and saves the data to the recruitment app database in modeled form (post-MVP, pending review)
  6. Recruitment funnel visualization at the site, site-network/client, and study levels beginning with excel tables and eventually user-friendly, internal dashboards (post-MVP)

Implementation

Ingestion options

Option 1: Implement a webhook in a new, ctms-integrations backend that will ingest patient updates from CRIO.

Pros:

  • We do not have to set up a system that performs requests to CRIO
  • With a webhook we will not miss any status updates

Cons:

  • There's a chance the webhook may have to support 8 different payloads which could be more difficult to manage

Option 2: Implement an endpoint in a new, CTMS-Integrations backend that periodically requests patient updates to CRIO’s API.

Pros:

  • We can can control how often we request data from CRIO

Cons:

  • If multiple updates are made to the patient’s status within the time period in which the endpoint does not perform a request, we may miss an intermediary status because statuses are overwritten.

Recommendation: Option 1 so that we do not miss out on intermediary patient updates that can bring the most accuracy to our KPIs.

Going with Option 1:

  1. Set up webhook to ingest patient updates that will include the patient’s status within a study. CRIO will hit this endpoint anytime there is an update to a patient. Trially will associate the siteId in the payload to the correct tenant bucket in GCS and then save the json to Trially’s data lake. Authress service clients will used to authenticate with the webhook.

  2. After receiving this information and finishing part 2 of this project, we will rework the web-app’s data schema to begin organizing the data for further analysis.

2a. ERD for web-app: https://link.excalidraw.com/readonly/gLQEkFeRo1QXMAzq2b8z

  1. From that new data schema architecture we can create simple tables like the one below at the site, site-network, and study levels.
  1. We can also use the data from the new data schema to create a dashboard of these waterfall Recruitment Funnel graphics at each of the aforementioned levels. A nice-to-have would be a simple frontend admin app that can generate these graphics (post-MVP).

Recruiting Funnel showing five levels of a candidate’s status in the recruiting process of a trial (Created by Ramon)

Launch Plan

Test in staging and production with Trially test site. Share auth credentials with CRIO.

Part 2 - Trially to CRIO Patient Syncing

Executive Summary

Trially clients will often use CTMSs to upload patient leads and manage patients’ statuses throughout a study. Currently, our clients have to download or receive a copy of Trially matches that they must then upload to the CTMS. We want to allow clients to manage the export of patients to the CTMS via the web recruitment app and have the patients get programmatically sent to CRIO via API. This necessitates the creation of an API integration that allows Trially to sync patients to CRIO seamlessly.

The same ctms-integrations backend as part 1 of this document will be used to implement a post endpoint to CRIO. The post endpoint will send a patient to CRIO after a Trially client triggers the post via a new “send to CRIO” button in the recruitment app. The endpoint will write patient data to CRIO that includes match percentages for a given study and Trially’s internal ID for that patient.

Motivation and Goals

We want to post patients to the CTMS so that our clients can see Trially matches and match percentages on their CTMS without having to manually export patients from Trially and import patients to CRIO. We will begin the CTMS Integration backend with a CRIO integration.

Goals

  • Post all patient matches that the recruitment app user decides to export.
  • Before posting a patient to CRIO, check if a patient already exists in the CRIO system (via the Patient Lookup API) and if so, to not override any existing values, do a get request to obtain existing patient values and only populate our new values in the actual post request. If the patient doesn’t exist, we can post all the patient data we have without concerns of overwriting. For both cases, we store the response.

Non-Goals

  • Posting percentage match to a match specfic field in CRIO on patient post. For the MVP, we will append the match percentage for a study to the patient’s notes string field in CRIO. Eventually, we may consider asking CRIO to create a new field where we can post a match percentage under a study object to prevent overrides from other vendors' API posts.

Proposal

The integration will consist of the following components:

  1. An “export to {CTMS}” button in the recruitment app for sites associated with a CTMS
  2. A small remodel of the web-recruitment app db schema to support associating a site with a CTMS (pending review)
  3. A web recruitment app endpoint (pending review) that receives the exported patient data from the “export to {CTMS}” button" in the recruitment app UI and posts the patients to the patient post endpoint in the ctms-integrations backend
  4. A new CTMS Integrations backend and repo (same as part 1 of the CRIO integration)
  5. A post endpoint that will receive the patient data from the web recruitment app and post patient data, match percentages, Trially's internal ID for that patient, and the study they are matched for to CRIO
  6. Raw patient json response data being saved to our data lake in a tenant specific bucket
  7. Eventual remodeling of the recruitment app’s data schema where the patient’s status in a study and CRIO ID will be saved (post-MVP, pending review)
  8. Eventual alteration of the endpoint so that it parses through the json response and saves the data to the recruitment app database (post-MVP, pending review)

Implementation

There are three main steps to implementing match syncing:

  1. The first is to update the recruitment app database to support associating a site to a CTMS.
  2. The second is small and involves building the export to CTMS button to be shown whenever a site is associated to a CTMS.
  3. The third is to create the post endpoint to CRIO within the ctms-integrations backend. This endpoint will have logic to check if a patient exists in CRIO (via the Patient Lookup API) before creating a new patient in CRIO (via the Patient Post API) or updating an existing patient in CRIO via first the get patient request and then the post patient request. There are many nuances because patients can already exist in the CRIO database if they were posted by a different vendor and because the patient API does a create or update which means there is a risk for overwriting. Check out the “patient data syncing flow” below where these details are captured and managed. In the end, we save the raw response from CRIO to our data lake in a tenant specific bucket.

Launch Plan

Next Stage, a user of CRIO, has asked to test with one site in staging and production before moving forward with all site data. We will deploy to staging first and test there with one site. Will do the same in production. We have a test site. We will need test patient data which CRIO can provide.

Full CRIO Integration Diagram