Going Beyond Cohort Discovery:

Current Limitations, Advanced Methodologies and Future Trends

Vojtech Huser Vikrant Deshmukh Adam Wilcox Henry Lowe
AMIA summit, 2012

Panelist
Vojtech Huser, Panel Moderator
Assistant Clinical Investigator, Laboratory for Informatics Development, National Institutes of Health, Clinical Center

Vikrant Deshmukh
Senior Data Warehouse Architect, U of Utah Hospital

Adam Wilcox
Assistant Professor at Columbia University and Director of the Clinical Databases, Information Services, New York Presbyterian Hospital

Henry Lowe
Director of the Center for Clinical Informatics, Stanford University

Vojtech Huser

Beyond Cohort Discovery

Background
Growth of data warehousing
CTSA IDR surveys in 2007,2008,2010

Increasing number of integrated sources Improved researchers access to data

IRB policies, user-friendly query tools Now an exception of a projects does not use an IDR

emergence of established platforms

i2b2, BI tools
Vojtech Huser Beyond Cohort Discovery 3

Topics
So you have your cohort estimate: now what? 1. self-service search tools 2. human-expert-assisted search 3. prospective, computer-assisted cohort recruitment
1

2 3

Vojtech Huser

Beyond Cohort Discovery

1. Self-service query tools overview

i2b2 (1.4 -> 1.6), SHRINE FURTHER (U of Utah) STRIDE (Stanford) Business intelligence based systems
(Business Objects, Cognos, MS, Pentaho, others)

BTRIS (NIH) DEDUCE (Duke) (+DISCERN) StarBrite (Vanderbilt) (SD)

Beyond Cohort Discovery 5

Vojtech Huser

Factors
User factor
Zero training system
vs. intro video vs. advanced training

User not versed in terminologies What is out there in the IDR?

granularity, source systems, IDR vs. EHR)

Level of support (helpdesk)

System factor
Underlying warehouse (data model, search engine) User interface metaphor
Vojtech Huser Beyond Cohort Discovery 6

Limits
Self service query tool
Average user limit Power user limit User interface/tool limit

Human-assisted search
Request form complexity limit Assigned analyst limit Analyst preferred query technology limit
(BI, SQL)

Any query technology limit

Vojtech Huser Beyond Cohort Discovery

Next steps
Cohort estimation
Feasibility review vs. full IRB approval

Results provide answer to the question Data extraction

Further manual review (narrowing the cohort) for retrospective study

Limited in-tool support, link to further systems

More complex query (human assisted)

May not be an option (not supported, too expensive)
Vojtech Huser Beyond Cohort Discovery 8

Challenges (self service query tools)

Temporal logic Ability to chain queries (Q1 output is input for Q2) Knowledge representation
SQL query + schema Phenotype modeling (HealthFlow, PheKB, eMerge) Eligibility criteria of clinical trials (OCRe, Arden Syntax)

Import/Export
within single platform sharing among users within organization
New inferred concepts in IDR

Search query elements, variables (interim constructs)

inter-element parameter passing
Depression, PHQ-9 score, therapy, 50% reduction in initial score
Vojtech Huser Beyond Cohort Discovery 9

2. Human-assisted search
DEFINITION: Ability of data requestors to ask for custom queries performed by human analyst Level of integration
No such option Designated analyst (within department) Data warehouse analyst (differing availability) Institutionalized entity with charge-back model
Response time, request submission (web-form with ticket # vs. informal request)

Vojtech Huser

Beyond Cohort Discovery

10

Factors
Data+Requestor Data+Analyst+Requestor
Misunderstanding Quality control (is this exactly the criteria requestor needed)

Not restricted by tool metaphor or UI Mode of communication platform

Web-form, emails, face-to-face meetings, iteration Ability to review the code (in addition to results)
Power requestor

Cycle time (interim results)

Week vs. month, interactive results

Ability to import search definition from self-service tool

Vojtech Huser Beyond Cohort Discovery 11

Limits
Self service query tool
Average user limit Power user limit User interface limit

Human-assisted search
Request form complexity limit (attachment) Assigned analyst limit Analyst preferred query technology limit
(SQL, Proc SQL, SAS, Xpath, SPARQL)

Any query technology limit

Vojtech Huser Beyond Cohort Discovery 12

3. prospective computer-assisted cohort recruitment

DEFINITION: informatics solution facilitating recruitment (eligibility criteria) (CTA) Data warehouse based alerts vs. EHR system alerts Different execution paradigm
Single patient (single event processing) Population (retrospective queries)

Ability to reuse the same retrospective logic

Reuse each search element criterion Flip a switch
Vojtech Huser Beyond Cohort Discovery 13

Example 1: Human-assisted search (Marshfield Clinic)

Flagship modality for providing access to IDR Request tracking since 2001, centralized team, pool of analyst (funded projects)
research team, administrative team

Parameters:
report title, reason, data criteria (attachment), desired columns, output (SAS, Excel), report frequency link to similar prior requests

Query code/results is archived (code re-use) between 2001-2008: 5507 queries

Use by various stakeholders Compliance perspective Research use (complex queries, analyst time, lines of resulting code)

Focus groups used for re-design of the request workflow

Analyst provides warehouse knowledge (what more can I ask?) Multiple iterations, face-to-face meetings
Vojtech Huser Beyond Cohort Discovery 14

Example 2: NIH Clinical Center

1. self service query tool:
BTRIS platform Flagship modality

2. human-assisted search
Limited degree (no elaborate financial model) Can also be limited to assistance with the query tool Provided to inform improvements within the self-service platform

3. prospective recruitment
Not fully applicable to Clinical Center as research hospital
Not within EHR Studies compete for participants ability to search on 12 parameters generate a list of phone numbers

In development phase around the IDR Database of healthy volunteers

Vojtech Huser

Beyond Cohort Discovery

15