Funnel view

Executive Summary

The goal of this project is to add a funnel visualization to the existing recruiting page to help users understand patient attrition through protocol criteria. This feature will complement the current table view by providing a clear visualization of how each criterion impacts the eligible patient pool. The implementation will maintain existing patterns and state management while adding the new visualization option.

Motivation and Goals

Key Objectives

• Add funnel visualization to existing recruiting page
• Maintain current protocol selection and criteria panel functionality
• Provide clear visualization of patient attrition
• Keep consistent UX with existing features

Benefits

• The filter view clearly shows the impact of each criterion
• Provides quick identification of restrictive criteria, and better visibility into how criteria affect patient eligibility

Non-Goals

• Other changes related to v0.2:
• Custom Criteria
• Navigation redesign

Proposal

Core Features

• Simple view toggle between:
• Existing table view
• New funnel visualization
• Statistics summarizing the applied criteria
• Total number of patients considered
• Criteria applied (out of the total available criteria)
• Sites selected (out of the total number of sites)
• Patients Matching (total number and percentage of total patients)
• Funnel visualization showing:
• Total initial patient pool
• Patient count after each criterion
• Percentage remaining at each step
• Number excluded by each criterion

Design and Implementation Details

Backend Implementation

The backend changes require adding a new endpoint to calculate funnel statistics for a given protocol. This will leverage existing database models but introduce new computation logic for tracking patient attrition through each criterion.

API Endpoint

• GET /protocols/{protocol_id}/funnel
• Uses existing database models
• Iteratively calculates exclusions relative to remaining patient pool
• Fetch initial patient pool for the given protocol
• Fetch currently active criteria for the given protocol
• Iterate through criteria and calculate exclusion statistics for each criterion applied

    # API Interface

    class CriteriaFunnelStats(BaseModel):

        criteria_id: int

        criteria_summary: str

        criteria_type: CriteriaType

        patients_excluded: int

        patients_remaining: int


    class ProtocolFunnelStats(BaseModel):

        total_patients: int

        final_matches: int

        criteria_stats: List[CriteriaFunnelStats]


    @router.get(
        "/protocols/{protocol_id}/funnel",
        response_model=ProtocolFunnelStats,
        response_model_exclude_none=True,
    )
    async def get_protocol_funnel(
        protocol_id: int,
        service: ProtocolsService = Depends(get_protocols_service),
    ) -> ProtocolFunnelStats:
        """Get funnel statistics for a protocol."""
        return service.get_protocol_funnel(protocol_id)

    # API Implementation

    def get_protocol_funnel(self, protocol_id: int) -> ProtocolFunnelStats:
        remaining_patient_ids = # Get initial patient pool for the protocol
        criteria_list = # Get list of active criteria for the protocol
        total_patients = len(remaining_patient_ids)
        criteria_stats = []

        for criterion in criteria_list:
            failing_patient_ids = # checks patients still in the remaining pool that fail this criterion

            # Remove failing patients from remaining pool
            remaining_patient_ids -= failing_patient_ids

            excluded_count = len(failing_patient_ids)
            remaining_count = len(remaining_patient_ids)
            percentage_remaining = (remaining_count / total_patients * 100) if total_patients > 0 else 0

            criteria_stats.append(CriteriaFunnelStats(
                criteria_id=criterion.id,
                criteria_summary=criterion.summary,
                criteria_type=criterion.type,
                patients_excluded=excluded_count,
                patients_remaining=remaining_count,
            ))

        return ProtocolFunnelStats(
            total_patients=total_patients,
            final_matches=len(remaining_patient_ids),
            criteria_stats=criteria_stats
        )

Frontend Implementation

The frontend changes introduce a new visualization option while maintaining the existing table view and criteria panel functionality. This implementation focuses on maintaining existing patterns while adding the new funnel view.

State Management

Option 1: Local State (Current Implementation)

- Keep existing protocol selection state
- Add simple view toggle state
- Use existing SWR pattern for fetching and caching funnel data
- Maintains current patterns
- Simpler implementation

Option 2: URL-based State

- Store protocol ID in URL
- Individual pages for table and funnel view
    - /recruiting/:protocolId/table
    - /recruiting/:protocolId/funnel
- Can bookmark/share more easily
- Would require more restructuring

Recommendation

I would recommend Option 1 for the following reasons:

• Matches existing implementation
• Simpler to implement
• Can be easily refactored later if needed

Data Flow

• Protocol selection drives both views
• Criteria updates affect both visualizations

Use existing external state management pattern to fetch and cache data:

const useProtocolFunnel = (protocolId?: number) => {
  const { data: funnelData, mutate, ...rest } = useSWR<ProtocolFunnel>(
    protocolId ? `/protocols/${protocolId}/funnel` : null
  )

  return {
    funnelData,
    ...rest,
  }
}

And update funnel data when criteria is changed:

const {
    funnelData,
    isLoading: isLoadingFunnel,
    mutate: mutateFunnel,
    error: funnelError,
} = useProtocolFunnel(selectedProtocolId)

...

const handleCriterionToggle = () => {
  mutateCriteria()
  mutatePatients()
  mutateFunnel()
}

graph TD
    %% Components
    RP[RecruitingPage]

    subgraph "Shared Components"
        PS[Protocol Selection]
        CP[Criteria Panel]
    end

    subgraph "Views"
        TV[Table View]
        FV[Funnel View]
    end

    subgraph "Data Hooks"
        PH["useProtocols()"]
        CH["useProtocolCriteria()"]
        TH["useProtocolMatches()"]
        FH["useProtocolFunnel()"]
    end

    subgraph "API Endpoints"
        PE["/protocols"]
        CE["/protocols/{id}/criteria"]
        TE["/protocols/{id}/matches"]
        FE["/protocols/{id}/funnel"]
    end

    RP --> PS
    RP --> CP
    RP --> TV
    RP --> FV

    PS --> PH
    CP --> CH
    TV --> TH
    FV --> FH

    PH --> PE
    CH --> CE
    TH --> TE
    FH --> FE

    class RP,PS,CP,TV,FV component
    class PH,CH,TH,FH hook
    class PE,CE,TE,FE api