Bulk Data


The Veradigm and Altera FHIR API is used to work with clinical data for a single patient or small group of patients. However, it is not technically feasible to use the usual Veradigm and Altera FHIR API requests to pass bulk data because of the number of requests and system resources involved. To support these cases, bulk data requests are available.

There are a variety of reasons why external systems would request bulk data from a Veradigm or Altera product. A few examples include:

  • Clinical studies based on conditions and treatment plans
  • Population health studies
  • Transfer of patient records from one clinical system to another

Only FHIR applications of the type System can send bulk data requests. Patient and User application types cannot send bulk data requests.

Note: For more information from HL7 on the FHIR specification for bulk data exports, see HL7's Bulk Data content.


Application Configuration Considerations

When the developer creates their FHIR application in the Veradigm Connect Portal, the following must be true in order for the application to request bulk data:

  • On the FHIR App page, the App Type must be set to System.
  • Backend authentication for access tokens via JWKS must be configured. For more information, see Testing System Applications.

Supported Resources

The Veradigm and Altera FHIR Bulk Data API can extract bulk data from the following FHIR resources:

  • AllergyIntolerance
  • CarePlan
  • CareTeam
  • Condition
  • Binary
  • Device
  • DiagnosticReport
  • DocumentReference
  • Encounter
  • Goal
  • Group
  • Immunization
  • Location
  • Medication
  • MedicationAdministration
  • MedicationRequest
  • MedicationStatement
  • Observation
  • Organization
  • Patient
  • Practitioner
  • PractitionerRole
  • Procedure
  • Provenance
  • RelatedPerson

Request and Data Flow

Unlike other Veradigm and Altera FHIR API requests, requesting bulk data is an asynchronous process; the Veradigm or Altera product does not immediately return information. The application makes the initial request and then waits for the Veradigm or Altera product to prepare the data.


For a detailed diagram, see here.


Before an application can request bulk data, they must know the ID of the Group resource they are requesting. Group resources are created by organizations in the individual Veradigm and Altera EHRs, and each EHR has a different method for creating Group resources. For example, Altera TouchWorks EHR uses the Patient List function to create Group resources, and Veradigm EHR uses segments in the Reporting module to create Group resources. For more information, see the individual EHR documentation. To obtain a specific Group resource ID, you can query the Group resource.


The process of requesting and receiving bulk data flows as follows:

  1. The application requests bulk data from a Veradigm or Altera product via FHIR. If successful, FHIR returns a 202 Accepted HTTP status code and a URL in the Content-Location header where status checks can be made. No actual data is included in the response.
  2. The application requests the status of the bulk data request by querying the URL in the Content-Location header.
    • If processing is not complete, FHIR returns status and progress (percentage complete) information.
    • When processing is complete, FHIR returns the URLs for the location of the data file.
  3. The application requests the data by sending a request to the file URLs.

  4. Creating an Export Request

    The application makes the initial bulk data request using $export. For example:

    [FHIR path]/Group/INF-101/$export

    Means: “Get all the patients in the Group resource with the ID INF-101.”

    The Veradigm and Altera FHIR API responds with the Content Location URL.

    Note: The requesting application must be authorized to access the data requested.

    Requesting an Export Request Status

    The application requests a status of the export package progress.

    GET [Content Location URL]

    The Veradigm and Altera FHIR API responds. If the package is still in progress, the response includes:

    • Status: Accepted.
    • X-Progress: Percentage complete.
    • Retry-After: Suggested duration of time until the next status request. This is measured in seconds. If a status request is made prior to the retry-after date/time, the FHIR API responds with a HTTP 429 Too Many Requests error.

    If the package is complete, the response includes:

    • Status: OK
    • Expires: Time when the export package will expire and thus will no longer be available.
    • Content-Type: JSON or XML.

    In the body of the response, a series of URLs are listed for individual packages for download depending on the size of the request. These URLs are included in the file request.

    If the export request has resulted in an error, the response includes:

    • Status: Error
    • OperationOutcome resource, which includes the severity, code, and description of the error.

    An export request can complete successfully when some of the data was successfully outputted but some was not. For example:

    {

    “transactionTime”: “[instant]”,

    “request” : “[base]/Patient/$export?_type=Patient,Observation,Provenance”,

    “requiresAccessToken” : true,

    “output” : [{

    “type” : “Patient”,

    “url” : http://serverpath2/patient_file_1.ndjson

    },{

    “type” : “Patient”,

    “url” : http://serverpath2/patient_file_2.ndjson

    },{

    “type” : “Observation”,

    “url” : http://serverpath2/observation_file_1.ndjson

    }],{

    “type” : “Provenance”,

    “url” : http://serverpath2/provenance_file_1.ndjson

    }],{

    “type” : “Provenance”,

    “url” : http://serverpath2/provenance_file_2.ndjson

    }],

    “error” : [{

    “type” : “OperationOutcome”,

    “url” : http://serverpath2/err_file_1.ndjson

    }],

    “extension”:{}

    }


    Deleting an Export Request

    The application can delete an export request if it was created in error or is no longer needed.

    DELETE [Content Location URL]


    Retrieving the Export File Packages

    After the application receives a status update that indicates the requested export files are complete, the application retrieves the files.

    GET [Content Location URL from the body of the completed request response]

    You can download the file packages as many times as necessary, but once they expire, they are no longer available.


    Testing

    To test bulk data requests, refer to the Testing section of the Process Overview page.


    Provenance Considerations

    The bulk data request supports the Provenance resource. The response file includes provenance data under the following conditions:

    • Provenance is included by default for those requests that do not specify which resource to include.
      [FHIR path]/Group/ABC/$export
      Means: "Get all the information for the patients in group ABC.” Because a specific resource (such as the MedicationRequest or Condition resource) is not included in the request, provenance data is included."
    • Provenance is included when explicitly listed in the request for a specific resource.
    • If provenance is passed as a requested resource, all other resources that are included in the request should then include provenance.
    • If provenance is not passed as a requested resource, no resources that are included in the request should include provenance.

    For more information on the Provenance resource, see the Searching page.