Big Data in Healthcare: Transforming Patient Experience and Access

Mayo Clinic in Rochester, MN, was the first health system to use an electronic health record (EHR) in 1960.1 Since then, electronic capture, storage and transmission of healthcare data has substantially increased. This digital shift has created an estimated 2.3 exabytes of data today2 — for reference, 5 exabytes is estimated to be equivalent to all words ever spoken by human beings.3 With emerging technologies like cloud storage, the ability to work with large data sets in healthcare is becoming increasingly accessible.

So where does this big data come from? And how should the healthcare industry use this data to transform patient experience and access? 

Read more about the significance of big data in healthcare and how data-driven solutions are transforming the healthcare industry.

What Exactly Is Big Data in Healthcare?


doctors laptop with medical data

Gartner defines big data as “high-volume, high-velocity and/or high-variety information assets that demand cost-effective, innovative forms of information processing that enable enhanced insight, decision making, and process automation.”4 Big data is unmanageable by conventional data storage methods and requires more agile methods to create insights, trends and models. 

Big data sources in healthcare primarily include:

      • Medical claims data (referred to as 837s), often sourced from clearing houses and third-party claims data aggregators
      • Remit data (referred to as 835s), sourced similarly to medical claims data
      • Patient medical records, primarily sourced from hospital and provider EHRs
      • Public health and consumer data, such as social determinants of health and demographic attributes that come from a variety of publicly and privately available consumer data sources
      • Clinical research data typically collected as part of clinical trials
      • Biometric data, often acquired by healthcare testing devices

    Although the digitization of medical records and the rapid advancement of medical technology generates big data, the industry has been slow to adopt the cloud-based computing environments necessary to work with this data. In recent years, health systems have started to recognize the importance of investing in analytics with the implementation of data visualization tools such as Tableau and PowerBI. They’ve also made the move from on-premise servers to cloud services such as Azure, AWS and Google. In addition, healthcare is investing in analytics leaders and teams outside of traditional IT who can take data from raw to insights.

    With these investments, it’s important to understand where big data can be used to improve patient experience and access.

    How Can Big Data Improve Care? 

    When done correctly, extensive data gathering and analysis can answer a variety of use cases that drive better decisions in healthcare, including making the right services available to the right patients at the right time. 

    The first step is to appropriately identify and define your use case — that is, what question are you trying to answer? For example, where are patients going for cancer care?

    The second step is to identify the data source(s) that best answer the question you are asking. Claims data, or 837s, can help you evaluate the entire market of cancer care. For instance, with Populi’s Oncology Insights product you can understand which procedures are done for any patients with a diagnosis of cancer.

    Once the data is identified and acquired, it needs to be transformed and structured in a way to facilitate robust analysis. Front-end dashboards can visualize your data, including reference data such as service line and provider network status, to bring you quick and easy answers.

    Finally, the analytics are provided to answer the specific questions posed by the use case. Insights are often packaged up into executive-level reports, and are also used to create actionable insights via a strategic plan, targeted provider list and consumer segmentation.

    How Claims Data Improves Patient Experience
    and Access


    doctor talking to happy patient in hospital bed 

    With internal data, payers, providers and health systems have limited insight into what existing patients may experience. However, with external data like claims data, you can achieve a greater and more complete understanding of the patient experience, including utilization, consumption patterns and care journeys. 

    Specific examples of how claims data and analytics can inform an improved experience: 

        • Understand where patients are going for care
        • Establish utilization benchmarks by service line
        • Assess future demand and need for healthcare services by market
        • Proactively identify clinical needs through propensity models
        • Target patients for preventive and primary care
        • Benchmark access to care for certain specialties, understanding how long it takes a patient to go between segments of care, i.e., primary care to orthopedics, medical oncology to imaging, office visit to surgery, etc.

      When evaluating external data sources, it’s important to understand data liquidity — that is, the ability to use the data flexibly and integrate with other data sources, typically in your own analytics environment. Modern analytics moves beyond front-end only access, to direct data access and the ability to build analytics specific to your use cases.

      By using available data to its fullest potential, healthcare delivery can be transformed to meet consumer and patient needs when, where and how they need it.

      1. “A history of EHRs: 10 things to know.” Becker’s Health IT. February 16, 2015.
      2. Stewart C. “Total amount of global healthcare data generated in 2013 and a projection for 2020.” Statista. September 24, 2020.
      3. “What is an Exabyte?” Teradata Glossary. Accessed April 11, 2023.
      4. “Big Data.” Gartner Glossary. Accessed April 11, 2023.

      Want to Know More?

      Let’s talk about connecting analytics you care about to the platforms you work in every day.

      Related News

      Andrea's headshot

      Media Contact

      Andrea Moschella