Back to Pilotfish Home
Data Analytics & Reporting Integration Case Study

Massive Data Analytics Integration of Administrative and Clinical Data

Adaptability, flexibility and high performance – that was all that was demanded by this new client. Piece of cake! This massive big data integration and complex implementation case study showcases the superiority of the PilotFish Integration Engine Solution. The client is focused on delivering a single source of healthcare industry financial, operational and quality metrics, coupled with proprietary, value-added analytical services.

The data sets are both high volume and exceptionally diverse. Batch data formats included administrative X12 EDI transactions – such as 837I (Institutional) and 837P (Professional) claims, CMS 1500, tabular batches of data (CSVs) as well as custom formats. Real-time messages include HL7 2.x standards messaging and XML. No matter what the data source or target, PilotFish offers one integrated solution that can handle it all.


The client provides strategic consulting, research, cloud-based software solutions and comprehensive implementation for revenue and other management services. Its business intelligence software and services are relied upon by customers in the healthcare ecosystem across the USA. Its solutions must aggregate significant volumes of extremely diverse administrative and clinical data into client-specific data repositories to deliver its valued high-quality metrics and actionable reporting.

The software solutions offered by the client require acquisition, validation, normalization and analysis of massive amounts of clinical and administrative patient data from a wide variety of healthcare and other sources. This information must be aggregated through numerous real-time and batch data channels, accurately parsed and cleansed, then loaded into the client’s proprietary analytics platform.


The client’s existing combination of homegrown and commercial software was proving incapable of readily handling the growing diversity and the number of data feeds required by its business. Dealing effectively with either alone would be difficult. Failing at handling both was untenable. 

Before selecting PilotFish, the client undertook an extensive evaluation process of all the major interface engines in the industry. This evaluation process culminated in a “bake-off” among the selected finalists. The finalists were tasked with demonstrating their solutions’ ability to solve the client’s integration challenges represented in a series of use cases. PilotFish proved to be the only solution capable of solving and completing the integration challenge for each specific use case. 

While the representation of the data took many forms, it typically includes a semantic payload from which the client could ascertain anonymous historical patient information.  They produced population health and quality reports based on these data sources. Once the data was received, it needed to be transformed into their specific format and then placed into normalized relational database tables. 

The client provides highly-valued real-time reports to their customers. So, throughput was also an integral part of the challenge. The data acquisition process absolutely needed to keep pace with the flood of input being received on a daily basis. 

Additionally, the distillation of this data was coming from widely differing sources. They had pushed to the limit the number of unique mappings and transformation rules that could feasibly be maintained using their current solution of homegrown and commercial software. The status quo had become terribly unwieldy to manage, and data quality issues were beginning to arise with increasing frequency.


After the selection of PilotFish, the client collaborated with the PilotFish professional services team to implement the PilotFish integration engine solution, consisting of the:

    1. eiPlatform: the Integration Engine production server
    2. eiConsole: the graphical IDE for configuring, managing and maintaining interfaces

PilotFish provided the architectural flexibility to adapt to diverse data flavors and formats, using a combination of out-of-the-box parsers and finely tuned custom components. The componentized nature of PilotFish software also maximizes the potential for reuse. Many times, new feeds can now be accepted by making only minor tweaks to existing interfaces. With reuse, the client has brought on new customers faster.


Executive support grew with bottom-line results via the benefit of leveraging PilotFish’s Interface Solution to automate processes. The client profited from PilotFish’s built-in flexibility to meet any new challenge. The vastly improved level of service achieved with PilotFish measurably increased existing customer satisfaction in the near term. Faster implementations and improved productivity allowed the client to expand more rapidly – without increasing implementation staff.

PilotFish’s advanced technology and rapid development tools delivered the adaptability the client urgently needed to effectively integrate with the ever-evolving array of IT systems, solutions and equipment found across the healthcare ecosystem.

With PilotFish, you gain the business and financial advantages of a single, consistent approach to integrations. Imagine the benefits of the acceleration of data integration from anything to anything. Then see the positive revenue impact of increased automation and streamlined business processes.

PilotFish is the only integration solution you’ll ever need. Let us prove the case. We love a challenge. Contact Us Now.

HL7 is the registered trademark of Health Level Seven International.
X12, chartered by the American National Standards Institute for more than 35 years, develops and maintains EDI standards and XML schemas.

This is a unique website which will require a more modern browser to work! Please upgrade today!