Data Lake
Our team developed an efficient, cost-effective alternative to an enterprise data warehouse by designing and building a data lake.
Our client needed to make better use of its data to streamline case processing, make quicker decisions, and improve program integrity; however, they were challenged by massive volumes of data from disparate sources, many trapped in siloed legacy systems, all in different formats.
Given the volume and complexity of data, and the high cost of developing an enterprise data warehouse, our team developed an efficient, cost-effective alternative by designing and building a data lake. Using an Oracle extract process, we daily transformed and stored more than 10 GB of structured and semi-structured data. The data was drawn from over 25,000 fields in 1,575 tables, into 1,790 fields in 61 tables in the data lake, and could then be used as needed for a variety of activities, including reporting, monitoring, and asynchronous predictive analytics.
The Problem
The advantage of a data lake is that it allowed incoming data to remain as close as possible to its original, native format, and then provided a “just in time” transformation and shaping of data, based on the unique data requirements for each project and/or activity. The data lake, and its terabytes of data, lowered costs and increased capabilities by reducing the impact on source systems and making data available much more quickly.
Storing and making available disparate and highly-complex data from multiple sources and legacy systems when needed
Transforming and normalizing stored data to meet the differing data requirements of new and ongoing projects and activities
Our Solution
Incorporating changing requirements and new data sources
Complying with a growing array of State and Federal regulations
Adhering to stringent privacy and security requirements
Challenges
Providing complete audit functionality
Quick and highly cost-effective solution
Capture and storage of key data from many different sources in original, native format
Benefits
“Just in time” transformation and normalization of data to meet specific project requirements
Access to, and better use of, existing data
Improved monitoring, reporting and analysis
LET'S WORK TOGETHER
Improved staff productivity and accuracy
Increased detection and deterrence of fraud, waste, and abuse