A commonly known example of data aggregation is the Consumer Price Index (CPI) from the Department of Labor which aggregates price changes in a wide variety of goods and services to track the fluctuation of the cost of living in the U.S. Unfortunately, despite the importance of data aggregation and its potential to improve decision-making, organizations still make major data aggregation mistakes.
Aggregated data is extremely valuable in a world with ever-increasing regulations on data sharing and security. That’s because aggregate data is anonymized and doesn’t carry the same restrictions or consent obligations as personal identifiable information (PII). This makes it easier to share, which can be vital in fields such as healthcare.
Data can only be as useful as the questions asked of it. This becomes apparent through poor query formation leading to discrepancies between what decision-makers think they’re seeing and what the collected data actually says. For example, a “running daily average” of energy consumption per customer would vary significantly depending on whether it was a weekly, monthly, or quarterly dataset.
This can create issues for data aggregation points, especially those being used in real-time dashboards. For example, imagine a network of IoT sensors measuring transformer throughput which then informs a centralized dashboard used throughout the utility. If one portion of the network is even 30 minutes behind, the aggregate data will constantly be off.
Data aggregation mistakes means also overcoming significant challenges in terms of data consistency, avoiding unnecessary migrations which lead to duplication, and giving admins greater control over how data is used and how datasets are created for analysis.
No comments yet