Yuya Jeremy Ong, Jay Pankaj Gala, et al.
IEEE CISOSE 2024
A comprehensive benchmark is crucial for evaluating automated Business Intelligence (BI) systems and their real-world effectiveness. We propose a holistic, end-to-end framework that assesses BI systems based on the quality, relevance, and depth of insights. It categorizes queries into descriptive, diagnostic, predictive, and prescriptive types, aligning with practical BI needs. Our fully automated approach enables custom benchmark generation tailored to specific datasets. Additionally, we introduce an automated evaluation mechanism that removes reliance on strict ground truth, ensuring scalable and adaptable assessments. By addressing key limitations, our user-centered framework offers a flexible and robust methodology for advancing next-generation BI systems.
Yuya Jeremy Ong, Jay Pankaj Gala, et al.
IEEE CISOSE 2024
Nitin Gupta, Hima Patel, et al.
arXiv
Masaki Ono, Takayuki Katsuki, et al.
MIE 2020
Daiki Kimura, Naomi Simumba, et al.
AGU Fall 2023