Big data queries eyed to head off Hadoop performance problems

Processing in big data systems can slow to a crawl if queries are not properly tuned or workloads not well balanced issues that call for careful monitoring of clusters.Data isn't the only thing that needs to be governed in big data systems. The queries run by data scientists and other users also have to be watched to make sure they don't bog down processing in Hadoop and Spark clusters.Hadoop performance problems became an issue at BT Group PLC after use of its data lake environment started rising rapidly in early 2016 as production applications began proliferating. "We had a bow wave of demand from users," said Jason Perkins, head of business insight and analytics architecture at the London-based company.