Taking the Front Office Beyond Traditional Business Intelligence

The global spread of the novel coronavirus (COVID-19) and the economic impact that followed has prompted many businesses to furlough the workforce or migrate from the traditional office to remote-working environments. The volatile landscape has created incremental risks, especially for organizations heavily relying on IT Sec/Ops teams to monitor security and privacy and enforce regulatory compliance.

Fanatics, a popular sports apparel website and fan gear merchandiser, needed to ingest terabytes of data from multiple historical and streaming sources transactional, e-commerce, and back-office systems to a data lake on Amazon S3. Once ingested, the data would be analyzed to better identify, predict, and fulfill customer needs related to the products Fanatics offers in over 300 online and offline stores. To accomplish this, Fanatics chose Attunity Replicate, a software solution featuring continuous data capture (CDC) and parallel threading for streaming data in real time from multiple sources into a data lake on Amazon S3. The data can then be consumed in Apache Kafka for real-time analytics. Attunity helps Fanatics avoid the heavy lifting of manually extracting data from disparate sources and enables the organization to see results in real time.

Companies continue to experience a dynamic shift in the growth of enterprise data. Fueled by a wave of emerging startups, innovative technologies and greater competition, the opportunity to drive faster decision-making has exploded – but so too have the questions on the best way to architect analytics-ready BI from the modern cloud-ready data warehouse.

Modern-day systems are transforming into complex, open-source, cloud-native services running on various environments and being developed/deployed at lightning speed by distributed teams. When working on these systems, identifying a broken link in the chain can be near impossible. Everything fails at one point or another, whether due to code bugs, infrastructure overload, or changes in end-user behavior or market driven factors or errors in data collection. This has led to the rise of DataOps with a focus on changing the organizational speed and trust in delivering data pipelines and the related artifacts by co-creating “decision quality” data with the consumers. This development has led to the idea of observability that includes monitoring, tracking, and triaging incidents to prevent downtime of the systems and around several factors such as freshness, distribution, volume, schema, lineage.