Home > Resources > Whitepapers > Data Lake: The Definitive Guide

Data Lake: The Definitive Guide

June 20, 2018

It would be an understatement to say that the hype surrounding the data lake is causing confusion in the industry. Perhaps, this is an inherent consequence of the data industry’s need for buzzwords: it’s not uncommon for a term to rise to popularity long before there is clear definition and repeatable business value. We have seen this phenomena many times when concepts including “big data,” “data reservoir,” and even the “data warehouse” first emerged in the industry. Today’s newcomer to the data world vernacular the “data lake”is a term that has endured both the scrutiny of pundits who harp on the risk of digging a data swamp and, likewise, the vision of those who see the potential of the concept to have a profound impact on enterprise data architecture. As the data lake term begins to come off its hype cycle and face the pressures of pragmatic IT and business stakeholders, the demand for clear data lake definitions, use cases, and best practices continues to grow.

Spotlight

Netskope

Netskope is the leader in cloud security. We help the world’s largest organizations take full advantage of the cloud and web without sacrificing security. Our patented Cloud XD technology eliminates blind spots by going deeper than any other security provider to quickly target and control activities across thousands of cloud services and millions of websites. With full control from one cloud, our customers benefit from 360-degree data protection that guards data everywhere and advanced threat protection that stops elusive attacks. At Netskope, we call this smart cloud security.

OTHER WHITEPAPERS

Accountability and Traceability White Paper & Research Roadmap

whitePaper | April 18, 2023

The MIT Future of Data Initiative is leading a multi-disciplinary research agenda to design and stimulate the deployment of consumer-empowering and accountable systems to provide trusted, traceable uses of personal data on an ecosystem-wide scale. The Initiative has gathered together computer science and Internet policy researchers as well as leading commercial enterprises in financial services, payment technology, cloud platforms, insurance and other sectors to discuss current challenges and opportunities in privacy and data governance. Today’s modern privacy laws place appropriately high expectations on organizations processing personal data. At the same time, consumers report declining trust in those who handle their personal data and regulators around the world struggle with the scale of the enforcement challenge. We aim to identify and put into service technical infrastructure for enterprises seeking to handle personal data in a trustworthy and lawful manner with guardrails to enable the traceable, accountable, and scalable use of data.

Tackling climate change with data science and AI

whitePaper | April 2, 2023

In this white paper, we share how The Alan Turing Institute’s AI for science and government (ASG) programme has been using collaborative and multidisciplinary data science and AI to help tackle climate change.

Data Drives the Future of Physician Practice Management

whitePaper | April 7, 2022

This white paper was created to outline best-practices for measuring, reporting, and building a constant stream of key performance indicators related to your physician network.

CDAIT Digital Transformation

whitePaper | November 26, 2019

There has been much said about the Internet of Things (IoT) and what it can bring to business and society - creating a more deeply connected world, informed by data sourced from connected things, made better by smarter, more meaningful actions and interactions. These views often focus on how IoT technical solutions can deliver this new future, emphasizing the technology DNA of IoT devices, networks and applications. We have found, however, both through research and experience, that while technology is, in fact, the core of IoT’s DNA; it is just a piece of the puzzle needed to deliver on IoT’s promise. With 60% of IoT projects still struggling to move out of proof-of-concept and scale successfully2 there is more to IoT success than technology alone. What is needed and how can business leaders plan better for IoT success?

Fault-Tolerant Components on AWS

whitePaper | November 15, 2019

Fault-tolerance is the ability for a system to remain in operation even if some of the components used to build the system fail. Even with very conservative assumptions, a busy e-commerce site may lose thousands of dollars for every minute it is unavailable. This is just one reason why businesses and organizations strive to develop software systems that can survive faults. Amazon Web Services (AWS) provides a platform that is ideally suited for building fault-tolerant software systems. The AWS platform enables you to build fault-tolerant systems that operate with a minimal amount of human interaction and up-front financial investment.

Protect personally identifiable information with data privacy measures

whitePaper | September 23, 2022

The COVID-19 pandemic times will go down in the annals of history as the ones which saw organizations face significant disruption in the way they executed their frontend business and backend operations. The pandemic waves have made organizations do the unthinkable – allowing their workforce to either operate exclusively from their homes, or at a minimum, distribute their working hours between home and office.

Data Lake: The Definitive Guide