Apache Apex Introduction part 2

Whether to take data ingestion cycles off the ETL tool and the Data Warehouse or to facilitate competitive Data Science and building algorithms in the organization, the Data Lake a place for unmodeled and vast data will be provisioned widely in 2019. Though it doesn’t have to be complicated, the Data Lake has a few key design points that are critical, and it does need to follow some principles for success. Avoid building the Data Swamp, but not the Data Lake! The tool ecosystem is building up around the Data Lake and soon many will have a robust Lake and Data Warehouse. We will discuss policy to keep them straight, send “horses to courses,” and keep up users’ confidence in the Data Platforms.

Metadata is more relevant than ever. It continues to be a powerful enabler for high-value data-driven business activities across operations, analytics, and compliance. However, to maintain this relevance, metadata management must deal with the increasing complexity of today’s business use cases and hybrid data environments. Metadata management is notoriously manual which makes it slow in development and error-prone in maintenance. Metadata management needs better tool automation. Metadata management is typically siloed due to users managing metadata per tool or platform. Metadata management needs a unified solution that is suited to sharing, reuse, governance, and comprehensive views of distributed data.

It’s no secret that trust provides a competitive advantage, with trusted companies outperforming their market peers. Boards, executives, and businesses across the globe want to find ways to build trust with consumers, employees, investors, and all stakeholders. But how do you define metrics, quantify, and measure trust?

As data and analytics environments become increasingly complex, organizations can no longer afford to perform many operations manually. According to TDWI research, automation (in general) is one of the top three priorities for analytics. We see automation occurring throughout the data and analytics life cycle. Automation increasingly leverages embedded AI/ML algorithms (i.e., infused in the software) to help perform tasks such as profiling and cleansing data, identifying sensitive data, data mapping, surfacing insights, or building machine learning models.