BIG DATA MANAGEMENT

Databricks Announces the Launch of SQL Analytics to Enable Cloud Data Warehousing

Databricks | November 12, 2020

Databricks, the data and AI organization, today reported the dispatch of SQL Analytics, which unexpectedly empowers data experts to perform outstanding burdens recently implied uniquely for a data distribution center on a data lake. This grows the conventional extent of the data lake from data science and AI to incorporate all data remaining tasks at hand including business insight (BI) and SQL. Presently, associations can engage data groups across data designing, data science, and data analytics to deal with a solitary wellspring of truth for data. SQL Analytics understands Databricks' vision for a lakehouse engineering that consolidates data warehousing execution with data lake financial matters, coming about in up to 9x better value/execution than customary cloud data distribution centers. SQL Analytics is accessible out in the open review successful on November 18.

A lakehouse engineering rearranges data and AI for associations. Previously, data groups needed to keep up restrictive data distribution centers for BI outstanding burdens and data lakes for data science and AI remaining tasks at hand, in light of the fact that no single data stage could meet the exhibition needs of BI and the adaptability needs of data science. Costly and muddled to keep up, this conjunction of heritage designs has made data storehouses that moderate advancement and smother data group profitability. A lakehouse addresses this by running all remaining tasks at hand through a solitary design.

Shell chose Databricks to be one of the foundational components of its Shell.ai platform. "Shell has been undergoing a digital transformation as part of our ambition to deliver more and cleaner energy solutions. As part of this, we have been investing heavily in our data lake architecture. Our ambition has been to enable our data teams to rapidly query our massive datasets in the simplest possible way. The ability to execute rapid queries on petabyte scale datasets using standard BI tools is a game changer for us. Our co-innovation approach with Databricks has allowed us to influence the product roadmap and we are excited to see this come to market." Dan Jeavons, GM Data Science

"It is no longer a matter of if organizations will move their data to the cloud, but when. A lakehouse architecture built on a data lake is the ideal data architecture for data-driven organizations and this launch gives our customers a far superior option when it comes to their data strategy," said Ali Ghodsi, CEO and co-founder of Databricks. "We've worked with thousands of customers to understand where they want to take their data strategy, and the answer is overwhelmingly in favor of data lakes. The fact is that they have massive amounts of data in their data lakes and with SQL Analytics, they now can actually query that data by connecting directly to their BI tools like Tableau."

SQL Analytics is based on Delta Lake, an open organization data motor that adds dependability, quality, and security, to a client's current data lake. Clients can try not to store numerous duplicates of data, just as securing data up exclusive arrangements. To convey BI-execution on a data lake, SQL Analytics utilizes two remarkable advancements. To start with, it gives simple to-utilize auto-scaling endpoints that keep question idleness reliably low under high client load. Second, it utilizes Delta Engine, Databricks' special polymorphic question execution motor, to finish inquiries rapidly against both huge and little data sets. With local connectors for all significant BI apparatuses, including Tableau and Microsoft Power BI, clients can undoubtedly coordinate SQL Analytics into their current BI work processes to lead analytics on a lot fresher, more complete data than any time in recent memory. SQL Analytics likewise gives a SQL-local question and perception interface to permit examiners, data researchers, and designers without admittance to conventional BI apparatuses to manufacture dashboards and reports that can be effectively shared inside their association.

"Now more than ever, organizations need a data strategy that enables speed and agility to be adaptable," said Francois Ajenstat, Chief Product Officer at Tableau. "As organizations are rapidly moving their data to the cloud, we're seeing growing interest in doing analytics on the data lake. The introduction of SQL Analytics delivers an entirely new experience for customers to tap into insights from massive volumes of data with the performance, reliability and scale they need. We're proud to partner with Databricks to bring that opportunity to life."

The lakehouse architecture is widely supported by Databricks partners including:

BI Partners: Tableau, Power BI, Qlik, Looker, Thoughtspot
Ingest Partners: Fivetran, Fishtown Analytics, Matillion, Talend
Catalog Partners: Collibra, Alation
Consulting Partners: Slalom, Thorogood, Advancing Analytics

"Databricks SQL Analytics is a critical step in the most important trend in the modern data stack: the unification of traditional SQL analytics with machine-learning and data science," said George Fraser, CEO at Fivetran. "Companies make huge investments in centralizing and curating data, and they should be able to make those investments once and then implement multiple analytical paradigms in a unified environment. The Lakehouse architecture supports that."

This declaration goes ahead the impact points of noteworthy energy Databricks has accomplished over the previous year. The organization accomplished a $350M+ income run rate as of Q3 2020, up from $200M in Q3 2019, and is presently among the quickest developing venture programming cloud organizations on record. It has accomplished worldwide development, multiplying its headcount in the UK, Netherlands, Germany, and Sweden, and becoming 5x in Australia and India in the course of the most recent year. Databricks has 1,500 representatives around the world, and a large number of data groups influence its Unified Data Analytics Platform over all businesses and verticals.

Spotlight

As the volume and types of business data have increased at a phenomenal pace, and the cost to store that data has plummeted, businesses have looked to data analytics to gain new insights into their customers and operations.


Other News
BIG DATA MANAGEMENT

Alibaba Cloud Forms Partnership with Starburst To Bring The Analytics Engine For Data Mesh to Asia-Pacific Region

Starburst | December 22, 2021

After a year of record financing and year-over-year growth across sales channels, hiring and the global customer base, Starburst, the analytics anywhere company, is announcing a partnership with Alibaba Cloud, the digital technology and intelligence backbone of Alibaba Group, to deliver Starburst Enterprise to the Greater China market. Through this partnership, Alibaba Cloud is providing engineering to integrate Starburst Enterprise on Alibaba Cloud, as well as providing the sales, services, and support resources needed to deliver a seamless customer experience. Starburst is currently available on major public and private cloud platforms, including AWS, Azure, GCP, Red Hat, and HPE, but this partnership is a key step in providing the analytics engine for data mesh to Alibaba Cloud customers in Greater China. According to IDC, over the past five years, the compound annual growth rate of China's public cloud market has reached 61.1%, which is significantly higher than the 23.8% growth in the United States. With 87.4% of Chinese firms using open-source technologies, making Chinese users the second most prolific group on GitHub after the United States, this expansion presents a huge market opportunity for Starburst to enable Alibaba Cloud customers to provide better data access through a data mesh architecture, deliver data as a product and empower business users to make more informed decisions. "China, like the rest of the world, has been impacted by the digital pressure spurred on by the pandemic," said Dr Jia Yangqing, VP of Alibaba Group, Senior Fellow of the Computing Platform, Alibaba Cloud Intelligence. "As long-time users and supporters of the Trino project, we're very excited to leverage the open-source community and now this enterprise distribution of Starburst on Alibaba Cloud so that large enterprises in China can take advantage of its power to accelerate their digital transformation initiatives in the face of the challenges created by the pandemic." With this partnership, Starburst is now uniquely available on nearly every major platform and can be seamlessly procured through their marketplaces. The Alibaba Cloud partnership comes on the heels of Starburst's recent release of Stargate, a new product offering that is intended to serve as a single point of access to data across borders, with fast query performance, while meeting data privacy and sovereignty requirements. Through this partnership, companies based in China and China-based subsidiaries of multinational companies can now easily leverage these powerful, global analytics capabilities to build architectures that reflect today's global nature of business. "Global companies find that their enterprise data is increasingly spread across multiple clouds and disparate geographic regions. Starburst's vision is to unlock the analyst within us all and allow access to all of a company's data, no matter where it resides. So we naturally want to be everywhere our customers want to be. Alibaba Cloud is the perfect partner to help us extend our analytics engine for this global data mesh to the greater China market and facilitate a deeper level of hybrid analytics that hasn't previously been possible in this region." Justin Borgman, CEO of Starburst After a year of explosive growth, Starburst is poised to continue its mission of bringing analytics anywhere. This partnership is a key step in achieving that mission. To learn more, please visit starburst.io. About Starburst Starburst is the analytics engine for the data mesh. We unlock the value of distributed data by making it fast and easy to access, no matter where it lives. Starburst queries data across any database, making it instantly actionable for data-driven organizations. With Starburst, teams can lower the total cost of their infrastructure and analytics investments, prevent vendor lock-in, and use the existing tools that work for their business. Trusted by companies like Comcast, FINRA, and Condé Nast, Starburst helps companies make better decisions faster on all data.

Read More

BIG DATA MANAGEMENT

RBC and Envestnet Data and Analytics announce agreement to provide clients with greater control over their financial data

RBC | June 14, 2022

RBC is pleased to announce a data access agreement with Envestnet | Yodlee, a leading data aggregation and analytics platform, to address the needs of an increasingly digital customer base. The agreement allows RBC clients to better manage their finances and build wealth by connecting to and sharing their RBC financial information with more than 1,500 third-party applications powered by the Envestnet | Yodlee platform. RBC becomes one of the largest banks in Canada to sign a data access agreement with Envestnet | Yodlee. The implementation of this agreement will empower RBC clients with the option to share their financial data, safely and securely with Envestnet | Yodlee through a direct application program interface (API) connection. This eliminates the need for them to share their RBC credentials, improves the accuracy of the data, and significantly accelerates financial data access. "RBC is committed to providing Canadians with industry-leading digital solutions that deliver more value, without compromising the security of their information. "This agreement is a great example of that commitment in action, and how the industry can come together to build new standards that better protect Canadians' data privacy." Peter Tilton, Chief Digital Officer, Personal & Commercial Banking, RBC This move to a direct API connection significantly reduces reliance on RBC client credentials for sharing financial information in Canada, and offers clients a better method to control the release of their account information, greater reliability, and higher speeds in accessing their account information. "Our relationship with RBC is vital for empowering their customers to make intelligent financial decisions," said Farouk Ferchichi, Group President, Envestnet Data and Analytics. "Through our agreement with RBC, we are giving their customers improved access to and control of their financial data, ultimately helping them grow, protect, and manage their wealth." Consumers often use mobile apps and tools to consolidate financial information and to assist with budgeting and managing their money. The implementation of this data access agreement with Envestnet | Yodlee will improve that customer experience. Additionally, RBC clients can now experience improved control and access when sharing their financial data with applications outside the bank. "Our clients want their primary banking relationship to be anchored with RBC, but they also want to be empowered to access, use and share their financial data with other applications," added Tilton. "As we deliver this added client value, it is more important than ever that we do so in a safe and secure manner. This agreement with Envestnet | Yodlee does just that. Not only do RBC clients gain secure access to the broad suite of apps and services Envestnet | Yodlee has to offer, but they also now have more confidence and control over the data that is shared." This new, industry partnership is a demonstration of both companies' long-standing commitment to add value, prioritize security and create peace of mind for clients as they manage their finances digitally. RBC clients also benefit from a wide range of RBC's digital security tools like PIN on Mobile, ID Verification, 2-Step Verification, Card Lock, two-way fraud alerts and fraud monitoring, in addition to the RBC Digital Banking Security Guarantee. These tools and this additional layer of protection through the data access agreement is timely, as privacy is a high priority right now with the rise of fraud attempts during the pandemic. According to RBC's 2022 Fraud Prevention Month Poll, 48% of respondents say fraudsters have increasingly targeted them since the start of the pandemic, compared to 22% in 2021. And Canadians aren't just feeling the increase in fraud attempts—the Canadian Anti-Fraud Centre has reported that incidents of ID fraud targeting financial credentials nearly doubled between 2019 and 2020, from about 9,000 to more than 17,000, and final numbers for 2021 are expected to double again. About RBC Royal Bank of Canada is a global financial institution with a purpose-driven, principles-led approach to delivering leading performance. Our success comes from the 89,000+ employees who leverage their imaginations and insights to bring our vision, values and strategy to life so we can help our clients thrive and communities prosper. As Canada's biggest bank and one of the largest in the world, based on market capitalization, we have a diversified business model with a focus on innovation and providing exceptional experiences to our 17 million clients in Canada, the U.S. and 27 other countries.

Read More

BIG DATA MANAGEMENT

Factored Partners With dbt Labs to Strengthen Data Analytics Workflows and Generate Informative Insights

Factored | June 30, 2022

Factored, a leader in data-centric AI helping tech unicorns and high-profile tech companies to build high-caliber data engineering, machine learning and data analytics teams, announced today that it has partnered with dbt Labs. The collaboration with this pioneer of analytics engineering will drive business value for Factored clients by allowing them to transform and test all of their data to help disseminate insight and organizational knowledge. Thanks to the partnership, Factored engineers and analysts can efficiently collaborate across data platforms to successfully transform, test and utilize data to generate dynamic and informative business insights and ultimately fuel growth. Factored engineers are experienced working with dbt, which allows Factored clients to apply this framework to their operations. The partnership enhances cross-functional participation, data interpretability and efficiency in decision making and solution implementation for Factored's clients. dbt is an analytics engineering framework that transforms raw data in the warehouse to make it easier to use for analysis. It allows data teams to shorten the transformation path from raw data in the warehouse to crucial business insights. dbt empowers data analysts and analytics engineers to orchestrate ELT pipelines using software engineering best practices like CI/CD, documentation, logging and alerting. The partnership will allow Factored engineers and analysts to leverage both dbt Core and dbt Cloud to bring the benefits of clean, documented and tested data to business users everywhere. "Our goal at dbt Labs is to make it easier for data teams to distill and disseminate organizational knowledge," said Amos Budde, Head of Services Partnerships at dbt Labs. "Having partners like Factored helps our customers get the most out of dbt and we are excited to be partnering with them." "We are excited to partner with dbt Labs and continue helping businesses clean, organize and glean insights from their data using the most vanguard tools. At Factored, we are committed to building and deploying the most powerful data and AI solutions for our clients and dbt Labs' analytics engineering tools are helping us accomplish this." Israel Niezen, Factored CEO Since its founding in 2019, Factored has seen fast-paced growth based on the world's increasing need for rigorously trained and highly expert data science professionals. Today, Factored is one of the biggest data science companies in Latin America. About Factored Factored (backed by Andrew Ng's AI Fund and deeplearning.ai) helps leading tech companies select and build world-class data science, machine learning and AI engineering teams much faster and more cost-effectively. Factored engineers have been personally vetted, educated and mentored by some of the most talented AI educators and engineers from Silicon Valley, Stanford University and deeplearning.ai.

Read More

DATA SCIENCE

Privacera Announces Most Complete Data Access Governance for Diverse Cloud Analytical Workloads

Privacera | May 13, 2022

Privacera, the unified data access governance leader founded by the creators of Apache Ranger™, today announced the release of Privacera Platform 6.3 and PrivaceraCloud 4.3, which offer complete data governance automation and expanded support across the cloud partner ecosystem. The new releases dramatically expand the Privacera Access Governance Platform's ability to secure a diverse and increasing number of data sources and analytical workloads offered by leading public and third-party cloud service providers, such as Google Cloud's BigQuery, Snowflake, AWS, and Starburst. "Privacera pioneered the industry's first SaaS-based data governance and security solution that integrates privacy and compliance across multiple cloud services. Our latest releases not only help organizations use data effectively and responsibly, but also accelerate digital transformation by helping organizations safely migrate data to the cloud, while minimizing compliance risk. By expanding the scope of our platforms, we're even better equipped to guide organizations along their data journeys." Privacera CEO Balaji Ganesan A key milestone with this new release is that Privacera now brings the industry's first comprehensive data access and security governance solution to Google BigQuery. BigQuery, Google's serverless data warehouse solution, is rapidly gaining popularity and with this release we provide fine grained access control and data masking capabilities, as well as the ability to discover and tag sensitive data to help manage data access to data stored and processed in BigQuery. Privacera has also enhanced access control by extending Attribute Based Access Control (ABAC) across all supported data and analytical sources. ABAC is a powerful approach to simplifying and automating data access governance, especially when combined with Privacera's Role Based Access Control (RBAC) and tag-based access control. Privacera's ABAC leverages user attributes, such as region, title, and department when creating data access policies, thus allowing data access based on those attributes. Integration with access management solutions, such as Okta, further automates the data access governance process. The new release also further expands support for Starburst Enterprise by adding encryption to accelerate and improve secure data sharing by automatically encrypting and decrypting data for authorized users or applications when they access it. As a result, data science and analytics teams can securely utilize more data, build and refine predictive and machine learning models, and increase their accuracy, utilizing the full array of features in their organization's data. "We partnered with Privacera two years ago to provide enterprises with centralized, secure access to data with end-to-end governance and compliance," said Starburst co-founder and CEO Justin Borgman. "In a world where data is fundamental to the success of every business, Privacera offers solutions to help move organizations into the future. We look forward to continuing our partnership for years to come." About Privacera Privacera's SaaS-based data security and governance platform enables analytics teams to access data without compromising compliance with regulations such as GDPR, CCPA, LGPD, and HIPAA. Privacera provides a single pane of glass for securing sensitive data across multiple cloud services such as AWS, Azure, Databricks, GCP, Starburst and Snowflake. Privacera's platform is utilized by Fortune 500 customers across finance, insurance, life sciences, retail, media, consumer industries and federal agencies/government to automate sensitive data discovery and easily manage high-fidelity policy management at petabyte scale on-prem and in the cloud. Headquartered in Fremont, California, Privacera was founded in 2016 by the creators of Apache Ranger™.

Read More