Big Data Management
PR Newswire | October 26, 2023
Databricks, the Data and AI company, today announced it has agreed to acquire Arcion, a Databricks Ventures portfolio company that helps enterprises quickly and reliably replicate data across on-prem, cloud databases and data platforms. This will enable Databricks to provide native solutions to ingest data from various databases and SaaS applications into the Databricks Lakehouse Platform. The transaction is valued at over $100 million, inclusive of incentives.
Data Lakehouse Platforms have emerged as the de facto standard for enterprise data and AI platforms. However, these data platforms are only as valuable as the data in them. Ingesting data from existing databases and applications remains complicated, fragile, and costly. Troves of important data sit not only in transactional databases such as Oracle, MySQL, and Postgres, but also in SaaS applications such as Salesforce, SAP, and Workday. According to a recent MIT Technology Review Insights and Databricks survey of senior data and technology executives ("Laying the foundation for data- and AI-led growth"), businesses still suffer from many siloed systems; 34% have 10+ systems, and of the largest companies, more than 80% have 10+ systems to juggle.
This acquisition will enable Databricks to natively provide a scalable, easy-to-use, and cost-effective solution to ingest data from various enterprise data sources. Building on a scalable change data capture (CDC) engine, Arcion offers connectors for over 20 enterprise databases and data warehouses. The integration will simplify ingesting such data either continuously or on-demand into the lakehouse, fully integrated with the enterprise security, governance, and compliance capabilities of the Databricks platform.
To build analytical dashboards, data applications, and AI models, data needs to be replicated from the systems of record like CRM, ERP, and enterprise apps to the Lakehouse, said Ali Ghodsi, Co-Founder and CEO at Databricks. Arcion's highly reliable and easy-to-use solution will enable our customers to make that data available almost instantly for faster and more informed decision-making. Arcion will be a great asset to Databricks, and we are excited to welcome the team and work with them to further develop solutions to help our customers accelerate their data and AI journeys.
"Arcion's real-time, large-scale CDC data pipeline technology extends Databricks' market-leading ETL solution to include replication of operational data in real-time," said Gary Hagmueller, CEO of Arcion. "Databricks has been a great partner and investor in Arcion, and we are very excited to join forces to help companies simplify and accelerate their data and AI business momentum."
Read More
Data Science
Business Wire | November 03, 2023
Snowflake (NYSE: SNOW), the Data Cloud company, today announced at its Snowday 2023 event new advancements that make it easier for developers to build machine learning (ML) models and full-stack apps in the Data Cloud. Snowflake is enhancing its Python capabilities through Snowpark to boost productivity, increase collaboration, and ultimately speed up end-to-end AI and ML workflows. In addition, with support for containerized workloads and expanded DevOps capabilities, developers can now accelerate development and run apps — all within Snowflake's secure and fully managed infrastructure.
“The rise of generative AI has made organizations’ most valuable asset, their data, even more indispensable. Snowflake is making it easier for developers to put that data to work so they can build powerful end-to-end machine learning models and full-stack apps natively in the Data Cloud,” said Prasanna Krishnan, Senior Director of Product Management, Snowflake. “With Snowflake Marketplace as the first cross-cloud marketplace for data and apps in the industry, customers can quickly and securely productionize what they’ve built to global end users, unlocking increased monetization, discoverability, and usage.”
Developers Gain Robust and Familiar Functionality for End-to-End Machine Learning
Snowflake is continuing to invest in Snowpark as its secure deployment and processing of non-SQL code, with over 35% of Snowflake customers using Snowpark on a weekly basis (as of September 2023). Developers increasingly look to Snowpark for complex ML model development and deployment, and Snowflake is introducing expanded functionality that makes Snowpark even more accessible and powerful for all Python developers. New advancements include:
Snowflake Notebooks (private preview): Snowflake Notebooks are a new development interface that offers an interactive, cell-based programming environment for Python and SQL users to explore, process, and experiment with data in Snowpark. Snowflake’s built-in notebooks allow developers to write and execute code, train and deploy models using Snowpark ML, visualize results with Streamlit chart elements, and much more — all within Snowflake’s unified, secure platform.
Snowpark ML Modeling API (general availability soon): Snowflake’s Snowpark ML Modeling API empowers developers and data scientists to scale out feature engineering and simplify model training for faster and more intuitive model development in Snowflake. Users can implement popular AI and ML frameworks natively on data in Snowflake, without having to create stored procedures.
Snowpark ML Operations Enhancements: The Snowpark Model Registry (public preview soon) now builds on a native Snowflake model entity and enables the scalable, secure deployment and management of models in Snowflake, including expanded support for deep learning models and open source large language models (LLMs) from Hugging Face. Snowflake is also providing developers with an integrated Snowflake Feature Store (private preview) that creates, stores, manages, and serves ML features for model training and inference.
Endeavor, the global sports and entertainment company that includes the WME Agency, IMG & On Location, UFC, and more, relies on Snowflake’s Snowpark for Python capabilities to build and deploy ML models that create highly personalized experiences and apps for fan engagement.
Snowpark serves as the driving force behind our end-to-end machine learning development, powering how we centralize and process data across our various entities, and then securely build and train models using that data to create hyper-personalized fan experiences at scale, said Saad Zaheer, VP of Data Science and Engineering, Endeavor. With Snowflake as our central data foundation bringing all of this development directly to our enterprise data, we can unlock even more ways to predict and forecast customer behavior to fuel our targeted sales and marketing engines.
Snowflake Advances Developer Capabilities Across the App Lifecycle
The Snowflake Native App Framework (general availability soon on AWS, public preview soon on Azure) now provides every organization with the necessary building blocks for app development, including distribution, operation, and monetization within Snowflake’s platform. Leading organizations are monetizing their Snowflake Native Apps through Snowflake Marketplace, with app listings more than doubling since Snowflake Summit 2023. This number is only growing as Snowflake continues to advance its developer capabilities across the app lifecycle so more organizations can unlock business impact.
For example, Cybersyn, a data-service provider, is developing Snowflake Native Apps exclusively for Snowflake Marketplace, with more than 40 customers running over 5,000 queries with its Financial & Economic Essentials Native App since June 2022. In addition, LiveRamp, a data collaboration platform, has seen the number of customers deploying its Identity Resolution and Transcoding Snowflake Native App through Snowflake Marketplace increase by more than 80% since June 2022. Lastly, SNP has been able to provide its customers with a 10x cost reduction in Snowflake data processing associated with SAP data ingestion, empowering them to drastically reduce data latency while improving SAP data availability in Snowflake through SNP’s Data Streaming for SAP - Snowflake Native App.
With Snowpark Container Services (public preview soon in select AWS regions), developers can run any component of their app — from ML training, to LLMs, to an API, and more — without needing to move data or manage complex container-based infrastructure.
Snowflake Automates DevOps for Apps, Data Pipelines, and Other Development
Snowflake is giving developers new ways to automate key DevOps and observability capabilities across testing, deploying, monitoring, and operating their apps and data pipelines — so they can take them from idea to production faster. With Snowflake’s new Database Change Management (private preview soon) features, developers can code declaratively and easily templatize their work to manage Snowflake objects across multiple environments. The Database Change Management features serve as a single source of truth for object creation across various environments, using the common “configuration as code” pattern in DevOps to automatically provision and update Snowflake objects.
Snowflake also unveiled a new Powered by Snowflake Funding Program, innovations that enable all users to securely tap into the power of generative AI with their enterprise data, enhancements to further eliminate data silos and strengthen Snowflake’s leading compliance and governance capabilities through Snowflake Horizon, and more at Snowday 2023.
Read More
Big Data Management
Business Wire | November 01, 2023
NICE Actimize, (Nasdaq: NICE) was named a winner in A-Team Group's Data Management Insight Awards USA 2023 in the category for Best Data Solution for Regulatory Compliance. NICE Actimize’s X-Sight DataIQ ClarityKYC was the recipient of the most online votes in its category derived from reader/online nominations from within the data management community and verified by A-Team Group editors and its advisory board.
NICE Actimize’s X-Sight DataIQ ClarityKYC is a SaaS workflow solution that automates data aggregation and simplifies KYC for financial services organization users. The solution facilitates compliance with KYC/Anti-Money Laundering (AML) requirements by integrating disparate datasets and streamlining the customer identification, due diligence, and credit investigation process.
Customer onboarding is a critical first step in any financial services organization’s risk management strategy. Onboarding new customers and conducting ongoing reviews presents numerous competitive challenges, which include manual and error-prone processes, long onboarding times which result in longer time to revenue for the banks, and no practical way to make sure the bank’s global regulatory policies are met in an auditable process, said Craig Costigan, CEO, NICE Actimize. NICE Actimize’s DataIQ ClarityKYC addresses these issues effectively. We thank the A-Team group and the data management community for recognizing the innovation we offer with X-Sight DataIQ.
“These awards recognize both established solution vendors and innovative newcomers providing leading data management solutions, services, and consultancy to capital markets participants across North America. Congratulations go to NICE Actimize for winning Best Data Solution for Regulatory Compliance,” said Angela Wilbraham, CEO of A-Team Group and host of the Data Management Insight Awards USA 2023.
X-Sight DataIQ ClarityKYC leverages AI-powered technologies to access traditional content while intelligently orchestrating data from various global data sources. X-Sight DataIQ Clarity reduces the amount of effort needed to conduct research. Long IT integration projects and tasks formerly done manually or requiring steps can be completed quickly, automatically saving time and effort while enabling teams to comply with confidence while reducing customer friction.
Read More