Big Data Management
NetApp | November 08, 2023
NetApp introduces StorageGRID for VMware Sovereign Cloud, enhancing data storage and security for sovereign cloud customers.
NetApp's Object Storage plugin for VMware Cloud Director enables seamless integration of StorageGRID for secure Object Storage for unstructured data.
NetApp's Sovereign Cloud integration ensures data sovereignty, security, and data value while adhering to regulatory standards.
NetApp, a prominent global cloud-led, data-centric software company, has recently introduced NetApp StorageGRID for VMware Sovereign Cloud. This NetApp plugin offering for VMware Cloud Director Object Storage Extension empowers sovereign cloud customers to cost-efficiently secure, store, protect, and preserve unstructured data while adhering to global data privacy and residency regulations. Additionally, NetApp has also unveiled the latest release of NetApp ONTAP Tools for VMware vSphere (OTV 10.0), which is designed to streamline and centralize enterprise data management within multi-tenant vSphere environments.
The concept of sovereignty has emerged as a vital facet of cloud computing for entities that handle highly sensitive data, including national and state governments, as well as tightly regulated sectors like finance and healthcare. In this context, national governments are increasingly exploring ways to enhance their digital economic capabilities and reduce their reliance on multinational corporations for cloud services.
NetApp's newly introduced Object Storage plugin for VMware Cloud Director offers Cloud Service Providers a seamless means to integrate StorageGRID as their primary Object Storage solution to provide secure Object Storage for unstructured data to their customers. This integration provides StorageGRID services into the familiar VMware Cloud Director user interface, thereby minimizing training requirements and accelerating time to revenue for partners. A noteworthy feature of StorageGRID is its universal compatibility and native support for industry-standard APIs, such as the Amazon S3 API, facilitating smooth interoperability across diverse cloud environments. Enhanced functionalities like automated lifecycle management further ensure cost-effective data protection, storage, and high availability for unstructured data within VMware environments.
The integration of NetApp's Sovereign Cloud with Cloud Director empowers providers to offer customers:
Robust assurance that sensitive data, including metadata, remains under sovereign control, safeguarding against potential access by foreign authorities that may infringe upon data privacy laws.
Heightened security and compliance measures that protect applications and data from evolving cybersecurity threats, all while maintaining continuous compliance with infrastructure, trusted local, established frameworks, and local experts.
A future-proof infrastructure capable of swiftly reacting to evolving data privacy regulations, security challenges, and geopolitical dynamics.
The ability to unlock the value of data through secure data sharing and analysis, fostering innovation without compromising privacy laws and ensuring data integrity to derive accurate insights.
VMware Sovereign Cloud providers are dedicated to designing and operating cloud solutions rooted in modern, software-defined architectures that embody the core principles and best practices outlined in the VMware Sovereign Cloud framework. Workloads within VMware Sovereign Cloud environments are often characterized by a diverse range of data sets, including transactional workloads and substantial volumes of unstructured data, all requiring cost-effective and integrated management that is compliant with regulated standards for sovereign and regulated customers.
In addition to the aforementioned advancements, NetApp also announced a collaborative effort with VMware aimed at modernizing API integrations between NetApp ONTAP and VMware vSphere. This integration empowers VMware administrators to streamline the management and operations of NetApp ONTAP-based data management platforms within multi-tenant vSphere environments, all while allowing users to leverage a new micro-services-based architecture that offers enhanced scalability and availability. With the latest releases of NetApp ONTAP and ONTAP Tools for vSphere, NetApp has significantly made protection, provisioning, and securing modern VMware environments at scale faster and easier, all while maintaining a centralized point of visibility and control through vSphere.
NetApp ONTAP Tools for VMware provides two key benefits to customers:
A redefined architecture featuring VMware vSphere APIs for Storage Awareness (VASA) integration, simplifying policy-driven operations and enabling cloud-like scalability.
An automation-enabled framework driven by an API-first approach, allowing IT teams to seamlessly integrate with existing tools and construct end-to-end workflows for easy consumption of features and capabilities.
Read More
Data Science
Business Wire | November 03, 2023
Snowflake (NYSE: SNOW), the Data Cloud company, today announced at its Snowday 2023 event new advancements that make it easier for developers to build machine learning (ML) models and full-stack apps in the Data Cloud. Snowflake is enhancing its Python capabilities through Snowpark to boost productivity, increase collaboration, and ultimately speed up end-to-end AI and ML workflows. In addition, with support for containerized workloads and expanded DevOps capabilities, developers can now accelerate development and run apps — all within Snowflake's secure and fully managed infrastructure.
“The rise of generative AI has made organizations’ most valuable asset, their data, even more indispensable. Snowflake is making it easier for developers to put that data to work so they can build powerful end-to-end machine learning models and full-stack apps natively in the Data Cloud,” said Prasanna Krishnan, Senior Director of Product Management, Snowflake. “With Snowflake Marketplace as the first cross-cloud marketplace for data and apps in the industry, customers can quickly and securely productionize what they’ve built to global end users, unlocking increased monetization, discoverability, and usage.”
Developers Gain Robust and Familiar Functionality for End-to-End Machine Learning
Snowflake is continuing to invest in Snowpark as its secure deployment and processing of non-SQL code, with over 35% of Snowflake customers using Snowpark on a weekly basis (as of September 2023). Developers increasingly look to Snowpark for complex ML model development and deployment, and Snowflake is introducing expanded functionality that makes Snowpark even more accessible and powerful for all Python developers. New advancements include:
Snowflake Notebooks (private preview): Snowflake Notebooks are a new development interface that offers an interactive, cell-based programming environment for Python and SQL users to explore, process, and experiment with data in Snowpark. Snowflake’s built-in notebooks allow developers to write and execute code, train and deploy models using Snowpark ML, visualize results with Streamlit chart elements, and much more — all within Snowflake’s unified, secure platform.
Snowpark ML Modeling API (general availability soon): Snowflake’s Snowpark ML Modeling API empowers developers and data scientists to scale out feature engineering and simplify model training for faster and more intuitive model development in Snowflake. Users can implement popular AI and ML frameworks natively on data in Snowflake, without having to create stored procedures.
Snowpark ML Operations Enhancements: The Snowpark Model Registry (public preview soon) now builds on a native Snowflake model entity and enables the scalable, secure deployment and management of models in Snowflake, including expanded support for deep learning models and open source large language models (LLMs) from Hugging Face. Snowflake is also providing developers with an integrated Snowflake Feature Store (private preview) that creates, stores, manages, and serves ML features for model training and inference.
Endeavor, the global sports and entertainment company that includes the WME Agency, IMG & On Location, UFC, and more, relies on Snowflake’s Snowpark for Python capabilities to build and deploy ML models that create highly personalized experiences and apps for fan engagement.
Snowpark serves as the driving force behind our end-to-end machine learning development, powering how we centralize and process data across our various entities, and then securely build and train models using that data to create hyper-personalized fan experiences at scale, said Saad Zaheer, VP of Data Science and Engineering, Endeavor. With Snowflake as our central data foundation bringing all of this development directly to our enterprise data, we can unlock even more ways to predict and forecast customer behavior to fuel our targeted sales and marketing engines.
Snowflake Advances Developer Capabilities Across the App Lifecycle
The Snowflake Native App Framework (general availability soon on AWS, public preview soon on Azure) now provides every organization with the necessary building blocks for app development, including distribution, operation, and monetization within Snowflake’s platform. Leading organizations are monetizing their Snowflake Native Apps through Snowflake Marketplace, with app listings more than doubling since Snowflake Summit 2023. This number is only growing as Snowflake continues to advance its developer capabilities across the app lifecycle so more organizations can unlock business impact.
For example, Cybersyn, a data-service provider, is developing Snowflake Native Apps exclusively for Snowflake Marketplace, with more than 40 customers running over 5,000 queries with its Financial & Economic Essentials Native App since June 2022. In addition, LiveRamp, a data collaboration platform, has seen the number of customers deploying its Identity Resolution and Transcoding Snowflake Native App through Snowflake Marketplace increase by more than 80% since June 2022. Lastly, SNP has been able to provide its customers with a 10x cost reduction in Snowflake data processing associated with SAP data ingestion, empowering them to drastically reduce data latency while improving SAP data availability in Snowflake through SNP’s Data Streaming for SAP - Snowflake Native App.
With Snowpark Container Services (public preview soon in select AWS regions), developers can run any component of their app — from ML training, to LLMs, to an API, and more — without needing to move data or manage complex container-based infrastructure.
Snowflake Automates DevOps for Apps, Data Pipelines, and Other Development
Snowflake is giving developers new ways to automate key DevOps and observability capabilities across testing, deploying, monitoring, and operating their apps and data pipelines — so they can take them from idea to production faster. With Snowflake’s new Database Change Management (private preview soon) features, developers can code declaratively and easily templatize their work to manage Snowflake objects across multiple environments. The Database Change Management features serve as a single source of truth for object creation across various environments, using the common “configuration as code” pattern in DevOps to automatically provision and update Snowflake objects.
Snowflake also unveiled a new Powered by Snowflake Funding Program, innovations that enable all users to securely tap into the power of generative AI with their enterprise data, enhancements to further eliminate data silos and strengthen Snowflake’s leading compliance and governance capabilities through Snowflake Horizon, and more at Snowday 2023.
Read More
Big Data Management
Bloomberg | November 06, 2023
Bloomberg and Google Cloud integrate Data License Plus (DL+) with BigQuery for efficient data access and analytics.
Customers can access fully modeled data within BigQuery, eliminating data preparation time.
Mackenzie Investments adopts DL+ ESG Manager to host the acquisition, management, and publishing of Multi-vendor ESG data.
Bloomberg has unveiled a new offering designed to accelerate the data strategies of Google Cloud customers by integrating Bloomberg's cloud-based data management solution, Data License Plus (DL+), with Google Cloud's fully managed, serverless data warehouse, BigQuery.
Now, with access to Bloomberg's extensive experience modeling, managing, and delivering vast quantities of complex content, mutual customers can receive their Bloomberg Data License (DL) data, entirely modeled and seamlessly combined within BigQuery. As a result, organizations can leverage the advanced analytics capabilities of Google Cloud to extract more value from critical business information quickly and efficiently with minimal data wrangling.
Through this extended collaboration, customers can harness the powerful analytics features of BigQuery and tap into Bloomberg's extensive collection of datasets available through Data License to power their most essential workloads. Bloomberg's Data License content offers a wide variety, including reference, pricing, ESG, regulatory, estimates, fundamentals, and historical data, supporting operational, quantitative, and investment research workflows, covering over 70 million securities and 40,000 data fields.
Key benefits include:
Direct Access to Bloomberg Data in BigQuery: Bloomberg customers can seamlessly access Bloomberg Data License content within BigQuery, allowing for scalable use across their organization. This eliminates the time-consuming tasks of ingesting and structuring third-party datasets, thereby accelerating the time-to-value for analytics projects.
Elimination of Data Barriers: Google Cloud and Bloomberg will make Bloomberg's DL+ solution available to mutual customers via BigQuery. This allows for the delivery of fully modeled Bloomberg data and multi-vendor ESG content within their analytics workloads.
In a recent announcement, Bloomberg revealed that Mackenzie Investments has selected DL+ ESG Manager to host the acquisition, management, and publishing of multi-vendor ESG data. This move positions Mackenzie Investments to implement ESG investing strategies more efficiently and develop sophisticated ESG-focused insights and investment products, with BigQuery playing a central role in powering these analytics workloads moving forward.
Don Huff, the Global Head of Client Services and Operations at Bloomberg Data Management Services, stated that as capital markets firms are in the process of migrating their workloads to the Cloud, their customers require efficient access to high-quality data in a preferred environment. He expressed excitement about extending their partnership with Google Cloud, aiming to stay at the forefront of innovation in financial data management and to enhance their customers' enterprise analytics capabilities.
Stephen Orban, the VP of Migrations, ISVs, and Marketplace at Google Cloud, stated that Google Cloud and Bloomberg share a common commitment to empowering customers making data-driven decisions to power their businesses. He mentioned that the expanded alliance between the two companies would allow customers to effortlessly integrate Bloomberg's leading datasets with their own data within BigQuery. This would simplify the process of conducting analytics with valuable insights related to financial markets, regulations, ESG, and other critical business information.
Read More