Q&A with James Lee, Managing Director and Head of Financial Services, Analytics and Cloud Transformation at PwC

James Lee, Managing Director and Head of Financial Services, Analytics and Cloud Transformation at PwC, is a well-recognized management consulting leader and senior technology executive specializing in advising global financial services organizations on “cloud-first, data-driven” digital transformation with data and analytics, AI, and intelligent automation. He has over 20 years of strategy consulting and technology operation experience in North America, Asia, and Europe that spanned across various industries including insurance, banking, asset and wealth management, private equity, and telecommunications.

In my opinion, the most forward-thinking digital leaders usually excel at constantly instilling a digital-first mindset into their workforce and encouraging employees to cherish every “fail fast” lesson learned opportunity.



MEDIA 7: Can you please take us through your 20 years of strategy consulting and technology operation professional experience? Where did your career begin?
JAMES LEE:
My pleasure. I started my career as a software developer in Asia (after graduating with an engineering degree in Hong Kong) and then spent the next 20 years in North America in various management consulting and digital executive roles. For the past decade, I have been working at PwC with a sheer focus on growing multi-competency teams to assist our global financial services clients with driving data-driven innovations and operationalizing high-impact digital transformations. From a consulting career perspective, it has definitely been a rewarding experience over the years as I was fortunate enough to participate in many large-scale transformations with companies across the globe and met a lot of amazing people along the journey. As a trained engineer with years of strategy consulting experience in the financial services world, I found my diverse background (culture, academics, work, international exposure) has given me a balanced perspective in approaching corporate digital challenges by helping organizations better leverage the power of cloud, analytics and automation within the context of their business.


M7: What is PwC’s new global strategy, The New Equation, about? How does it address some of the major challenges in the world today?
JL:
The New Equation is our new PwC global strategy that responds to fundamental changes in the world, including technological disruption, climate change, fractured geopolitics, social tension, and the continuing effects of the COVID-19 pandemic. It focuses on two interconnected needs clients face in the coming years:

Build Trust - to meet rising expectations of transparency and stakeholder engagement and

Deliver Sustained Outcomes - with an integrated and multidisciplinary approach that combine expertise in strategy, digital, data & cloud services, people & organization, ESG, cybersecurity, and AI.

Our firm will be making over US$12 billion investment over the next 5 years ($3 billion will be in the Asia Pacific to further expand business in this high growth region) with initial commitments including new ESG Centres of Excellence, Leadership Institutes, accelerated deployment of emerging technologies and increased investment to support audit quality and advisory capacity. As a result, we will be creating over 100,000 new jobs so it is definitely an exciting time to be in the professional services industry.


There have been increasingly pressing needs about ESG(especially the “E”) as evidenced by the recent United Nations IPCC report that the earth climate crisis is unequivocally caused by human activities.



M7: What are some of the successful digital and sustainable transformation strategies that you have implemented through your work at PwC?
JL:
For this question, I think it is very important to define what success means for a digital transformation journey and the answer certainly varies for different organizations. Yet from my years of experience helping global firms strategize and operationalize their digital transformation programs, there are a few key success factors in common and will continue to be crucial:

A top-down approach with clear and measurable goals: Digital transformation is not just another big project but instead a multi-year corporate journey that requires all management teams and staff to stick together to go through the ups and downs associated with the changes. Every successful digital transformation starts with a strongly supportive board with laser-focused strategic vision and an experienced, highly collaborative squad(s) that masters delivery execution with disciplined change management, outcomes-driven budgeting model, and agile-based implementation governance.

‘Employees-Led’, ‘Customer-Focused’, ‘Technology-Enabled’: I know it sounds a bit generic but these are golden rules/guiding principles a digital executive could always rely on to make sensible decisions during ambiguous situations in a transformation. Is this budget decision going to help to upskill our staff to promote the digital-first culture? Which 2 out of these 10 proposed digital servicing ideas will provide the most ‘distinctive’ customer experience? Does this seemingly cheaper or convenient technology option align with our “cloud-first, API-connected” strategy?

“Communicate, Celebrate and Communicate again”: ‘Transformation Fatigue’ is one of the biggest threats for any digital transformation. Over the years, I found the smartest digital leaders were always the ones who mastered the art of active communications and agile goal-based celebrations to continuously instill positive momentum to the organization throughout the transformation journey.

‘Data-driven’: Regardless of AI, IoT, or intelligent automation, data is the fuel that powers any high-speed digital transformation train. A solid enterprise data strategy and a secure cloud data platform are fundamental to every successful digital transformation.

‘Digital (& Sustainable) Transformation’: As ESG is gaining more attention in the boardroom, I am seeing more and more corporate digital innovators leveraging emerging technologies to try to improve their ESG outcomes. From the early digital age of automating manual, paper-based processes with RPA and OCR technologies to more recently using AI, 5G, and augmented reality to facilitate remote work to minimize air travel (oil fuel is a major source of carbon emission), I can foresee a growing trend that companies will increasingly integrate ESG factors into their digital transformation agenda to drive sustainability benefits.


M7: What do you see as the most noticeable change right now happening in the workforce, encouraged by the rise of digital technologies?
JL:
This is a great question. I was recently invited to speak at a global digital banking summit with a few fellow C-level panelists and it is interesting that we all shared similar observations regarding some of the implications for the workforce:

The importance of “Thinking Digital, not just Being Digital”: Digital technologies bring inevitable changes to operation as well as a business model but still many companies didn’t do enough to evolve their corporate culture and business processes (“Excel is still my best friend regardless!”) after having invested millions in technology infrastructure. In my opinion, the most forward-thinking digital leaders usually excel at constantly instilling a digital-first mindset into their workforce and encouraging employees to cherish every “fail fast” lesson learned opportunity.

Upskilling in data literacy and risk governance awareness: For most companies, the primary objectives of digital transformation are to improve customer experience or time to market for products and services. While capitalizing on these benefits as part of what I called the offensive aspects of a transformation, corporations should not overlook the flip side of the coin which are the defensive duties to mitigate risks associated with the use of emerging technologies. Are your AI algorithms making decisions that align with your company’s values? How is your brand affected if you cannot explain how AI systems work to your customers and regulators? Do customers trust you with their data and did you provide adequate customer data protection training to your workforce? These are all legitimate questions that require business leaders to revisit their risk management plan and formulate an upskilling strategy for the workforce (e.g. curriculum to improve overall data and analytics literacy as well as risk governance awareness).


In this ever-changing digital world, we always emphasize to our clients the importance of upskilling so you can probably imagine most of us are vivid lifelong learners ourselves too.



M7: James, I understand you had a lot of experience helping financial services institutions with their ESG data management, analytics, and reporting needs. Would you be able to share your perspective in this space given this is such a hot topic in the industry?
JL:
Certainly. There have been increasingly pressing needs about ESG(especially the “E”) as evidenced by the recent United Nations IPCC report that the earth climate crisis is unequivocally caused by human activities. It is a sad reality but it could get worse if the slim chance remaining to stave off heating above 1.5C is not immediately addressed. It definitely requires stronger global collaborations and I think the financial services industry (as an allocator of capital) will continue to play a pivotal role in the following areas:

Sustainable Finance – promote sustainable economy through prioritizing financing decisions such as sustainability-linked loans, green bonds or carbon tax optimization,

Responsible Investment – promote ESG through thematic investing to achieve better risk-adjusted returns and value creation,

Climate Risk Analysis – mitigate climate-related operational risks (physical, transition) and achieve eco-efficiency by reducing the carbon footprint (e.g. through climate risk scenario analysis) and better managing other environmental risks (water, land, waste),

Resilient Supply Chain – transform supply chain with ESG considerations to enhance operational resilience.

From a data and AI strategy perspective, there are a few key ESG trends that I believe will continue to drive the acceleration of digital transformation in the financial services industry:

Mandatory ESG disclosure: While EU regulators were early movers in leading climate disclosures, other international financial hubs (e.g. Hong Kong Stock Exchange, London Stock Exchange) have already followed the footsteps to demand TCFD disclosures for listed companies. In addition, SEC has repeatedly expressed a new, heightened focus on disclosures about climate change. I think this compliance-driven wave will push more companies to revamp their data and reporting infrastructure to cope with the complexity of the 4Vs (volume/variety/veracity/velocity) associated with ESG data. For example, many companies are currently still using manual processes to collect internal ESG data (water consumption, carbon emissions, workforce demographics) and these data sources are usually scattered in disparate databases in various formats. Adding to the complexity of data collection is the need to bring in relevant (and material) external data sources and alternative datasets for ESG analysis.

Correlation of non-financial KPIs and financial performance: It is not uncommon these days CFOs are asked by an investor on an earning call about their firm‘s climate risk exposure or progress in diversity and inclusion. Indeed many ESG materiality assessments indicated that there are correlations between non-financial KPIs and the financial performance of a company. Given that ESG data has to be ‘Investor grade’ and audit-ready for disclosure purposes, I believe more companies will begin to incorporate ESG data and analytics into their digital finance transformation agenda by centralizing data on a cloud data lake coupled with an agile data governance process to enhance both data access and data quality.

Continuous monitoring of ESG performance: I think it is important to point out that the ultimate goal of ESG should not be just a series of sustainability reports for PR purposes. But rather an ability for companies to measure and report ESG progress in a timely manner and for investors and other stakeholders to be able to verify it with data. As I mentioned earlier, with the tightened regulatory requirements and increased complexity of ESG data, I think companies will continue to invest in emerging technologies to address the following ESG data and reporting challenges:

-      Ensure accuracy, agility, and data integrity of the relevant metrics.

-      Reduce the time and cost to prepare ESG disclosures.

-      Improve ESG industry ranking by actively monitoring sustainability KPIs to drive actionable insights.

This is an immense topic that I have a lot of passion about and can easily talk about for another hour with a beer (chuckles). If you are interested in learning more about potential ESG data and AI use cases pertaining to different sectors in financial services (insurance, asset and wealth management, banking, and capital market), I have published a few blog articles that you can easily find in my LinkedIn profile.


M7: This is the last question but it is an important one for our readers. Knowing what you know now, what advice would you give to the young people who are aspiring to pursue a successful management consulting career in digital transformation?
JL:
I would summarize into 4 key traits which I personally found are important for every management consultant: 

Listening: Management consulting is all about problem-solving for our clients. Regardless of how many world-class strategy frameworks or talented digital/data/cloud individuals a top-tier consulting firm possesses, these are all irrelevant if we don’t listen effectively to the unique challenges faced by each of our clients. Indeed the ultimate responsibility of our digital consulting practitioners is to help clients analyze their problems through active listening and then tailor a ‘fit for purpose’ digital solution approach through our global network of experienced professionals, strategic technology alliances, and intellectual assets.

Proactiveness: Management Consulting is a people business or more precisely a team sport that involves multiple stakeholders in the ecosystem: customers, employees, alliance partners, regulators, industry analysts, etc. I found almost every successful digital consulting leader has a highly proactive attitude and they never shies away from taking initiatives in their approach for innovations, stakeholders management, and professional networking,

Lifelong Learning: In this ever-changing digital world, we always emphasize to our clients the importance of upskilling so you can probably imagine most of us are vivid lifelong learners ourselves too. This trait usually goes side by side with ‘curiosity’ and I think, both are key characteristics for any individual who wants to pursue a sustainable management consulting career.

Empathy: This is my favorite one. As I mentioned before, management consulting is a professional team sport and stress management is an inevitable part of our work life. Throughout my global consulting career, I have assembled many cultural-diverse, multi-competency teams to cope with countless stressful client situations. I found ‘Empathy’ was always my best companion in those situations (for both clients and teams) and it was amazing that every time we were able to emerge as a more united and inclusive team with ever-stronger and trusted client relationships.

ABOUT PWC

With offices in 155 countries and more than 285,000 people, PwC is one of the leading professional services networks in the world that provides assurance, tax, and advisory services.

More THOUGHT LEADERS

Q&A with Charles Southwood, Vice President, N. Europe and MEA at Denodo

Media 7 | September 15, 2021

Charles Southwood, Regional VP at Denodo Technologies is responsible for the company’s business revenues in Northern Europe, Middle East and South Africa. He is passionate about working in rapidly moving and innovative markets to support customer success and to align IT solutions that meet the changing business needs. With a degree in engineering from Imperial College London, Charles has over 20 years of experience in data integration, big data, IT infrastructure/IT operations and Business Analytics....

Read More

Q&A with Vishal Srivastava, Vice President (Model Validation) at Citi

Media 7 | September 8, 2021

Vishal Srivastava, Vice President (Model Validation) at Citi was invited as a keynote speaker to present on Fraud Analytics using Machine Learning at the International Automation in Banking Summit in New York in November 2019. Vishal has experience in quantitative risk modeling using advanced engineering, statistical, and machine learning technologies. His academic qualifications in combination with a Ph.D. in Chemical Engineering and an MBA in Finance have enabled him to challenge quantitative risk models with scientific rigor. Vishal’s doctoral thesis included the development of statistical and machine learning-based risk models—some of which are currently being used commercially. Vishal has 120+ peer-reviewed citations in areas such as risk management, quantitative modeling, machine learning, and predictive analytics....

Read More

Q&A with Sadiqah Musa, Co-Founder at Black In Data

Media 7 | September 1, 2021

Sadiqah Musa, Co-Founder at Black In Data, is also an experienced Senior Data Analyst at Guardian News and Media with a demonstrated history of working in the energy and publishing sectors. She is skilled in Advanced Excel, SQL, Python, data visualization, project management, and Data Analysis and has a strong professional background with a Master of Science (MSc) from The University of Manchester....

Read More

Q&A with Charles Southwood, Vice President, N. Europe and MEA at Denodo

Media 7 | September 15, 2021

Charles Southwood, Regional VP at Denodo Technologies is responsible for the company’s business revenues in Northern Europe, Middle East and South Africa. He is passionate about working in rapidly moving and innovative markets to support customer success and to align IT solutions that meet the changing business needs. With a degree in engineering from Imperial College London, Charles has over 20 years of experience in data integration, big data, IT infrastructure/IT operations and Business Analytics....

Read More

Q&A with Vishal Srivastava, Vice President (Model Validation) at Citi

Media 7 | September 8, 2021

Vishal Srivastava, Vice President (Model Validation) at Citi was invited as a keynote speaker to present on Fraud Analytics using Machine Learning at the International Automation in Banking Summit in New York in November 2019. Vishal has experience in quantitative risk modeling using advanced engineering, statistical, and machine learning technologies. His academic qualifications in combination with a Ph.D. in Chemical Engineering and an MBA in Finance have enabled him to challenge quantitative risk models with scientific rigor. Vishal’s doctoral thesis included the development of statistical and machine learning-based risk models—some of which are currently being used commercially. Vishal has 120+ peer-reviewed citations in areas such as risk management, quantitative modeling, machine learning, and predictive analytics....

Read More

Q&A with Sadiqah Musa, Co-Founder at Black In Data

Media 7 | September 1, 2021

Sadiqah Musa, Co-Founder at Black In Data, is also an experienced Senior Data Analyst at Guardian News and Media with a demonstrated history of working in the energy and publishing sectors. She is skilled in Advanced Excel, SQL, Python, data visualization, project management, and Data Analysis and has a strong professional background with a Master of Science (MSc) from The University of Manchester....

Read More

Related News

Big Data Management

Immuta Launches Immuta Discover for Enhanced Data Security and Compliance

PR Newswire | October 31, 2023

Immuta, a data security leader, today announced the release of Immuta Discover, a new product for automated tagging and classification on cloud data platforms. Immuta Discover enables data teams to establish and maintain highly accurate metadata for the primary purpose of data access control, monitoring and regulatory compliance. Immuta Discover automatically and continuously discovers structured data in cloud data platforms. It can also leverage existing metadata pulled from enterprise data catalogs like Alation. Leveraging all of this metadata, Immuta Discover provides visibility of all sensitive data, which is critical for building data policies to protect the data and analyzing its usage. As a core pillar of Immuta's comprehensive Data Security Platform, Immuta Discover eliminates manual, error-prone processes and helps organizations improve data security, gain full visibility into their data and unlock its value. Discovering, tagging and classifying sensitive data with highly accurate and descriptive metadata is a critical component of a successful data security strategy. However, today's organizations are burdened with increasingly dispersed data as the number of data sources and users continues to grow. Existing solutions are not designed to tag and classify data across all of these sources with the accuracy and granularity required for security and governance. The lack of accurate metadata creates security gaps and blind spots, making it difficult to meet stringent compliance regulations. Immuta Discover allows data teams to easily and accurately scan, classify and tag data so they can apply data control policies and monitor data access. This allows teams to gain full visibility and context into all of their data assets, and better protect and monitor their sensitive data for enhanced data security posture management. Immuta offers 60+ pre-built and domain-specific classifiers, providing organizations with the flexibility to tailor data classifications to their unique business needs and based on their desired confidence levels. Immuta Discover is the foundational component to making any data security program work within your organization and to remain compliant, said Mo Plassnig, Chief Product Officer at Immuta. It's the first step that must happen in order to protect your data. Data is constantly changing in today's digital age. Organizations must fully understand what data they have – with accurate classification – and maintain a trustworthy inventory of sensitive data across cloud platforms in order to meet growing data security and access demands. And Immuta Discover helps our customers accomplish just this. With Immuta Discover, customers can automatically identify and classify sensitive data by leveraging the following new capabilities: Accurate and actionable metadata for data security – Immuta Discover provides accurate, granular metadata and actionable tags on sensitive data so users can seamlessly secure data across cloud platforms at scale. The product enables full metadata visibility into what data requires access controls and auditing. The metadata can be leveraged directly in Immuta Secure to govern the data and in Immuta Detect to monitor data access. Agentless, multi-cloud data discovery – Immuta Discover scans and classifies data natively, with no software to install and no agent running next to the cloud data platform. The data never leaves the data platform. This approach simplifies operations and is secure and compliant, enabling data teams to be more efficient with an always up-to-date data inventory. Out-of-the-box and customizable regulatory frameworks – Immuta Discover automatically categorizes data that is subject to leading regulatory frameworks such as CCPA, GDPR, HIPAA, or PCI, to implement compliant access control policies. These frameworks are created and maintained by Immuta's in-house Legal Engineering Team, and additionally enable customers to customize frameworks based on their interpretations of specific regulations for accelerated compliance. "I am thrilled to see the addition and announcement of Immuta Discover to Immuta's Data Security Platform," said Sanjeev Mohan, former Gartner analyst and founder of SanjMo. "Efficiently tagging and classifying data is a critical first step in protecting it against the growing number of threats in today's data management landscape. Being able to scan data natively in the cloud data platform is an innovation that enables data discovery at scale." Immuta offers a comprehensive Data Security Platform that enables sensitive data discovery, security and access control and activity monitoring. Immuta Discover automatically discovers and classifies sensitive data relevant to the leading regulatory frameworks. Immuta Detect adds critical user and data access behavior monitoring and auditing. And, Immuta Secure provides scalable data access control and enforcement across cloud data platforms. Together, these capabilities give customers comprehensive data security and governance for analytic workloads. About Immuta Immuta enables organizations to unlock value from their cloud data by protecting it and providing secure access. The Immuta Data Security Platform provides sensitive data discovery, security and access control, data activity monitoring, and has deep integrations with the leading cloud data platforms. Immuta is now trusted by Fortune 500 companies and government agencies around the world to secure their data. Founded in 2015, Immuta is headquartered in Boston, MA. To learn more about Immuta, click here.

Read More

Data Science

Snowflake Accelerates How Users Build Next Generation Apps and Machine Learning Models in the Data Cloud

Business Wire | November 03, 2023

Snowflake (NYSE: SNOW), the Data Cloud company, today announced at its Snowday 2023 event new advancements that make it easier for developers to build machine learning (ML) models and full-stack apps in the Data Cloud. Snowflake is enhancing its Python capabilities through Snowpark to boost productivity, increase collaboration, and ultimately speed up end-to-end AI and ML workflows. In addition, with support for containerized workloads and expanded DevOps capabilities, developers can now accelerate development and run apps — all within Snowflake's secure and fully managed infrastructure. “The rise of generative AI has made organizations’ most valuable asset, their data, even more indispensable. Snowflake is making it easier for developers to put that data to work so they can build powerful end-to-end machine learning models and full-stack apps natively in the Data Cloud,” said Prasanna Krishnan, Senior Director of Product Management, Snowflake. “With Snowflake Marketplace as the first cross-cloud marketplace for data and apps in the industry, customers can quickly and securely productionize what they’ve built to global end users, unlocking increased monetization, discoverability, and usage.” Developers Gain Robust and Familiar Functionality for End-to-End Machine Learning Snowflake is continuing to invest in Snowpark as its secure deployment and processing of non-SQL code, with over 35% of Snowflake customers using Snowpark on a weekly basis (as of September 2023). Developers increasingly look to Snowpark for complex ML model development and deployment, and Snowflake is introducing expanded functionality that makes Snowpark even more accessible and powerful for all Python developers. New advancements include: Snowflake Notebooks (private preview): Snowflake Notebooks are a new development interface that offers an interactive, cell-based programming environment for Python and SQL users to explore, process, and experiment with data in Snowpark. Snowflake’s built-in notebooks allow developers to write and execute code, train and deploy models using Snowpark ML, visualize results with Streamlit chart elements, and much more — all within Snowflake’s unified, secure platform. Snowpark ML Modeling API (general availability soon): Snowflake’s Snowpark ML Modeling API empowers developers and data scientists to scale out feature engineering and simplify model training for faster and more intuitive model development in Snowflake. Users can implement popular AI and ML frameworks natively on data in Snowflake, without having to create stored procedures. Snowpark ML Operations Enhancements: The Snowpark Model Registry (public preview soon) now builds on a native Snowflake model entity and enables the scalable, secure deployment and management of models in Snowflake, including expanded support for deep learning models and open source large language models (LLMs) from Hugging Face. Snowflake is also providing developers with an integrated Snowflake Feature Store (private preview) that creates, stores, manages, and serves ML features for model training and inference. Endeavor, the global sports and entertainment company that includes the WME Agency, IMG & On Location, UFC, and more, relies on Snowflake’s Snowpark for Python capabilities to build and deploy ML models that create highly personalized experiences and apps for fan engagement. Snowpark serves as the driving force behind our end-to-end machine learning development, powering how we centralize and process data across our various entities, and then securely build and train models using that data to create hyper-personalized fan experiences at scale, said Saad Zaheer, VP of Data Science and Engineering, Endeavor. With Snowflake as our central data foundation bringing all of this development directly to our enterprise data, we can unlock even more ways to predict and forecast customer behavior to fuel our targeted sales and marketing engines. Snowflake Advances Developer Capabilities Across the App Lifecycle The Snowflake Native App Framework (general availability soon on AWS, public preview soon on Azure) now provides every organization with the necessary building blocks for app development, including distribution, operation, and monetization within Snowflake’s platform. Leading organizations are monetizing their Snowflake Native Apps through Snowflake Marketplace, with app listings more than doubling since Snowflake Summit 2023. This number is only growing as Snowflake continues to advance its developer capabilities across the app lifecycle so more organizations can unlock business impact. For example, Cybersyn, a data-service provider, is developing Snowflake Native Apps exclusively for Snowflake Marketplace, with more than 40 customers running over 5,000 queries with its Financial & Economic Essentials Native App since June 2022. In addition, LiveRamp, a data collaboration platform, has seen the number of customers deploying its Identity Resolution and Transcoding Snowflake Native App through Snowflake Marketplace increase by more than 80% since June 2022. Lastly, SNP has been able to provide its customers with a 10x cost reduction in Snowflake data processing associated with SAP data ingestion, empowering them to drastically reduce data latency while improving SAP data availability in Snowflake through SNP’s Data Streaming for SAP - Snowflake Native App. With Snowpark Container Services (public preview soon in select AWS regions), developers can run any component of their app — from ML training, to LLMs, to an API, and more — without needing to move data or manage complex container-based infrastructure. Snowflake Automates DevOps for Apps, Data Pipelines, and Other Development Snowflake is giving developers new ways to automate key DevOps and observability capabilities across testing, deploying, monitoring, and operating their apps and data pipelines — so they can take them from idea to production faster. With Snowflake’s new Database Change Management (private preview soon) features, developers can code declaratively and easily templatize their work to manage Snowflake objects across multiple environments. The Database Change Management features serve as a single source of truth for object creation across various environments, using the common “configuration as code” pattern in DevOps to automatically provision and update Snowflake objects. Snowflake also unveiled a new Powered by Snowflake Funding Program, innovations that enable all users to securely tap into the power of generative AI with their enterprise data, enhancements to further eliminate data silos and strengthen Snowflake’s leading compliance and governance capabilities through Snowflake Horizon, and more at Snowday 2023.

Read More

Big Data Management

Databricks Agrees to Acquire Arcion, the Leading Provider for Real-Time Enterprise Data Replication Technology

PR Newswire | October 26, 2023

Databricks, the Data and AI company, today announced it has agreed to acquire Arcion, a Databricks Ventures portfolio company that helps enterprises quickly and reliably replicate data across on-prem, cloud databases and data platforms. This will enable Databricks to provide native solutions to ingest data from various databases and SaaS applications into the Databricks Lakehouse Platform. The transaction is valued at over $100 million, inclusive of incentives. Data Lakehouse Platforms have emerged as the de facto standard for enterprise data and AI platforms. However, these data platforms are only as valuable as the data in them. Ingesting data from existing databases and applications remains complicated, fragile, and costly. Troves of important data sit not only in transactional databases such as Oracle, MySQL, and Postgres, but also in SaaS applications such as Salesforce, SAP, and Workday. According to a recent MIT Technology Review Insights and Databricks survey of senior data and technology executives ("Laying the foundation for data- and AI-led growth"), businesses still suffer from many siloed systems; 34% have 10+ systems, and of the largest companies, more than 80% have 10+ systems to juggle. This acquisition will enable Databricks to natively provide a scalable, easy-to-use, and cost-effective solution to ingest data from various enterprise data sources. Building on a scalable change data capture (CDC) engine, Arcion offers connectors for over 20 enterprise databases and data warehouses. The integration will simplify ingesting such data either continuously or on-demand into the lakehouse, fully integrated with the enterprise security, governance, and compliance capabilities of the Databricks platform. To build analytical dashboards, data applications, and AI models, data needs to be replicated from the systems of record like CRM, ERP, and enterprise apps to the Lakehouse, said Ali Ghodsi, Co-Founder and CEO at Databricks. Arcion's highly reliable and easy-to-use solution will enable our customers to make that data available almost instantly for faster and more informed decision-making. Arcion will be a great asset to Databricks, and we are excited to welcome the team and work with them to further develop solutions to help our customers accelerate their data and AI journeys. "Arcion's real-time, large-scale CDC data pipeline technology extends Databricks' market-leading ETL solution to include replication of operational data in real-time," said Gary Hagmueller, CEO of Arcion. "Databricks has been a great partner and investor in Arcion, and we are very excited to join forces to help companies simplify and accelerate their data and AI business momentum."

Read More

Big Data Management

Immuta Launches Immuta Discover for Enhanced Data Security and Compliance

PR Newswire | October 31, 2023

Immuta, a data security leader, today announced the release of Immuta Discover, a new product for automated tagging and classification on cloud data platforms. Immuta Discover enables data teams to establish and maintain highly accurate metadata for the primary purpose of data access control, monitoring and regulatory compliance. Immuta Discover automatically and continuously discovers structured data in cloud data platforms. It can also leverage existing metadata pulled from enterprise data catalogs like Alation. Leveraging all of this metadata, Immuta Discover provides visibility of all sensitive data, which is critical for building data policies to protect the data and analyzing its usage. As a core pillar of Immuta's comprehensive Data Security Platform, Immuta Discover eliminates manual, error-prone processes and helps organizations improve data security, gain full visibility into their data and unlock its value. Discovering, tagging and classifying sensitive data with highly accurate and descriptive metadata is a critical component of a successful data security strategy. However, today's organizations are burdened with increasingly dispersed data as the number of data sources and users continues to grow. Existing solutions are not designed to tag and classify data across all of these sources with the accuracy and granularity required for security and governance. The lack of accurate metadata creates security gaps and blind spots, making it difficult to meet stringent compliance regulations. Immuta Discover allows data teams to easily and accurately scan, classify and tag data so they can apply data control policies and monitor data access. This allows teams to gain full visibility and context into all of their data assets, and better protect and monitor their sensitive data for enhanced data security posture management. Immuta offers 60+ pre-built and domain-specific classifiers, providing organizations with the flexibility to tailor data classifications to their unique business needs and based on their desired confidence levels. Immuta Discover is the foundational component to making any data security program work within your organization and to remain compliant, said Mo Plassnig, Chief Product Officer at Immuta. It's the first step that must happen in order to protect your data. Data is constantly changing in today's digital age. Organizations must fully understand what data they have – with accurate classification – and maintain a trustworthy inventory of sensitive data across cloud platforms in order to meet growing data security and access demands. And Immuta Discover helps our customers accomplish just this. With Immuta Discover, customers can automatically identify and classify sensitive data by leveraging the following new capabilities: Accurate and actionable metadata for data security – Immuta Discover provides accurate, granular metadata and actionable tags on sensitive data so users can seamlessly secure data across cloud platforms at scale. The product enables full metadata visibility into what data requires access controls and auditing. The metadata can be leveraged directly in Immuta Secure to govern the data and in Immuta Detect to monitor data access. Agentless, multi-cloud data discovery – Immuta Discover scans and classifies data natively, with no software to install and no agent running next to the cloud data platform. The data never leaves the data platform. This approach simplifies operations and is secure and compliant, enabling data teams to be more efficient with an always up-to-date data inventory. Out-of-the-box and customizable regulatory frameworks – Immuta Discover automatically categorizes data that is subject to leading regulatory frameworks such as CCPA, GDPR, HIPAA, or PCI, to implement compliant access control policies. These frameworks are created and maintained by Immuta's in-house Legal Engineering Team, and additionally enable customers to customize frameworks based on their interpretations of specific regulations for accelerated compliance. "I am thrilled to see the addition and announcement of Immuta Discover to Immuta's Data Security Platform," said Sanjeev Mohan, former Gartner analyst and founder of SanjMo. "Efficiently tagging and classifying data is a critical first step in protecting it against the growing number of threats in today's data management landscape. Being able to scan data natively in the cloud data platform is an innovation that enables data discovery at scale." Immuta offers a comprehensive Data Security Platform that enables sensitive data discovery, security and access control and activity monitoring. Immuta Discover automatically discovers and classifies sensitive data relevant to the leading regulatory frameworks. Immuta Detect adds critical user and data access behavior monitoring and auditing. And, Immuta Secure provides scalable data access control and enforcement across cloud data platforms. Together, these capabilities give customers comprehensive data security and governance for analytic workloads. About Immuta Immuta enables organizations to unlock value from their cloud data by protecting it and providing secure access. The Immuta Data Security Platform provides sensitive data discovery, security and access control, data activity monitoring, and has deep integrations with the leading cloud data platforms. Immuta is now trusted by Fortune 500 companies and government agencies around the world to secure their data. Founded in 2015, Immuta is headquartered in Boston, MA. To learn more about Immuta, click here.

Read More

Data Science

Snowflake Accelerates How Users Build Next Generation Apps and Machine Learning Models in the Data Cloud

Business Wire | November 03, 2023

Snowflake (NYSE: SNOW), the Data Cloud company, today announced at its Snowday 2023 event new advancements that make it easier for developers to build machine learning (ML) models and full-stack apps in the Data Cloud. Snowflake is enhancing its Python capabilities through Snowpark to boost productivity, increase collaboration, and ultimately speed up end-to-end AI and ML workflows. In addition, with support for containerized workloads and expanded DevOps capabilities, developers can now accelerate development and run apps — all within Snowflake's secure and fully managed infrastructure. “The rise of generative AI has made organizations’ most valuable asset, their data, even more indispensable. Snowflake is making it easier for developers to put that data to work so they can build powerful end-to-end machine learning models and full-stack apps natively in the Data Cloud,” said Prasanna Krishnan, Senior Director of Product Management, Snowflake. “With Snowflake Marketplace as the first cross-cloud marketplace for data and apps in the industry, customers can quickly and securely productionize what they’ve built to global end users, unlocking increased monetization, discoverability, and usage.” Developers Gain Robust and Familiar Functionality for End-to-End Machine Learning Snowflake is continuing to invest in Snowpark as its secure deployment and processing of non-SQL code, with over 35% of Snowflake customers using Snowpark on a weekly basis (as of September 2023). Developers increasingly look to Snowpark for complex ML model development and deployment, and Snowflake is introducing expanded functionality that makes Snowpark even more accessible and powerful for all Python developers. New advancements include: Snowflake Notebooks (private preview): Snowflake Notebooks are a new development interface that offers an interactive, cell-based programming environment for Python and SQL users to explore, process, and experiment with data in Snowpark. Snowflake’s built-in notebooks allow developers to write and execute code, train and deploy models using Snowpark ML, visualize results with Streamlit chart elements, and much more — all within Snowflake’s unified, secure platform. Snowpark ML Modeling API (general availability soon): Snowflake’s Snowpark ML Modeling API empowers developers and data scientists to scale out feature engineering and simplify model training for faster and more intuitive model development in Snowflake. Users can implement popular AI and ML frameworks natively on data in Snowflake, without having to create stored procedures. Snowpark ML Operations Enhancements: The Snowpark Model Registry (public preview soon) now builds on a native Snowflake model entity and enables the scalable, secure deployment and management of models in Snowflake, including expanded support for deep learning models and open source large language models (LLMs) from Hugging Face. Snowflake is also providing developers with an integrated Snowflake Feature Store (private preview) that creates, stores, manages, and serves ML features for model training and inference. Endeavor, the global sports and entertainment company that includes the WME Agency, IMG & On Location, UFC, and more, relies on Snowflake’s Snowpark for Python capabilities to build and deploy ML models that create highly personalized experiences and apps for fan engagement. Snowpark serves as the driving force behind our end-to-end machine learning development, powering how we centralize and process data across our various entities, and then securely build and train models using that data to create hyper-personalized fan experiences at scale, said Saad Zaheer, VP of Data Science and Engineering, Endeavor. With Snowflake as our central data foundation bringing all of this development directly to our enterprise data, we can unlock even more ways to predict and forecast customer behavior to fuel our targeted sales and marketing engines. Snowflake Advances Developer Capabilities Across the App Lifecycle The Snowflake Native App Framework (general availability soon on AWS, public preview soon on Azure) now provides every organization with the necessary building blocks for app development, including distribution, operation, and monetization within Snowflake’s platform. Leading organizations are monetizing their Snowflake Native Apps through Snowflake Marketplace, with app listings more than doubling since Snowflake Summit 2023. This number is only growing as Snowflake continues to advance its developer capabilities across the app lifecycle so more organizations can unlock business impact. For example, Cybersyn, a data-service provider, is developing Snowflake Native Apps exclusively for Snowflake Marketplace, with more than 40 customers running over 5,000 queries with its Financial & Economic Essentials Native App since June 2022. In addition, LiveRamp, a data collaboration platform, has seen the number of customers deploying its Identity Resolution and Transcoding Snowflake Native App through Snowflake Marketplace increase by more than 80% since June 2022. Lastly, SNP has been able to provide its customers with a 10x cost reduction in Snowflake data processing associated with SAP data ingestion, empowering them to drastically reduce data latency while improving SAP data availability in Snowflake through SNP’s Data Streaming for SAP - Snowflake Native App. With Snowpark Container Services (public preview soon in select AWS regions), developers can run any component of their app — from ML training, to LLMs, to an API, and more — without needing to move data or manage complex container-based infrastructure. Snowflake Automates DevOps for Apps, Data Pipelines, and Other Development Snowflake is giving developers new ways to automate key DevOps and observability capabilities across testing, deploying, monitoring, and operating their apps and data pipelines — so they can take them from idea to production faster. With Snowflake’s new Database Change Management (private preview soon) features, developers can code declaratively and easily templatize their work to manage Snowflake objects across multiple environments. The Database Change Management features serve as a single source of truth for object creation across various environments, using the common “configuration as code” pattern in DevOps to automatically provision and update Snowflake objects. Snowflake also unveiled a new Powered by Snowflake Funding Program, innovations that enable all users to securely tap into the power of generative AI with their enterprise data, enhancements to further eliminate data silos and strengthen Snowflake’s leading compliance and governance capabilities through Snowflake Horizon, and more at Snowday 2023.

Read More

Big Data Management

Databricks Agrees to Acquire Arcion, the Leading Provider for Real-Time Enterprise Data Replication Technology

PR Newswire | October 26, 2023

Databricks, the Data and AI company, today announced it has agreed to acquire Arcion, a Databricks Ventures portfolio company that helps enterprises quickly and reliably replicate data across on-prem, cloud databases and data platforms. This will enable Databricks to provide native solutions to ingest data from various databases and SaaS applications into the Databricks Lakehouse Platform. The transaction is valued at over $100 million, inclusive of incentives. Data Lakehouse Platforms have emerged as the de facto standard for enterprise data and AI platforms. However, these data platforms are only as valuable as the data in them. Ingesting data from existing databases and applications remains complicated, fragile, and costly. Troves of important data sit not only in transactional databases such as Oracle, MySQL, and Postgres, but also in SaaS applications such as Salesforce, SAP, and Workday. According to a recent MIT Technology Review Insights and Databricks survey of senior data and technology executives ("Laying the foundation for data- and AI-led growth"), businesses still suffer from many siloed systems; 34% have 10+ systems, and of the largest companies, more than 80% have 10+ systems to juggle. This acquisition will enable Databricks to natively provide a scalable, easy-to-use, and cost-effective solution to ingest data from various enterprise data sources. Building on a scalable change data capture (CDC) engine, Arcion offers connectors for over 20 enterprise databases and data warehouses. The integration will simplify ingesting such data either continuously or on-demand into the lakehouse, fully integrated with the enterprise security, governance, and compliance capabilities of the Databricks platform. To build analytical dashboards, data applications, and AI models, data needs to be replicated from the systems of record like CRM, ERP, and enterprise apps to the Lakehouse, said Ali Ghodsi, Co-Founder and CEO at Databricks. Arcion's highly reliable and easy-to-use solution will enable our customers to make that data available almost instantly for faster and more informed decision-making. Arcion will be a great asset to Databricks, and we are excited to welcome the team and work with them to further develop solutions to help our customers accelerate their data and AI journeys. "Arcion's real-time, large-scale CDC data pipeline technology extends Databricks' market-leading ETL solution to include replication of operational data in real-time," said Gary Hagmueller, CEO of Arcion. "Databricks has been a great partner and investor in Arcion, and we are very excited to join forces to help companies simplify and accelerate their data and AI business momentum."

Read More