BIG DATA MANAGEMENT

Arcion Partners With Databricks for Real-Time Data Replication on the Lakehouse

Arcion | April 21, 2022

Arcion
Arcion today announced a partnership to bring the world’s only cloud-native, CDC-based data replication platform to Databricks. Arcion is the first partner to offer preconfigured, validated data replication for users of Databricks through that company’s new Partner Connect program.

Arcion’s product enables faster, more agile analytics and AI/ML by empowering enterprises to integrate mission-critical transactional systems with their Databricks Lakehouse in real time, at scale, and with guaranteed transactional integrity. It is the only fully managed, distributed data replication as a service on the market today, offering zero-code, zero-maintenance change data capture (CDC) pipelines that can be deployed in just minutes. It empowers data teams to move high-volume data from transactional databases like Oracle and MySQL, without a single line of code.

Partner Connect makes it possible for customers to implement Arcion’s technology directly within their Databricks Lakehouse. With just a few clicks, Partner Connect automatically configures the resources necessary to begin using streaming data pipelines. Enable real-time data ingestion with powerful pipelines between Oracle, MySQL, and Snowflake (additional sources coming soon) to the Databricks Lakehouse.

“Through Partner Connect, Arcion and Databricks are deepening our thriving relationship and working together to deliver a unified experience for our customers that offers simplicity, security, rock-solid reliability, and scale. Companies across the globe are using ML and advanced analytics to turn raw data into tangible business value, but they need the right tools to help them get there. Arcion helps companies unify their data by delivering it to Databricks, where everything is available in one place, with zero delay.”

Arcion’s CEO Gary Hagmueller

Arcion Cloud uses CDC to identify and track changes to data in transactional systems, whether they are deployed on-premise, in the cloud, or across a hybrid landscape. Arcion detects any changes made within those systems and replicates them to Databricks in real time. Capable of handling petabyte-scale integration, Arcion handles high transaction volumes easily, without adversely impacting the source system’s performance.

“Arcion’s replication for Databricks’ Lakehouse provides extraordinarily rapid time to value for analytics and AI/ML,” said Adam Conway, SVP of Products at Databricks. “By making Arcion available via Partner Connect, we’re enabling thousands of Databricks customers to discover and take advantage of Arcion’s highly scalable, efficient and flexible CDC technology. With just a few clicks, users can set up a trial account and start streaming real-time data from transactional systems to their Lakehouse.”

About Arcion
Fortune 500 companies around the world rely on Arcion’s distributed, CDC-based data replication solution to drive fast and accurate data insights. Arcion helps enterprises eliminate slow, brittle data pipelines and high-maintenance overheads. Break down data silos through high-volume, scalable change data capture pipelines with guaranteed transactional integrity.

Spotlight

While some experts say data lakes are turning into "data swamps," new research from Eckerson Group shows otherwise. A survey of 400+ professionals reveals 55% have data lakes in production. Dive in to see how users of all types are finding valuable insights in data lakes, and what gaps remain.


Other News
DATA SCIENCE

Saturn Cloud and Bodo.ai Partner to Bring Extreme Performance Python to Data Scientists

Saturn Cloud | June 02, 2022

Saturn Cloud, the data science and machine learning platform and bodo.ai, a parallel data compute platform providing extreme scale and speed for Python, have announced their partnership to take Python analytics performance to the next level for data science teams. Data scientists develop multiple workflows across teams, and rely on Saturn Cloud to provide a collaborative environment and computing resources. With this partnership, those teams now have seamless access to the Bodo platform - allowing them to scale prototypes to petabyte-scale parallel-processing production without any tuning or re-coding. Saturn Cloud's pre-built tools allow data science teams to collaborate and scale easily, without locking users into patterns. Instead, the platform encourages the workflow the user already has, while providing an environment where they don't need to rely on dev sources or manage compute environments. It prioritizes keeping the data scientist self-sufficient, while being able to collaborate and share work more efficiently. Bodo offers a parallel compute platform providing extreme scale and speed, but with the simplicity and flexibility of using native Python. In contrast to using libraries and frameworks like Spark, Bodo is a new type of compiler offering automatic parallelism and high efficiency surpassing 10,000+ cores. Bodo can also be used natively with analytics packages such as Pandas, NumPy, SciKit Learn, and more. The joint solution is available immediately, with bodo.ai software running within Saturn Cloud resources. Saturn Cloud provides a pre-built template with Bodo already installed and configured. Then, users are able to access the functionality of bodo.ai within JupyterLab or via SSH from VSCode, PyCharm, or the terminal. By using Saturn Cloud, users are able to get up to 4TB of RAM and 128 vCPUs, all backing the powerful software of Bodo. You can try the following examples right away here: Use Bodo to speed up feature engineering and model training or use Bodo to speed up data manipulation and analysis. "Our partnership is focused on providing massive speed and productivity improvements to data scientists struggling with large-scale analytics projects. Bodo's platform adds terabyte-scale processing with unheard-of infrastructure efficiencies for Saturn Cloud users." Behzad Nasre, CEO, Bodo "We not only want to provide a flexible workspace for data science teams, but enable greater Python scaling capabilities to increase productivity in projects that are more demanding. This joint offering with Bodo will give users an opportunity to take their work to the next level with automatic parallelization for better overall performance," says Sebastian Metti, one of the Saturn Cloud founders. About Saturn Cloud Saturn Cloud is a data science and machine learning platform flexible enough for any team. Collaborate together in the cloud on analyses and model training, then deploy your code. All using the same patterns you're used to, but with cloud scale. Learn more here. About Bodo Founded in 2019, Bodo.ai is an extreme-performance parallel compute platform for data analytics, scaling past 10,000 cores and petabytes of data with unprecedented efficiency and linear scaling. Leveraging automatic parallelization and the first inferential compiler, Bodo is helping F500 customers solve some of the world's largest data analysis problems. And doing so in a fraction of traditional time, complexity, and cost, all while leveraging the simplicity and flexibility of native Python. Developers can deploy Bodo on any infrastructure, from a laptop to a public cloud.

Read More

BIG DATA MANAGEMENT

Dynamo Software Enhances Data Automation for Private Investment Industry by Acquiring Smonik Systems

Dynamo Software | August 02, 2022

Dynamo Software, Inc., a market-leading provider of end-to-end cloud software solutions for the alternative investment management industry, announced today its acquisition of Smonik Systems, a leader in data management. Smonik provides cutting-edge software that closes the gap for Limited Partners (LPs) to process both structured and unstructured data. Dynamo will enhance its world-class platform with Smonik’s automated products for data collection, extraction, validation, and reconciliation. “As the financial markets undergo further stress, Dynamo has calibrated its business to provide robust, best-in-class software for alternative asset managers and institutional investors. “At Dynamo, we have built our stack to do the heavy lifting – removing several repetitive and manual processes with configurable dashboards, workflows, and reports. Augmenting our business with Smonik’s proprietary data management and reconciliation software will further fuel our mission of being the leading global, end-to-end software platform for the alternative investments ecosystem. We are excited to welcome their impressive team to the Dynamo family.” Dynamo’s CEO Hank Boughner The Smonik acquisition, combined with the Dynamo Data Automation (DDA) software, underscores Dynamo’s steadfast commitment to focus on removing labor-intensive processes, especially those around data extraction and validation. Endowments, foundations, pensions, family offices, and funds of funds (FOF), will benefit from Smonik’s ability to reduce or completely eliminate manual data entry and processing. For example, with Smonik’s software, LPs can automate the collection and extraction of data for all investment types – including alternatives – while also adopting a data-agnostic approach to reconcile any two data sets. “Ultimately, the combined power of Dynamo and Smonik is unprecedented,” said Sethu Bijumalla, CEO and Co-Founder of Smonik Systems. “By joining Dynamo, a company backed by Blackstone and Francisco Partners, we are excited by the additional resources and support to further extend Smonik’s value to private investment clients. Additionally, our two companies are built on a culture of excellence, with seasoned professionals who are fueling high-growth, innovative software that simultaneously empowers clients to increase efficiency, reduce cost, and minimize operational risk.” “This is truly a win-win for both Dynamo and Smonik’s current and prospective clients,” added Stephen Hixon, principal and co-owner of Smonik. “Dynamo is leading this industry’s race in delivering compelling end-to-end software, and now, with Smonik, the company is further cementing its leadership in data automation for LPs. We are proud to be a part of the next chapter in Dynamo’s story.” Both Dynamo and Smonik are headquartered in the Boston, MA area, allowing the operations, sales, marketing, and product innovation teams to work closely together to ensure a seamless integration. About Smonik Systems, LLC. Smonik Systems, now a Dynamo brand, provides data management and reconciliation services to the financial services industry. Using its extensive background in investment operations, Smonik prides itself in developing tools to automate back-office manual processes. The proprietary, best-in-class software delivered by Smonik focuses on automating the entire data management workflow. This includes the collection, extraction, transformation, reconciliation, and integration of both structured and unstructured data. About Dynamo Software, Inc. Dynamo Software’s mission is to be the leading global, end-to-end cloud software platform for the alternatives ecosystem, serving the information sharing and analytical data needs of our constituents. Since 1998, the company has been providing industry-tailored, highly-configurable investment management, reporting, and data management cloud software solutions to the global alternative investment industry. Dynamo’s cloud-based solutions serve the private investment landscape including private equity and venture capital funds, real estate investment firms, infrastructure, hedge funds, endowments, pensions, foundations, prime brokers, funds of funds, family offices, and fund administrators. The Dynamo™ platform has improved productivity across the alternatives ecosystem, including CRM, fundraising, deal management, research management, investor servicing, portfolio management, and compliance teams worldwide. Dynamo has a global footprint with operations across North America, EMEA, and APAC.

Read More

BIG DATA MANAGEMENT

Clarity AI’s Sustainability Data and Capabilities Will Support BlackRock’s Enterprise SFDR Reporting

Clarity AI | June 01, 2022

Clarity AI, the global sustainability tech platform, announced today that their sustainability data, integrated into BlackRock’s Aladdin platform, is being utilized in preparation for BlackRock’s enterprise reporting for the Sustainable Finance Disclosure Regulation (SFDR) framework. BlackRock will leverage Clarity AI capabilities, data and expertise to facilitate efficient and accurate reporting on Principal Adverse Impact (PAI) indicators. PAI indicators are a set of specific ESG metrics mandated by the European Union as part of SFDR, which imposes granular sustainability disclosure obligations for asset managers and other financial market participants. “We are thrilled to deepen our client relationship with BlackRock. “As part of our comprehensive sustainability tech kit, we are uniquely positioned in the market to deliver everything required for regulatory reporting, including SFDR, EU Taxonomy, UK Taxonomy, TCFD and MiFID II. Financial market participants of any size can leverage these capabilities via custom, easy integrations or our off-the-shelf web app.” Rebeca Minguela, Founder & CEO of Clarity AI Clarity AI’s market-leading SFDR coverage encompasses more than 49,000 companies, and its capabilities allow for portfolio aggregation and multi-asset look-through to more than 220,000 funds, including ETFs. All data is fully granular, which allows for better understanding of underlying calculations for each SFDR PAI. Should they choose to, financial market participants will be able to access these capabilities and data within their own configuration of Aladdin and leverage them as an input according to their own portfolio and reporting needs. "Deepening our partnership with Clarity AI is an exciting step forward for BlackRock and will provide us the ability to offer Aladdin users enterprise level reporting for SFDR,” said Stéphane Lapiquonne, Managing Director at BlackRock and Head of Sustainability for Europe, Middle East and Africa. “The depth and transparency behind Clarity AI data can help Aladdin users better understand exposures to the PAI metrics across their portfolios.” About Clarity AI Clarity AI is a sustainability technology platform that uses machine learning and big data to deliver environmental and social insights to investors, organizations, and consumers. As of May 2022, Clarity AI’s platform analyzes more than 49,000 companies, 220,000 funds, 198 countries and 188 local governments, and delivers data and analytics for investing, corporate research and reporting. Clarity AI has offices in North America, Europe and the Middle East, and its client network manages tens of trillions in assets under management. Clarity AI’s minority investors include, but are not limited to, Deutsche Börse, BlackRock, and SoftBank.

Read More

BUSINESS STRATEGY

Vertica Announces Vertica 12 for Future-Proof Analytics

Vertica | June 08, 2022

Vertica, a Micro Focus line of business, today announced the release of version 12 of the Vertica analytical database. Vertica 12 includes new major features and enhancements for analytics and machine learning across multi-cloud, hybrid on-premises and cloud, and multi-regional deployments. The announcement was made during Vertica Unify 2022, the organization's annual user conference, where attendees learned that Vertica 12 users can now choose from the broadest range of deployment options on the market, with improved automation capabilities as well, to future-proof analytics against constantly changing technology requirements. "While many companies are being forced to choose their analytics deployment strategy, to commit to one thing –public cloud, on-premises, or hybrid –no one knows exactly what the future may hold. "With Vertica 12, we have developed a completely flexible platform that is seamlessly hybrid. It is as capable of deploying in a SaaS model as it is on-premises. The continuous advancement of our analytical capabilities means that no matter what your future data strategies may hold, Vertica brings powerful analytics to your data." Scott Richards, Senior Vice President and General Manager, Vertica at Micro Focus In addition to supporting more on-premises object stores, Vertica 12 expands its Kubernetes support beyond AWS S3 to Google Cloud Storage (GCS), Azure Blob Storage and Hadoop Distributed Filesystem Storage (HDFS), making it fully cloud-native in any environment. Vertica's cloud-optimized architecture also has been enhanced with intelligent subclustering to better manage variable workloads and data sharing, helping to assign costs to owners in a logical way. On the integration front, Vertica 12 increases the interaction with the data analytics ecosystem. Customers will benefit because key proprietary and open-source technologies work seamlessly, including a new version of VerticaPy, the Vertica Python and Jupyter Notebook interface, as well as an enhanced Spark connector and broadened PMML support. About Vertica The core analytical database within the Micro Focus software portfolio, Vertica is the Unified Analytics Platform, based on a massively scalable architecture with the broadest set of analytical functions spanning event and time series, pattern matching, geospatial, and end-to-end in-database machine learning. Vertica enables many customers – from Agoda to Philips to many others – to easily apply these powerful functions to the largest and most demanding analytical workloads, arming businesses and their customers with predictive business insights faster than any analytical database or data warehouse in the market.

Read More

Spotlight

While some experts say data lakes are turning into "data swamps," new research from Eckerson Group shows otherwise. A survey of 400+ professionals reveals 55% have data lakes in production. Dive in to see how users of all types are finding valuable insights in data lakes, and what gaps remain.

Resources