Prophecy Accelerates Adoption of Lakehouse Technology

Prophecy | June 22, 2022

Prophecy, the premier low-code platform for data engineering, today announced the release of Prophecy for Databricks, a powerful new service that enables building data pipelines for business intelligence and machine learning simpler and quicker. With a visual drag-and-drop canvas, this platform allows anybody interested in data engineering to visually and interactively create, deploy, and monitor data pipelines on Apache Spark.

Prophecy for Databricks, designed for usage by both seasoned data engineering teams and non-programmer data citizens, allows several more people to simply develop pipelines, transfer them to production, and expedite the transformation of enterprises to become data-driven. With 10x users enabled, data teams see a dramatic improvement in operational efficiency and data quality, allowing them to manage more pipelines than ever before.

According to IDC, data is being created at a 23% annual growth rate, which means that 181 zettabytes of data will be developed by 2025. Businesses are struggling to keep up with the rate at which data is growing. According to Gartner, the DBMS market is nearly $80 billion and has grown 22% in the last year, with cloud DBMS rising even faster than the overall DBMS market.

Existing data engineering products do not address the requirements of businesses and have proven to be overly complex and inefficient. Businesses can 10x data engineering with Prophecy for Databricks, resulting in dramatic increases in data practitioners' doing data engineering, individual productivity, data pipeline reliability, and data quality.

"The industry need for data & analytics far outstrips what can be produced by data engineers programming in notebooks. With this release of Prophecy for Databricks, we're providing powerful, visual tools that enable an order of magnitude more data users to quickly develop data pipelines, at the same level as programmers. This expansion of data engineering to non-programmers is the only way to realize the potential of data at scale."

Raj Bains, CEO and co-founder of Prophecy


It goes without doubt that big data is clearly delivering significant value to users. Larger enterprises are leading the way and getting results from big data with focused initiatives. Big Data Analytics Solutions have the potential to be not just disruptive, but also potentially transformational. The consensus is clear: big data brings disruption that can revolutionize business. Successful big data companies are leveraging big data tools and big data technologies to drive results across large, complex enterprises in many different industries.


It goes without doubt that big data is clearly delivering significant value to users. Larger enterprises are leading the way and getting results from big data with focused initiatives. Big Data Analytics Solutions have the potential to be not just disruptive, but also potentially transformational. The consensus is clear: big data brings disruption that can revolutionize business. Successful big data companies are leveraging big data tools and big data technologies to drive results across large, complex enterprises in many different industries.

Related News


Wipro Launches Wipro Data Intelligence Suite

Wipro | December 02, 2022

At AWS re:Invent today, Wipro Limited announced the launch of Wipro Data Intelligence Suite, a one-stop solution for accelerating cloud modernization and data monetization, focused on modernizing data estates, including data stores, pipelines and visualizations, running on Amazon Web Services (AWS). Wipro Data Intelligence Suite offers reliable and secure means to migrate from existing platforms and fragmented legacy systems to the cloud. “Today, cloud architectures are enabling companies to transform their legacy systems, allowing them to develop business innovations, optimize costs, and enhance agility,” said Sudhir Kesavan, Senior Vice President & Global Head of Cloud Transformation, Wipro FullStride Cloud Services, Wipro Limited. “We’re excited to put these benefits in reach for our clients using Wipro’s FullStride Cloud Services’ full-stack approach in collaboration with AWS.” In conjunction with a clearly defined cloud migration strategy, Wipro Data Intelligence Suite offers a standardized platform that supports each phase of the migration process. With a comprehensive approach, enterprises can unlock new opportunities while avoiding stability, latency, or data loss issues. Wipro Data Intelligence Suite enables an end-to-end automation of a company’s cloud migration journey, ensuring that businesses can gain business value quickly while mitigating migration risks. It delivers an efficient, reliable pathway to cloud-driven modernizations that set the stage for accelerated workflows and enhanced data analytics operations. Its innovative suite of accelerators and ready-to-deploy modules allow enterprises to develop an effective strategy for migrating legacy applications and data to the cloud, as well as prioritizing their cloud investments. “The scalability, security, and stability of AWS’s cloud-based architecture, together with Wipro’s Data Intelligence Suite, increases productivity, reduces migration costs, and drives faster time to market. In today’s rapidly changing world, we take pride in the ability to deliver seamless transformation that simplifies technology complexities and enhance business value for our clients.” -Sriram Narasimhan, Senior Vice President and Global Head of Data & Analytics, Wipro Limited. About Wipro Limited Wipro Limited is a leading technology services and consulting company focused on building innovative solutions that address clients’ most complex digital transformation needs. Leveraging our holistic portfolio of capabilities in consulting, design, engineering, and operations, we help clients realize their boldest ambitions and build future-ready, sustainable businesses. With over 250,000 employees and business partners across 66 countries, we deliver on the promise of helping our customers, colleagues, and communities thrive in an ever-changing world. For additional information, visit us at

Read More


Hammerspace Shatters Expectations for High-Performance File Data Architectures

Hammerspace | November 14, 2022

Hammerspace, the pioneer of the global data environment, today unveiled the performance capabilities that many of the most data-intensive organizations in the world depend on for high-performance data and storage in decentralized workflows. Hammerspace completely changes previously held notions of how unstructured data architectures can work, delivering the performance needed to free workloads from data silos, eliminate copy proliferation, and provide direct data access to applications and users, no matter where the data is stored. Hammerspace allows organizations to take full advantage of the performance capabilities of any server, storage system and network anywhere in the world. This capability enables a unified, fast, and efficient global data environment for the entire workflow, from data creation to processing, collaboration, and archiving across edge devices, data centers, and public and private clouds. 1) High-Performance Across Data Centers and to the Cloud: Saturate the Available Internet or Private Links Instruments, applications, compute clusters and the workforce are increasingly decentralized. With Hammerspace, all users and applications have globally shared, secured access, to all data no matter which storage platform or location it is on, as if it were all on a local NAS. Hammerspace overcomes data gravity to make remote data fast to use locally. Modern data architectures require data placement to be as local as possible to match the user or application’s latency and performance requirements. Hammerspace’s Parallel Global File System orchestrates data automatically and by policy in advance to make data present locally without wasting time waiting for data placement. And data placement occurs fast! Using dual, 100Gb/E networks, Hammerspace can intelligently orchestrate data at 22.5GB/second to where it is needed. This performance level enables workflow automation to orchestrate data in the background on a file-granular basis directly, by policy, making it possible to start working with the data as soon as the first file is transferred and without needing to wait for the entire data set to be moved locally. Unstructured data workloads in the cloud can take full advantage of as many compute cores as allocated and take advantage of as much bandwidth as is needed for the job, even saturating the network within the cloud when desired to connect the compute environment with applications. A recent analysis of EDA workloads in Microsoft Azure showed that Hammerspace scales performance linearly, taking full advantage of the network configuration available in Azure. This high-performance cloud file access is necessary for compute-intensive use cases, including processing genomics data, rendering visual effects, training machine learning models and implementing high-performance computing architectures in the cloud. High-performance across data centers and to the cloud in the Release 5 software include: Backblaze, Zadara, and Wasabi support Continual system-wide optimization to increase scalability, improve back-end performance, and improve resilience in very large, distributed environments New Hammerspace Management GUI, with user-customizable tiles, better administrator experience, and increased observability of activity within shares Increased scale, increasing the number of Hammerspace clusters supported in a single global data environment from 8 to 16 locations 2) High-Performance Across Interconnect within the Data Center: Saturate Ethernet or InfiniBand Networks within the Data Center Data centers need massive performance to ingest data from instruments and large compute clusters. Hammerspace makes it possible to reduce the friction between resources, to get the most out of both your compute and storage environment, reducing the idle time waiting on data to ingest into storage. Hammerspace supports a wide range of high-performance storage platforms that organizations have in place today. The power of the Hammerspace architecture is its ability to saturate even the fastest storage and network infrastructures, orchestrating direct I/O and scaling linearly across otherwise incompatible platforms to maximize aggregate throughput and IOPS. It does this while providing the performance of a parallel file system coupled with the ease of standards-based global NAS connectivity and out-of-band metadata updates. In one recent test with moderately sized server configurations deploying just 16 DSX nodes, the Hammerspace file system took advantage of the full storage performance to hit 1.17 Tbits/second, which was the max throughput the NVMe storage could handle, and with 32kb file sizes and low CPU utilization. The tests demonstrated that the performance would scale linearly to extreme levels if additional storage and networking were added. High-performance across interconnect within the data center enhancements in the Release 5 software include: 20 percent increase in metadata performance to accelerate file creation in primary storage use cases Accelerated collaboration on shared files in high client count environments RDMA support for global data over NFS v4.2, providing high-performance, coupled with the simplicity and open standards of NAS protocols to all data in the global data environment, no matter where it is located 3) High-Performance Server-local IO: Deliver to Applications Near Theoretical I/O Subsystem Maximum Performance of Cloud Instances, VMs, and Bare Metal Servers High-performance use cases, edge environments and DevOps workloads all benefit from leveraging the full performance of the local server. Hammerspace takes full advantage of the underlying infrastructure, delivering 73.12 Gbits/sec performance from a single NVMe-based server, providing nearly the same performance through the file system that would be achieved on the same server hardware with direct-to-kernel access. The Hammerspace Parallel Global File System architecture separates the metadata control plane from the data path and can use embedded parallel file system clients with NFS v4.2 in Linux, resulting in minimal overhead in the data path. For servers running at the Edge, Hammerspace elegantly handles situations where edge or remote sites become disconnected. Since file metadata is global across all sites, local read/write continues until the site reconnects, at which time the metadata synchronizes with the rest of the global data environment. Quotes: David Flynn, founder and CEO of Hammerspace and previous co-founder and CEO of Fusion-IO “Technology typically follows a continuum of incremental advancements over previous generations. But every once in a while, a quantum leap forward is taken with innovation that changes paradigms. This was the case at Fusion-IO when we invented the concept of highly-reliable high-performance SSDs that ultimately became the NVMe technology. Another paradigm shift is upon us to create high-performance global data architectures incorporating instruments and sensors, edge sites, data centers, and diverse cloud regions.” Eyal Waldman, co-founder and previous CEO of Mellanox Technologies, Hammerspace Advisory Board Member “The innovation at Mellanox was focused on increasing data center efficiency by providing the highest throughput and lowest latency possible in the data center and in the cloud to deliver data faster to applications and unlock system performance capability. I see high-performance access to global data as the next step in innovation for high-performance environments. The challenge of fast networks and fast computers has been well solved for years but making remote data available to these environments was a poorly solved problem until Hammerspace came into the market. Hammerspace makes it possible to take cloud and data utilization to the next level of decentralization, where data resides.” Trond Myklebust, Maintainer for the Linux Kernel NFS Client and Chief Technology Officer of Hammerspace “Hammerspace helped drive the IETF process and wrote enterprise quality code based on the standard, making NFS4.2 enterprise-grade parallel performance NAS a reality.” Jeremy Smith, CTO of Jellyfish Pictures "We wanted to see if the technology really stood up to all the hype about RDMA to NFS4.2 performance. The interconnectivity that RoCE/RDMA provides is really outstanding. When looking to get the maximum amount of performance for our clients, enabling this was an obvious choice.” Mark Nossokoff, Research Director at Hyperion Research “Data being consumed by both traditional HPC modeling and simulation workloads and modern AI and HPDA workloads is being generated, stored, and shared between a disparate range of resources, such as the edge, HPC data centers, and the cloud. Current HPC architectures are struggling to keep up with the challenges presented by such a distributed data environment. By addressing the key areas of collaboration at scale while supporting system performance capabilities and minimizing potential costly data movement in HPC cloud environments, Hammerspace aims to deliver a key missing ingredient that many HPC users and system architects are looking for.” About Hammerspace Hammerspace delivers a Global Data Environment that spans across on-prem data centers and public cloud infrastructure enabling the decentralized cloud. With origins in Linux, NFS, open standards, flash and deep file system and data management technology leadership, Hammerspace delivers the world’s first and only solution to connect global users with their data and applications, on any existing data center infrastructure or public cloud services including AWS, Google Cloud, Microsoft Azure and Seagate Lyve Cloud.

Read More


Amazon Unveils Additional Analytics and Data to Empower Seller Success

Amazon | September 19, 2022

Today at Accelerate, Amazon’s annual seller conference, Amazon (NASDAQ: AMZN) announced new features to Manage Your Experiments, a tool that helps sellers optimize content on product detail pages to drive higher rates of conversion, increasing their sales by up to 25%. Amazon also enhanced the Product Opportunity Explorer and Search Analytics Dashboard with new capabilities that help brands analyze marketing campaigns and identify areas to acquire new customers and drive repeat purchases. This new set of industry-leading tools makes it easier for sellers to tap into customer insights and analytics data to launch new products and increase sales. “We’re focused on supporting sellers as they work to build and grow their business. “The tools we’re announcing today are a direct result of seller feedback and target every step of their Amazon sales funnel, from new customer acquisition to increased lifetime value. We’re committed to continuing to develop tools and features that deliver actionable insights for sellers.” Benjamin Hartman, vice president of Amazon North America Selling Partner Services “We have been working with Amazon since the beginning, leveraging data to build our business into one of the largest jewelry sellers on Amazon,” said Tal Masica, founder of PAVOI Jewelry. “Thanks to enhancements to the Search Analytics Dashboard and Product Opportunity Explorer, we now have the ability to analyze search trends at a granular level, giving us actionable insights to improve both trend forecasting and design for future collections – so we can continue delivering quality sustainable jewelry that our customers love to wear every day.” Amazon offers a range of industry-leading tools that empower sellers to optimize their listings, better understand customers, differentiate their brands, and grow their business. The following new tools were announced at Accelerate 2022: Manage Your Experiments is designed to increase the quality of product detail pages and drive higher conversion. With Manage Your Experiments, brands are able to run A/B tests on their titles, main images, and A+ content to see what performs best. Now, brands can also A/B test bullet points and descriptions, and review machine learning-based recommendations for product images and titles to drive better conversion. Additionally, brands can now opt-in to auto-publish winning experiments to the product detail page, automating their A/B tests. Sellers benefit from traffic from hundreds of millions of Amazon customers, and the new Manage Your Experiments features make it easier to test more content, faster. Search Analytics Dashboard has expanded since its launch in early 2022 to offer a new insights dashboard that provides sellers with anonymized data to better understand customers’ interests and shopping habits. For the first time, brands can download Search Query and Catalog Performance data and new ASIN-level details. This new capability enables brands to easily assess marketing campaigns to identify areas to drive repeat purchases and acquire new customers—either directly from within Amazon’s tools or by combining Amazon data with the seller’s own business data. The enhanced Search Analytics Dashboard is launching worldwide in September. Product Opportunity Explorer builds on its successful beta introduction in 2021, continuing to offer rich, accurate data that helps sellers understand, gauge, and evaluate product opportunities in the Amazon store. Sellers can assess the likelihood of a new product gaining traction with customers and forecast sales potential. For the first time, Amazon has now introduced an enhanced Product Opportunity Explorer with a new feature, Customer Reviews Insight. This feature helps sellers work backward from the customer, using customer feedback from product review insights and product star ratings, to help brands determine what features they should build and prioritize as they launch new products or modify existing ones. Marketplace Product Guidance, initially announced in 2021, has been enhanced to provide Selection Recommendations—products in high demand—for U.S. sellers looking to expand to France, Italy, and Spain. Selection Recommendations give sellers insight as to products not currently offered that fit a seller’s portfolio, surfacing new growth opportunities. The tool takes the guesswork out of which products should be considered in those stores, based on customer demand. These recommendations are personalized and ranked based on their opportunity score as calculated by machine learning models that are designed to predict the best opportunities for new selection. Every year, Amazon invests billions of dollars to improve the infrastructure, tools, services, fulfillment solutions, and resources dedicated to helping sellers succeed. Sellers are responsible for more than half of Amazon’s physical product sales; sellers in our store employed and provided jobs for more than 1.5 million people in the United States. About Amazon Amazon is guided by four principles: customer obsession rather than competitor focus, passion for invention, commitment to operational excellence, and long-term thinking. Amazon strives to be Earth’s Most Customer-Centric Company, Earth’s Best Employer, and Earth’s Safest Place to Work. Customer reviews, 1-Click shopping, personalized recommendations, Prime, Fulfillment by Amazon, AWS, Kindle Direct Publishing, Kindle, Career Choice, Fire tablets, Fire TV, Amazon Echo, Alexa, Just Walk Out technology, Amazon Studios, and The Climate Pledge are some of the things pioneered by Amazon.

Read More