Home > Resources > Whitepapers > Spark Framework for Big Data Analysis on Pseudo Distributed Clusters

Spark Framework for Big Data Analysis on Pseudo Distributed Clusters

Many ”big data” applications have been designed which track statistics about page views in real time, train a machine learning model and automatically detect anomalies. But these applications often require different set of tools like Map-Reduce on Hadoop (MR), Hive, Hadoop Streaming, Weka and Mahout to create models and classifiers. This document talks about streaming data operated on various layers of the Spark stack, such as Spark Streaming, Spark SQL, Spark Machine Learning libraries (MLlib).

Spotlight

NewBrand (acquired by Sprinklr)

NewBrand's customer experience software does more than just social listening, analytics and reputation management. It enables you to quickly identify, prioritize and implement changes that will have the greatest impact on your brand--at the local, regional and global level. The company serves clients across food and beverage, hospitality, retail, and government industries, including Five Guys, Hyatt, Dick's Sporting Goods, Kohl's, Subway and many more.

OTHER WHITEPAPERS

Our Approach to Data Analytics

whitePaper | November 4, 2022

Truveta offers the most complete, timely, and highest-quality data on US health, enabling actionable insights by researchers in life sciences and healthcare.

Architecting for HIPAA Security and Compliance on Amazon Web Services

whitePaper | January 27, 2020

AWS maintains a standards-based risk management program to ensure that the HIPAA-eligible services specifically support the administrative, technical, and physical safeguards required under HIPAA. Using these services to store, process, and transmit PHI allows our customers and AWS to address the HIPAA requirements applicable to the AWS utility-based operating model.

Are Predictive Analytics Truly Predictive?

whitePaper | April 19, 2023

Unplanned downtime is one of the most significant pain points for industrial manufacturers today, costing them an estimated $50 billion each year. The risk is even greater for process manufacturing, where a critical equipment failure could result in the loss of an entire batch, environmental hazards, or safety risks. The adoption of digital technologies, such as the industrial internet of things (IIoT), promises to mitigate these threats by forecasting equipment failures and catching faults before they lead to unscheduled shutdowns. However, in practice, several challenges arise when maintenance personnel and operations leaders work to implement an IIoT solution aimed at eliminating unplanned downtime.

The Intersection of Big Data, Data Governance and MDM

whitePaper | June 29, 2022

For (oh so many) years, we’ve been hearing about the “promise” of big data. Much of the buzz focused on getting people prepared for the onslaught of bigger data sets – and what IT needed to do to help the business make sense of this information.

Customization of Access Control in Microsoft Azure

whitePaper | December 6, 2022

Datamatics provides intelligent solutions for data-driven businesses to increase productivity and enhance the customer experience. With a complete digital approach, Datamatics portfolio spans across Information Technology Services, Business Process Management, Engineering Services and Big Data & Analytics all powered by Artificial Intelligence

Cisco HyperFlex HX Data Platform

whitePaper | September 23, 2022

The Cisco HyperFlex™ HX Data Platform revolutionizes data storage for hyperconverged infrastructure deployments and makes Cisco HyperFlex Systems ready for your enterprise applications—whether they run in virtualized environments such as Microsoft Windows 2016 Hyper-V or VMware vSphere, in containerized applications using Docker and Kubernetes, or in your private or public cloud. Learn about the platform’s architecture and software-defined storage approach and how you can use it to eliminate the storage silos that complicate your data center.

Spark Framework for Big Data Analysis on Pseudo Distributed Clusters