How Dutch Railways uses big data to keep schedules on track

techhq.com | January 13, 2020

Transporting one million commuters each day, 92 percent of Dutch Railways Nederlandse Spoorwegen’s (NS) trains arrive on time, making it the third-most punctual rail service in the world, behind Switzerland and Japan. While certain delays are unavoidable results of severe weather or obstructions on the line, for exampleNS owes its industry-leading scheduling to big data analytics, thanks to thousands of sensors on its locomotives. As reported in TNW, NS has been leveraging the power of predictive maintenance since 4G became available in the Netherlands, allowing data to be processed at much higher speeds. Not only is this data based on the health of the vehicles themselves, but it’s also combined with check-in data and status measurement points on tracks. In total, 140 data sources converge to provide an instant snapshot of the state of operation across the company’s entire network.

Spotlight

Today’s public cloud computing infrastructures are not architected to support truly large “Big Data” applications. While it make sense to start up some big data projects in the cloud, this whitepaper tells you when it is time to leave the cloud and come home. DriveScale can help you get out of the cloud; we offer a software-defined Big Data infrastructure that gives you control, predictability, and efficiency without losing the flexibility that cloud solutions provide.

Spotlight

Today’s public cloud computing infrastructures are not architected to support truly large “Big Data” applications. While it make sense to start up some big data projects in the cloud, this whitepaper tells you when it is time to leave the cloud and come home. DriveScale can help you get out of the cloud; we offer a software-defined Big Data infrastructure that gives you control, predictability, and efficiency without losing the flexibility that cloud solutions provide.

Related News

BUSINESS INTELLIGENCE, BIG DATA MANAGEMENT

Hevo Data Partners with Databricks to Bring Effortless and Automated Data Integration to More Data-Driven Businesses

Hevo Data | October 19, 2022

Hevo Data, the leading no-code ETL platform, announced their official partnership with Databricks, the lakehouse company, to simplify and accelerate data integration into the lakehouse for mutual customers. This new partnership allows users to access Hevo Data directly in Databricks and redefines data integration at scale. As data continues to enable operational and strategic business decisions, data lakehouses like Databricks, which combine the best of data warehouses and data lakes, offer businesses a single platform to combine all their data, analytics, and AI operations. Yet, data teams struggle to reliably bring together the data that's siloed in dozens of business softwares, often requiring months of effort in setting up and maintaining ETL pipelines. Hevo Data's intuitive, no-code platform enables data teams to automate data integration at scale with near-zero latency, and transform the raw data into analytics-ready state in a matter of minutes. With this new partnership, Hevo Data will be available as part of Databricks Partner Connect. As a validated partner that's natively integrated, Databricks customers can now easily access and deploy their ETL pipelines on Hevo Data right within Databricks. "We're excited to build on our partnership with Hevo Data and bring another powerful lakehouse integration to our customers. "With Partner Connect, customers can quickly discover and easily leverage Hevo Data directly from Databricks, taking advantage of a seamless connection that's optimized for the lakehouse." Ariel Amster, Director of Strategic Technology Partners at Databricks "With this partnership with Databricks, we're staying true to our goal of simplifying data-driven decisions for businesses," added Manish Jethani, co-founder and CEO of Hevo Data. "We hope that we're able to increase accessibility, reduce time-consuming and unnecessary maintenance, and enable innovation based on data for the growing Databricks customer base." About Hevo Data Hevo Data is an intuitive data pipeline platform that modern data analytics teams across 40+ countries rely on to fuel timely analytics and data-driven decisions. Hevo Data helps them reliably and effortlessly sync data from 150+ SaaS apps and other data sources to any cloud warehouse or data lake and turn it analytics-ready through intuitive models and workflows.

Read More

BUSINESS INTELLIGENCE, BIG DATA MANAGEMENT

Plotly Releases the Next Generation of Its Low-Code Data App Platform

Plotly | October 18, 2022

Plotly has released Dash Enterprise 5.0, the leading low-code platform for building, scaling, and deploying Python data apps. Version 5.0 grants customers a sophisticated, fully scalable architecture and features that greatly improve the user experience of both IT administrators and app authors. The platform is built on Kubernetes architecture and is compatible across Google Cloud Platform, Amazon Web Services, and Microsoft Azure cloud service providers. Dash Enterprise 5.0 equips IT platform administrators with new tools for user management, resource management, and security compliance: Multi-server architecture can run hundreds of apps on the same platform. App services scale redundantly across servers, which can easily be added as usage expands. Scripted installation enables customers to easily install Dash Enterprise within their private cloud. It requires little Kubernetes management experience, addressing common pain points around infrastructure provisioning while keeping customers' information behind their firewall. The identity access integration wizard accommodates 10 identity providers, plus custom connections via SAML, OpenID Connect (OIDC) or LDAP, bypassing hours of configuration in seconds and ensuring single sign-on (SSO) and access management for all apps. Platform observability from cluster to app level empowers administrators to view app logs, delete stale apps, rank when apps were recently viewed, or remotely terminate unauthorized user activity. Dash Enterprise 5.0 grants app authors secure collaboration and powerful, low-code data science workflows: Easily scale apps across multiple nodes in the cluster, enabling apps to handle more users and most apps to run two to eight times faster. Co-owner and group permissions enable app authors to collaborate on, manage, and deploy the same app. A modern UI reduces app deployment time by 50% with a built-in Workspace IDE, drag-and-drop data uploads, one-click app templates, and a simple deploy button. Upload, read, update, and cache datasets up to 25 GB per app in a fast, secure, isolated filesystem attached to the app and Workspace, overcoming storage limitations of deployment images and Git repositories without introducing network costs or delays. Visualize 120 million rows of data in less than four seconds. "The Dash framework is downloaded over 1 million times monthly, and data apps are becoming essential to every company's analytics strategy. Dash Enterprise 5.0 combines industry-standard, cloud-native Kubernetes architecture with cutting-edge product design that removes common IT barriers to bringing data apps into production," said Chris Parmer, Chief Product Officer and Co-Founder at Plotly.

Read More

BUSINESS INTELLIGENCE, BIG DATA MANAGEMENT, DATA SCIENCE

UiPath Partners with Snowflake to Launch Data Integration on the Data Cloud

UiPath | September 08, 2022

UiPath , a leading enterprise automation software company, today announced it has strengthened its partnership with Snowflake, the Data Cloud company, by launching a new bi-directional integration that will extend the value of automation across the enterprise. UiPath and Snowflake are enabling joint customers to design and build workflows based on 360-degree views of trusted and accessible data on Snowflake’s platform. By leveraging the Snowflake Data Cloud, UiPath robots can quickly connect data directly to business processes in the Data Cloud without using complex code, speeding up time to value. Automation is helping organizations around the world become faster and more agile in the face of increased demand and rapidly changing environments. The UiPath end-to-end platform provides robotic process automation (RPA) at its core, removing manual work so users can focus on what matters most. In fact, IDC projects that by 2025, RPA-driven economic benefits from revenues will almost equal economic benefits from cost savings and operational efficiencies.* With this expanded integration, UiPath and Snowflake will continue to enable customers to accelerate digital transformation and simplify the automation of data pipelines for data-driven decision making. “The partnership between UiPath represents customers’ needs for both data and business processes that should be fast, frictionless, and automated. “As demand for automation on data increases, partnerships with organizations, such as UiPath, help Snowflake serve our joint customers globally and support our mission of mobilizing the world’s data.” Tarik Dwiek, Head of Technology Alliances at Snowflake Automate the data pipeline: With the ability to both retrieve and push data in and out of the Snowflake Data Cloud, UiPath can orchestrate and automate data pipelines creating workflows that deliver increased quality, access, analytics, and business insights for data-driven decision making. Broaden workflows with data from multiple sources: By incorporating Snowflake query activities such as ‘native search’, RPA developers can quickly access the data they need from Snowflake to create decision logic for building better workflows and use cases across multiple lines-of-businesses. Enrich their data: Combined with the UiPath ability to bring together both UI and API automation, this allows UiPath customers to quickly unlock and access the data necessary to maximize their automations with the data quality they require.a “We believe that accessing reliable and trusted data from the Snowflake Data Cloud is key to delivering a fully automated enterprise,” said Dhruv Asher, SVP of Business Development and Product Alliances at UiPath. “With this new integration, UiPath and Snowflake collaborate to seamlessly centralize valuable data that is locked away in data silos, documents, legacy systems, and other hard to reach places. This enables our joint customers to find new insights, unlock new opportunities, and drive innovation with more of the data they already have.” As part of today's announcement, UiPath is now recognized as a Snowflake Technology Partner within the Snowflake Partner Program. * IDC Thought Leadership White Paper, sponsored by UiPath, “The Economic Impact of UiPath Robotic Process Automation,” Doc. #US47905721, September 2021 About UiPath UiPath has a vision to deliver the Fully Automated Enterprise™, one where companies use automation to unlock their greatest potential. UiPath offers an end-to-end platform for automation, combining the leading Robotic Process Automation (RPA) solution with a full suite of capabilities that enable every organization to rapidly scale digital business operations.

Read More