Big Data
Airbyte | January 30, 2024
Airbyte, creators of the leading open-source data movement infrastructure, today announced a series of accomplishments and awards reinforcing its standing as the largest and fastest-growing data movement community. With a focus on innovation, community engagement, and performance enhancement, Airbyte continues to revolutionize the way data is handled and processed across industries.
“Airbyte proudly stands as the front-runner in the data movement landscape with the largest community of more than 5,000 daily users and over 125,000 deployments, with monthly data synchronizations of over 2 petabytes,” said Michel Tricot, co-founder and CEO, Airbyte. “This unparalleled growth is a testament to Airbyte's widespread adoption by users and the trust placed in its capabilities.”
The Airbyte community has more than 800 code contributors and 12,000 stars on GitHub. Recently, the company held its second annual virtual conference called move(data), which attracted over 5,000 attendees.
Airbyte was named an InfoWorld Technology of the Year Award finalist: Data Management – Integration (in October) for cutting-edge products that are changing how IT organizations work and how companies do business. And, at the start of this year, was named to the Built In 2024 Best Places To Work Award in San Francisco – Best Startups to Work For, recognizing the company's commitment to fostering a positive work environment, remote and flexible work opportunities, and programs for diversity, equity, and inclusion. Today, the company received the BigDATAwire Readers/Editors Choice Award – Big Data and AI Startup, which recognizes companies and products that have made a difference.
Other key milestones in 2023 include the following.
Availability of more than 350 data connectors, making Airbyte the platform with the most connectors in the industry. The company aims to increase that to 500 high-quality connectors supported by the end of this year.
More than 2,000 custom connectors were created with the Airbyte No-Code Connector Builder, which enables data connectors to be made in minutes.
Significant performance improvement with database replication speed increased by 10 times to support larger datasets.
Added support for five vector databases, in addition to unstructured data sources, as the first company to build a bridge between data movement platforms and artificial intelligence (AI).
Looking ahead, Airbyte will introduce data lakehouse destinations, as well as a new Publish feature to push data to API destinations.
About Airbyte
Airbyte is the open-source data movement infrastructure leader running in the safety of your cloud and syncing data from applications, APIs, and databases to data warehouses, lakes, and other destinations. Airbyte offers four products: Airbyte Open Source, Airbyte Self-Managed, Airbyte Cloud, and Powered by Airbyte. Airbyte was co-founded by Michel Tricot (former director of engineering and head of integrations at Liveramp and RideOS) and John Lafleur (serial entrepreneur of dev tools and B2B). The company is headquartered in San Francisco with a distributed team around the world. To learn more, visit airbyte.com.
Read More
Data Architecture
SingleStore | January 25, 2024
SingleStore, the database that allows you to transact, analyze and contextualize data, today announced powerful new capabilities — making it the industry’s only real-time data platform. With its latest release, dubbed SingleStore Pro Max, the company announced ground-breaking features like indexed vector search, an on-demand compute service for GPUs/ CPUs and a new free shared tier, among several other innovative new products. Together, these capabilities shrink development cycles while providing the performance and scale that customers need for building applications.
In an explosive generative AI landscape, companies are looking for a modern data platform that’s ready for enterprise AI use cases — one with best-available tooling to accelerate development, simultaneously allowing them to marry structured or semi-structured data residing in enterprise systems with unstructured data lying in data lakes.
“We believe that a data platform should both create new revenue streams while also decreasing technological costs and complexity for customers. And this can only happen with simplicity at the core,” said Raj Verma, CEO, SingleStore. “This isn’t just a product update, it’s a quantum leap… SingleStore is offering truly transformative capabilities in a single platform for customers to build all kinds of real-time applications, AI or otherwise.”
“At Adobe, we aim to change the world through digital experiences,” said Matt Newman, Principal Data Architect, Adobe. “SingleStore’s latest release is exciting as it pushes what is possible when it comes to database technology, real-time analytics and building modern applications that support AI workloads. We’re looking forward to these new features as more and more of our customers are seeking ways to take full advantage of generative Al capabilities.”
Key new features launched include:
Indexed vector search. SingleStore has announced support for vector search using Approximate Nearest Neighbor (ANN) vector indexing algorithms, leading to 800-1,000x faster vector search performance than precise methods (KNN). With both full-text and indexed vector search capabilities, SingleStore offers developers true hybrid search that takes advantage of the full power of SQL for queries, joins, filters and aggregations. These capabilities firmly place SingleStore above vector-only databases that require niche query languages and are not designed to meet enterprise security and resiliency needs.
Free shared tier. SingleStore has announced a new cloud-based Free Shared Tier that’s designed for startups and developers to quickly bring their ideas to life — without the need to commit to a paid plan.
On-demand compute service for GPUs and CPUs. SingleStore announces a compute service that works alongside SingleStore’s native Notebooks to let developers spin up GPUs and CPUs to run database-adjacent workloads including data preparation, ETL, third-party native application frameworks, etc. This capability brings compute to algorithms, rather than the other way around, enabling developers to build highly performant AI applications safely and securely using SingleStore — without unnecessary data movement.
New CDC capabilities for data ingest and egress. To ease the burden and costs of moving data in and out of SingleStore, SingleStore is adding native capabilities for real-time Change Data Capture (CDC) in for MongoDB®, MySQL and ingestion from Apache Iceberg without requiring other third party CDC tools. SingleStore will also support CDC out capabilities that ease migrations and enable the use of SingleStore as a source for other applications and databases like data warehouses and lakehouses.
SingleStore Kai™. Now generally available, and ready for both analytical and transactional processing for apps originally built on MongoDB. Announced in public preview in early 2023, SingleStore Kai is an API to deliver over 100x faster analytics on MongoDB® with no query changes or data transformations required. Today, SingleStore Kai supports BSON data format natively, has improved transactional performance, increased performance for arrays and offers industry-leading compatibility with MongoDB query language.
Projections: To further advance as the world’s fastest HTAP database, SingleStore has added Projections. Projections allow developers to greatly speed up range filters and group by operations by introducing secondary sort and shard keys. Query performance improvements range from 2-3x or more, depending on the size of the table.
With this latest release, SingleStore becomes the industry’s first and only real-time data platform designed for all applications, analytics and AI. SingleStore supports high-throughput ingest performance, ACID transactions and low-latency analytics; and structured, semi-structured (JSON, BSON, text) and unstructured data (vector embeddings of audio, video, images, PDFs, etc.). Finally, SingleStore’s data platform is designed not just with developers in mind, but also ML engineers, data engineers and data scientists.
“Our new features and capabilities advance SingleStore’s mission of offering a real-time data platform for the next wave of gen AI and data applications,” said Nadeem Asghar, SVP, Product Management + Strategy at SingleStore. “New features, including vector search, Projections, Apache Iceberg, Scheduled Notebooks, autoscaling, GPU compute services, SingleStore Kai™, and the Free Shared Tier allow startups — as well as global enterprises — to quickly build and scale enterprise-grade real-time AI applications. We make data integration with third-party databases easy with both CDC in and CDC out support.”
"Although generative AI, LLM, and vector search capabilities are early stage, they promise to deliver a richer data experience with translytical architecture," states the 2023 report, “Translytical Architecture 2.0 Evolves To Support Distributed, Multimodel, And AI Capabilities,” authored by Noel Yuhanna, Vice President and Principal Analyst at Forrester Research. "Generative AI and LLM can help democratize data through natural language query (NLQ), offering a ChatGPT-like interface. Also, vector storage and index can be leveraged to perform similarity searches to support data intelligence."
SingleStore has been on a fast track leading innovation around generative AI. The company’s product evolution has been accompanied by high-momentum growth in customers and surpassing $100M in ARR late last year. SingleStore also recently ranked #2 in the emerging category of vector databases, and was recognized by TrustRadius as a top vector database in 2023. Finally, SingleStore was a winner of InfoWorld’s Technology of the year in the database category. To learn more about SingleStore visit here.
About SingleStore
SingleStore empowers the world’s leading organizations to build and scale modern applications using the only database that allows you to transact, analyze and contextualize data in real time. With streaming data ingestion, support for both transactions and analytics, horizontal scalability and hybrid vector search capabilities, SingleStore helps deliver 10-100x better performance at 1/3 the costs compared to legacy architectures. Hundreds of customers worldwide — including Fortune 500 companies and global data leaders — use SingleStore to power real-time applications and analytics. Learn more at singlestore.com. Follow us @SingleStoreDB on Twitter or visit www.singlestore.com.
Read More
Big Data Management
Radiant Logic | January 11, 2024
Radiant Logic, the Identity Data Fabric company, today unveiled RadiantOne AI, its data lake powered Artificial Intelligence engine, and AIDA, its Generative AI Data Assistant. RadiantOne AI is designed to complement your existing tech stack and governance products by correlating data across multiple sources and providing contextual information to drive better decision making. The result is a radical reduction in the time and resources needed to gather the data required to effectively meet audit demands—meaning fewer security gaps and increased compliance with organizational policies.
The first capability to be unveiled on RadiantOne AI is a truly automated user access review (UAR) process, expertly guided by AIDA. Many business leaders are familiar with the tedious UAR process – it’s crucial for demonstrating compliance and improving organizational security posture. But laborious processes can often end in a “bulk approval” to save time and check an audit box instead of accurately reviewing access rights to ensure the right business outcomes. RadiantOne’s AI-driven approach will be a paradigm shift in the way people work, forever transforming and streamlining the usually time-consuming UAR process down to days and minutes instead of months.
“Historically, user access reviews are a highly manual process–a ‘necessary evil’ within security practices. This approach not only creates fatigue for the team but also introduces a considerable amount of risk,” says Dr. John Pritchard, Chief Product Officer at Radiant Logic. “While this may work in the short term to satisfy auditor requirements, the company’s assets are never truly protected. There is also still the risk that something may be overlooked, or someone within the business has retained access to something they shouldn’t. With RadiantOne AI and AIDA, existing IAM and IGA processes can be automated and simplified for overworked teams trying to comb through mountains of user access data to make the right decisions to protect their organizations.”
With RadiantOne AI, conducting a user access review becomes as easy as following AIDA’s guidance. Using the power of large language models to drive advanced data correlation, contextualization and analysis, combined with an intuitive data visualization dashboard, AIDA will reinvent the user access review ritual. Based on an organization's proprietary data, the fully guided UAR experience will allow reviewers to interact and pose questions to AIDA using natural human language, like “where does this access come from?” or “show me who else has these access rights?” AIDA will highlight any potential user access risks, offer expert insight, and suggest remediations or access modifications based on an organization’s policies. Any changes, such as low risk bulk access approvals or revoking atypical access rights, are completed via a click of a button, so there’s less training required to complete the reviews and less risk of human error during the process.
RadiantOne AI’s AIDA-guided user access review capability works to provide enterprise organizations with:
Automated workflows: Leverage vast data sets and contextual insights to make intelligent and confident decisions about access rights.
Simplified compliance: Easily detect over-privileged accounts or atypical access rights with intuitive data visualization techniques.
Greater visibility into user actions: Get beyond roles quickly to see who has access to what and how they received that access so insights and remediations are easily actionable.
Click-button remediation: Based on the insights and recommendations from AIDA, reviewers can approve or revoke access or atypical rights individually or take bulk approval/rejection actions with the click of a button.
Data into the hands of business owners: Put relevant, risk-based identity data insights into the hands of business users in the language they understand to make it a breeze to adhere to compliance policies.
“User access reviews with AIDA are just the beginning,” comments Joe Sander, Radiant Logic’s CEO. “Using RadiantOne’s AI engine, we see potential to revolutionize identity data management, governance, risk, compliance and cybersecurity processes by removing complexity as a roadblock. This frees up critical IT and security resources to focus on other business-critical tasks and expands the role of identity to truly be a business enabler.”
RadiantOne AI comes on the heels of the completion of the integration of Brainwave Identity Analytics into the RadiantOne Identity Data Platform. AIDA will initially be available as a complement to the RadiantOne Identity Analytics solution.
About Radiant Logic
Radiant Logic, the identity data experts, helps organizations turn identity data into a strategic asset that drives automated governance, enhanced security, and operational efficiency.
Our RadiantOne Identity Data Platform removes complexity as a roadblock to identity-first strategies by creating an authoritative data source for real-time, context-aware controls. We provide visibility and actionable insights to intelligently detect and remediate risk using AI/ML-powered identity analytics.
With RadiantOne, organizations are able to tap into the wealth of information across the infrastructure, combining context and analytics to deploy governance that works for the most advanced use cases.
Read More