https://www.datarobot.com/

DataRobot combines a trusted enterprise AI platform and a trusted AI-native strategic partnership for global enterprises that want to harness the power of AI and their existing teams to succeed in today's Intelligence Revolution.

"We lived and breathed data science,"

Forbes 50 AI companies 2019

Workflow orchestration

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

https://www.h2o.ai/

H2O.ai is the creator of H2O the leading open source machine learning and artificial intelligence platform trusted by data scientists across 14K enterprises

https://thehive.ai/

Hive is a full-stack deep learning company focused on solving visual intelligence problems. Let us help you join the AI Revolution. End-To-End Solutions. Full-Stack Approach.

https://databricks.com/

Unified Data Analytics Platform - One cloud platform for massive scale data engineering and collaborative data science.

https://www.iguazio.com/

The Iguazio Data Science Platform automates your machine learning pipeline, transforming AI projects into real-world business outcomes.

Workflow orchestration

https://airflow.apache.org/

Airflow is a platform created by community to programmatically author, schedule and monitor workflows.

https://polyaxon.com/

A platform for reproducing and managing the whole life cycle of machine learning and deep learning applications.

https://www.dessa.com/

Create more with machine learning. Build, run & monitor 1000s of ML experiments with Foundations

https://petuum.com/

Petuum accelerates and simplifies AI solutions so your enterprise can deploy it easily and maintain it effortlessly.

https://supervise.ly/

First available ecosystem to cover all aspects of training data development. Manage, annotate, validate and experiment with your data without coding.

Workflow orchestration

Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way.

Workflow orchestration

Michelangelo, Uber’s machine learning (ML) platform, supports the training and serving of thousands of models in production across the company. Designed to cover the end-to-end ML workflow, the system currently supports classical machine learning, time series forecasting, and deep learning models that span a myriad of use cases ranging from generating marketplace forecasts, responding to customer support tickets, to calculating accurate estimated times of arrival (ETAs) and powering our One-Click Chat feature using natural language processing (NLP) models on the driver app.

https://mlflow.org/

An open source platform for the machine learning lifecycle

https://www.aible.com/

Create AI that delivers impact, not accuracy, with cost-benefit tradeoffs & operational constraints, in a friendly, intuitive UI designed for real business.

https://dotdata.com/

When AutoML is enhanced with AI-powered feature engineering, the result is dotData. We focus on delivering data science automation for the enterprise. End-to-end data science automation platform accelerates, democratizes, and operationalizes the entire data science process.

Workflow orchestration

https://www.prefect.io/

The Global Leader in Dataflow Automation

Workflow orchestration

https://metaflow.org/

Metaflow makes it quick and easy to build and manage real-life data science projects. Metaflow is built for data scientists, not just for machines.

Workflow orchestration

https://flyte.org/

Lyft’s Cloud Native Machine Learning and Data Processing Platform, Now Open Sourced

We're on a mission to create a world without waste. We push the limits of data science, helping plan, make, and move goods and resources for manufacturers and complex supply chains.

addresses each failure point in the data pipeline from edge device to on-prem and cloud

Forbes 50 AI companies 2019

Workflow orchestration

Kedro is an open source development workflow tool that helps structure reproducible, scaleable, deployable, robust and versioned data pipelines.

Workflow orchestration

https://valohai.com/

The MLOps platform for the whole team. Valohai takes you from POC to production while managing the whole model lifecycle.

Focus on deep learning. Tooling, technology, framework, and cloud-agnostic.

https://tecton.ai/

The Data Platform for Machine Learning. Build a library of great features. Serve them in production. Do it at scale.

From the creators of Michaelangelo

https://datagrok.ai/

Datagrok: Swiss Army Knife for Data. A platform for turning data into actionable insights

can interactively visualize datasets with millions of rows completely in the browse

https://www.figure-eight.com/

Figure Eight combines the best of human and machine intelligence to provide high-quality annotated training data that powers the world's most innovative machine learning and business solutions

https://spark.apache.org/

Apache Spark is a unified analytics engine for big data processing, with built-in modules for streaming, SQL, machine learning and graph processing.

https://scrapinghub.com/

Turn websites into data with the world's leading web scraping services & tools from the creators of Scrapy. Data extraction trusted by industry leaders.

https://www.alteryx.com/

We are a leader in the self-service data analytics movement with a platform that can discover, prep, and analyze all your data, then deploy and share analytics at scale for deeper insights faster than you ever thought possible.

Control Meets Freedom: Unlock the Data Vault and Unleash Your Data Gurus in a Secure Way.

https://www.tamr.com/

Tamr's leading data management system and services work to create a data migration strategy that simplifies your data unification process. Talk with us today.

Forbes 50 AI companies 2019

https://aircloak.com/

Aircloak's unique approach ensures the existing primary database is not modified in any way. Aircloak handles all data types including unstructured text.

https://prometheus.io/

An open-source monitoring system with a dimensional data model, flexible query language, efficient time series database and modern alerting approach.

https://imerit.net/

iMerit specializes in data labeling and annotation for purposes of training models for Machine Learning and Artificial Intelligence.

https://prestodb.io/

Presto is an open source distributed SQL query engine for running interactive analytic queries against data sources of all sizes ranging from gigabytes to petabytes.

https://aws.amazon.com/redshift/

Amazon Redshift is a fast, fully managed, and cost-effective data warehouse that gives you petabyte scale data warehousing and exabyte scale data lake analytics together in one service. Amazon Redshift is up to ten times faster than traditional on-premises data warehouses.

https://druid.apache.org/

Apache Druid is a high performance real-time analytics database

column-oriented database

Hitachi Vantara

https://www.waterlinedata.com/

Waterline's enterprise data catalog enables data professionals to discover, govern, and rationalize an organization's data lake.

https://incorta.com/

Incorta aggregates large complex business data in real time, eliminating the need to reshape it. No Data Warehouse. No Transformations. Real-Time Insight.

https://www.igneous.io/

Igneous Unstructured Data Protection offers the scalability to handle hundreds of file systems, billions of files, and exabytes of enterprise data requiring backup

Unstructured data

https://www.rubrik.com/en

We provide a powerful, policy-driven platform to simplify recovery and unlock insights from data residing in the data center and cloud.

https://www.quobyte.com/

Quobyte is software defined storage that turns commodity servers into a reliable and highly automated data center file system.

https://www.elastifile.com/

Elastifile's cloud-native file storage helps organizations adapt and accelerate their business in the cloud era. Powered by a scalable, enterprise-grade distributed file system with intelligent object tiering, Elastifile augments existing public cloud services with a scalable, POSIX-compliant NAS, facilitating frictionless cloud adoption. With Elastifile, organizations enjoy low-touch file storage services, or deploy and manage cloud-native file storage themselves, eliminating the need for manual storage management and IT forecasting. Elastifile's unique combination of features and flexibility empowers organizations to seamlessly integrate cloud resources, with no application refactoring… thereby modernizing their infrastructure and achieving IT agility and efficiency goals.

https://datera.io/

Get sub-200µS latency & millions of IOPS with 100% software-defined data automation. Save up to 70% on data infrastructure total-cost-of-ownership.

https://www.cohesity.com/

Eliminate mass data fragmentation with Cohesity's modern approach to data management, beginning with backup. Gain instant recovery. Learn more today.

https://www.atscale.com/

Freedom of choice for the enterprise. Break free the complexities and security risks associated with cloud migration and self-service analytics with Intelligent Data Virtualization—no matter where dat.

https://orc.apache.org/

the smallest, fastest columnar storage for Hadoop workloads.

Twitter, Cloudera

Apache Parquet is a columnar storage format available to any project in the Hadoop ecosystem, regardless of the choice of data processing framework, data model or programming language.

https://www.cazena.com/

First Data Lake with a SaaS Experience. Cazena empowers enterprises to collect, store and analyze any data in the cloud, without any DevOps resources or admin time. Cazena's Data Lake as a Service includes everything, and is delivered as secure SaaS, ready to load, store and analyze data with any method: SQL, Spark, R, Python, and many more.

https://www.confluent.io/

Confluent is a fully managed Kafka service and enterprise stream processing platform. Real-time data streaming for AWS, GCP, Azure or serverless. Try free!

founded by the original creators of Apache Kafka

https://yellowbrick.com/

The ultimate solution for your data warehouse. Quick to deploy, easy to expand, and simple to manage. Yellowbrick Data can solve your data problems.

https://www.naveego.com/

A leading provider of cloud-first, distributed data accuracy solutions for seamless, end-to-end data cleansing, Naveego enables organizations to proactively manage, detect and eliminate data accuracy issues across all enterprise data sources in real-time–regardless of structure or schema.

https://gluent.com/

Data virtualization software eliminates data silos. Gluent's transparent data virtualization provides virtual access to all enterprise data, with zero code changes.

https://www.vexata.com/

Vexata is an active data infrastructure company that accelerates database and analytic platforms via groundbreaking storage solutions.

http://storbyte.com/

Storbyte designs and manufactures all-flash & hybrid flash enterprise storage arrays that offer performance, power management, availability, reliability, density, efficiency, flexibility, expandability, and affordability. Storbyte is providing innovative data storage solutions and has not lost sight of what is important to end users: a responsible, cost-correct price point.

https://www.komprise.com/

In 15 minutes, our free data management software trial will show you how you can save 70% on data management costs, on-premises and in the cloud.

https://www.excelero.com/

Local NVMe performance at data center scale through true convergence. Software-defined block storage for Cloud and Enterprise applications at any scale.

https://www.clearskydata.com/

ClearSky Data offers enterprise storage as a hybrid cloud service delivering on-demand primary storage, offsite backup, and DR as a single service.

https://www.pachyderm.com/

Data Lineage with End-to-End Pipelines on Kubernetes, engineered for the enterprise. And… It's open source!

http://www.kimonolabs.com/

Kimono Labs is an online platform that allows its users to convert their websites into APIs.

Atlassian, GitHub

https://git-lfs.github.com/

Git Large File Storage (LFS) replaces large files such as audio samples, videos, datasets, and graphics with text pointers inside Git, while storing the file contents on a remote server like GitHub.com or GitHub Enterprise.

An open source Git extension for versioning large files - 7.9k stars

https://www.alluxio.io/

an open source data orchestration layer that brings data close to compute for big data and AI/ML workloads in the cloud.

https://www.dremio.com/

Get more value from your data, faster. Dremio makes your data engineers more productive, and your data consumers more self-sufficient.

founders of the Apache Arrow and Apache Drill

https://hammerspace.com/

Hammerspace allows data to move freely, like the air you breathe, across clouds and services. Make data accessible exactly where you need it, when you need it – on demand.

Data-as-a-Service

https://www.octopai.com/

An automated, centralized, cross-platform metadata search engine that enables BI groups to quickly and precisely discover and govern shared metadata.

https://www.kyvosinsights.com/

Kyvos accelerates BI on trillions of rows of data on the cloud and on-premise platforms with a semantic layer powered by its next-generation OLAP technology.

It pre-calculates aggregates at multiple levels of dimensional hierarchies to improve query response times as compared to SQL-on-Hadoop platforms

https://www.geminidata.com/

Gemini Data provides Data Availability for AI/ML driven analysis and applications to enable unified enterprise knowledge and access.

https://www.definedcrowd.com/

Leverage machine learning technology and human intelligence to source, structure, and enrich high quality training data in speech, NLP, and computer vision.

Forbes 50 AI companies 2019

https://www.ascend.io/

Experience continuously optimized data pipelines with less code and fewer breakages. Enter the new era of data engineering with Ascend's autonomous dataflow service.

https://dask.org/

Dask natively scales Python. Dask provides advanced parallelism for analytics, enabling performance at scale for the tools you love

https://quiltdata.com/

Quilt is a versioned data portal for AWS

https://imply.io/

Imply delivers real-time analytics powered by Apache Druid. ... Stream or batch load data into Druid for high performance, ad-hoc analytic queries.

https://vaex.io/

Power up your business with our data driven solutions. With our unique, state-of-the-art technology, we provide fast and scalable solutions that will make you more agile, while limiting unnecessary resources.

Parallax Capital Partners

https://erwin.com/

Integrated enterprise architecture, business process and data modeling with data cataloging and data literacy for risk management and digital transformation.

https://www.aparavi.com/

Aparavi's highly scalable data intelligence and automation solutions enable organizations to easily discover, classify, protect, and optimize their data.

https://scale.com

Trusted by world class companies, Scale delivers high quality training data for AI applications such as self-driving cars, mapping, AR/VR, robotics, and more.

LabelImg is a graphical image annotation tool and label object bounding boxes in images

Independent tool

https://segments.ai/

Deep learning-fueled labeling technology with a focus on instance and semantic segmentation.

https://playment.io/

Build high-quality ground truth datasets with ML-assisted tools, sophisticated project management software, expert human workforce, and much more.

Programmatically Building and Managing Training Data

Bigger than a spreadsheet, smaller than a database, datasets are all around us. Use Qri to browse, download, create, fork, & publish datasets across a network of peers.

https://hudi.apache.org/

Apache Hudi ingests & manages storage of large analytical datasets over DFS (hdfs or cloud stores)

https://www.starburstdata.com/

Limitless Queries. Break boundaries and harness the power of the world's fastest SQL query engine.

https://flur.ee/

Welcome to better data management. The Fluree platform organizes blockchain-secured data in a highly-scalable, highly-insightful graph database.

https://dvc.org/

Open-source version control system for Data Science and Machine Learning projects. Git-like experience to organize your data, models, and experiments.

https://www.pilosa.com/

Pilosa is an open source, distributed bitmap index that dramatically accelerates continuous analysis across multiple, massive data sets.

https://prodi.gy/

Prodigy is a scriptable annotation tool so efficient that data scientists can do the annotation themselves, enabling a new level of rapid iteration. ... With Prodigy you can take full advantage of modern machine learning by adopting a more agile approach to data collection.

Python library for efficient multi-threaded data processing, with the support for out-of-memory datasets.

https://www.hycu.com/

Keep hyper-converged infrastructure running with HYCU's powerful, simple backup & recovery and monitoring solutions. Deploy in seconds for superior results.

https://www.liquidata.co/

Liqiudata's mission is to make data move more efficiently. We built Dolt, an an open-source version-controlled SQL database with Git-like semantics.

SQL database: We have a SQL database with Git versioning semantics called Dolt. As far as we know it's the only database with branch and merge functionality.

https://dataturks.com/

ML data annotations made super easy for teams. Just upload data, add your team and build training/evaluation dataset in hours.

https://voxel51.com/scoop/

Quickly Build Insights into Your Video Datasets. Scoop enables you to make sense of your video datasets quickly and effectively. Scoop's faceted search is one-of-a-kind in the industry to let you quickly distill large amounts of video into the answers you need.

Label Studio is a multi-type data labeling and annotation tool with standardized output format

Text annotation for Human. Just create project, upload data and start annotation. You can build dataset in hours.

https://labelbox.com/

A complete solution for your training data problem with fast labeling tools, human workforce, data management, a powerful API and automation features.

https://rapids.ai/

Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

https://github.com/modin-project/modin

Modin uses Ray to provide an effortless way to speed up your pandas notebooks, scripts, and libraries. Unlike other distributed DataFrame libraries, Modin provides seamless integration and compatibility with existing pandas code. Even using the DataFrame constructor is identical.

https://feast.dev/

Feast (Feature Store) is a tool for managing and serving machine learning features. Feast is the bridge between models and data.

https://www.tmlt.io/

Unleashing the power of data with ironclad privacy protection

Differential privacy

https://github.com/uber/aresdb

A GPU-powered real-time analytics storage and query engine.

https://sql-machine-learning.github.io/

Extends SQL to support AI. Extract knowledge from Data. Currently support MySQL, Apache Hive, Alibaba MaxCompute, XGBoost and TensorFlow.