1 of 14

Milvus Project: 2021 Update and Request for Graduation

Jun Gu, Milvus project evangelist

2 of 14

Milvus Vector Database

https://milvus.io

  • The embedding serving infrastructure for production AI applications
  • Common application scenarios:
    • Computer vision: Content based image retrieval system, etc
    • NLP: Chat bot, document semantic search, etc
    • Personalized recommender system
    • Computational chemistry: molecular analysis
    • etc

3 of 14

The graduation requirements

  • Have a healthy number of code contributions coming from at least five organizations.
  • Have reached a minimum of 1000 stars on GitHub.
  • Have achieved and maintained a Core Infrastructure Initiative Best Practices Gold Badge.
  • Have demonstrated a substantial ongoing flow of commits and merged contributions for the past 12 months*.
  • Have completed at least one collaboration with another LF AI & Data hosted project.
  • Have a technical lead appointed for the representation of the project on the LF AI & Data Technical Advisory Council.

4 of 14

Milvus community (till May 12th, 2021)

  • Contributors: 137
    • Zilliz
    • Beike
    • iQiyi
    • Yandex
    • Montage Tech
    • Arkie
    • ...
  • GitHub stars: 6.1K
  • GitHub forks: 887
  • Docker hub downloads: 320K
  • Known community users: 2.3K
    • Slack: 700+
    • Wechat group: 1,700+

5 of 14

Best Practices Badge

https://bestpractices.coreinfrastructure.org/en/projects/3563

​

6 of 14

Commit History

7 of 14

Governance and Procedures

  • Contributors Guide

https://github.com/milvus-io/milvus/blob/master/CONTRIBUTING.md

​

  • Governance Information

https://github.com/milvus-io/milvus/blob/master/COMMUNITY.md

​

8 of 14

Milvus Project Leader

Xiaofan, Luan

7 years working experience focused on open source database and big data system design. After graduation from Cornell university with a master degree, Xiaofan works for Oracle, Hedvig and Alibaba cloud. Xiaofan has built several distributed storage and database systems in public cloud.

Xiaofan is currently working at Zilliz as the head of engineering, in charge of R&D of Milvus open source project and the Zilliz cloud platform.

9 of 14

Collaboration with LF AI & Data projects and members

  • Adopted ONNX (graduated project) to manage deep learning models in the Milvus bootcamp applications
  • Working with GSI Technology (general member) to support GSI APU processor in Milvus
  • Working with Baidu (premier member) in multiple Paddlepaddle and EDL cases
  • Initial discussion with Vulkan Kompute (sandbox project) about the integration

10 of 14

2021 Roadmap - Completed

  • Adopted most popular ANNS algorithm libraries
  • Supported near real time data injection
  • Implemented the 1st sharding proxy
  • Provided SDK for the most popular programming language
  • Milvus 1.0 - The first long-term support release

​

11 of 14

2021 Roadmap

​

  • 6.30.2021 Milvus 2.0 - Deliver Distributed Vector database
    • Fully managed Failure Recovery and service discovery
    • Support Vector search with scalar filtering/ point query
    • Support new orm-style apis
    • Tunable consistency
    • Grafana | Prometheus support
  • 10.31.2021 Milvus 2.1 - Deliver complete set of DML functions
    • Support deletes/updates
    • Support string/varbinary type
    • Segment Compaction implementation
    • Memory/Load based Balancer
    • K8s deployment support
    • Apache Spark/Flink Connector

12 of 14

2022 Roadmap

  • 2.28.2022 Milvus 2.2 - Cloud Native Vector database
    • Multi tenant support
    • QueryNode Group isolation
    • BulkLoad insert
    • Traffic Control/ back pressure
    • Distributed Tracing/Log Aggregation
    • Embedded milvus for data scientist
    • Kafka Support

​

13 of 14

Long-term Roadmap

  • Federated Queries by Spark/Presto
  • Time Traverse
  • Traffic Control/ back pressure
  • Hot/Cold Data separation
  • Performance boosting
  • Incremental Backup
  • Cost based Query Optimization
  • On Disk vector Indexing
  • Change Data capture
  • Data Importer/Transformer

14 of 14

Executive Summary

  • The Milvus project meets all graduation criteria for LF AI & Data graduating projects
  • We have been in incubation for 15 months and grew the project to become a strong player in the open source AI ecosystem
  • The Milvus project has no requirements for any spending at this time
  • We look forward to your positive vote to graduate the project

​