Skip to content

Latest commit

 

History

History
executable file
·
86 lines (61 loc) · 5.65 KB

README.md

File metadata and controls

executable file
·
86 lines (61 loc) · 5.65 KB
Raw

PyPI version Python Version License Downloads

arXiv CM test CM script automation features test MLPerf inference resnet50 CMX: image classification with ONNX

About

Collective Knowledge (CK) in an educational project to help researchers and engineers automate their repetitive, tedious and time-consuming tasks to build, run, benchmark and optimize AI, ML and other applications and systems across diverse and continuously changing models, data, software and hardware.

CK consists of several sub-projects:

  • Collective Mind framework (CM) - a very lightweight Python-based framework with minimal dependencies to help users implement, share and reuse cross-platform automation recipes to build, benchmark and optimize applications on any platform with any software and hardware.

    • CM interface to run MLPerf inference benchmarks

    • CM4MLOPS - a collection of portable, extensible and technology-agnostic automation recipes with a human-friendly interface (aka CM scripts) to unify and automate all the manual steps required to compose, run, benchmark and optimize complex ML/AI applications on diverse platforms with any software and hardware: see online catalog at CK playground, online MLCommons catalog

    • CM4ABTF - a unified CM interface and automation recipes to run automotive benchmark across different models, data sets, software and hardware from different vendors.

  • CMX (the next generation of CM) - we are developing the next generation of CM to make it simpler and more flexible based on user feedback. Please follow this project here.

  • Collective Knowledge Playground - a unified platform to list CM scripts similar to PYPI, aggregate AI/ML Systems benchmarking results in a reproducible format with CM workflows, and organize public optimization challenges and reproducibility initiatives to co-design more efficient and cost-effiective software and hardware for emerging workloads.

  • Artifact Evaluation - automating artifact evaluation and reproducibility initiatives at ML and systems conferences.

License

Apache 2.0

Copyright

  • Copyright (c) 2021-2024 MLCommons
  • Copyright (c) 2014-2021 cTuning foundation

Motivation and long-term vision

You can learn more about the motivation behind these projects from the following articles and presentations:

  • "Enabling more efficient and cost-effective AI/ML systems with Collective Mind, virtualized MLOps, MLPerf, Collective Knowledge Playground and reproducible optimization tournaments": [ ArXiv ]
  • ACM REP'23 keynote about the MLCommons CM automation framework: [ slides ]
  • ACM TechTalk'21 about automating research projects: [ YouTube ] [ slides ]

CM Documentation

Acknowledgments

Collective Knowledge (CK) and Collective Mind (CM) were created by Grigori Fursin, sponsored by cKnowledge.org and cTuning.org, and donated to MLCommons to benefit everyone. Since then, this open-source technology (CM, CM4MLOps, CM4MLPerf, CM4ABTF, CM4Research, etc) is being developed as a community effort thanks to all our volunteers, collaborators and contributors!