From d79bdd570af8981e134e43554a9ff0f6b0335536 Mon Sep 17 00:00:00 2001 From: Grigori Fursin Date: Fri, 22 Sep 2023 10:35:32 +0200 Subject: [PATCH] updated docs --- README.md | 53 ++++++++++++++++++++++++++++++++-------------------- cm/README.md | 44 +++++++++++++++++++++++++------------------ 2 files changed, 59 insertions(+), 38 deletions(-) diff --git a/README.md b/README.md index da808f75a0..088923ce8b 100755 --- a/README.md +++ b/README.md @@ -5,30 +5,43 @@ [![CM test](https://github.com/mlcommons/ck/actions/workflows/test-cm.yml/badge.svg)](https://github.com/mlcommons/ck/actions/workflows/test-cm.yml) [![CM script automation features test](https://github.com/mlcommons/ck/actions/workflows/test-cm-script-features.yml/badge.svg)](https://github.com/mlcommons/ck/actions/workflows/test-cm-script-features.yml) +### Breaking news -### About - -The [MLCommons Task Force on Automation and Reproducibility](https://github.com/mlcommons/ck/blob/master/docs/taskforce.md), -[cTuning foundation](https://cTuning.org) and [cKnowledge.org](https://cKnowledge.org) -are developing Collective Knowledge v3 - an open-source technology -enabling collaborative, reproducible, automated and unified benchmarking, optimization and comparison of AI, ML and other emerging workloads -across diverse and rapidly evolving models, data sets, software and hardware from different vendors. +Breaking news: our open-source CM automation language and CK playground +enabled the 1st mass-scale community submission of 10000+ MLPerf +inference benchmarking results (representing more than 90% of all v3.1 submissions) +across diverse models, data sets, software and hardware +from different vendors via the [cTuning foundation](https://cTuning.org) - +see the [HPC Wire article](https://www.hpcwire.com/2023/09/13/mlperf-releases-latest-inference-results-and-new-storage-benchmark) +for more details and get in touch via our [public Discord server](https://discord.gg/JjWNWXKxwT) +if you want to automate and optimize your future MLPerf submissions +using our open-source technology! -Collective Knowledge v3 includes: -* [Non-intrusive, technology-agnostic and plugin-based Collective Mind automation language](cm) -* [Collective Knowledge Playground](https://access.cKnowledge.org) -* [Modular Inference Library](https://cknowledge.org/mil) -[The community](https://access.cknowledge.org/playground/?action=challenges) successfully validated CM automation language and CK playground to automate > 90% of all [MLPerf inference v3.1 results](https://mlcommons.org/en/news/mlperf-inference-storage-q323/) -and cross 10000 submissions in one round for the first time (submitted via [cTuning foundation](https://cTuning.org))! -Here is the [list of the new CM/CK capabilities](docs/news-mlperf-v3.1.md) available to everyone -to prepare and automate their future MLPerf submissions. +### About -See related [HPC Wire'23 article](https://www.hpcwire.com/2023/09/13/mlperf-releases-latest-inference-results-and-new-storage-benchmark), -[ACM REP'23 keynote](https://doi.org/10.5281/zenodo.8105339), +Collective Knowledge (CK) is an open-source, non-intrusive and technology-agnostic toolset +to facilitate reproducible research and automate development, benchmarking, optimization, comparison and +deployment of Pareto-efficient AI/ML Systems across diverse and rapidly evolving models, data sets, +software and hardware from different vendors and users. + +Collective Knowledge v3 has been developed from scratch by the [MLCommons Task Force on Automation and Reproducibility](https://github.com/mlcommons/ck/blob/master/docs/taskforce.md), +[cTuning foundation](https://cTuning.org), [cKnowledge.org](https://cKnowledge.org) and [the community](CONTRIBUTING.md) +and includes the following sub-projects: + +* [Collective Mind automation Language (CM)](cm) - a simple automation language + that helps to implement modular, portable and technology-agnostic + benchmarks and applications with a common API that can automatically + plug in diverse and rapidly evolving models, data sets, software and hardware + from different vendors and users. +* [CM scripts](cm-mlops/scripts) - a database of portable, reusable and technology-agnostic automations to modularize benchmarks, software projects and AI/ML Systems. +* [Modular Inference Library (MIL)](https://cknowledge.org/mil) - a universal and modular C++ implementation of MLPerf inference benchmarks. +* [Collective Knowledge Playground](https://access.cKnowledge.org) - an open platform to benchmark and optimize AI and ML Systems via community challenges. + +See related [ACM REP'23 keynote](https://doi.org/10.5281/zenodo.8105339), [ACM Tech Talk](https://learning.acm.org/techtalks/reproducibility) and [MLPerf submitters orientation](https://doi.org/10.5281/zenodo.8144274) -to learn more about our open-source developments and long-term vision. +to learn more about our open-source technology and long-term vision. Join our [public Discord server](https://discord.gg/JjWNWXKxwT) to learn how to run and extend MLPerf benchmarks, participate in future MLPerf submissions, automate reproducibility initiatives at ACM/IEEE/NeurIPS conferences and co-design efficient AI Systems. @@ -51,7 +64,7 @@ automate reproducibility initiatives at ACM/IEEE/NeurIPS conferences and co-desi * [CM installation](https://github.com/mlcommons/ck/blob/master/docs/installation.md) * [All CM tutorials](https://github.com/mlcommons/ck/blob/master/docs/tutorials) -#### Run Python Hello World app +#### Run Python Hello World app on Linux, Windows and MacOS ```bash python3 -m pip install cmind @@ -118,7 +131,7 @@ We invite the community to participate in collaborative benchmarking and optimiz ### Acknowledgments -Collective Knowledge Technology v3 (including Collective Mind automation language and Collective Knowledge playground) +Collective Knowledge Technology v3 (including Collective Mind automation language and Collective Knowledge Playground) was developed from scratch by [Grigori Fursin](https://cKnowledge.org/gfursin) and [Arjun Suresh](https://www.linkedin.com/in/arjunsuresh) in 2022-2023 within the [MLCommons Task Force on Automation and Reproducibility](docs/taskforce.md) diff --git a/cm/README.md b/cm/README.md index 47aa18a7e8..6d8581facf 100644 --- a/cm/README.md +++ b/cm/README.md @@ -6,27 +6,35 @@ [![CM test](https://github.com/mlcommons/ck/actions/workflows/test-cm.yml/badge.svg)](https://github.com/mlcommons/ck/actions/workflows/test-cm.yml) [![CM script automation features test](https://github.com/mlcommons/ck/actions/workflows/test-cm-script-features.yml/badge.svg)](https://github.com/mlcommons/ck/actions/workflows/test-cm-script-features.yml) +### Breaking news + +Our open-source CM automation language and CK playground +enabled the 1st mass-scale community submission of 10000+ MLPerf +inference benchmarking results (representing > 90% of all v3.1 submissions) +across diverse models, data sets, software and hardware +from different vendors via the [cTuning foundation](https://cTuning.org) - +see the [HPC Wire article](https://www.hpcwire.com/2023/09/13/mlperf-releases-latest-inference-results-and-new-storage-benchmark) +for more details and get in touch via our [public Discord server](https://discord.gg/JjWNWXKxwT) +if you want to automate and optimize your future MLPerf submissions +using our open-source technology! + + ### About -The [MLCommons Task Force on Automation and Reproducibility](https://github.com/mlcommons/ck/blob/master/docs/taskforce.md), -[cTuning foundation](https://cTuning.org) and [cKnowledge.org](https://cKnowledge.org) have developed -a non-intrusive, technology-agnostic and plugin-based [Collective Mind automation language (CM)](https://doi.org/10.5281/zenodo.8105339) -to enable collaborative, reproducible, automated and unified benchmarking, optimization and comparison of AI, ML and other emerging workloads -across diverse and rapidly evolving models, data sets, software and hardware from different vendors. - -The CM automation language was successfully validated by the community and [cTuning foundation](https://cTuning.org) -via [public optimization and reproducibility challenges at the CK playground](https://access.cknowledge.org/playground/?action=challenges) -to automate > 90% of all [MLPerf inference v3.1 results](https://mlcommons.org/en/news/mlperf-inference-storage-q323/) -and cross 10000 submissions in one round for the first time - please read this [HPC Wire article](https://www.hpcwire.com/2023/09/13/mlperf-releases-latest-inference-results-and-new-storage-benchmark) -about the cTuning's community submission and check the [list of the new CM/CK capabilities](https://github.com/mlcommons/ck/blob/master/docs/news-mlperf-v3.1.md) -available to everyone to prepare and automate their future MLPerf submissions. - -See related [ACM REP'23 keynote](https://doi.org/10.5281/zenodo.8105339), -[ACM Tech Talk](https://learning.acm.org/techtalks/reproducibility) +Collective Mind automation Language (CM) is a simple automation language that helps to implement modular, portable and technology-agnostic +benchmarks and applications with a common API that can automatically plug in diverse and rapidly evolving models, data sets, software and hardware +from different vendors and users. It is extended via [CM scripts](cm-mlops/scripts) - a database of portable, reusable and technology-agnostic automations +to modularize benchmarks, software projects and AI/ML Systems. + +CM is used by the community to provide a common interface to all shared knowledge, facilitate reproducible research +and automate development, benchmarking, optimization, comparison and deployment of AI/ML Systems. +See related [ACM REP'23 keynote](https://doi.org/10.5281/zenodo.8105339) and [MLPerf submitters orientation](https://doi.org/10.5281/zenodo.8144274) -to learn more about our open-source developments and long-term vision. +to learn more about our open-source technology and long-term vision. -Join our [Discord server](https://discord.gg/JjWNWXKxwT) to ask questions, provide feedback and participate in collaborative developments. +Join our [public Discord server](https://discord.gg/JjWNWXKxwT) to learn how to use CM, +modularize your projects and benchmarks, run and extend MLPerf benchmarks, participate in future MLPerf submissions, +automate reproducibility initiatives at ACM/IEEE/NeurIPS conferences and co-design efficient AI/ML Systems. ### Documentation @@ -37,7 +45,7 @@ Join our [Discord server](https://discord.gg/JjWNWXKxwT) to ask questions, provi * [CM installation](https://github.com/mlcommons/ck/blob/master/docs/installation.md) * [All CM tutorials](https://github.com/mlcommons/ck/blob/master/docs/tutorials) -#### Run Python Hello World app +#### Run Python Hello World app on Linux, Windows and MacOS ```bash python3 -m pip install cmind