diff --git a/docs/community_articles.md b/docs/community_articles.md index 110c25ed..f9777e7d 100644 --- a/docs/community_articles.md +++ b/docs/community_articles.md @@ -7,6 +7,26 @@ hide: Below, you'll find a collection of community articles showcasing the implementation of generative AI on the Jetson platform. +## Dave's Armoury [Bringing GLaDOS to life with Robotics and AI](https://www.youtube.com/watch?v=yNcKTZsHyfA){:target="_blank"} (2-8-2024) + +See how DIY robotics legend Dave Niewinski from [davesarmoury.com](https://www.davesarmoury.com/){:target="_blank"} brings GLaDOS to life using Jetson AGX Orin, running LLMs onboard alongside object + depth tracking, and RIVA ASR/TTS with a custom-trained voice model for speech recognition and synthesis! Using [Unitree Z1](https://shop.unitree.com/){:target="_blank"} arm with 3D printing and [StereoLabs ZED2](https://www.stereolabs.com/products/zed-2){:target="_blank"}. + + + +> :fontawesome-solid-arrow-up-right-from-square:   Find more resources about this project here: [[Forums]](https://forums.developer.nvidia.com/t/robotic-ai-driven-glados-from-portal/282278){:target="_blank"} [[GitHub]](https://github.com/davesarmoury/GLaDOS){:target="_blank"} + +## Hackster [Seeed Studio's Local Voice Chatbot Puts a Speech-Recognizing LLaMa-2 LLM on Your Jetson](https://www.hackster.io/news/seeed-studio-s-local-voice-chatbot-puts-a-speech-recognizing-llama-2-llm-on-your-nvidia-jetson-a82bb3c5ec1d){:target="_blank"} (2-7-2024) + +[Seeed Studio](https://www.seeedstudio.com/tag/nvidia.html){:target="_blank"} has announced the launch of the [**Local Voice Chatbot**](https://wiki.seeedstudio.com/Local_Voice_Chatbot/){:target="_blank"}, an NVIDIA Riva- and LLaMa-2-based large language model (LLM) chatbot with voice recognition capabilities — running entirely locally on NVIDIA Jetson devices, including the company's own [reComputer](https://www.seeedstudio.com/Nvidia-Jetson-c-2016.html){:target="_blank"} range. Follow the step-by-step guide on the [Seeed Studio wiki](https://wiki.seeedstudio.com/Local_Voice_Chatbot/){:target="_blank"}. + + + +## YouTube [GenAI Nerds React - Insider Look at NVIDIA's Newest Generative AI](https://www.youtube.com/watch?v=OjI_p5a85c4){:target="_blank"} (2-6-2024) + +Watch this panel about the latest trends & tech in edge AI, featuring Kerry Shih from [OStream](https://www.ostream.com/){:target="_blank"}, Jim Benson from [JetsonHacks](https://jetsonhacks.com/){:target="_blank"}, and Dusty from NVIDIA. + + + ## NVIDIA [Bringing Generative AI to Life with NVIDIA Jetson](https://youtu.be/6mCFzDatGGc){:target="_blank"} (11-7-2023) Watch this webinar about deploying LLMs, VLMs, ViTs, and vector databases onboard Jetson Orin for building next-generation applications using Generative AI: @@ -21,9 +41,9 @@ JetsonHacks publishes an insightful video that walks developers through the typi ## Hackster [Vision2Audio - Giving the blind an understanding through AI](https://www.hackster.io/shahizat/vision2audio-giving-the-blind-an-understanding-through-ai-33f929){:target="_blank"} (10-15-2023) -Nurgaliyev Shakhizat demonstrates Vision2Audio running on Jetson AGX Orin 64GB Developer Kit to harness the power of LLaVA to help visually impaired people. +Nurgaliyev Shakhizat demonstrates [Vision2Audio](https://www.hackster.io/shahizat/vision2audio-giving-the-blind-an-understanding-through-ai-33f929){:target="_blank"} running on Jetson AGX Orin 64GB Developer Kit to harness the power of LLaVA to help visually impaired people: -![](https://hackster.imgix.net/uploads/attachments/1652753/_qKOfQUA6JP.blob?auto=compress%2Cformat&w=640&h=458&fit=min) +[![](https://hackster.imgix.net/uploads/attachments/1652753/_qKOfQUA6JP.blob?auto=compress%2Cformat&w=640&h=458&fit=min)](https://www.hackster.io/shahizat/vision2audio-giving-the-blind-an-understanding-through-ai-33f929){:target="_blank"} ## NVIDIA [Generative AI Models at the Edge](https://developer.nvidia.com/blog/bringing-generative-ai-to-life-with-jetson/){:target="_blank"} (10-19-2023) @@ -31,7 +51,7 @@ Follow this walkthrough of the Jetson AI Lab tutorials along with coverage of th -> :fontawesome-solid-arrow-up-right-from-square:   [https://developer.nvidia.com/blog/bringing-generative-ai-to-life-with-jetson/](https://developer.nvidia.com/blog/bringing-generative-ai-to-life-with-jetson/){:target="_blank"} +> :fontawesome-solid-arrow-up-right-from-square:   Technical Blog - [https://developer.nvidia.com/blog/bringing-generative-ai-to-life-with-jetson/](https://developer.nvidia.com/blog/bringing-generative-ai-to-life-with-jetson/){:target="_blank"} ## Medium [How to set up your Jetson device for LLM inference and fine-tuning](https://medium.com/@michaelyuan_88928/how-to-set-up-your-jetson-device-for-llm-inference-and-fine-tuning-682e36444d43){:target="_blank"} (10-02-2023) @@ -43,70 +63,52 @@ Michael Yuan's guide demonstrating how to set up the Hackster [Getting Started with AI on Nvidia Jetson AGX Orin Dev Kit](https://www.hackster.io/shahizat/getting-started-with-ai-on-nvidia-jetson-agx-orin-dev-kit-5a55b5){:target="_blank"} (09-16-2023) -Nurgaliyev Shakhizat demonstrates `llamaspeak` on Jetson AGX Orin 64GB Developer Kit. - -![](https://hackster.imgix.net/uploads/attachments/1634209/_YUIZvPChHh.blob?auto=compress%2Cformat&w=640&h=458&fit=min) - -> :fontawesome-solid-arrow-up-right-from-square:   [https://www.hackster.io/shahizat/getting-started-with-ai-on-nvidia-jetson-agx-orin-dev-kit-5a55b5](https://www.hackster.io/shahizat/getting-started-with-ai-on-nvidia-jetson-agx-orin-dev-kit-5a55b5){:target="_blank"} +Nurgaliyev Shakhizat demonstrates `llamaspeak` on Jetson AGX Orin 64GB Developer Kit in this [Hackster post](https://www.hackster.io/shahizat/getting-started-with-ai-on-nvidia-jetson-agx-orin-dev-kit-5a55b5){:target="_blank"}: +[![](https://hackster.imgix.net/uploads/attachments/1634209/_YUIZvPChHh.blob?auto=compress%2Cformat&w=640&h=458&fit=min)](https://www.hackster.io/shahizat/getting-started-with-ai-on-nvidia-jetson-agx-orin-dev-kit-5a55b5){:target="_blank"} ## Hackster [New AI Tool Is Generating a Lot of Buzz](https://www.hackster.io/news/new-ai-tool-is-generating-a-lot-of-buzz-3cc5f23a3598){:target="_blank"} (09-13-2023) -Nick Bild provides an insightful introduction to the Jetson Generative AI Playground. +Nick Bild provides an insightful introduction to the Jetson Generative AI Playground: > :fontawesome-solid-arrow-up-right-from-square:   [https://www.hackster.io/news/new-ai-tool-is-generating-a-lot-of-buzz-3cc5f23a3598](https://www.hackster.io/news/new-ai-tool-is-generating-a-lot-of-buzz-3cc5f23a3598){:target="_blank"} ## JetsonHacks [Use These! Jetson Docker Containers Tutorial](https://jetsonhacks.com/2023/09/04/use-these-jetson-docker-containers-tutorial/){:target="_blank"} (09-04-2023) -JetsonHacks has a in-depth tutorial on how to use `jetson-containers` and even show `text-generation-webui` and `stable-diffusion-webui` containers in action! +JetsonHacks has a [in-depth tutorial](https://jetsonhacks.com/2023/09/04/use-these-jetson-docker-containers-tutorial/){:target="_blank"} on how to use `jetson-containers` and even show `text-generation-webui` and `stable-diffusion-webui` containers in action! -> :fontawesome-solid-arrow-up-right-from-square:   [https://jetsonhacks.com/2023/09/04/use-these-jetson-docker-containers-tutorial/](https://jetsonhacks.com/2023/09/04/use-these-jetson-docker-containers-tutorial/){:target="_blank"} - - ## Hackster [LLaMa 2 LLMs w/ NVIDIA Jetson and textgeneration-web-ui](https://www.hackster.io/pjdecarlo/llama-2-llms-w-nvidia-jetson-and-textgeneration-web-ui-96b070){:target="_blank"} (08-17-2023) -Paul DeCarlo demonstrates 13B and 70B parameter LLama 2 models running locally on Jetson AGX Orin 64GB Developer Kit. +Paul DeCarlo demonstrates 13B and 70B parameter LLama 2 models running locally on Jetson AGX Orin 64GB Developer Kit in this [Hackster post](https://www.hackster.io/pjdecarlo/llama-2-llms-w-nvidia-jetson-and-textgeneration-web-ui-96b070){:target="_blank"}: -![](https://hackster.imgix.net/uploads/attachments/1621246/img_1231_RqW30jjm1q.gif?auto=format%2Ccompress&gifq=35&w=640&h=458&fit=min&fm=mp4) - -> :fontawesome-solid-arrow-up-right-from-square:   [https://www.hackster.io/pjdecarlo/llama-2-llms-w-nvidia-jetson-and-textgeneration-web-ui-96b070](https://www.hackster.io/pjdecarlo/llama-2-llms-w-nvidia-jetson-and-textgeneration-web-ui-96b070){:target="_blank"} +[![](https://hackster.imgix.net/uploads/attachments/1621246/img_1231_RqW30jjm1q.gif?auto=format%2Ccompress&gifq=35&w=640&h=458&fit=min&fm=mp4)](https://www.hackster.io/pjdecarlo/llama-2-llms-w-nvidia-jetson-and-textgeneration-web-ui-96b070){:target="_blank"} ## Hackster [Running a ChatGPT-Like LLM-LLaMA2 on a Nvidia Jetson Cluster](https://www.hackster.io/shahizat/running-a-chatgpt-like-llm-llama2-on-a-nvidia-jetson-cluster-cbc7d4){:target="_blank"} (08-14-2023) -Discover how to run a LLaMA-2 7B model on an NVIDIA Jetson cluster in this insightful tutorial by Nurgaliyev Shakhizat. - -![](https://hackster.imgix.net/uploads/attachments/1621367/_NioNNBZrnm.blob?auto=compress%2Cformat&w=640&h=458&fit=min) +Discover how to run a LLaMA-2 7B model on an [NVIDIA Jetson cluster](https://www.hackster.io/shahizat/running-a-chatgpt-like-llm-llama2-on-a-nvidia-jetson-cluster-cbc7d4){:target="_blank"} in this insightful tutorial by Nurgaliyev Shakhizat: -> :fontawesome-solid-arrow-up-right-from-square:   [https://www.hackster.io/shahizat/running-a-chatgpt-like-llm-llama2-on-a-nvidia-jetson-cluster-cbc7d4](https://www.hackster.io/shahizat/running-a-chatgpt-like-llm-llama2-on-a-nvidia-jetson-cluster-cbc7d4){:target="_blank"} +[![](https://hackster.imgix.net/uploads/attachments/1621367/_NioNNBZrnm.blob?auto=compress%2Cformat&w=640&h=458&fit=min)](https://www.hackster.io/shahizat/running-a-chatgpt-like-llm-llama2-on-a-nvidia-jetson-cluster-cbc7d4){:target="_blank"} ## JetsonHacks [Speech AI on NVIDIA Jetson Tutorial](https://jetsonhacks.com/2023/08/07/speech-ai-on-nvidia-jetson-tutorial/){:target="_blank"} (08-07-2023) -JetsonHacks gives a nice introduction to NVIDIA RIVA SDK and demonstrate its automated speech recognition (ASR) capability on Jetson Orin Nano Developer Kit. +JetsonHacks gives a [nice introduction](https://jetsonhacks.com/2023/08/07/speech-ai-on-nvidia-jetson-tutorial/){:target="_blank"} to NVIDIA RIVA SDK and demonstrate its automated speech recognition (ASR) capability on Jetson Orin Nano Developer Kit. -> :fontawesome-solid-arrow-up-right-from-square:   [https://jetsonhacks.com/2023/08/07/speech-ai-on-nvidia-jetson-tutorial/](https://jetsonhacks.com/2023/08/07/speech-ai-on-nvidia-jetson-tutorial/){:target="_blank"} - ## Hackster [LLM based Multimodal AI w/ Azure Open AI & NVIDIA Jetson](https://www.hackster.io/pjdecarlo/llm-based-multimodal-ai-w-azure-open-ai-nvidia-jetson-135ff2){:target="_blank"} (07-12-2023) -Learn how to harness the power of Multimodal AI by running Microsoft JARVIS on an Jetson AGX Orin 64GB Developer Kit, enabling a wide range of AI tasks with ChatGPT-like capabilities, image generation, and more, in this comprehensive guide by Paul DeCarlo. - -![](https://hackster.imgix.net/uploads/attachments/1610773/2023-07-12_21-40-55_YHOalYh1xU.gif?auto=format%2Ccompress&gifq=35&w=640&h=458&fit=min&fm=mp4) - -> :fontawesome-solid-arrow-up-right-from-square:   [https://www.hackster.io/pjdecarlo/llm-based-multimodal-ai-w-azure-open-ai-nvidia-jetson-135ff2](https://www.hackster.io/pjdecarlo/llm-based-multimodal-ai-w-azure-open-ai-nvidia-jetson-135ff2){:target="_blank"} +Learn how to harness the power of Multimodal AI by running Microsoft JARVIS on an Jetson AGX Orin 64GB Developer Kit, enabling a wide range of AI tasks with ChatGPT-like capabilities, image generation, and more, in this [comprehensive guide](https://www.hackster.io/pjdecarlo/llm-based-multimodal-ai-w-azure-open-ai-nvidia-jetson-135ff2){:target="_blank"} by Paul DeCarlo. +[![](https://hackster.imgix.net/uploads/attachments/1610773/2023-07-12_21-40-55_YHOalYh1xU.gif?auto=format%2Ccompress&gifq=35&w=640&h=458&fit=min&fm=mp4)](https://www.hackster.io/pjdecarlo/llm-based-multimodal-ai-w-azure-open-ai-nvidia-jetson-135ff2){:target="_blank"} ## Hackster [How to Run a ChatGPT-Like LLM on NVIDIA Jetson board](https://www.hackster.io/shahizat/how-to-run-a-chatgpt-like-llm-on-nvidia-jetson-board-41fd79){:target="_blank"} (06-13-2023) -Nurgaliyev Shakhizat explores voice AI assistant on Jetson using FastChat and VoskAPI. - -![](https://hackster.imgix.net/uploads/attachments/1599748/_51IDaX5X5J.blob?auto=compress%2Cformat&w=640&h=458&fit=min) - -> :fontawesome-solid-arrow-up-right-from-square:   [https://www.hackster.io/shahizat/how-to-run-a-chatgpt-like-llm-on-nvidia-jetson-board-41fd79](https://www.hackster.io/shahizat/how-to-run-a-chatgpt-like-llm-on-nvidia-jetson-board-41fd79){:target="_blank"} +Nurgaliyev Shakhizat explores [voice AI assistant](https://www.hackster.io/shahizat/how-to-run-a-chatgpt-like-llm-on-nvidia-jetson-board-41fd79){:target="_blank"} on Jetson using FastChat and VoskAPI. +[![](https://hackster.imgix.net/uploads/attachments/1599748/_51IDaX5X5J.blob?auto=compress%2Cformat&w=640&h=458&fit=min)](https://www.hackster.io/shahizat/how-to-run-a-chatgpt-like-llm-on-nvidia-jetson-board-41fd79){:target="_blank"} diff --git a/docs/css/colors.css b/docs/css/colors.css index 96bb9a6a..a8764b22 100644 --- a/docs/css/colors.css +++ b/docs/css/colors.css @@ -210,6 +210,10 @@ a.nv-buy-link:hover,.load-buy-link:hover { background: #394657; color: #fff; font-size: 0.8em; border-radius: 0.1em; padding-left: 0.3em; padding-right: 0.3em; margin: 0.2em; } +.blobDavesArmoury { + background: #f7b700; color: #fff; font-size: 0.8em; border-radius: 0.1em; padding-left: 0.3em; padding-right: 0.3em; margin: 0.2em; +} + .blobHackster { background: #2e9fe6; color: #fff; font-size: 0.8em; border-radius: 0.1em; padding-left: 0.3em; padding-right: 0.3em; margin: 0.2em; } @@ -222,6 +226,10 @@ a.nv-buy-link:hover,.load-buy-link:hover { background: #76b900; color: #fff; font-size: 0.8em; border-radius: 0.1em; padding-left: 0.3em; padding-right: 0.3em; margin: 0.2em; } +.blobYouTube { + background: #FF0000; color: #fff; font-size: 0.8em; border-radius: 0.1em; padding-left: 0.3em; padding-right: 0.3em; margin: 0.2em; +} + .highlightYellow { background: #ffc105; border-radius: 0.5em; diff --git a/docs/tutorial-intro.md b/docs/tutorial-intro.md index 8e176b18..96b98b91 100644 --- a/docs/tutorial-intro.md +++ b/docs/tutorial-intro.md @@ -19,7 +19,8 @@ Give your locally running LLM an access to vision! | | | | :---------- | :----------------------------------- | | **[Mini-GPT4](./tutorial_minigpt4.md)** | [Mini-GPT4](https://minigpt-4.github.io/), an open-source model that demonstrate vision-language capabilities.| -| **[LLaVA](./tutorial_llava.md)** | [Large Language and Vision Assistant](https://llava-vl.github.io/), multimodal model that combines a vision encoder and Vicuna LLM for general-purpose visual and language understanding. | +| **[LLaVA](./tutorial_llava.md)** | [Large Language and Vision Assistant](https://llava-vl.github.io/), multimodal model that combines a vision encoder and LLM for visual and language understanding. | +| **[Live LLaVA](./tutorial_live-llava.md)** | Run multimodal models interactively on live video streams over a repeating set of prompts. | ### Image Generation @@ -32,11 +33,11 @@ Give your locally running LLM an access to vision! | | | | :---------- | :----------------------------------- | -| **[EfficientVIT](./tutorial_efficientvit.md)** | MIT Han Lab's [EfficientViT](https://github.com/mit-han-lab/efficientvit), Multi-Scale Linear Attention for High-Resolution Dense Prediction | -| **[NanoSAM](./tutorial_nanosam.md)** | [NanoSAM](https://github.com/NVIDIA-AI-IOT/nanosam), SAM model variant capable of running in real-time on Jetson | -| **[NanoOWL](./tutorial_nanoowl.md)** | [OWL-ViT](https://huggingface.co/docs/transformers/model_doc/owlvit) optimized to run real-time on Jetson with NVIDIA TensorRT | -| **[SAM](./tutorial_sam.md)** | Meta's [SAM](https://github.com/facebookresearch/segment-anything), Segment Anything model | -| **[TAM](./tutorial_tam.md)** | [TAM](https://github.com/gaomingqi/Track-Anything), Track-Anything model, is an interactive tool for video object tracking and segmentation | +| **[EfficientVIT](./vit/tutorial_efficientvit.md)** | MIT Han Lab's [EfficientViT](https://github.com/mit-han-lab/efficientvit), Multi-Scale Linear Attention for High-Resolution Dense Prediction | +| **[NanoSAM](./vit/tutorial_nanosam.md)** | [NanoSAM](https://github.com/NVIDIA-AI-IOT/nanosam), SAM model variant capable of running in real-time on Jetson | +| **[NanoOWL](./vit/tutorial_nanoowl.md)** | [OWL-ViT](https://huggingface.co/docs/transformers/model_doc/owlvit) optimized to run real-time on Jetson with NVIDIA TensorRT | +| **[SAM](./vit/tutorial_sam.md)** | Meta's [SAM](https://github.com/facebookresearch/segment-anything), Segment Anything model | +| **[TAM](./vit/tutorial_tam.md)** | [TAM](https://github.com/gaomingqi/Track-Anything), Track-Anything model, is an interactive tool for video object tracking and segmentation | ### Vector Database