Skip to content

Commit

Permalink
Have a proper tutorial introduction
Browse files Browse the repository at this point in the history
  • Loading branch information
tokk-nv committed Nov 29, 2023
1 parent 20458ec commit 445d23f
Show file tree
Hide file tree
Showing 2 changed files with 64 additions and 3 deletions.
63 changes: 62 additions & 1 deletion docs/tutorial-intro.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,65 @@
# Tutorial - Intro
# Tutorial - Introduction

## Overview

Our tutorials are divided into categories roughly based on model modality, the type of data to be processed or generated.


### Text (LLM)

| | |
| :---------- | :----------------------------------- |
| **[text-generation-webui](./tutorial_text-generation.md)** | Interact with a local AI assistant by running a LLM with oobabooga's text-generaton-webui |
| **[llamaspeak](./tutorial_llamaspeak.md)** | Talk live with Llama using Riva ASR/TTS, and chat about images with Llava! |

### Text + Vision (VLM)

Give your locally running LLM an access to vision!

| | |
| :---------- | :----------------------------------- |
| **[Mini-GPT4](./tutorial_minigpt4.md)** | [Mini-GPT4](https://minigpt-4.github.io/), an open-source model that demonstrate vision-language capabilities.|
| **[LLaVA](./tutorial_llava.md)** | [Large Language and Vision Assistant](https://llava-vl.github.io/), multimodal model that combines a vision encoder and Vicuna LLM for general-purpose visual and language understanding. |

### Image Generation

| | |
| :---------- | :----------------------------------- |
| **[Stable Diffusion](./tutorial_stable-diffusion.md)** | Run AUTOMATIC1111's [`stable-diffusion-webui`](https://github.com/AUTOMATIC1111/stable-diffusion-webui) to generate images from prompts |
| **[Stable Diffusion XL](./tutorial_stable-diffusion-xl.md)** | A newer ensemble pipeline consisting of a base model and refiner that results in significantly enhanced and detailed image generation capabilities.|

### Vision Transformers (ViT)

| | |
| :---------- | :----------------------------------- |
| **[EfficientVIT](./tutorial_efficientvit.md)** | MIT Han Lab's [EfficientViT](https://github.com/mit-han-lab/efficientvit), Multi-Scale Linear Attention for High-Resolution Dense Prediction |
| **[NanoSAM](./tutorial_nanosam.md)** | [NanoSAM](https://github.com/NVIDIA-AI-IOT/nanosam), SAM model variant capable of running in real-time on Jetson |
| **[NanoOWL](./tutorial_nanoowl.md)** | [OWL-ViT](https://huggingface.co/docs/transformers/model_doc/owlvit) optimized to run real-time on Jetson with NVIDIA TensorRT |
| **[SAM](./tutorial_sam.md)** | Meta's [SAM](https://github.com/facebookresearch/segment-anything), Segment Anything model |
| **[TAM](./tutorial_tam.md)** | [TAM](https://github.com/gaomingqi/Track-Anything), Track-Anything model, is an interactive tool for video object tracking and segmentation |

### Vector Database

| | |
| :---------- | :----------------------------------- |
| **[NanoDB](./tutorial_nanodb.md)** | Interactive demo to witness the impact of Vector Database that handles multimodal data |


### Audio

| | |
| :---------- | :----------------------------------- |
| **[AudioCraft](./tutorial_audiocraft.md)** | Meta's [AudioCraft](https://github.com/facebookresearch/audiocraft), to produce high-quality audio and music |
| **[Whisper](./tutorial_whisper.md)** | OpenAI's [Whisper](https://github.com/openai/whisper), pre-trained model for automatic speech recognition (ASR) |

## Tips

| | |
| :---------- | :----------------------------------- |
| Knowledge Distillation | |
| SSD + Docker | |
| Memory optimization | |


## About NVIDIA Jetson

Expand Down
4 changes: 2 additions & 2 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ extra_css:
nav:
- Home: index.md
- Tutorials:
- About NVIDIA Jetson: tutorial-intro.md
- Introduction: tutorial-intro.md
- Text (LLM):
- text-generation-webui: tutorial_text-generation.md
- llamaspeak πŸ†•: tutorial_llamaspeak.md
Expand All @@ -87,7 +87,7 @@ nav:
- Vector Database:
- NanoDB: tutorial_nanodb.md
- Audio:
- Audiocraft πŸ†•: tutorial_audiocraft.md
- AudioCraft πŸ†•: tutorial_audiocraft.md
- Whisper πŸ†•: tutorial_whisper.md
# - Tools:
# - LangChain: tutorial_distillation.md
Expand Down

0 comments on commit 445d23f

Please sign in to comment.