diff --git a/docs/community_articles.md b/docs/community_articles.md index 41ee94ef..6a85f06d 100644 --- a/docs/community_articles.md +++ b/docs/community_articles.md @@ -1,4 +1,5 @@ --- +title: Community Projects hide: - navigation --- diff --git a/docs/overrides/home.html b/docs/overrides/home.html index c6e705bc..aa65a4ee 100644 --- a/docs/overrides/home.html +++ b/docs/overrides/home.html @@ -453,9 +453,6 @@

Generative AI at the Edge

Walkthrough - - DLI Lab -
diff --git a/docs/overrides/main.html b/docs/overrides/main.html index b1c131ac..7f25a67f 100644 --- a/docs/overrides/main.html +++ b/docs/overrides/main.html @@ -8,7 +8,7 @@ .md-announce a:focus { color: hsl(82, 100%, 72%); text-decoration: underline; } .md-announce a:hover { color: hsl(82, 100%, 72%); text-decoration: underline;} -
Check out the new tutorials on Small Language Models and Mini VLMs on Orin Nano!
+
Attend the Jetson AI Lab Research Group meeting on Wednesday, 4/3 at 9am PST!
{% endblock %} diff --git a/docs/research.md b/docs/research.md new file mode 100644 index 00000000..a1daa23f --- /dev/null +++ b/docs/research.md @@ -0,0 +1,267 @@ +--- +title: Jetson AI Lab Research Group +hide: + - navigation +--- + +# Jetson AI Lab Research Group + +The Jetson AI Lab Research Group is a global collective for advancing open-source Edge ML, open to anyone in the world to join and collaborate with others from the community and leverage each other's work. Our goal is using advanced AI for good in real-world applications in accessible and responsible ways. By coordinating together as a group, we can keep up with the rapid pace of AI and more quickly arrive at deploying intelligent multimodal agents and autonomous robots into the field. + +There are virtual [meetings](#meeting-schedule) that anyone is welcome to join, offline discussion on the [Jetson Projects](https://forums.developer.nvidia.com/c/agx-autonomous-machines/jetson-embedded-systems/jetson-projects/78){:target="_blank"} forum, and guidelines for upstreaming open-source [contributions](#contribution-guidelines). + +!!! abstract "Next Meeting" + + The first team meeting is on Wednesday, April 3rd at 9am PST - see the [calendar invite](#meeting-schedule) below or click [here](https://teams.microsoft.com/l/meetup-join/19%3ameeting_YzczZmJmZmItYzRiNi00ZWU0LWFmY2MtNTc0ZmM5NzA5NDVm%40thread.v2/0?context=%7b%22Tid%22%3a%2243083d15-7273-40c1-b7db-39efd9ccc17a%22%2c%22Oid%22%3a%221f165bb6-326c-4610-b292-af9159272b08%22%7d){:target="_blank"} to attend! + +## Key Members + +Below are shown some of the sustaining members of the group who have been working on generative AI in edge computing: + + + +
+
+
+ + + + + Dustin Franklin, NVIDIA
+ + Principal Engineer | Pittsburgh, PA
+ (jetson-inference, jetson-containers) +
+
+
+
+
+ + + + Nurgaliyev Shakhizat
+ + Institute of Smart Systems and AI | Kazakhstan
+ (Assistive Devices, Vision2Audio, HPC Clusters) +
+
+
+
+
+ + + + Kris Kersey, Kersey Fabs
+ + Embedded Software Engineer | Atlanta, GA
+ (Project OASIS, AR/VR, 3D Fabrication) +
+
+
+
+
+ + + + Johnny Núñez Cano
+ + PhD Researcher in CV/AI | Catalonia, Spain
+ (Recurrent LLMs, Pose & Behavior Analysis) +
+
+
+
+
+
+ + + Doruk Sönmez, Open Zeka
+ + Intelligent Video Analytics Engineer | Turkey
+ (NVIDIA DLI Certified Instructor, IVA, VLM) +
+
+
+
+
+
+ + + + Akash James, Spark Cognition
+ + AI Architect, UC Berkeley Researcher | Oakland
+ (NVIDIA AI Ambassador, Personal Assistants) +
+
+
+
+
+
+ + + Mieszko Syty, MS/1 Design
+ + AI/ML Engineer | Warsaw, Poland
+ (LLM, Home Assistants, DevOps) +
+
+
+
+
+
+ + + Jim Benson, JetsonHacks
+ + DIY Extraordinaire | Los Angeles, CA
+ (AI in Education, RACECAR/J) +
+
+
+
+ +## Topics of Interest + +These are some initial research topics for us to discuss. This list will vary over time as experiments evolve and the SOTA progresses: + + + + + + + + + + + + + + + + + + + + + + + + + + +
• Controller LLMs for pipeline codegen• Fine-tuning LLM/VLM onboard Jetson AGX Orin 64GB
• HomeAssistant.io integration for smart home [1] [2]• Continuous multi-image VLM streaming and change detection
• Recurrent LLM architectures (Mamba, RKVW, ect) [1]• Lightweight low-memory streaming ASR/TTS models
• Diffusion models for image processing and enhancement• Time Series Forecasting with Transformers [1] [2]
• Guidance, grammars, and guardrails for constrained output• Inline LLM function calling / plugins from API definitions
• ML DevOps, edge deployment, and orchestration• Robotics, IoT, and cyberphysical systems integration
+ +New topics can be raised to the group either during the meetings or on the [forums](https://forums.developer.nvidia.com/c/agx-autonomous-machines/jetson-embedded-systems/jetson-projects/78){:target="_blank"} (people are welcome to work on whatever they want of course) + +## Contribution Guidelines + +When experiments are successful, ideally the results will be packaged in such a way that they are easily reusable for others to integrate into their own projects: + +* Open-source libraries & code on GitHub +* Models on [HuggingFace Hub](https://huggingface.co/models){:target="_blank"} +* Containers provided by [jetson-containers](https://github.com/dusty-nv/jetson-containers){:target="_blank"} +* Documentation / tutorials on Jetson AI Lab +* [Hackster.io](https://www.hackster.io/){:target="_blank"} for hardware-centric builds + +Ongoing technical discussions can occur on the [Jetson Projects](https://forums.developer.nvidia.com/c/agx-autonomous-machines/jetson-embedded-systems/jetson-projects/78){:target="_blank"} forum (or GitHub Issues), with status updates given during the meetings. + +## Meeting Schedule + +We'll aim to meet monthly or bi-weekly as a team in virtual meetings that anyone is welcome to join and speak during. We'll discuss the latest updates and experiments that we want to explore. Please remain courteous to others during the calls. We'll stick around after for anyone who has questions or didn't get the chance to be heard. + +* Wednesday, April 3 at 9am PST (4/3/24) + - Microsoft Teams - [Meeting Link](https://teams.microsoft.com/l/meetup-join/19%3ameeting_YzczZmJmZmItYzRiNi00ZWU0LWFmY2MtNTc0ZmM5NzA5NDVm%40thread.v2/0?context=%7b%22Tid%22%3a%2243083d15-7273-40c1-b7db-39efd9ccc17a%22%2c%22Oid%22%3a%221f165bb6-326c-4610-b292-af9159272b08%22%7d){:target="_blank"} + - Meeting ID: `223 573 467 074` + - Passcode: `6ybvCg` + - Outlook Invite: [`Jetson AI Lab Research Group (4324).ics`](research/invites/Jetson AI Lab Research Group (4324).ics){:target="_blank"} + +The meetings will be recorded and posted so that anyone unable to attend live will be able to watch them after. diff --git a/docs/research/images/Akash_James.jpg b/docs/research/images/Akash_James.jpg new file mode 100644 index 00000000..b8ccc947 Binary files /dev/null and b/docs/research/images/Akash_James.jpg differ diff --git a/docs/research/images/Doruk_Sonmez.png b/docs/research/images/Doruk_Sonmez.png new file mode 100644 index 00000000..a1dd08cc Binary files /dev/null and b/docs/research/images/Doruk_Sonmez.png differ diff --git a/docs/research/images/Dustin_Franklin.jpg b/docs/research/images/Dustin_Franklin.jpg new file mode 100644 index 00000000..702e9495 Binary files /dev/null and b/docs/research/images/Dustin_Franklin.jpg differ diff --git a/docs/research/images/Jim_Benson.png b/docs/research/images/Jim_Benson.png new file mode 100644 index 00000000..3054da91 Binary files /dev/null and b/docs/research/images/Jim_Benson.png differ diff --git a/docs/research/images/Johnny_Cano.jpg b/docs/research/images/Johnny_Cano.jpg new file mode 100644 index 00000000..f59569f7 Binary files /dev/null and b/docs/research/images/Johnny_Cano.jpg differ diff --git a/docs/research/images/Kris_Kersey.jpg b/docs/research/images/Kris_Kersey.jpg new file mode 100644 index 00000000..252d28df Binary files /dev/null and b/docs/research/images/Kris_Kersey.jpg differ diff --git a/docs/research/images/Mieszko_Syty.jpg b/docs/research/images/Mieszko_Syty.jpg new file mode 100644 index 00000000..276499eb Binary files /dev/null and b/docs/research/images/Mieszko_Syty.jpg differ diff --git a/docs/research/images/Shakhizat_Nurgaliyev.jpg b/docs/research/images/Shakhizat_Nurgaliyev.jpg new file mode 100644 index 00000000..69640de6 Binary files /dev/null and b/docs/research/images/Shakhizat_Nurgaliyev.jpg differ diff --git a/docs/research/images/github.png b/docs/research/images/github.png new file mode 100644 index 00000000..d6ec7c80 Binary files /dev/null and b/docs/research/images/github.png differ diff --git a/docs/research/images/hackster.png b/docs/research/images/hackster.png new file mode 100644 index 00000000..f0cf8254 Binary files /dev/null and b/docs/research/images/hackster.png differ diff --git a/docs/research/images/linkedin.png b/docs/research/images/linkedin.png new file mode 100644 index 00000000..db3b973f Binary files /dev/null and b/docs/research/images/linkedin.png differ diff --git a/docs/research/images/youtube.png b/docs/research/images/youtube.png new file mode 100644 index 00000000..715ffcd5 Binary files /dev/null and b/docs/research/images/youtube.png differ diff --git a/docs/research/invites/Jetson AI Lab Research Group (4324).ics b/docs/research/invites/Jetson AI Lab Research Group (4324).ics new file mode 100644 index 00000000..1ad96981 --- /dev/null +++ b/docs/research/invites/Jetson AI Lab Research Group (4324).ics @@ -0,0 +1,712 @@ +BEGIN:VCALENDAR +PRODID:-//Microsoft Corporation//Outlook 16.0 MIMEDIR//EN +VERSION:2.0 +METHOD:REQUEST +X-MS-OLK-FORCEINSPECTOROPEN:TRUE +BEGIN:VTIMEZONE +TZID:Eastern Standard Time +BEGIN:STANDARD +DTSTART:16011104T020000 +RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=11 +TZOFFSETFROM:-0400 +TZOFFSETTO:-0500 +END:STANDARD +BEGIN:DAYLIGHT +DTSTART:16010311T020000 +RRULE:FREQ=YEARLY;BYDAY=2SU;BYMONTH=3 +TZOFFSETFROM:-0500 +TZOFFSETTO:-0400 +END:DAYLIGHT +END:VTIMEZONE +BEGIN:VEVENT +CLASS:PUBLIC +CREATED:20240325T144923Z +DESCRIPTION:Join us for the inaugural team meeting of the Jetson AI Lab Res + earch Group ! \n \nOn the + agenda\, we will discuss the latest updates\, GTC recap\, and experiments + that we want to explore.\n \nAnyone is welcome to join and speak. Please r + emain courteous of others during the call\, and stay muted if you aren’t + speaking. We’ll stick around after for anyone who has questions or did + n't get the chance to be heard. This meeting will be recorded and posted + afterwards for anyone unable to attend.\n \n______________________________ + __________________________________________________\nMicrosoft Teams Need h + elp? \nJoin the meeting now + \nMeeting ID: 223 573 467 074 \nPasscode: 6ybvCg + \n________________________________\n\nDial-in by phone \n+1 949-570-1120\, + \,479114164# United States\, Irvine \nFin + d a local number \nPhone conference ID: 479 114 164# \nJoin + on a video conferencing device \nTenant key: teams@vc.nvidia.com \nVideo ID: 117 895 093 3 \nMore info \nFor organizers: Meeting options | Reset dial-in PIN \n_____________________________________________ + ___________________________________\n \n +DTEND;TZID="Eastern Standard Time":20240403T130000 +DTSTAMP:20240325T144923Z +DTSTART;TZID="Eastern Standard Time":20240403T120000 +LAST-MODIFIED:20240325T144923Z +LOCATION:Microsoft Teams Meeting +ORGANIZER;CN="Dustin Franklin":mailto:dustinf@nvidia.com +PRIORITY:5 +SEQUENCE:0 +SUMMARY;LANGUAGE=en-us:Jetson AI Lab Research Group (4/3/24) +TRANSP:OPAQUE +UID:040000008200E00074C5B7101A82E008000000000073DD919F7EDA01000000000000000 + 010000000FD71ED1ACE6CD042864AC0D1532EE5F1 +X-ALT-DESC;FMTTYPE=text/html:< + style>

Join us for the ina + ugural team meeting of the Jetson AI Lab Research Group!&nb + sp\; \;

 \;

On the agenda\, we will discuss the latest updates\, G + TC recap\, and experiments that we want to explore.

 \;

Anyone is welcome to j + oin and speak. Please remain courteous of others during the call\, and sta + y muted if you aren’\;t speaking.  + \; We’\;ll stick around after for anyone who has questions or + didn't get the chance to be heard. \; < + /span>This meeting will be recorded and posted afterwards for anyone unabl + e to attend.

 \;

< + div style='margin-bottom:.25in\;overflow:hidden'>

_______________________________________________ + _________________________________

Mic + rosoft Teams Need help?

Meeting ID: 223 573 467 074

Passcode: 6ybv + Cg


Dial-in by phone

+1 949-570-1120\,\,4 + 79114164# United States\, Irvine

Phone conference ID: 479 114 164#

Join on a v + ideo conferencing device

Tenant key: teams@vc.nvidia.com +

Video ID: 117 895 093 3< + span style='font-family:"Segoe UI"\,sans-serif\;mso-fareast-font-family:"T + imes New Roman"\;color:#242424'>

For organizers: Meeting options | Reset dial-in PIN +

____________________________________ + ____________________________________________

 \;

+X-MICROSOFT-CDO-BUSYSTATUS:BUSY +X-MICROSOFT-CDO-IMPORTANCE:1 +X-MICROSOFT-DISALLOW-COUNTER:FALSE +X-MS-OLK-AUTOFILLLOCATION:FALSE +X-MS-OLK-CONFTYPE:0 +BEGIN:VALARM +TRIGGER:-PT15M +ACTION:DISPLAY +DESCRIPTION:Reminder +END:VALARM +END:VEVENT +END:VCALENDAR diff --git a/mkdocs.yml b/mkdocs.yml index 45fe0d79..38dd4203 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -112,8 +112,9 @@ nav: - 🔖 SSD + Docker: tips_ssd-docker.md - 🔖 Memory optimization: tips_ram-optimization.md - Benchmarks: benchmarks.md - - Community Projects: community_articles.md - - Try: try.md + - Projects: community_articles.md + - Research Group: research.md + #- Try: try.md extra: generator: false