CuriGPT: Interactive Humanoid Manipulation by Multimodal Large Language Models

• CuriGPT: Interactive Humanoid Manipulation by Multimodal Large Language Models
  • Installation
    • System requirements
    • Installation
  • Functions
    • Multimodal Voice Q&A

• The Team

• Acknowledge

Installation

System requirements

The package is only tested by the following configuration.

1. Ubuntu 20.04
2. Ros Neotic

Installation

Please refer to the installation manual.

Functions

Multimodal Voice Q&A

For this function, we implemented a Voice Q&A robot based on multimodal large language modelsqwen_vl_chat_v1.

This whole pipeline can be activated by calling

cd ~/CuriGPT/scripts/
python curigpt.py

The Team

CuriGPT is developed and maintained by the CLOVER Lab (Collaborative and Versatile Robots Laboratory), CUHK.

Acknowledge

We would like to acknowledge the following projects:

1. wpr_chatgpt
2. ros-vosk
3. tts-ros1