From 6aace49e405257f8fd9aa49e8ec3808839574df9 Mon Sep 17 00:00:00 2001 From: Brenton Carter <99493283+Inserian@users.noreply.github.com> Date: Sat, 28 Dec 2024 12:46:44 -0900 Subject: [PATCH] Update README.md --- README.md | 260 ++++++++++++++++++++---------------------------------- 1 file changed, 96 insertions(+), 164 deletions(-) diff --git a/README.md b/README.md index f0a5ec3..d5860fe 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,5 @@ ---- - -# Quantum-Enhanced Language Model (QELM) +```markdown +# Quantum-Enhanced Language Model (QELM) (QLM) ![License](https://img.shields.io/github/license/R-D-BioTech-Alaska/QELM) ![Python Version](https://img.shields.io/badge/python-3.7%2B-blue) @@ -9,9 +8,8 @@ ![GitHub Stars](https://img.shields.io/github/stars/R-D-BioTech-Alaska/QELM?style=social) ## Table of Contents - - [Overview](#overview) -- [Comparison](#comparison-with-regular-llms) +- [Comparison with Regular LLMs](#comparison-with-regular-llms) - [Features](#features) - [Installation](#installation) - [Prerequisites](#prerequisites) @@ -21,8 +19,11 @@ - [Usage](#usage) - [Training the Model](#training-the-model) - [Performing Inference](#performing-inference) - - [Graphical User Interface (GUI)](#graphical-user-interface-gui) - - [Viewing Help/Usage Information](#viewing-helpusage-information) + - [Graphical Interfaces](#graphical-interfaces) + - [1. QelmGUI (Training + Inference)](#1-qelmgui-training--inference) + - [2. QELMChatUI (Conversational UI)](#2-qelmchatui-conversational-ui) + - [Legacy Command Line (Older Script)](#legacy-command-line-older-script) + - [Viewing Help/Usage](#viewing-helpusage) - [Project Structure](#project-structure) - [Credits](#credits) - [License](#license) @@ -31,115 +32,65 @@ --- ## Overview - -Welcome to the **Quantum-Enhanced Language Model (QELM)** project; an innovative project that merges the power of quantum computing with natural language processing to create a next-generation language model. QELM creates and reduces LLMs to ultra-compact models using qubits without compromising capabilities, enabling them to run instantly on small devices without data centers. - -Here’s the revised **Comparison with Regular LLMs** section, rewritten to maintain the facts but with improved readability and flow: +The **Quantum-Enhanced Language Model (QELM)** merges quantum computing with natural language processing to produce extremely **compact** yet powerful language models. By encoding token embeddings into **quantum states** and leveraging **entanglement**, QELM drastically reduces storage requirements compared to classical LLMs. This makes QELM an excellent choice for edge devices or memory-limited environments. --- ## Comparison with Regular LLMs +Classical LLMs often reach **6 - 60 GB** (or more) even for modest architectures. In current comparison tests QELM, by contrast, typically yields models around **2 MB** when a classical llm would be **15-20 MB, delivering: +- **8–9x size reduction** +- Similar perplexity/performance (e.g., perplexity ~100) +- Efficient parameter usage through quantum ansätze and ring entanglement -One of the standout features of the Quantum-Enhanced Language Model (QELM) is its compact file size. For instance, a small QELM (qlm) model trained with the following settings resulted in an exceptionally small file size of just 2,165 KB: - -### Key QELM Settings: -- **Vocabulary Size:** 100 -- **Embedding Dimensions:** 128 -- **Attention Heads:** 4 -- **Hidden Dimensions:** 256 -- **Learning Rate:** 0.05 -- **Epochs:** 2 -- **Output File Size:** 2,165 KB - -In contrast, a traditional language model (LLM) trained with similar configurations would typically be far larger (15-20mb's). Here’s a breakdown of how QELM achieves its efficiency and the advantages it offers: - ---- - -### Why QELM Outperforms Regular LLMs: - -1. **Smaller File Size Through Quantum Encoding:** - - Quantum models utilize quantum states to encode information, significantly reducing the need for large, redundant weight matrices common in classical LLMs. - - This results in models that are inherently more compact, without compromising on performance. - -2. **Expected Size of Regular LLMs:** - - A comparable classical LLM would produce a file size between **15 MB and 20 MB**, depending on factors like precision (e.g., FP16 vs. FP32) and storage format. - - This makes QELM approximately **8 to 9 times smaller** than its classical counterparts out of the box. - -3. **Efficient Use of Parameters:** - - Classical LLMs store parameters in a direct 1:1 format, leading to storage inefficiencies. - - In contrast, QELM leverages quantum circuits and entanglement to represent these parameters more efficiently, requiring fewer physical resources. - -4. **Performance Metrics:** - - Despite its smaller size, QELM achieves robust performance, with metrics like perplexity (99.94–100.28) and loss (4.60–4.61) aligning with expectations for models of this scale. - ---- - -### The QELM Advantage: - -- **Compactness:** QELM’s file size makes it an excellent choice for deployment on devices with limited storage, such as edge devices or embedded systems. -- **Resource Efficiency:** By reducing redundancy and optimizing parameter storage, QELM minimizes memory and computational overhead. -- **Scalability Potential:** As model size and complexity grow, QELM’s quantum architecture may offer even more pronounced efficiency gains compared to classical LLMs. - ---- - -### Summary: - -The QELM demonstrates a paradigm shift in model design, showcasing how quantum computing principles can deliver compact and powerful language models. Where a traditional LLM of similar complexity might take up 15–20 MB, QELM achieves the same functionality at just **2 MB**, making it a compelling choice for resource-constrained environments. With further scaling, QELM could redefine the expectations for size and efficiency in NLP models. - -With current settings a 100gb model would scale to 10gb with only an increase in performance. +In short, quantum-based “compression” can significantly reduce overhead without compromising on capabilities. --- ## Features - -- **Quantum Parameter Optimization**: Gradient-based optimization via the Parameter Shift Rule. -- **Advanced Quantum Circuits**: Implements entangling gates with multiple layers. -- **Thread-Safe GUI with Training Feedback**: New GUI-based interface for training, inference, and model management. -- **Synthetic and Real Dataset Support**: Train with synthetic datasets for testing or real-world text data. -- **Resource Monitoring**: Integrated system resource usage monitoring (CPU/GPU). -- **Enhanced Logging**: Threaded logging ensures training progress and errors are visible in real-time. +- **Sophisticated Quantum Circuits** + - **Advanced Ansatz**: RY, RZ, ring entanglement patterns, optional data reuploading + - **Multi-Block Transformers**: Stack attention+FFN blocks for deeper language understanding + - **Parameter Shift Gradient** training for quantum gates +- **GUI Support** + - **QelmGUI**: Train/infer on quantum LLMs with real-time logs, progress bars, resource tracking + - **QELMChatUI**: Chat-like interface for multi-turn conversations, model selection, and conversation saving +- **Multi-Threaded / Multiprocessing** + - Parallel parameter-shift evaluations + - CPU/GPU/both simulation modes +- **Dataset Flexibility** + - Load real text or generate synthetic tokens + - Manage token mappings easily +- **Resource Monitoring** + - CPU usage via `psutil` + - GPU usage (if available) with `nvidia-smi` --- -![QELM Diagram](docs/images/QELM_Diagram.png) - ---- ## Installation ### Prerequisites - -- **Python 3.7 to 3.11** -- **Qiskit** -- **Qiskit Aer** +- **Python 3.7+** (up to 3.11 tested) +- **Qiskit** + **Qiskit Aer** - **TensorFlow** -- **Numpy** -- **Tkinter** -- **psutil** (optional for resource monitoring) - ---- +- **NumPy** +- **Tkinter** (standard in most Python distributions) +- **psutil** *(optional for resource usage)* ### Clone the Repository - ```bash git clone https://github.com/R-D-BioTech-Alaska/QELM.git cd QELM ``` ---- - ### Setup Virtual Environment - ```bash python -m venv qiskit_env -# Activate the virtual environment -source qiskit_env/bin/activate # For Linux/macOS -qiskit_env\Scripts\activate # For Windows +# Activate the env: +source qiskit_env/bin/activate # Linux/macOS +qiskit_env\Scripts\activate # Windows ``` ---- - ### Install Dependencies - ```bash pip install --upgrade pip pip install -r requirements.txt @@ -150,129 +101,110 @@ pip install -r requirements.txt ## Usage ### Training the Model +1. **Prepare your dataset**: real text or synthetic (auto-generated). +2. **Set hyperparameters**: vocabulary size, embed dim, #heads, #blocks, advanced ansatz toggles, etc. +3. **Run training**: + - **GUI**: Launch `QelmGUI.py`, fill in parameters, press **Start Training**. + - **CLI**: Use `Qelm2.py --train --epochs N --lr 0.05` (older approach). -QELM allows you to train using **synthetic** or **real datasets**. Use the command-line interface (CLI) for basic runs or the GUI for an enhanced experience. - -#### CLI Example: +### Performing Inference +- **GUI**: Inference tab allows user to provide a token, set `max_length`, temperature, and generate. +- **CLI**: Use `Qelm2.py --inference --input_id 5 --load_path your_model.qelm`. -```bash -python Qelm2.py --train --epochs 02 --lr 0.05 -``` +--- -#### GUI Model with Threading +## Graphical Interfaces -To run the **QELM GUI**, execute: +### 1. QelmGUI (Training + Inference) +`QelmGUI.py` offers: +- **Dataset Selection** (real .txt or synthetic) +- **Hyperparameter Entry** (embed dim, #heads, #blocks, advanced ansatz, etc.) +- **Live Logs & Progress Bars** (epoch progress, gradient progress) +- **Error & Resource Monitoring** (CPU%, GPU usage if available) +- **Model Save/Load** + **Token Mapping** management +- **Inference** interface (token-based text generation) +**Run**: ```bash python QelmGUI.py ``` +You’ll see a tabbed window for training, inference, and token mapping. Advanced toggles let you experiment with ring entanglement, RZ gates, data reuploading, multi-block architectures, etc. -This launches an intuitive interface for: - -- Selecting datasets -- Training with live progress updates -- Running inference -- Managing token mappings - -**Key GUI Features**: - -- Real-time progress bars for gradient computations and training -- System resource usage (CPU/GPU) display -- Interactive logs for feedback and error monitoring -- Buttons for saving/loading models, stopping training gracefully, or halting immediately. - ---- - -### Performing Inference - -Run predictions from the GUI or CLI. - -#### CLI Inference Example: +### 2. QELMChatUI (Conversational UI) +`QELMChatUI.py` provides a **ChatGPT-like** experience: +- **Multi-session**: Keep track of multiple conversation threads +- **Center Chat Panel**: Type messages, get QELM’s replies +- **Advanced Layout**: Avoids duplication errors from older prototypes +- **Model Loading & Token Mapping**: Quickly switch or update quantum LLMs +- **Save Chat**: Archive entire dialogues to text +**Run**: ```bash -python Qelm2.py --inference --input_id 5 --load_path quantum_llm_model_enhanced.json +python QELMChatUI.py ``` - -#### GUI Inference: - -1. Enter an **input token**. -2. Set parameters like `Max Length` and `Temperature`. -3. Click **Run Inference**. - -The GUI outputs the generated sequence. +Engage in interactive conversation with your quantum model. Great for testing QELM’s dialogue capabilities or showcasing quantum-based reasoning in a chat interface. --- -## Graphical User Interface (GUI) +## Legacy Command Line (Older Script) +We retain the **original** CLI script `Qelm2.py` for those who want a simpler, command-line-driven approach: +- **Training** (`--train`) +- **Inference** (`--inference`) +- Basic model load/save +However, it lacks the robust features of the GUIs. For a more comprehensive experience, use **QelmGUI**. -![QELM](docs/images/Qelm.png) --- -The **QELM GUI** offers an easy-to-use tool for all functionalities, including: - -- **Training**: Monitor training logs, set hyperparameters, and view progress visually. -- **Inference**: Input tokens and generate outputs interactively. -- **Model Management**: Save and load models with token mappings. -- **Resource Monitoring**: View real-time CPU usage and estimate remaining training time. - -**Launching the GUI**: - -```bash -python QelmGUI.py -``` +### Viewing Help/Usage +- **GUI** usage: intuitive once launched; each tab explains itself. +- **CLI** usage: + ```bash + python Qelm2.py --help + ``` --- ## Project Structure - ```plaintext QELM/ -├── Qelm2.py # CLI-based model training and inference -├── QelmGUI.py # GUI model with threading -├── requirements.txt # Python dependencies -├── README.md # Project documentation -├── quantum_llm_model_enhanced.json # Default model file +├── Qelm2.py # Legacy CLI script for training & inference +├── QelmGUI.py # Graphical interface for training & inference +├── QELMChatUI.py # Chat-style interface (like ChatGPT) +├── requirements.txt # Dependencies +├── README.md # This documentation └── docs/ └── images/ ├── QELM_Diagram.png ├── quantum.png - └── Qelm.png + └── Qelm.png ``` --- ## Credits +If you build upon QELM, please acknowledge: -If you use or build upon this project, provide proper credit to the original developer: - -- Include the following attribution in your project: - **"Based on Quantum-Enhanced Language Model (QELM) by Brenton Carter (Inserian)"** - -- Provide a link back to the original repository: [R-D-BioTech-Alaska/QELM](https://github.com/R-D-BioTech-Alaska/QELM) +- **"Based on Quantum-Enhanced Language Model (QELM) by Brenton Carter (Inserian)"** -- Include a mention to the Qiskit community. [Qiskit](https://qiskit.org/) +- [R-D-BioTech-Alaska/QELM](https://github.com/R-D-BioTech-Alaska/QELM) -- [IBM Quantum](https://www.ibm.com/quantum) +- [Qiskit](https://qiskit.org) community & [IBM Quantum](https://www.ibm.com/quantum) --- ## License - -This project is licensed under the MIT License. +Licensed under the MIT License. See [LICENSE](LICENSE) for details. --- ## Contact +For questions, suggestions, or collaborations: -For inquiries, suggestions, or contributions: - -- **Email**: [contact@rdbiotechalaska.com](mailto:contact@rdbiotechalaska.com) -- **GitHub**: [R-D-BioTech-Alaska](https://github.com/R-D-BioTech-Alaska) -- **Website**: [RDBioTech.org](http://RDBioTech.org) +- **Email**: [contact@rdbiotechalaska.com](mailto:contact@rdbiotechalaska.com) ---- +- **GitHub**: [R-D-BioTech-Alaska](https://github.com/R-D-BioTech-Alaska) -> **Disclaimer**: QELM is an experimental project integrating quantum principles with NLP. Contributions are welcome to advance this pioneering effort in quantum-enhanced computing! +- **Website**: [RDBioTech.org](http://RDBioTech.org) or [Qelm.net](http://www.Qelm.net) ---- +> *Disclaimer: QELM is mostly experimental. Community feedback & contributions are welcome and needed to advance this exciting field.*