The original intention of this project is to implement deep learning models I have learned over the years, providing notebooks and Medium articles for learning. This project is not a production-ready library for a specific domain but serves as a resource for learning.
The development direction will extend towards a monorepo.
package name: common
| Model | Article Link | Package |
|---|---|---|
| K-Means | - | kmeans |
| Kernel PCA | medium | kernel_pca |
package name: image_classification
| Model | Article Link | Package |
|---|---|---|
| ResNet50, ResNet-B, ResNet-C, ResNet-D | medium | resnet |
| ResNeXt50-32x4d | medium | resnext |
| ResNeSt50 | medium | resnest |
| Res2Net50 | medium | res2net |
| SE-ResNet50, SE-ResNeXt50 | medium | senet |
| SKNet | medium | sknet |
| Residual attention network(AttentionNet56) | medium | residual_attention |
| DenseNet | medium | densenet |
| Highway Network | medium | highway |
| MobileNet v1 | medium | mobilenet |
| MobileNet v2 | medium | mobilenetv2 |
| MobileNet v3 | medium | mobilenetv3 |
| EfficientNet v1 | medium | efficientnet |
| EfficientNet v2 | medium | efficientnetv2 |
| GhostNet | medium | ghostnet |
| GhostNet v2 | - | ghostnetv2 |
| ShuffleNet | medium | shufflenet |
| ShuffleNetV2 | medium | shufflenetv2 |
| SqueezeNet | medium | squeezenet |
| SqueezeNeXt | medium | squeezenext |
| Extraction | medium | extraction |
| DarkNet19 | medium | darknet19 |
| Vision transformer (ViT) | medium | vit |
| Distillated Vision transformer (DeiT) | medium | deit |
package name: object_detection
| Model | Article Link | Package |
|---|---|---|
| You only look once (YOLO) | medium | yolov1 |
| Tiny YOLOv1 | - | yolov1_tiny |
| YOLOv2 | medium | yolov2 |
| Single shot multibox detector (SSD) | - | ssd |
| RetinaNet | - | retinanet |
package name: semantic_segmentation
| Model | Article Link | Package |
|---|---|---|
| UNet | medium | unet |
| Fully convolution network (FCN) | medium | fcn |
| Pyramid spatial pooling network (PSPNet) | medium | pspnet |
| Dual attention(DANet) | medium | dual_attention |
| DeepLab v1 | - | deeplabv1 |
| DeepLab v2 | - | deeplabv2 |
| DeepLab v3 | - | deeplabv3 |
| Lite reduced ASPP (LR-ASPP) | - | lr_aspp |
| Reduced ASPP (R-ASPP) | - | r_aspp |
| Multi-scale context aggregation by dilation | - | mscad |
| ParseNet | - | parsenet |
| Segmentation transformer (SETR) | medium | setr |
package name: ocr
| Model | Article Link | Package |
|---|---|---|
| Convolution recurrent neural network (CRNN) | medium | crnn |
package name: image_generation
| Model | Article Link | Package |
|---|---|---|
| Variational autoencoder (VAE) | medium | vae |
| Generative adversarial network (GAN) | medium | gan |
| Deep convolution GAN (DCGAN) | medium | dcgan |
package name: style_transfer
| Model | Article Link | Package |
|---|---|---|
| Neural style transfer | medium | neural_style_transfer |
| Neural Doodle | medium | neural_doodle |
| Fast style transfer | medium | fast_style_transfer |
| Adaptive instance normalization (AdaIN) | medium | adain |
| Pix2Pix | medium | pix2pix |
package name: sequence_data
| Model | Article Link | Package |
|---|---|---|
| Long and short term memory (LSTM) | medium | lstm |
| Bidirectional LSTM (BiLSTM) | medium | lstm |
| Gated recurrent unit (GRU) | medium | gru |
| Temporal convolution network (TCN) | medium | tcn |
| Seq2Seq | medium | seq2seq |
| Transformer | - | transformer |
package name: language_model
| Model | Article Link | Package |
|---|---|---|
| Word2vec | medium | word2vec |
| GloVe | medium | glove |
| Vector log-bilinear language model (vLBL/ivLBL) | medium | vlbl |
package name: text_classification
| Model | Article Link | Package |
|---|---|---|
| TextCNN | medium | textcnn |
| Character CNN (CharCNN) | medium | charcnn |
| Very deep CNN (VDCNN) | medium | vdcnn |
| Recurrent CNN (RCNN) | medium | rcnn |
| Dynamic CNN (DCNN) | medium | dcnn |
| FastText | - | fasttext |
package name: tag_prediction
| Model | Article Link | Package |
|---|---|---|
| Bidirectional LSTM - Conditional random field (BiLSTM-CRF) | medium | bilstm_crf |
package name: text_generation
| Model | Article Link | Package |
|---|---|---|
| Show and tell | - | show_and_tell |
package name: few_shot
| Model | Article Link | Package |
|---|---|---|
| Siamese network | medium | siamese |
| Prototypical network | medium | prototypical |
package name: representation
| Model | Article Link | Package |
|---|---|---|
| Positive pairwise mutual information (PPMI) | medium | ppmi |
| Hellinger PCA | medium | hellinger |
package name: graph
| Model | Article Link | Package |
|---|---|---|
| Graph Convolution Network (GCN) | medium | gcn |
| Graph Attention Network (GAT) | medium | attention |
| Graph Attention Network v2 (GAT v2) | medium | attention |
package name: reinforcement_learning
| Model | Article Link | Package |
|---|---|---|
| Deep Q Network | medium | dqn |
pip install git+https://www.github.com/gite0z9/pytorch-implementations.git@main#torchlake --target=/torchlake
notebooks: demonstrate how to use torchlake.
torchlake: deep learning models composed of different domains.
In general, each domain will have a structure similar to the following:
├───adapter
├───artifacts
│ └───model_name
├───configs
│ └───model_name
├───constants
├───controller
├───datasets
│ └───dataset_name
| └───datasets.py
├───models
│ ├───base
│ └───model_name
| |───reference
| | └───paper
│ ├───model.py
│ ├───network.py
│ ├───loss.py
│ ├───helper.py
│ └───decode.py
├───runs
├───scripts
│ └───debug
├───tests
├───utils
└───requirements.txt
adapter: Interface between the controller and other resources (model, model.loss, etc.).
configs: Model configuration files, including device definition, model definition, training definition, and inference definition, covering four aspects.
constants: Fixed values, including constants and enums.
controller: Controller, where controller is a shared base, and trainer, evaluator, predictor are responsible for training, evaluation, and prediction tasks, respectively.
models: Model definitions, where network.py represents the model blocks, model.py is the assembled model, and loss.py is the loss function.
datasets: Datasets, currently categorized by domain, with raw datasets and CSV datasets. The former reads raw data, while the latter reads processed CSV data (such as normalized coordinates).
runs: Folder documenting TensorBoard logs for trainer and evaluator results.
tests: Unit tests using pytest.
utils: Stores functions with low dependency and high reusability.
Model and dataset configurations are controlled using pydantic.
