Introduction to Data Science

Instructor: Renato Rocha Souza

This is the repository of code for the "Introduction to Data Science"

This class is about the Data Science process, in which we seek to gain useful predictions and insights from data. Through real-world examples and code snippets, we introduce methods for:

• data munging, scraping, sampling andcleaning in order to get an informative, manageable data set;
• data storage and management in order to be able to access data (even if big data);
• exploratory data analysis (EDA) to generate hypotheses and intuition about the data;
• prediction based on statistical learning tools;
• communication of results through visualization, stories, and interpretable summaries

Detailed Syllabus:

• Propaedeutic ref1, ref2, ref3, ref4, ref5, ref6

• Related Courses cs109, cs229, ML Andrew Ng, free courses

• Books ref1

• Data Science Concepts ref1, ref2, ref3, ref4, book1, book2, book3

  • Data Scientists ref1, re2, ref3, ref4, ref5, ref6, ref7, ref8, ref9
  • Statistics book1, book2, ref1, ref2, ref3, ref4
  • AI ref1, ref2, ref3
    • AI and Game Theory ref1
  • Machine Learning ref1, ref2, ref3, ref4, book, movies
  • Business Intelligence ref1

• Data Science Process ref1, ref2, ref3

  • Data Formats ref1
  • Data Acquisition ref1
  • Data Engineering ref1, ref2, ref3
    • Handling Missing Values ref1
    • Changing Datatypes ref1
  • Exploratory Data Analysis ref1, ref2, ref3, ref4, ref5, book
  • Data Visualization & Storytelling ref1, ref2, ref3

• Model Selection ref1

  • Feature Engineering ref1, ref2, book
    • Automated Feature Engineering featuretools
  • Feature Selection ref1, ref2, ref3
    • Numeric Data ref1
    • Discrete/Categorical Data ref1
    • Textual Data ref1, ref2
  • Hiperparameter Search ref1
  • Cross Validation ref1, video1
  • Oversampling and Undersampling ref1
  • Regularization ref1, ref2
  • Bias and Variance ref1
  • Overfitting and Underfitting ref1, ref2
  • Evaluation Metrics and Explainability ref1, ref2, ref3, ref4, ref5, ref6, ref7

• Machine Learning Algorithms ref1, ref2, ref3, ref4, ref5, ref6, ref7

  • Unsupervised ref1
    • Dimensionality reduction ref1
      • PCA ref1
      • SVD ref1
      • t-SNE ref1
    • Clustering ref1, ref2, ref3, ref4, ref5,ref6, ref7
      • K-Means ref1, ref2
      • Hierarchical Clustering ref1
      • K-Modes ref1
    • Unsupervised Deep Learning ref1
  • Supervised

    • Regression ref1, ref2, ref3, ref4, ref5

    • Classification ref1, ref2, ref3, ref3

    • Linear Models ref1

      • Linear Regression ref1

        • Polynomial Regression ref1 ref2
        • Stepwise Regression ref1
        • Ridge Regression ref1, ref2
        • Lasso Regression ref1, package
        • ElasticNet Regression ref1

      • Logistic Regression (Categorical) ref1, ref2, ref3, ref4

      • Support Vector Machines ref1, ref2, ref3, ref4, paper, video

        • SVR ref1, ref2
        • SVC ref1, ref2

      • Passive-Aggressive ref1

      • Perceptron ref1, ref2

    • Bayesian Models

      • Naive Bayes ref1
      • Gaussian Naive Bayes ref1
      • Bernoulli Naive Bayes ref1

    • Guassian Mixtures ref1, ref2, package

    • k Nearest Neighbors (kNN) ref1

    • Decision Tree ref1, ref2

    • Ensemble Models ref1, ref2 ref3, package

      • Bagging ref1
        • Random Forest ref1, ref2, ref3
        • Extremely Randomized Trees ref1
        • Bagging Classifier ref1, package
      • Boosting ref1, ref2
        • Adaboost ref1
        • GBM ref1
        • XGBoost ref1, ref2, package
        • LightGBM ref1
        • CatBoost ref1
        • Regularized Greedy Forests ref1, package
      • Stacking ref1
      • Voting ref1

    • Genetic Algorithms ref1, ref2, ref3

      • TPOT,
      • Auto SKLearn
      • Auto Keras
      • AutoML

    • Neural Networks and Deep Learning ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, ref9, ref10, simple implementation, book, viz, video, meme

      • Deep Learning and NLP ref1

    • Neural Network concepts

      • General Math
      • Linear and dense layers
      • Weight Initialization ref1, ref2
      • Weight Averaging ref1
      • Hyperparameter Tuning ref1, ref2
      • Gradient Descent ref1, ref2, ref3, video
      • Backpropagation ref1
      • Loss Functions ref1
      • Convolutional Neural Networks ref1, ref2, ref3, ref4, ref5, ref6, ref6, ref7, ref8, ref9, ref10, architectures, viz
        • CNNs for NLP ref1, ref2
        • CNNs for object detection ref1
        • CNNs for Time Series ref1
        • Capsule Networks ref1
      • RNNs (Sequence Models) ref1, ref2, ref3
        • Attention Models ref1
        • LSTMs and GRUs ref1, ref2, ref3, ref4, ref5, ref6
        • Word Embeedings ref1, ref2, ref3, ref4, ref5, ref6, ref6
          • Word2vec ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref, video, en-us trained models, pt-br trained models, ge-de trained models, pre-trained models
          • Char2vec ref1
          • Sentence Embeddings ref1, ref2,
          • Doc2vec ref1, ref2, ref3
          • Beyond Word Embeddings ref1, ref2, ref3, ref4, ref5, ref6, ref7
          • Glove ref1, ref2, ref3
          • FastText ref1, ref2
          • Misspelling Oblivious Word Embeddings (MOE) ref1, ref2
        • Transformers ref1, ref2, ref3, ref4, ref5
          • BERT ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, ref9, ref10, ref11
          • ALBERT ref1, ref2
          • XLNet ref1
          • ULMFiT ref1
          • GPT-3 ref1
      • Reinforcement Learning ref1, ref2, ref3, ref4, ref5, ref6, programming resource
      • Transfer Learning ref1, ref2, ref3, ref4, ref5, ref6, ref6, ref7
      • Autoencoders ref1, ref2, ref3, ref4, ref5
        • Generative Adversarial Networks ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, GANS and Deepfakes, Colab Notebooks

• Data Science Tasks

  • NLP tasks ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, ref9, ref10

    • Text Classification ref1, ref2, model interpretability, pretrained models, other pretrained models
    • Vector Representation ref1, ref2a, ref2b, ref3, ref4
    • OCR ref1, ref2
    • Topic Modeling ref1, ref2, ref3, ref4, ref5, ref5
      • LSA ref1
      • LDA ref1
    • Text Mining and Information Extraction ref1, ref2, ref3, ref4, ref5
    • Keyword Extraction and Text Summarization ref1, ref2, ref3, ref4, ref5, ref6, ref7
    • Collocation Extraction ref1
    • Text Generation ref1
    • Regular Expressions ref1, ref2 , ref3
    • Named Entity Recognition ref1, ref2, ref3, ref4, ref5, ref6, ref7
    • Coreference Resolution ref1,
    • Document and Sentence Similarity ref1, ref2, ref3, ref4
    • Sentiment Analysis ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, ref9, ref10
    • Sarcasm Detection ref1
    • Chatbots ref1, ref2
    • Labeling ref1, ref2

  • Information Retrieval ref1, ref2

  • Graphs and Network Analysis ref1, ref2, ref3, ref4, ref5, ref7

    • Knowledge Graphs ref1, ref2, ref3, ref4, ref5

  • Time Series Analysis ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, ref9, video

  • Recommender Systems ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8

  • Music Classification ref1

  • WebScraping ref1, ref2, ref3, ref4, ref5

• Preparing the Environment

  • Linux, Python, Virtualenv

  • Jupyter Notebook on AWS setup

  • Versioning Tools

    • Git ref1, ref2
    • Github ref1, , ref2
    • GIT Large File Versioning ref1
    • Gitlab ref1

  • Exploratory Data Analysis Tools

    • Jupyter ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8
    • Numpy ref1, ref2, ref3
    • Pandas ref1, ref2, ref3, ref4, styling
      • Pandas Machine Learning
      • SKLearn Pandas
      • Geopandas
      • Framequery
    • Statsmodels ref1

  • Machine Learning Tools

    • Scikit-Learn ref1, ref2
      • Inbalanced Learn
      • ForestCI
    • Tensor Flow ref1, ref2, ref3, ref4, ref5
      • TF.Text ref1
      • TF Datasets ref1
    • Keras ref1, ref2, ref3
    • PyTorch ref1, Comparison Tensorflow vs PyTorch, Comparison Tensorflow vs PyTorch 2
    • Hugging Face ref1
    • Gensim
    • Orange

  • NLP Tools

    • NLTK ref1, ref2
    • Spacy ref1, ref2, ref3, ref4, ref5, ref6
    • TextBlob ref1, ref6, ref7
    • StanfordNLP Lib ref1

  • Visualization Tools ref1

    • Matplotlib ref1, ref2, ref3
    • Seaborn ref1
    • Bokeh ref1, example
    • Plotly
    • Altair
    • GGPlot2
    • missingno
    • MPLD3
    • d3.js ref1,ref2
    • HoloViews
    • Folium
      • Leaflet
    • Autoviz ref1, ref2

  • Graph Analysis Tools

    • NetworkX, DataShader, Igraph

  • Dashboards and UIs

    • Django ref1
    • Flask ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8
    • Streamlit ref2, ref2, ref3, ref4, ref5, ref6
    • Panel ref1, ref2
    • Python GUI ref1
    • Dash ref1, ref2, ref3
    • FastAPI ref1

  • Neural Networks visualization

    • TensorBoard ref1, ref2
    • Tensor Flow Playground ref1
    • Tensor Flow Projector ref1
    • Tensor Flow Hub ref1
    • Deep Replay ref1

  • Relational databases and SQL

    • SQL ref1, ref2, ref3, ref4, ref5

  • NoSQL / Graph Databases

    • Elastic Search ref1
    • SPARQL ref1
    • MongoDB ref1

  • Data Wrangling and Distributed computing

    • Map Reduce
    • Spark ref1
    • Dask ref1
    • ETE Toolkit ref1
    • ODO ref1
    • Docker ref1, ref2,ref3
    • Google Colab ref1, ref2

  • Analytical Pipelines

    • Scikit-Learn Pipelines ref1
    • Luigi ref1
    • Airflow ref1, ref2

  • Machine Learning Datasets ref1

We are using https://git-lfs.github.com because the /datasets files can be large. Install it before the git clone.