Skip to content
This repository has been archived by the owner on Mar 14, 2023. It is now read-only.

Add Category model and introduce it to the Spider class #1

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

aniav
Copy link
Owner

@aniav aniav commented Sep 8, 2019

Hello future self 👋

After at least three attempts at this PR spread across 2 years I decided to finally write down what has already been done.

What is this PR about?

This PR is supposed to introduce a concept of categories which would be connected to both the web page and the crawlers. Right now the crawler would always search within one specific category (dresses) in all the crawled pages, which has proven to not be enough. But has also proven to be a successful MVP which is running on heroku without any supervision for 2+ years 💃

Progress

  • Add the Category model
  • Add migrations for the Category model
  • Add the categories structure in a static way (file?)
  • Add a data migration loading the structure to the database into the Category model
  • Add the categories to 1 crawler and check it works
  • Connect the categories to the web page sidebar
  • Make the products page category aware and make it load proper products
  • Add the categories to all other crawlers and check it works

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant