Ahoy, nakama! Ready to manage your One Piece Card Game collection with Merry and Olop Price Scraping?
This script will set you up with both applications running outside Docker, using PostgreSQL as your database. It was initially designed to be deployed on a Raspberry Pi 3B+.
-
Gather Your Supplies:
- Git: Download the latest version of Git (https://git-scm.com/)
- Python: Install Python 3.12 (https://www.python.org/)
- Tesseract OCR: Install Tesseract OCR (https://tesseract-ocr.github.io/)
- A Strong Will: (Optional, but always helpful!)
-
Clone the Repository:
git clone https://github.com/GomuGomuu/mery-pod.git
-
Set Up Your Treasure Map:
- Navigate to the repository:
cd mery-pod
- Make the script executable:
chmod +x setup.sh
- Navigate to the repository:
-
Edit the
.env
Files:- Merry:
- Navigate to the
merry
directory. - Open
.env
and set the following environment variables:POSTGRES_HOST
(usuallylocalhost
)POSTGRES_DB
(the name of your PostgreSQL database)POSTGRES_USER
(the name of your PostgreSQL user)POSTGRES_PASSWORD
(the password for your PostgreSQL user)DJANGO_SECRET_KEY
(set a strong, unique value)
- Navigate to the
- Olop Price Scraping:
- Navigate to the
olop-price-scraping
directory. - Open
.env
and set the following environment variables:REDIS_PORT_HOST
(usuallylocalhost
)REDIS_PORT
(the Redis port, usually6379
)REDIS_DB
(the Redis database number, usually0
)SELENIUM_HOST
(usuallylocalhost
)OCR_PATH
(the path to your Tesseract OCR executable)
- Navigate to the
- Merry:
-
Run the Setup Script:
To run the setup script, follow these instructions:
-
Make the script executable (if you haven't done so already):
chmod +x setup.sh
-
Run the setup script:
You can start the installation process by running the script from the command line. You can specify a starting step if needed.
-
To run the entire setup from the beginning:
./setup.sh
-
To start the setup from a specific step, provide the step number as an argument. For example, to start from step 5 (Redis installation):
./setup.sh 5
-
The script performs the following steps:
- Step 1: Clones the necessary repositories (
merry
andolop-price-scraping
) or updates them if they already exist. - Step 2: Installs essential dependencies, including Python, Pip, and Tesseract OCR.
- Step 3: Installs and configures PostgreSQL.
- Step 4: Configures environment variables for PostgreSQL and creates the database and user.
- Step 5: Installs and configures Redis.
- Step 6: Installs Chromium for the scraping process.
- Step 7: Downloads and configures ChromeDriver for web automation.
- Step 8: Installs Flower, a real-time monitoring tool for Celery.
- Step 9: Installs additional dependencies for both Merry and Olop Price Scraping.
- Step 10: Configures environment variables for Redis and Selenium.
- Step 11: Initializes the Merry application and applies database migrations.
- Step 12: Installs Screen, allowing you to run the services in the background.
- Step 13: Starts the Merry and Olop services in separate screen sessions, enabling you to manage them easily.
-
-
Access the Applications:
- Merry: Open your browser and go to
http://localhost:8000
- Olop Price Scraping API: Access the API endpoints documented in the
olop-price-scraping
repository. - Chromium: Launch Chromium using the command:
chromium-browser
- PostgreSQL: You can access the PostgreSQL database using tools like
psql
orpgAdmin
.
- Merry: Open your browser and go to
screen
is a terminal multiplexer that allows you to run multiple terminal sessions within a single window. This is particularly useful for running long-running processes without keeping your terminal open.
-
Start a New Screen Session: To start a new screen session, use the following command:
screen -S <session_name>
Replace
<session_name>
with a name of your choice. For example:screen -S merry
-
Detach from a Screen Session: To detach from the session (leave it running in the background), press:
Ctrl + A, then D
-
List Active Screen Sessions: To see all running screen sessions, type:
screen -ls
-
Reattach to a Screen Session: To reattach to a detached session, use:
screen -r <session_name>
For example:
screen -r merry
-
Exit a Screen Session: To exit a session completely, type
exit
within the screen session, or pressCtrl + D
.
Here's an example of how to start and manage your services using screen
:
-
Start Merry:
screen -S merry bash -c "cd merry && source venv/bin/activate && python manage.py runserver"
-
Start Flower:
screen -S flower bash -c "cd merry && source venv/bin/activate && flower"
-
Start Olop Price Scraping:
screen -S olop bash -c "cd olop-price-scraping && source venv/bin/activate && celery -A olop worker --loglevel=INFO"
-
Detach from any session: Press
Ctrl + A
, thenD
. -
Reattach to the Merry session:
screen -r merry
- Screen Magic: Use
screen -r <session_name>
to reattach to themerry
,flower
, orolop
screen sessions. - Troubleshooting: Check your environment variables, install any missing dependencies, and consult the repositories' documentation for more guidance.
- ChromeDriver: Ensure that you have the correct version of ChromeDriver for your Chromium installation.
[Explain how to use the project, how to run the scraping process, how to access the results, etc. Be specific and provide examples.]
Ahoy! Want to join our crew? Contributions are most welcome! Please see the CONTRIBUTING.md file for details on how to contribute to this project.
This project is licensed under the MIT License - see the LICENSE file for details.