If you're looking to automate digital product updates efficiently, this project is for you! It automates the retrieval, processing, and publishing of digital product data for a Shopify store and thus, helps reduce manual work, and ensures seamless product updates. It integrates Google Cloud Platform (GCP), Amazon Web Service (AWS), Terraform (Tofu), Ansible, Python, Bash and GitHub Actions to manage data pipelines efficiently.
✅ Data Extraction & Cleaning:
- Uses Google Cloud Storage to store raw product datasets.
- Python & Pandas process, clean, and update product information.
✅ Automation & Infrastructure:
- Terraform (Tofu) provisions GCP resources (buckets, IAM, etc.).
- GitHub Actions automates Terraform deployments and Python scripts.
✅ Product Updates & Publishing:
-
Data is transformed and published back to GCP for further use.
-
Shopify API integration allows automated product updates.
-
AWS Lambda triggers automate workflows.
-
Tech Stack
🔹 Cloud: Google Cloud Storage, IAM, AWS Lambda
🔹 IaC: Terraform (Tofu), Ansible
🔹 Automation: GitHub Actions
🔹 Data Processing: Python, Pandas
🔹 E-commerce: Shopify API
🚀 Automate & streamline digital product updates for an existing online store (it's been upt for 4 years !), reducing manual work and improving data accuracy.
We are a DevOps Engineer (@aglorhythm) and a Data Analyst (@azezu), entrepreneurs, working together to build efficient, automated solutions for our e-commerce. Passionate about cloud infrastructure, data processing, and automation, we created this project to streamline digital product updates for Shopify. You probably guessed, we are very busy ! Our goal is to eliminate manual work, improve data accuracy, and optimize workflows using modern DevOps and data engineering practices.