Skip to content

Commit

Permalink
chore: replace elastic with postgresql (#63)
Browse files Browse the repository at this point in the history
* replace ES with postgresql

* update vagrant setup documentation

* update Kubernetes setup documentation

* fix pr typos
  • Loading branch information
devadvocado authored Apr 14, 2020
1 parent e6d2d12 commit 592ff98
Show file tree
Hide file tree
Showing 71 changed files with 735 additions and 857 deletions.
5 changes: 3 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
.DS_Store
node_modules
# dist

# local env files
.env.local
Expand All @@ -27,7 +26,7 @@ yarn-error.log*
!/vagrant/frontend/
!/vagrant/backend/
!/vagrant/backend-v1/
!/vagrant/elastic/
!/vagrant/postgresql/
!/vagrant/control-plane/
!/vagrant/redis/
!/vagrant/metrics/
Expand All @@ -36,3 +35,5 @@ yarn-error.log*
!/vagrant/README.md
!Vagrantfile

# postgres data
pgdata/
8 changes: 4 additions & 4 deletions DEVELOPER.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

First and foremost, make sure to run all commands to get each endpoint setup. Please refer to the [README](README.md) to see those commands.

#### Elasticsearch - Products endpoint
#### PostgreSQL - Products endpoint

The products endpoint is located at: `http://localhost:3001/items`. This will not return any product data unless `?q=` is appended to it (see below).

Expand Down Expand Up @@ -44,9 +44,9 @@ curl http://localhost:3001/items/6/reviews/
- `Modal.vue` - A generic component for displaying modals
- `Product.vue` - Displays all of the product details. It's the first component seen when loading the app, and has the most importance in `SearchResults.vue`
- `Reviews.vue` - This component displays a product's reviews. It's used within a modal in `Product.vue`
- `Search.vue` - The main search bar referenced in `GlobalHeader.vue`. It triggers the Elasticsearch endpoint queries as the user types, but will ignore the BACKSPACE and DEL keys as to prevent needless repeat queries. Using the search will emit an action to `App.vue`, which will trigger the Elasticsearch queries based on what the user searches for
- `SearchResults.vue` - This is what is immediately displayed on app load. It will consume the Elasticsearch endpoint and display its data. It is housed within `/src/App.vue`, which is where the Elasticsearch products data is called
- `Search.vue` - The main search bar referenced in `GlobalHeader.vue`. It triggers the PostgreSQL endpoint queries as the user types, but will ignore the BACKSPACE and DEL keys as to prevent needless repeat queries. Using the search will emit an action to `App.vue`, which will trigger the PostgreSQL queries based on what the user searches for
- `SearchResults.vue` - This is what is immediately displayed on app load. It will consume the PostgreSQL endpoint and display its data. It is housed within `/src/App.vue`, which is where the PostgreSQL products data is called

#### App.vue

This is where all of the magic happens. The Elasticsearch endpoint is called on page load and displays all of the products. As the user searches, the search method is called and queries the Elasticsearch endpoint for the query the user enters.
This is where all of the magic happens. The PostgreSQL endpoint is called on page load and displays all of the products. As the user searches, the search method is called and queries the PostgreSQL endpoint for the query the user enters.
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,26 +15,26 @@ This repository houses the demo application used to illustrate Kuma's extensive
- [Introduction](#introduction)
- [Frontend](#frontend)
- [Backend](#backend)
- [Elasticsearch](#elasticsearch)
- [PostgreSQL](#postgresql)
- [Redis](#redis)
- [Deployment](#deployment)

## Introduction
[![][diagram]][diagram]

The Kuma Demo Application is a clothing marketplace where you can browse listed items along with the reviews left by users. It consists of four components: [Vue frontend UI](#frontend), [Node backend API](#backend), [Elasticsearch](#Elasticsearch), and [Redis](#Redis).
The Kuma Demo Application is a clothing marketplace where you can browse listed items along with the reviews left by users. It consists of four components: [Vue frontend UI](#frontend), [Node backend API](#backend), [PostgreSQL](#PostgreSQL), and [Redis](#Redis).

### Frontend

The frontend UI is built using [VuePress](https://vuepress.vuejs.org/) and the source code can be found in the [app directory](app/README.md). It gives the users a webpage where they can browse items and reviews.

### Backend

The backend API is built using [Node.js](https://nodejs.org/en/) and the source code can be found in the [api directory](api/README.md). It contains endpoints that enables the user to query the Elasticsearch and Redis databases.
The backend API is built using [Node.js](https://nodejs.org/en/) and the source code can be found in the [api directory](api/README.md). It contains endpoints that enables the user to query the PostgreSQL and Redis databases.

### Elasticsearch
### PostgreSQL

The Elasticsearch database is used to store all the items. THe list of items can be found in this [JSON file](api/db/items.json). Here is a sample of how each object in our list of items look:
The PostgreSQL database is used to store all the items. THe list of items can be found in this [JSON file](api/db/items.json). Here is a sample of how each object in our list of items look:

```json
...
Expand Down Expand Up @@ -85,7 +85,7 @@ The Elasticsearch database is used to store all the items. THe list of items can
...
```

All this information will be saved in Elasticsearch **EXCEPT** the reviews. Reviews will be separated out and stored in Redis.
All this information will be saved in PostgreSQL **EXCEPT** the reviews. Reviews will be separated out and stored in Redis.

### Redis

Expand Down
4 changes: 2 additions & 2 deletions api/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# docker build . -t kvn0218/kuma-demo-be:v1
# docker push kvn0218/kuma-demo-be:v1
# docker build . -t kvn0218/kuma-demo-be:latest
# docker push kvn0218/kuma-demo-be:latest

FROM node:lts-alpine

Expand Down
6 changes: 3 additions & 3 deletions api/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# API

The backend API is built using [Node.js](https://nodejs.org/en/). It contains endpoints that enables the user to query the Elasticsearch and Redis databases.
The backend API is built using [Node.js](https://nodejs.org/en/). It contains endpoints that enables the user to query the PostgreSQL and Redis databases.

## Local Installation

Expand All @@ -9,9 +9,9 @@ The backend API is built using [Node.js](https://nodejs.org/en/). It contains en
npm install
```

2. Run ElasticSearch on Docker:
2. Run PostgreSQL on Docker:
```sh
docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.3.2
docker run --rm -p 5432:5432 --name kuma-postgres -e POSTGRES_USER=kumademo -e POSTGRES_PASSWORD=kumademo -e POSTGRES_DB=kumademo kvn0218/postgres:latest
```

3. Run Redis on Docker:
Expand Down
144 changes: 0 additions & 144 deletions api/app/elastic.js

This file was deleted.

58 changes: 58 additions & 0 deletions api/app/postgresql.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
const items = require("../db/items.json");
const { Pool } = require("pg");

const pool = new Pool({
user: process.env.POSTGRES_USER || "kumademo",
host: process.env.POSTGRES_HOST || "localhost",
database: process.env.POSTGRES_DB || "kumademo",
password: process.env.POSTGRES_PASSWORD || "kumademo",
port: process.env.POSTGRES_PORT_NUM || 5432, //POSTGRES_PORT environmental variable is taken on K8S
});

pool.on("error", (err, clients) => {
console.error("Unexpected error on idle client", err);
process.exit(-1);
});

const search = async (itemName) => {
return await pool.query(
`SELECT data FROM marketItems WHERE name ILIKE '%${itemName}%'`
);
};

const importData = () => {
(async () => {
const client = await pool.connect();
try {
await client.query("BEGIN");
await client.query("DROP TABLE IF EXISTS marketItems");
await client.query(
`CREATE TABLE marketItems(
index INTEGER PRIMARY KEY,
name TEXT,
data JSONB
)`
);
items.forEach((item) => {
client.query(
`INSERT INTO marketItems VALUES(${item.index},'${
item.name
}','${JSON.stringify(item)}')`
);
});
await client.query("COMMIT");
} catch (e) {
console.log("Error");
await client.query("ROLLBACK");
throw e;
} finally {
console.log("Release");
client.release();
}
})().catch((e) => console.error(e.stack));
};

module.exports = Object.assign({
search,
importData,
});
15 changes: 0 additions & 15 deletions api/db/elasticsearch/Dockerfile

This file was deleted.

Binary file removed api/db/elasticsearch/data/nodes/0/_state/_j.cfe
Binary file not shown.
Binary file removed api/db/elasticsearch/data/nodes/0/_state/_j.cfs
Binary file not shown.
Binary file removed api/db/elasticsearch/data/nodes/0/_state/_j.si
Binary file not shown.
Binary file removed api/db/elasticsearch/data/nodes/0/_state/_m.cfe
Binary file not shown.
Binary file removed api/db/elasticsearch/data/nodes/0/_state/_m.cfs
Binary file not shown.
Binary file removed api/db/elasticsearch/data/nodes/0/_state/_m.si
Binary file not shown.
Binary file removed api/db/elasticsearch/data/nodes/0/_state/manifest-0.st
Binary file not shown.
Binary file removed api/db/elasticsearch/data/nodes/0/_state/node-0.st
Binary file not shown.
Binary file removed api/db/elasticsearch/data/nodes/0/_state/segments_q
Binary file not shown.
Empty file.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Empty file.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Empty file.
14 changes: 14 additions & 0 deletions api/db/postgresql/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# docker build . -t kvn0218/postgres:latest
# docker push kvn0218/postgres:latest

FROM postgres:alpine

RUN mkdir -p /tmp/psql_data/

RUN echo "host all all 0.0.0.0/0 md5" >> /var/lib/postgresql/data/pg_hba.conf

# Expose the PostgreSQL port
EXPOSE 5432

COPY ./database.sql /tmp/psql_data/
COPY ./init_docker_postgres.sh /docker-entrypoint-initdb.d/
Loading

0 comments on commit 592ff98

Please sign in to comment.