Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Two phase download #675

Merged
merged 39 commits into from
Feb 8, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
b5355c3
util for date formatting
Nurou Jan 12, 2021
009bca4
add util shared form creation for downloads
Nurou Jan 13, 2021
efdd6c9
download quiz info in 2 phases
Nurou Jan 13, 2021
618997b
extract token generation to function
Nurou Jan 13, 2021
47167bb
quiz answer info refactor
Nurou Jan 17, 2021
7efefe3
fix name of peer review info
Nurou Jan 17, 2021
68f2970
further refactoring
Nurou Jan 17, 2021
c01dc2f
swap redis for ioredis
Nurou Jan 17, 2021
2dc53cb
Merge branch 'master' into two-phase-download
Nurou Jan 21, 2021
783eda6
flush redis after each test
Nurou Jan 22, 2021
4d15989
fix admin spelling and remove redundant function in acc control
Nurou Jan 22, 2021
25bc294
add missing redis flush after widget tests
Nurou Jan 22, 2021
9a617e6
fix tests
Nurou Jan 22, 2021
2e4eabd
..
Nurou Jan 22, 2021
f193a5b
attempt ts fix
Nurou Jan 22, 2021
0bdf032
rm obsolote import
Nurou Jan 22, 2021
355707b
try again
Nurou Jan 22, 2021
cb25a70
ci install redis
Nurou Jan 22, 2021
4072f9a
rm c.log
Nurou Jan 24, 2021
19005d6
redis test config
Nurou Jan 25, 2021
d875e62
define redis port in cc env variables
Nurou Jan 25, 2021
e9eec4f
try another config
Nurou Jan 25, 2021
f6690eb
redis host
Nurou Jan 25, 2021
c2fdf9c
try another tag
Nurou Jan 25, 2021
20534b9
log redis
Nurou Jan 25, 2021
79ca1fb
move env redis
Nurou Jan 25, 2021
49ad069
remove useless test
Nurou Jan 30, 2021
5f79e02
add missing download tests
Nurou Jan 30, 2021
81de51b
Merge branch 'master' into two-phase-download
Nurou Feb 1, 2021
b2b9def
add redis log when unititialized
Nurou Feb 1, 2021
76bfb3a
use user id instead of username for redis key
Nurou Feb 1, 2021
d220b8b
rm non-existent import
Nurou Feb 3, 2021
ba1cb68
Merge branch 'master' into two-phase-download
Nurou Feb 3, 2021
c9f3b04
fix tests
Nurou Feb 3, 2021
6a5fadc
rm console.log
Nurou Feb 3, 2021
b73ec71
add download tests
Nurou Feb 3, 2021
c4bc37d
Trigger Build
Nurou Feb 6, 2021
60e9946
fix hanging test
Nurou Feb 6, 2021
0529fec
add prefix to download token
Nurou Feb 7, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,8 @@ jobs:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: quizzes_test
- image: circleci/redis:6.0.8-alpine

steps:
- checkout
- run:
Expand All @@ -81,15 +83,39 @@ jobs:
sleep 1
done
echo Failed waiting for Postgres && exit 1
- run:
name: "Waiting for Redis to be ready"
command: |
for i in `seq 1 10`;
do
nc -z localhost 6379 && echo Success && exit 0
echo -n .
sleep 1
done
echo Failed waiting for Redis && exit 1
- run:
name: "install dependencies"
command: cd packages/backendv2 && npm ci
- run:
name: "run migrations"
command: cd packages/backendv2 && export NODE_ENV=test && npx knex migrate:latest
- run:
name: install dockerize
command: wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz && sudo tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz && rm dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
environment:
DOCKERIZE_VERSION: v0.3.0
- run:
name: Wait for redis
command: dockerize -wait tcp://localhost:6379 -timeout 1m
- run:
name: Wait for db
command: dockerize -wait tcp://localhost:5432 -timeout 1m
- run:
name: "run tests"
command: "cd packages/backendv2 && npm run test -- --coverage"
environment:
REDIS_HOST: localhost
REDIS_PORT: 6379
- run:
name: "Upload coverage"
command: "cd packages/backendv2 && bash <(curl -s https://codecov.io/bash)"
Expand Down
41 changes: 10 additions & 31 deletions packages/backendv2/config/redis.ts
Original file line number Diff line number Diff line change
@@ -1,39 +1,18 @@
import dotenv from "dotenv"
import redis from "redis"
import Redis from "ioredis"

import { promisify } from "util"

if (process.env.NODE_ENV !== "production") {
dotenv.config({ path: ".env" })
}

let client = null

/* Create client if variables available */
let client
if (process.env.REDIS_HOST && process.env.REDIS_PORT) {
client = redis.createClient({
client = new Redis({
port: Number(process.env.REDIS_PORT),
host: process.env.REDIS_HOST,
port: Number.parseInt(process.env.REDIS_PORT as string, 10),
password: process.env.REDIS_PASSWORD && process.env.REDIS_PASSWORD,
})
client.on("error", err => {
console.log(err)
})

client.on("connect", () => {
if (client) {
console.log(
`Connected to Redis running on - HOST: ${process.env.REDIS_HOST} PORT: ${process.env.REDIS_PORT} `,
`Redis client initialized and running on ${process.env.REDIS_HOST}:${process.env.REDIS_PORT}`,
)
})
}
} else {
console.log("Redis uninitialized. Configuration missing.")
}

let get = client ? promisify(client.get).bind(client) : null
let set = client ? promisify(client.get).bind(client) : null
let setex = client ? promisify(client.setex).bind(client) : null

export default {
client,
get,
set,
setex,
}
export default { client }
Loading