Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: refactor mock api server to fail on error #141

Merged
merged 2 commits into from
Oct 31, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .env
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,4 @@ DB_PASS=change-this-sqlpassword
WIKIBASE_PINGBACK=false
# wikibase.svc is the internal docker hostname, change this value to the public hostname
WIKIBASE_HOST=wikibase.svc
WIKIBASE_PORT=80
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this has some background, but I cannot bind to port 80 locally using docker compose unless I elevate priviliges.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is no reason to keep this like it is; I guess it was just picked at some point to look closest to prod. After all these port names end up in the URIs of the triples that go in the queryservice

WIKIBASE_PORT=8000
15 changes: 1 addition & 14 deletions docker-compose.ci.yml
Original file line number Diff line number Diff line change
@@ -1,18 +1,5 @@
version: '2'
services:

wikibase:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really know what these overrides are good for. Containers can use the Docker network (instead of the host one) in both CI and local testing.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no clue either.. let's not keep it

networks:
default:
aliases:
- some-cool-internal-hostname.svc

api:
environment:
- API_WIKIBASE_DOMAIN=wikibase.svc
- API_WDQS_BACKEND=wdqs.svc:9999
- WBSTACK_PROXYMAP_INGRESS="some-cool-internal-hostname.svc:80"

wdqs-updater:
build:
context: .
Expand All @@ -27,7 +14,7 @@ services:
environment:
- WBSTACK_WIKIBASE_SCHEME=http
- WBSTACK_LOOP_LIMIT=100
- WBSTACK_API_ENDPOINT=http://api.svc:3030
- WBSTACK_API_ENDPOINT=http://api.svc:3030/getBatches
- WBSTACK_BATCH_SLEEP=1
- WIKIBASE_HOST=wikibase.svc
- HEAP_SIZE=32m
18 changes: 14 additions & 4 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
version: '2'
version: '3'

services:
wdqs:
image: ghcr.io/wbstack/queryservice:main
Expand Down Expand Up @@ -31,9 +32,12 @@ services:
default:
aliases:
- api.svc
depends_on:
wikibase:
condition: service_healthy
environment:
- API_WIKIBASE_DOMAIN=localhost
- API_WDQS_BACKEND=localhost:9999
- API_WIKIBASE_DOMAIN=wikibase.svc
- API_WDQS_BACKEND=wdqs.svc:9999

wikibase:
image: "${WIKIBASE_IMAGE_NAME}"
Expand All @@ -48,6 +52,11 @@ services:
default:
aliases:
- wikibase.svc
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:80/w/api.php"]
interval: 1s
timeout: 3s
retries: 30
volumes:
- ./LocalSettings.ci.php:/var/www/html/LocalSettings.d/LocalSettings.override.php
environment:
Expand All @@ -59,7 +68,7 @@ services:
DB_USER: ${DB_USER}
DB_PASS: ${DB_PASS}
DB_NAME: ${DB_NAME}
WIKIBASEHOST:
WIKIBASE_HOST: ${WIKIBASE_HOST}

mysql:
image: "${MYSQL_IMAGE_NAME}"
Expand All @@ -76,6 +85,7 @@ services:
default:
aliases:
- mysql.svc

volumes:
query-service-data:
mediawiki-mysql-data:
73 changes: 42 additions & 31 deletions seeder/server.js
Original file line number Diff line number Diff line change
@@ -1,41 +1,52 @@
var http = require('http');
const wbEdit = require( 'wikibase-edit' )( require( './wikibase-edit.config' ) );

var x = 0
http.createServer(async function (req, res) {
res.writeHead(200, {'Content-Type': 'text/json'});
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having this as the first line meant we'd 200 even when wikibase was still down.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch

http.createServer(function (req, res) {
(async () => {
switch (req.url) {
case '/getBatches':
const numEntities = 20;
const entities = [];

numEntities = 20;
entities = [];
for (let i = 1; i <= numEntities; ++i) {
const { entity } = await wbEdit.entity.create({
type: 'item',
labels: {
'en': new Date().toISOString()
},
descriptions: {
'en': new Date().toDateString() + new Date().toISOString()
}
});

for(var i=1; i <= numEntities; ++i){
const { entity } = await wbEdit.entity.create({
type: 'item',
labels: {
'en': new Date().toISOString()
},
descriptions: {
'en': new Date().toDateString() + new Date().toISOString()
console.log('created item id', entity.id)
entities.push(entity.id)
}
});

console.log('created item id', entity.id)
entities.push(entity.id)
}
console.log(new Date().toISOString());

x += numEntities;
console.log(new Date().toISOString());
responseObject = {
'entityIds': entities.join(','),
'wiki': {
'domain': process.env.API_WIKIBASE_DOMAIN,
'wiki_queryservice_namespace': {
'backend': process.env.API_WDQS_BACKEND,
'namespace': 'wdq'
}
},

responseObject = {
'entityIds': entities.join(','),
'wiki': {
'domain': process.env.API_WIKIBASE_DOMAIN,
'wiki_queryservice_namespace': {
'backend': process.env.API_WDQS_BACKEND,
'namespace': 'wdq'
}
},

};
res.end(JSON.stringify([responseObject]));
};
res.writeHead(200, {'Content-Type': 'text/json'});
res.end(JSON.stringify([responseObject]));
return;
default:
res.writeHead(404);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this; good idea to ensure it 404s everywhere else

res.end('Not found');
}
})()
.catch((err) => {
console.error('Failed handling request: %s', err.message);
res.writeHead(500);
res.end(err.message);
})
}).listen(3030);
Loading