Skip to content

Commit

Permalink
add quote schedule
Browse files Browse the repository at this point in the history
  • Loading branch information
nrandriamanana committed Apr 18, 2024
1 parent 770ce52 commit aa00aea
Show file tree
Hide file tree
Showing 3 changed files with 4 additions and 3 deletions.
1 change: 1 addition & 0 deletions etl-sampling/configmap.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ metadata:
namespace: projet-ape
data:
NAMESPACE: projet-ape
AWS_S3_ENDPOINT: minio.lab.sspcloud.fr
NUMBER_TO_ANNOTATE: "1701"
MONTH_INTERVAL: "1"
DATA_SAMPLED_QUEUE_PATH: s3/projet-ape/label-studio/annotation-campaign-2024/NAF2008/data-samples/queue/
Expand Down
2 changes: 1 addition & 1 deletion etl-sampling/cronjob.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ metadata:
name: etl-sampling
spec:
timeZone: "Europe/Paris"
schedule: "13 01 * * 5"
schedule: "23 01 * * 5"
concurrencyPolicy: Forbid
jobTemplate:
spec:
Expand Down
4 changes: 2 additions & 2 deletions utils/extract_db.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@
pip install -r requirements.txt
# sampling from extracted db
EXTRACT_DB=$(python extract_db.py "$NAMESPACE/extractions" $NUMBER_TO_ANNOTATE $MONTH_INTERVAL)
mc mv "$EXTRACT_DB" "$DATA_SAMPLED_QUEUE_PATH"
mc cp --recursive "$DATA_SAMPLED_QUEUE_PATH" "$DATA_SAMPLED_ARCHIVE_PATH"
mc mv $EXTRACT_DB $DATA_SAMPLED_QUEUE_PATH
mc cp --recursive $DATA_SAMPLED_QUEUE_PATH $DATA_SAMPLED_ARCHIVE_PATH

0 comments on commit aa00aea

Please sign in to comment.