Trying to make getting OCDS data as easy as possible, in any format you want.
For more informaion see https://ocds-downloads.opendata.coop/ which uses this tool to create all the downloads.
Contains a CLI tool to download OCDS data into a PostgreSQL database and an Airflow pipline to make exports of this database to various formats.
Clone this repo.
pip install -r requirements.txt
A PostgreSQL database is needed with a connection string to it.
Set environment variable DATABASE_URL as the connection string)
export DATABASE_URL=postgresql://user:password@localhost/dbname
Only one CLI command is needed:
python ocdsdata.py import_scraper <scraper> <schema>
This will scrape data from a defined in Kingfisher Collect Documentation and import it into a the PostgreSQL . For example to import zambia OCDS data into a PostgreSQL schema named my_zambia_data run:
python ocdsdata.py import_scraper zambia my_zambia_data
This process may take a long time.
TODO
TODO
To update requirements run:
pip-compile requirements.in kingfisher-collect/requirements.txt --output-file=requirements.txt