CLI : Add data directly without running server.

[suryatejreddy]

Fixes #167

Checklist

• My branch is up-to-date with upstream/develop branch.
• Everything works and tested for Python 3.5.2 and above.

Description

Added a new option to the cli called additems that allows the user to load contents to the database directly without starting the server. The option expects a json file and the API doc .

As of now the following format of data is supported,
[{"Prop1":"Value1","Prop2":"Value2","@type":"dummyClass"},{"Prop1":"Value1","Prop2":"Value2","@type":"dummyClass"}]

Support for more data formats can be added later.

Test Logs

[chrizandr]

Please make sure your code is PEP8 compliant.

[suryatejreddy]

@chrizandr I have made the required changes. Although, I have one doubt regarding the SQL Session. I am using the crud methods to add data to the database. So does the DB get reset every time we start the server, or uses the previous state?

[chrizandr]

That depends on the database that you use. If the user had not specified the database, we use a temporary one using sqlite. This one will be deleted once the session is closed. This is an issue, good that you noticed it. We need to come up with a workaround for this. Maybe use the same session with which you add data, for the server.

[suryatejreddy]

@chrizandr to fix the database issue, i have added another option to the CLI while adding data from json, the user will have to mention the database url to which the data must be added. This way, the data will not be lost. What are your views on this?

[Mec-iS]

Note: we don't have a database configuration file yet? Database choice should be defined at deploy time (sqlite by default). The app should know already where its storage is, without the CLI user to specify.

[suryatejreddy]

@Mec-iS this is only for the adddata option. When a user wants to start the server using the CLI, the default sqlite is used in case he doesn't explicitly mention a database. Is that what you meant or am i missing something?

[Mec-iS]

Something like:

1. Load the database configuration from the database configuration file (the same that tells the app where the database is, database credentials, etc.)
2. Open the database with SQLAlchemy and insert the data after reading from a file

First thing to do: have/locate the variable/module where the database configuration is stored.
Do we have a module from which we load database connection information?

[suryatejreddy]

As of now, we take input of the database url, for example, sqlite:///:memory:, and then access it using

hydrus/main.py

Line 30 in 42612ef

engine = create_engine(DB_URL)

.

[Mec-iS]

That's what I meant. Database's uri and credentials have to be moved to a configuration file where all the other modules can read them. This configuration variables have to be set at deploy time (by Docker?). Please open the issues to accomplish this layout.

@xadahiya @chrizandr ^

[EDIT] For example, they can also be stored in a text file, loaded into variables at runtime for now.

[xadahiya]

Ideally, the configuration variables should be stored as environment variables. You can provide defaults but they should only be used when environment variables are not present.
Using env variables is most appropriate because in most cases these variables will contain sensitive data like database credentials, database URI etc.

One interesting thing that can be done is using the CLI to set env variables.
For example -
Suppose the database credential env variable is not set then if someone tries to run hydrus we can show a warning just like git does when global user.name and user.email are not set.

This could easily be implemented with Docker as well.

[suryatejreddy]

Are there any other changes that need to be made? @chrizandr

[Mec-iS]

1. Every method shall have a compelling documentation as defined here
2. Type annotations has to be added to methods' signature

[ashwani99]

@suryatejreddy Please consider adding a "Changes log" section in your PR comment to follow this PR template

Also you can consider squashing your commits as there are multiple commits with same commit message.

[suryatejreddy]

Thanks for the review @ashwani99 , @Mec-iS . I am working on the changes.

[py-ranoid]

@suryatejreddy Have the changes been made ?

[suryatejreddy]

@py-ranoid All the necessary changes have been made. I am in the testing phase now.