Reading SQL database(Currently just MsSql) data and forwarding it as an event on UDP socket. This app is able to run in two different mode(Master&Slave). Master node will try to query the database according to config.toml specification. Another agent can be run on different node in slave mode. slave node at first try to find it's master and sync it's config by master. so any change in config should just be done on the master node.
If for any reason master node goes down, slave node will go to master mode and continue the operation until master node come back. These three files should be created for proper operation:
touch /var/siem-db-connector/config.toml
touch /var/siem-db-connector/log/log
touch /var/siem-db-connector/db_track_chnage.json #
Sample config.toml configuration is available in this repo. This app is able to create logs by complicated query specification in its config.toml. This kind of config allows the user to merge the result of multiple query in one event. All query and network operation of this app is handled by async function so this app is able to run multiple queries and forward their results with minimum resource usage.
Indicate IP address and listening port for the app, The app use this port to communicate with its partner
listening_addr = "10.235.20.1:8080"
IP address and listening port of the partner
peer_addr= "10.235.20.20:8080"
Unix socket path to prevent running multiple instance
app_socket="/tmp/db"
Pause time between agent communication, and config sync
ping_duration = 2000
Default app role it could be {master, slave}
default_role = "master"
If the user did not specify log server per log_source the app use this address as default log server
log_server = "127.0.0.1:10100"
Log source name
name="test"
Mssql server ip address
addr = "192.168.77.129"
Mssql server port number
port= 1433
Username with proper access to run query
username= "test"
Password
pass= "123"
The query that should run on DB, with notations like id>?? we can aware the app for using this field as counter
query= "select * FROM [test].[dbo].[identity] where id>??"
Counter field name
counter_field = "id"
For first time the app use this value as counter default value, after that the new value will be store in db_track_change.json
counter_default_value="0"
Any row readed from DB will be sent on udp to this list, it's possible to define multiple server by comma(,) separation
log_server="127.0.0.1:10200,127.0.0.1:10201"
Defining pause time between query call
pause_duration = 2000
Log mode can have these values {Both,Net,File} Net just sent log over udp, File store recieved log as csv on specified path in path property
log_mode="Both"
Specify the location of csv file for saving log, this propert only works if log_mode set to File or Both.
path="/tmp/test.csv"
Add a date field at the begenning of each event by the value of current local time&date
set_current_time=1
Comp stands for complicated, this property define the name of complicated query this name should be unique
name="comp1"
Define result format, comp1 will be replace with the result of [[comp.log_sources]].name="comp1" and so on
result="compq1, cat=compq2,"
This is a log_sources object, full description of properties is available at above(num 2). There is only 1 difference, in complicated queries log_sources the app just use first returned row to generated final result even if the query return more than 1 row.
[[comp.log_sources]]
name="compq1"
addr = "192.168.77.129"
port= 1433
username= "test"
pass= "123"
query= "SELECT title as log_field_name FROM [test].[dbo].[cat] where id=1"
set_current_time=1