Merge branch 'develop'

README.md

@@ -46,6 +46,14 @@ Getting Started
 4. Install TANNER: ``python3 setup.py install``
 5. Run TANNER: ``sudo tanner``
 
+### Run Tanner Api
+
+Run ``sudo tannerapi``
+
+### Run Tanner WebUI
+
+Run ``sudo tannerweb``
+
 You obviously want to bind to 0.0.0.0 when running in <i>production</i> and on a different host than SNARE (recommended).
 
 [See the docs for more info](docs/source/index.rst)

bin/tannerapi

@@ -0,0 +1,9 @@
+#!/usr/bin/python3.5
+from tanner.api import server
+
+def main():
+    api = server.ApiServer()
+    api.start()
+
+if __name__ == "__main__":
+    main()

bin/tannerweb

@@ -0,0 +1,9 @@
+#!/usr/bin/python3.5
+from tanner.web import server
+
+def main():
+    tannerweb = server.TannerWebServer()
+    tannerweb.start()
+
+if __name__ == "__main__":
+    main()

docs/source/api.rst

@@ -0,0 +1,43 @@
+Tanner API
+==========
+Tanner api provides various stats related to traffic captured by snare. It can be accessed at ``locahost:8092/``.
+
+/
+~~~~
+This is the index page which shows ``tanner api``.
+
+/snares
+~~~~~~~~~~
+This shows all the snares' uuid.
+
+/snare/<snare-uuid>
+~~~~~~~~~~~~~~~~~~~~~~
+Replace ``<snare-uuid>`` with a valid `snare-uuid` and it will show all the sessions related to that ``snare-uuid`` and their details.
+
+/snare-stats/<snare-uuid>
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Replace ``<snare-uuid>`` with a valid `snare-uuid` and it will show some stats.
+
+	* No of sessions in the sanre
+	* Total duration for which snare remains active
+	* Attack frequency, which shows no of sessions which face different attacks.
+
+/<snare-uuid>/sessions?filters=<filters>
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+This shows all the sessions' uuid which follow the filters.
+Filters are sepatated by ``white-space`` and name-value pair are separated by ``:``. E.g ``?filters=filter1:value1 filter2:value2``.
+
+It supports 5 filters:
+
+	* **peer_ip** -- Sessions with given ip. E.g ``peer_ip:127.0.0.1 ``
+	* **user-agent** -- Sessions with given user-agent. E.g ``user-agent:Chrome``
+	* **attack_types** -- Sessions with given attack type such as lfi, rfi, xss, cmd_exec, sqli. E.g ``attack_types:lfi``
+	* **possible_owners** -- Sessions with given owner type such as user, tool, crawler, attacker. E.g ``possible_owners:attacker``
+	* **start_time** -- Sessions which started after `start_time`. E.g ``start_time:1480560``
+	* **end_time** -- Sessions which ended before `end_time`. E.g ``end_time:1480560``
+
+Multiple filters can be applied as ``peer_ip:127.0.0.1 start_time:1480560 possible_owners:attacker``
+
+/api/session/<sess-uuid>
+~~~~~~~~~~~~~~~~~~~~~~~~
+It gives all information about the session with given uuid.

docs/source/config.rst

@@ -13,6 +13,14 @@ There are 8 different sections :
 
     :Host: The host at which Tanner is running
     :Port: The port at which Tanner is running
+  * **WEB**
+
+    :Host: The host at which Tanner Web UI is running
+    :Port: The port at which Tanner Web UI is running
+  * **API**
+
+    :Host: The host at which Tanner API is running
+    :Port: The port at which Tanner API is running
   * **REDIS**
 
     :Host: The host address at which redis is running
@@ -22,6 +30,7 @@ There are 8 different sections :
   * **EMULATORS**
 
     :root_dir: The root directory for emulators that need data storing such as SQLI and LFI. Data will be stored in this directory
+    :emulator_enabled: This tells which emulators are enabled.
   * **SQLI**
 
     :db_name: THe name of database used in SQLI emulator
@@ -53,8 +62,12 @@ If no file is specified, following json will be used as default:
               'user_dorks': '/opt/tanner/data/user_dorks.pickle',
               'vdocs': '/opt/tanner/data/vdocs.json'},
      'TANNER': {'host': '0.0.0.0', 'port': 8090},
+     'WEB': {'host': '0.0.0.0', 'port': 8091},
+     'API': {'host': '0.0.0.0', 'port': 8092},
      'REDIS': {'host': 'localhost', 'port': 6379, 'poolsize': 80, 'timeout': 1},
-     'EMULATORS': {'root_dir': '/opt/tanner'},
+     'EMULATORS': {'root_dir': '/opt/tanner',
+                    'emulator_enabled': {'sqli': True, 'rfi': True, 'lfi': True, 'xss': True, 'cmd_exec': True}
+                  },
      'SQLI': {'type':'SQLITE', 'db_name': 'tanner_db', 'host':'localhost', 'user':'root', 'password':'user_pass'},
      'DOCKER': {'host_image': 'busybox:latest'},
      'LOGGER': {'log_file': '/opt/tanner/tanner.log'},

docs/source/index.rst

@@ -17,6 +17,8 @@ Contents:
    storage
    dorks
    config
+   api
+   web
 
 
 

docs/source/quick-start.rst

@@ -33,4 +33,14 @@ Setup and run TANNER
 #. Go to the tanner source directory ``cd tanner``
 #. Install requirements: ``pip3 install -r requirements.txt``
 #. Install tanner ``python3 setup.py install``
-#. Run TANNER: ``sudo tanner``
+#. Run TANNER: ``sudo tanner``
+
+Run Tanner Api
+""""""""""""""
+
+#. Run ``sudo tannerapi``
+
+Run Tanner WebUI
+""""""""""""""""
+
+#. Run ``sudo tannerweb``

docs/source/sessions.rst

@@ -11,7 +11,7 @@ Session class accepts ``data`` as a parameter. The ``data`` came from SNARE and
     * **ip** -- peer ip address.
     * **port** -- peer port.
     * **user_agent** -- peer user agent.
-    * **sensor** -- SNARE sensor uuid.
+    * **snare_uuid** -- SNARE sensor uuid.
     * **paths** -- list of dictionaries. Contains ``path``, ``timestamp``, ``attack_type`` and SNARE ``response status``.
     * **sess_uuid** -- randomly generated session uuid.
     * **start_timestamp** -- session start time.
@@ -50,7 +50,7 @@ The result contains next fields:
     * **peer_ip**
     * **peer_port**
     * **user_agent**
-    * **sensor_uuid**
+    * **snare_uuid**
     * **start_time**
     * **cookies**
 * **end_time** -- last session timestamp

docs/source/web.rst

@@ -0,0 +1,46 @@
+Tanner WEB
+==========
+Tanner WEB provides various stats related to traffic captured by snare in UI form. It can be accessed at ``locahost:8091/``.
+
+/
+~~~~
+This is the index page which shows ``Tanner Web``.
+
+/snares
+~~~~~~~~~~ 
+This shows all the snares' uuid. Each snare object is clickable. Clicking displays the page **/snare/<snare-uuid>**
+
+/snare/<snare-uuid>
+~~~~~~~~~~~~~~~~~~~~~~
+Replace ``<snare-uuid>`` with a valid `snare-uuid` and it will provide two options:
+	* **Snare-Stats** -- It will move you to **/snare-stats/<snare-uuid>**
+	* **Sessions** -- It will move you to **/<snare-uuid>/sessions**
+
+/snare-stats/<snare-uuid>
+~~~~~~~~~~~~~~~~~~~~~~~~~
+This page shows some general stats about the snare
+
+	* **No of Sessions** - Total no of sessions of the snare
+	* **Total Duration** - Total durations during which sessions remain active
+	* **Attack Frequency** - Frequency of different attacks made on the snare
+
+/<snare-uuid>/sessions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+This shows all the sessions' uuid. Each is clickable. Clicking displays **/session/<sess-uuid>**
+Filters can be on the sessions using the input box and clicking the ``Apply`` button.
+Filters are sepatated by ``white-space`` and name-value pair are separated by ``:``. E.g ``filter1:value1 filter2:value2``.
+
+It supports 5 filters:
+	* **peer_ip** -- Sessions with given ip. E.g ``peer_ip:127.0.0.1 ``
+	* **user-agent** -- Sessions with given user-agent. E.g ``user-agent:Chrome``
+	* **attack_types** -- Sessions with given attack type such as lfi, rfi, xss, cmd_exec, sqli. E.g ``attack_types:lfi``
+	* **possible_owners** -- Sessions with given owner type such as user, tool, crawler, attacker. E.g ``possible_owners:attacker``
+	* **start_time** -- Sessions which started after `start_time`. E.g ``start_time:1480560``
+	* **end_time** -- Sessions which ended before `end_time`. E.g ``end_time:1480560``
+
+Multiple filters can be applied as ``peer_ip:127.0.0.1 start_time:1480560 possible_owners:attacker``
+
+/session/<sess-uuid>
+~~~~~~~~~~~~~~~~~~~~~~~~
+It gives all information about the session with given uuid. Here you may find some of the text clickable such as 
+``peer_ip``,``possible_owners``, ``start_time``, ``end_time``, ``attack_types``. Clicking on them will display all the sessions will same attribute value.

requirements.txt

@@ -1,5 +1,6 @@
 aiohttp>=2.0
 aiomysql
+aiohttp_jinja2
 docker
 elizabeth==0.3.27
 yarl
@@ -8,3 +9,4 @@ asyncio_redis
 uvloop
 pymongo
 pylibinjection
+jinja2

setup.py

@@ -9,6 +9,6 @@
       author_email='[email protected]',
       url='https://github.com/mushorg/tanner',
       packages=find_packages(exclude=['*.pyc']),
-      scripts=['bin/tanner'],
+      scripts=['bin/tanner', 'bin/tannerweb', 'bin/tannerapi'],
       data_files=[('/opt/tanner/data/',['tanner/data/dorks.pickle'])]
       )

tanner/api/api.py

@@ -0,0 +1,111 @@
+import json
+import logging
+import operator
+import asyncio
+import asyncio_redis
+
+class Api:
+    def __init__(self, redis_client):
+        self.logger = logging.getLogger('tanner.api.Api')
+        self.redis_client = redis_client
+
+    async def return_snares(self):
+        query_res = []
+        try:
+            query_res = await self.redis_client.smembers('snare_ids')
+            query_res = await query_res.asset()
+        except asyncio_redis.NotConnectedError as connection_error:
+            self.logger.error('Can not connect to redis %s', connection_error)
+        return list(query_res)
+
+    async def return_snare_stats(self, snare_uuid):
+        result = {}
+        sessions = await self.return_snare_info(snare_uuid)
+        if sessions == 'Invalid SNARE UUID':
+            return result
+
+        result['total_sessions'] = len(sessions)
+        result['total_duration'] = 0
+        result['attack_frequency'] = {'sqli' : 0,
+                                      'lfi' : 0,
+                                      'xss' : 0,
+                                      'rfi' : 0,
+                                      'cmd_exec' : 0}
+
+        for sess in sessions:
+            result['total_duration'] += sess['end_time'] - sess['start_time']
+            for attack in sess['attack_types']:
+                if attack in result['attack_frequency']:
+                    result['attack_frequency'][attack] += 1
+
+        return result
+
+    async def return_snare_info(self, uuid, count=-1):
+        query_res = []
+        try:
+            query_res = await self.redis_client.lrange_aslist(uuid, 0, count)
+        except asyncio_redis.NotConnectedError as connection_error:
+            self.logger.error('Can not connect to redis %s', connection_error)
+        else:
+            if not query_res:
+                return 'Invalid SNARE UUID'
+            for (i, val) in enumerate(query_res):
+                query_res[i] = json.loads(val)
+        return query_res
+
+    async def return_session_info(self, sess_uuid, snare_uuid= None):
+        query_res = []
+        if snare_uuid:
+            snare_uuids = [snare_uuid]
+        else:
+            snare_uuids = await self.return_snares()
+
+        for snare_id in snare_uuids:
+            sessions = await self.return_snare_info(snare_id)
+            if sessions == 'Invalid SNARE UUID':
+                continue
+            for sess in sessions:
+                if sess['sess_uuid'] == sess_uuid:
+                    return sess
+
+    async def return_sessions(self, filters):
+        query_res = []
+        snare_uuids = await self.return_snares()
+
+        matching_sessions = []
+        for snare_id in snare_uuids:
+            result = await self.return_snare_info(snare_id)
+            if result == 'Invalid SNARE UUID':
+                return 'Invalid filter : SNARE UUID'
+            sessions = result
+            for sess in sessions:
+                match_count = 0
+                for filter_name, filter_value in filters.items():
+                    try:
+                        if(self.apply_filter(filter_name, filter_value, sess)):
+                            match_count += 1
+                    except KeyError:
+                        return 'Invalid filter : %s' % filter_name
+
+                if match_count == len(filters):
+                    matching_sessions.append(sess) 
+
+        return matching_sessions
+
+    def apply_filter(self, filter_name, filter_value, sess):
+        available_filters = {'user_agent' : operator.contains,
+                             'peer_ip' : operator.eq,
+                             'attack_types' : operator.contains,
+                             'possible_owners' : operator.contains,
+                             'start_time' : operator.le,
+                             'end_time': operator.ge,
+                             'snare_uuid' : operator.eq
+                            }
+
+        try:
+            if available_filters[filter_name] is operator.contains:
+                return available_filters[filter_name](sess[filter_name], filter_value)
+            else:
+                return available_filters[filter_name](filter_value, sess[filter_name])
+        except KeyError:
+            raise