Prepare 2020.12.31 Release (#72)

- Make log messages for textfsm and genie parsers failing to parse consistent as `log.warning`
- Add factory example
- Add "root" priv level to junos driver -- probably should be considered experimental for now :)
- Fix issue where `send_config` unified result did not have finish time set
- **POSSIBLY BREAKING CHANGE:** logger names have changed to be easier to get/make more sense -- the logger for each instance used to look like: "scrapli-channel-{{ HOST }}" which kinda was not really smart :). Loggers now look like: "scrapli.{{ HOST }}:{{ PORT }}.channel" -- can be channel|driver|transport!
- Changes to test environment:
  - Support running devices on localhost w/ nat'd management ports -- in "vrouter" mode (poorly named) -- this is enabled with the `SCRAPLI_VROUTER` environment variable set to on/true/something
  - Added bootvar into nxos base config -- when missing causes qemu nxosv to boot into loader prompt so thats no good
  - Replace resource settings in vdc in nxos to account for nxos instances with differing resources (memory/cpu)
  - Got rid of static license udi in iosxe config, replaced more certificate stuff so show run comparisons are easier on iosxe
- **NEW TRANSPORT** `asynctelnet` transport is built using standard library asyncio, as such it is part of scrapli core
  - Should be considered beta for a while :)
  - Added a bunch of tests mocking streamreader/writer to ensure that this driver is well tested
- Added asynctelnet support in nxos and juniper drivers (to change prompt for those platforms)
- Support asynctelnet in base driver
- `auth_bypass` for both telnet drivers completely bypasses not only auth (as it did previously) but also the auth validation where we confirm we got logged in successfully -- reason being is that for console servers and such you may not care about that, you may just want to log in and read data.
- Removed unnecessary re-checking/verifying of ssh config file in system transport (was basically duplicated from base transport, so was pointless!)
- Bumped all the default timeout values up as they were probably a bit on the aggressive side
- Added `eager` argument to send commands/commands from file and config/configs/configs from file methods -- basically this `eager` mode will *not* look for a prompt between lines of commands/configs. This means that things have potential to get out of whack because we will just send things as fast as possible. In order to not totally break things we *will* (whether you like it or not!) wait and find the prompt on the last command/config in the list though -- that way we dont get too out of whack. This now means we can use `eager` to configure banners and macros and things and we no longer need to do the dirty send interactive workaround.
- Added `ScrapliConnectionLost` exception and raise it if we get EOF in system transport -- with a message that is more clear than just "EOF" and some obscure line in ptyprocess!
- Added `tclsh` privilege level for IOSXE
- Fixed a bug that would prevent going to "parallel" privilege levels -- i.e. going from tclsh to configuration or visa versa in IOSXE or from configuration to configuration_exclusive in IOSXR
- If no `failed_when_contains` is passed to `send_interactive` network drivers will now use the network drivers `failed_when_contains` attribute to bring it inline with the normal command/config methods
- Added `timeout_ops` to `send_interactive` and wrap those methods with the `TimeoutModifier` decorator
- Add logic to properly fetch socket address family type so we can handle IPv6 hosts (w/ scrapli-ssh2/scrapli-paramiko)
- Added `tclsh` privilege level for NXOS, didn't even know that existed before!

CHANGELOG.md

@@ -1,6 +1,50 @@
 CHANGELOG
 =======
 
+# 2020.12.31
+- Make log messages for textfsm and genie parsers failing to parse consistent as `log.warning`
+- Add factory example
+- Add "root" priv level to junos driver -- probably should be considered experimental for now :)
+- Fix issue where `send_config` unified result did not have finish time set
+- **POSSIBLY BREAKING CHANGE:** logger names have changed to be easier to get/make more sense -- the logger for each 
+  instance used to look like: "scrapli-channel-{{ HOST }}" which kinda was not really smart :). Loggers now look 
+  like: "scrapli.{{ HOST }}:{{ PORT }}.channel" -- can be channel|driver|transport!
+- Changes to test environment:
+  - Support running devices on localhost w/ nat'd management ports -- in "vrouter" mode (poorly named) -- this is 
+    enabled with the `SCRAPLI_VROUTER` environment variable set to on/true/something
+  - Added bootvar into nxos base config -- when missing causes qemu nxosv to boot into loader prompt so thats no good
+  - Replace resource settings in vdc in nxos to account for nxos instances with differing resources (memory/cpu)
+  - Got rid of static license udi in iosxe config, replaced more certificate stuff so show run comparisons are 
+    easier on iosxe
+- **NEW TRANSPORT** `asynctelnet` transport is built using standard library asyncio, as such it is part of scrapli core
+  - Should be considered beta for a while :)
+  - Added a bunch of tests mocking streamreader/writer to ensure that this driver is well tested
+- Added asynctelnet support in nxos and juniper drivers (to change prompt for those platforms)
+- Support asynctelnet in base driver
+- `auth_bypass` for both telnet drivers completely bypasses not only auth (as it did previously) but also the auth 
+  validation where we confirm we got logged in successfully -- reason being is that for console servers and such you 
+  may not care about that, you may just want to log in and read data.
+- Removed unnecessary re-checking/verifying of ssh config file in system transport (was basically duplicated from 
+  base transport, so was pointless!)
+- Bumped all the default timeout values up as they were probably a bit on the aggressive side
+- Added `eager` argument to send commands/commands from file and config/configs/configs from file methods -- 
+  basically this `eager` mode will *not* look for a prompt between lines of commands/configs. This means that things 
+  have a tiny potential to get out of whack because we will just send things as fast as possible. In order to not 
+  totally break things we *will* (whether you like it or not!) wait and find the prompt on the last command/config 
+  in the list though -- that way we dont get too out of whack. This now means we can use `eager` to configure 
+  banners and macros and things and we no longer need to do the dirty send interactive workaround.
+- Added `ScrapliConnectionLost` exception and raise it if we get EOF in system transport -- with a message that is 
+  more clear than just "EOF" and some obscure line in ptyprocess!
+- Added `tclsh` privilege level for IOSXE
+- Fixed a bug that would prevent going to "parallel" privilege levels -- i.e. going from tclsh to configuration or 
+  visa versa in IOSXE or from configuration to configuration_exclusive in IOSXR
+- If no `failed_when_contains` is passed to `send_interactive` network drivers will now use the network drivers 
+  `failed_when_contains` attribute to bring it inline with the normal command/config methods
+- Added `timeout_ops` to `send_interactive` and wrap those methods with the `TimeoutModifier` decorator
+- Add logic to properly fetch socket address family type so we can handle IPv6 hosts (w/ scrapli-ssh2/scrapli-paramiko)
+- Added `tclsh` privilege level for NXOS, didn't even know that existed before!
+
+
 # 2020.11.15
 - Fix a regex that sometimes caused a failed functional IOSXR test
 - Add `ptyprocess` transport options for system transport -- sounds like this may be needed for huawei community

README.md

@@ -183,19 +183,27 @@ scrapli is built primarily in three parts: transport, channel, and driver. The t
  providing a file-like interface to the target server. The channel layer is responsible for reading and writing
   to the provided file-like interface. Finally, the driver provides the user facing API/interface to scrapli.
 
-There are two available "transports" in scrapli "core" -- both of which inherit from a base transport class
+There are three available "transports" in scrapli "core" -- all of which inherit from a base transport class
  and provide the same file-like interface to the upstream channel. There are also (currently!) three transport plugins
   available -- all of which are installable as optional extras. The transport options are:
 
-- [paramiko](https://github.com/paramiko/paramiko) (optional extra)
-- [ssh2-python](https://github.com/ParallelSSH/ssh2-python) (optional extra)
-- OpenSSH/System available SSH (scrapli core)
-- telnetlib (scrapli core)
-- [asyncssh](https://github.com/ronf/asyncssh) (optional extra)
+#### "Core"
+
+- "system" -- OpenSSH/System available SSH binary
+- "telnet" -- Python standard library telnetlib
+- "asynctelnet" -- Python standard library asyncio stream
+
+
+#### Optional Extra/Plugin
+
+- [asyncssh](https://github.com/ronf/asyncssh)
+- [ssh2-python](https://github.com/ParallelSSH/ssh2-python)
+- [paramiko](https://github.com/paramiko/paramiko)
+
 
 A good question to ask at this point is probably "why?". Why multiple transport options? Why not just use paramiko
  like most folks do? Historically the reason for moving away from paramiko was simply speed. ssh2-python is a wrapper
-  around the libssh2 C library, and as such is very very fast. In a prior project
+  around the libssh2 C library, and as such is very, very fast. In a prior project
    ([ssh2net](https://github.com/carlmontanari/ssh2net)), of which scrapli is the successor/evolution, ssh2-python
     was used with great success, however, it is a bit feature-limited, and development had stalled around the same
      time scrapli was getting going.
@@ -217,10 +225,12 @@ With the goal of supporting all of the OpenSSH configuration options the primary
 Adding telnet support via telnetlib was trivial, as the interface is basically the same as SystemSSH, and it turns out
  telnet is still actually useful for things like terminal servers and the like!
 
-Finally, the most recent scrapli transport plugin is the `asyncssh` transport. This transport option represents a
- very big change for scrapli as the entire "backend" was basically re-worked in order to provide the exact same API
-  for both synchronous and asynchronous applications. Currently asyncssh is the only asynchronous transport supported
-  , but of course there could be additional transports (telnetlib3 perhaps?) in the future!
+Next, perhaps the most interesting scrapli transport plugin is the `asyncssh` transport. This transport option 
+represented a very big change for scrapli as the entire "backend" was basically re-worked in order to provide the 
+exact same API for both synchronous and asynchronous applications.
+
+Lastly, the `asynctelnet` transport is the latest (and perhaps last?!) transport plugin. This transport plugin was 
+built with only the python standard library (just like system/telnet) and as such it is part of scrapli "core".
 
 The final piece of scrapli is the actual "driver" -- or the component that binds the transport and channel together and
  deals with instantiation of a scrapli object. There is a "base" driver object -- `Scrape` -- which provides essentially

codecov.yaml

@@ -6,4 +6,4 @@ codecov:
 
 coverage:
   precision: 2
-  range: "85...100"
+  range: "75...90"

docs/PUBLIC_API_STATUS.md

@@ -30,10 +30,10 @@ Note that all public methods, unless otherwise noted, are available in sync and
 |-------------------------------|-------------|-------------|--------------------------------------------------------------|
 | get_prompt                    | 2020.03.29  |             |                                                              |
 | send_command                  | 2020.03.29  | 2020.08.09  | added `timeout_ops` keyword argument to modify timeout       |
-| send_commands                 | 2020.03.29  | 2020.08.09  | added `timeout_ops` keyword argument to modify timeout       |
-| send_commands_from_file       | 2020.04.30  | 2020.08.09  | added `timeout_ops` keyword argument to modify timeout       |
-| send_interactive              | 2020.03.29  | 2020.04.11  | changed to support list of "events" to interact with         |
-| send_and_read                 | 2020.08.28  |             |                                                              |                            
+| send_commands                 | 2020.03.29  | 2020.12.31  | added `eager` keyword argument                               |
+| send_commands_from_file       | 2020.04.30  | 2020.12.31  | added `eager` keyword argument                               |
+| send_interactive              | 2020.03.29  | 2020.12.31  | added `timeout_ops` keyword argument                         |
+| send_and_read                 | 2020.08.28  |             |                                                              |
 
 
 ### AsyncGenericDriver (and NetworkDriver sub-classes unless overridden)
@@ -42,9 +42,9 @@ Note that all public methods, unless otherwise noted, are available in sync and
 |-------------------------------|-------------|-------------|--------------------------------------------------------------|
 | get_prompt                    | 2020.06.06  |             |                                                              |
 | send_command                  | 2020.06.06  | 2020.08.09  | added `timeout_ops` keyword argument to modify timeout       |
-| send_commands                 | 2020.06.06  | 2020.08.09  | added `timeout_ops` keyword argument to modify timeout       |
-| send_commands_from_file       | 2020.06.06  | 2020.08.09  | added `timeout_ops` keyword argument to modify timeout       |
-| send_interactive              | 2020.06.06  |             |                                                              |
+| send_commands                 | 2020.06.06  | 2020.12.31  | added `eager` keyword argument                               |
+| send_commands_from_file       | 2020.06.06  | 2020.12.31  | added `eager` keyword argument                               |
+| send_interactive              | 2020.06.06  | 2020.12.31  | added `timeout_ops` keyword argument                         |
 | send_and_read                 | 2020.08.28  |             |                                                              |
 
 
@@ -55,10 +55,10 @@ Note that all public methods, unless otherwise noted, are available in sync and
 | update_privilege_levels       | 2020.05.09  |             | update priv map/all prompt pattern if adding/modifying privs |
 | acquire_priv                  | 2020.03.29  |             |                                                              |
 | register_configuration_session| 2020.05.09  |             | register a config session so the priv level can be tracked   |
-| send_config                   | 2020.05.09  | 2020.08.09  | added `timeout_ops` keyword argument to modify timeout       |
-| send_configs                  | 2020.03.29  | 2020.08.09  | added `timeout_ops` keyword argument to modify timeout       |
-| send_configs_from_file        | 2020.04.30  | 2020.08.09  | added `timeout_ops` keyword argument to modify timeout       |
-| send_interactive              | 2020.03.29  | 2020.05.09  | added `privilege_level` argument                             |
+| send_config                   | 2020.05.09  | 2020.12.31  | added `eager` keyword argument                               |
+| send_configs                  | 2020.03.29  | 2020.12.31  | added `eager` keyword argument                               |
+| send_configs_from_file        | 2020.04.30  | 2020.12.31  | added `eager` keyword argument                               |
+| send_interactive              | 2020.03.29  | 2020.12.31  | added `timeout_ops` keyword argument                         |
 
 
 ### AsyncNetworkDriver (and Platform driver sub-classes unless overridden)
@@ -68,18 +68,18 @@ Note that all public methods, unless otherwise noted, are available in sync and
 | update_privilege_levels       | 2020.06.06  |             |                                                              |
 | acquire_priv                  | 2020.06.06  |             |                                                              |
 | register_configuration_session| 2020.06.06  |             |                                                              |
-| send_config                   | 2020.06.06  | 2020.08.09  | added `timeout_ops` keyword argument to modify timeout       |
-| send_configs                  | 2020.06.06  | 2020.08.09  | added `timeout_ops` keyword argument to modify timeout       |
-| send_configs_from_file        | 2020.06.06  | 2020.08.09  | added `timeout_ops` keyword argument to modify timeout       |
-| send_interactive              | 2020.06.06  |             |                                                              |
+| send_config                   | 2020.06.06  | 2020.12.31  | added `eager` keyword argument                               |
+| send_configs                  | 2020.06.06  | 2020.12.31  | added `eager` keyword argument                               |
+| send_configs_from_file        | 2020.06.06  | 2020.12.31  | added `eager` keyword argument                               |
+| send_interactive              | 2020.06.06  | 2020.12.31  | added `timeout_ops` keyword argument                         |
 
 
 ## Channel
 
 | Method                        | Implemented | Last Change | Notes                                                        |
 |-------------------------------|-------------|-------------|--------------------------------------------------------------|
 | get_prompt                    | 2020.03.29  |             |                                                              |
-| send_input                    | 2020.03.29  |             |                                                              |
+| send_input                    | 2020.03.29  | 2020.12.31  | added `eager` keyword argument                               |
 | send_inputs_interact          | 2020.03.29  | 2020.04.11  | changed to support list of "events" to interact with         |
 | send_input_and_read           | 2020.08.28  |             |                                                              |
 
@@ -89,7 +89,7 @@ Note that all public methods, unless otherwise noted, are available in sync and
 | Method                        | Implemented | Last Change | Notes                                                        |
 |-------------------------------|-------------|-------------|--------------------------------------------------------------|
 | get_prompt                    | 2020.06.06  |             |                                                              |
-| send_input                    | 2020.06.06  |             |                                                              |
+| send_input                    | 2020.06.06  | 2020.12.31  | added `eager` keyword argument                               |
 | send_inputs_interact          | 2020.06.06  |             |                                                              |
 | send_input_and_read           | 2020.08.28  |             |                                                              |
 

docs/scrapli/channel/async_channel.html

@@ -3,7 +3,7 @@
 <head>
 <meta charset="utf-8">
 <meta name="viewport" content="width=device-width, initial-scale=1, minimum-scale=1" />
-<meta name="generator" content="pdoc 0.9.1" />
+<meta name="generator" content="pdoc 0.9.2" />
 <title>scrapli.channel.async_channel API documentation</title>
 <meta name="description" content="scrapli.channel.async_channel" />
 <link rel="preload stylesheet" as="style" href="https://cdnjs.cloudflare.com/ajax/libs/10up-sanitize.css/11.0.1/sanitize.min.css" integrity="sha256-PK9q560IAAa6WVRRh76LtCaI8pjTJ2z11v0miyNNjrs=" crossorigin>
@@ -983,7 +983,7 @@ <h4><code><a title="scrapli.channel.async_channel.AsyncChannel" href="#scrapli.c
 </nav>
 </main>
 <footer id="footer">
-<p>Generated by <a href="https://pdoc3.github.io/pdoc"><cite>pdoc</cite> 0.9.1</a>.</p>
+<p>Generated by <a href="https://pdoc3.github.io/pdoc"><cite>pdoc</cite> 0.9.2</a>.</p>
 </footer>
 </body>
 </html>

docs/scrapli/channel/base_channel.html

@@ -3,7 +3,7 @@
 <head>
 <meta charset="utf-8">
 <meta name="viewport" content="width=device-width, initial-scale=1, minimum-scale=1" />
-<meta name="generator" content="pdoc 0.9.1" />
+<meta name="generator" content="pdoc 0.9.2" />
 <title>scrapli.channel.base_channel API documentation</title>
 <meta name="description" content="scrapli.channel.base_channel" />
 <link rel="preload stylesheet" as="style" href="https://cdnjs.cloudflare.com/ajax/libs/10up-sanitize.css/11.0.1/sanitize.min.css" integrity="sha256-PK9q560IAAa6WVRRh76LtCaI8pjTJ2z11v0miyNNjrs=" crossorigin>
@@ -91,7 +91,7 @@ <h1 class="title">Module <code>scrapli.channel.base_channel</code></h1>
             N/A
 
         &#34;&#34;&#34;
-        self.logger = getLogger(f&#34;scrapli.channel-{transport.host}&#34;)
+        self.logger = getLogger(f&#34;scrapli.{transport.host}:{transport.port}.channel&#34;)
         attach_duplicate_log_filter(logger=self.logger)
 
         self.transport = transport
@@ -361,7 +361,7 @@ <h2 id="raises">Raises</h2>
             N/A
 
         &#34;&#34;&#34;
-        self.logger = getLogger(f&#34;scrapli.channel-{transport.host}&#34;)
+        self.logger = getLogger(f&#34;scrapli.{transport.host}:{transport.port}.channel&#34;)
         attach_duplicate_log_filter(logger=self.logger)
 
         self.transport = transport
@@ -565,7 +565,7 @@ <h4><code><a title="scrapli.channel.base_channel.ChannelBase" href="#scrapli.cha
 </nav>
 </main>
 <footer id="footer">
-<p>Generated by <a href="https://pdoc3.github.io/pdoc"><cite>pdoc</cite> 0.9.1</a>.</p>
+<p>Generated by <a href="https://pdoc3.github.io/pdoc"><cite>pdoc</cite> 0.9.2</a>.</p>
 </footer>
 </body>
 </html>

docs/scrapli/channel/channel.html

@@ -3,7 +3,7 @@
 <head>
 <meta charset="utf-8">
 <meta name="viewport" content="width=device-width, initial-scale=1, minimum-scale=1" />
-<meta name="generator" content="pdoc 0.9.1" />
+<meta name="generator" content="pdoc 0.9.2" />
 <title>scrapli.channel.channel API documentation</title>
 <meta name="description" content="scrapli.channel.channel" />
 <link rel="preload stylesheet" as="style" href="https://cdnjs.cloudflare.com/ajax/libs/10up-sanitize.css/11.0.1/sanitize.min.css" integrity="sha256-PK9q560IAAa6WVRRh76LtCaI8pjTJ2z11v0miyNNjrs=" crossorigin>
@@ -995,7 +995,7 @@ <h4><code><a title="scrapli.channel.channel.Channel" href="#scrapli.channel.chan
 </nav>
 </main>
 <footer id="footer">
-<p>Generated by <a href="https://pdoc3.github.io/pdoc"><cite>pdoc</cite> 0.9.1</a>.</p>
+<p>Generated by <a href="https://pdoc3.github.io/pdoc"><cite>pdoc</cite> 0.9.2</a>.</p>
 </footer>
 </body>
 </html>

docs/scrapli/channel/index.html

@@ -3,7 +3,7 @@
 <head>
 <meta charset="utf-8">
 <meta name="viewport" content="width=device-width, initial-scale=1, minimum-scale=1" />
-<meta name="generator" content="pdoc 0.9.1" />
+<meta name="generator" content="pdoc 0.9.2" />
 <title>scrapli.channel API documentation</title>
 <meta name="description" content="scrapli.channel" />
 <link rel="preload stylesheet" as="style" href="https://cdnjs.cloudflare.com/ajax/libs/10up-sanitize.css/11.0.1/sanitize.min.css" integrity="sha256-PK9q560IAAa6WVRRh76LtCaI8pjTJ2z11v0miyNNjrs=" crossorigin>
@@ -1160,7 +1160,7 @@ <h4><code><a title="scrapli.channel.Channel" href="#scrapli.channel.Channel">Cha
 </nav>
 </main>
 <footer id="footer">
-<p>Generated by <a href="https://pdoc3.github.io/pdoc"><cite>pdoc</cite> 0.9.1</a>.</p>
+<p>Generated by <a href="https://pdoc3.github.io/pdoc"><cite>pdoc</cite> 0.9.2</a>.</p>
 </footer>
 </body>
 </html>