2022.01.30a2

Fixes broken channel log in a1 release.

2022.01.30a1

Pre release to unblock scrapli-netconf/cfg updates

• Removed deprecated comms_ansi argument
• Improved error handling/error message for insufficient permissions when opening ssh config/known hosts file
  (system transport)
• Added support for hashed entries in known hosts file thanks to @kangtastic work in #174
• Improved "in channel" SSH and Telnet authentication handling; better consistency between sync and async, patterns
  are now compiled only if/when needed
• Added option to enable echo in PTYProcess (was originally removed from vendor'd code) -- should only be
  useful/necessary with netconf #165
• Allow users to build their own open_cmd for system transport -- users can override this to do things like
  kubectl exec -it args args args or docker exec -it args args args to connect to containers in k8s/docker #166
• Updated/fixed(?) Juniper shell patterns for "normal" and root shells #170
• Support transport options being passed to asyncssh transport thanks to @cuong-nguyenduy work in #178 and #183

2021.07.30

• Added "% Unavailable command" to EOS failed_when_contains
• Moved core platform failed_when_contains to base to not have to duplicate them in sync and async platforms
• Add file_mode to the enable_basic_logging function, can now choose "append" or "write" for logfile
• Add channel_log_mode to the base driver arguments; you can now choose "append" or "write" for this as well!
• Improve reading until prompt methods; no longer use re.search on the entire received byte string, now only checks for prompt on the last N chars where N is governed by the base channel args comms_prompt_search_depth attribute... this fixes an issue where scrapli could be wayyyyyy slow for very very large outputs (like full tables show bgp)
• Fix bug (or just terrible initial idea!?) in asynctelnet that reset a timer back to a very small value that was used for testing; most people shouldn't have noticed an issue here, but if you had slow devices this could cause issues that "looked" like an authentication issue due to scrapli not having responded to all telnet control characters before punting to auth
• Added commandeer to driver object; this is used to "commandeer" an existing connection but treat it like the new connection object (prompt patterns, methods, etc.) -- generally this would be used for using GenericDriver to connect to a console server, then "commandeering" that connection and turning it into an IOSXR/IOSXE/etc. connection object so you have all the "normal" behavior of scrapli
• Add missing timeout on the asynctelnet open method
• Add py.typed to hopefully do typing more correctly :P
• BUGFIX: network drivers aborted configuration sessions if responses were failed even if the stop_on_failed arg was set to False; this has been fixed now so that sessions are only aborted if the response is failed and
• Improved typing for send_interactive
• Remove napalm dev requirement -- switch to scrapli-cfg for dev environment config management; something something eating dog food or whatever.
• Deprecate comms_ansi -- if there is an ANSI escape sequence we will now just strip it out automagically; this is not currently a breaking change, but will be -- there is a deprecation warning now and comms_ansi will be fully removed in the 2022.01.30 release (and pre-releases).
• Removed a sleep that was in the default on_open for IOSXR devices... this has been there a while and I think it was just a hold over from early early versions of scrapli that perhaps had a less robust in channel authentication handler. 1 second faster IOSXR for free! Yay!
• Fixed an issue with system transport where the transport would get closed twice causing an unhandled exception -- thank you to Alex Lardschneider for finding this!
• Added an example for the enable_basic_logging function as well as the commandeer method
• Improved priv level handling -- if you try to acquire "parallel" privileges (ex. configuration and configuration exclusive in IOSXR) previously we would say things worked, but we would just stay in configuration mode. This has been fixed (hopefully)!
• Move ansi escape pattern to compile globally, so it only compiles once (why it was never like that before... who knows)
• Simplify the collect bits for integration tests... this is still not used heavily but hopefully will be soon!
• Replace vrnetlab creds in examples with scrapli (felt confusing to have vrnetlab creds everywhere, plus functional testing is moving away from (but still supporting) vrnetlab test environment)
• Crank up the rows/cols for system transport -> 80 rows, 256 cols -- this to align with scrapligo and to make it less common that users need to modify these values.
• BUGFIX: fixed blocking read in async channel telnet authentication (thank you Dmitry Figol!)
• Added not_contains field to privilege levels... this will help greatly simplify the necessary regex patterns, as well as allow us to ditch look arounds which go does not support... step one to a standardized community platform that works with python -or- go!
• Simplified (at least a little... more would be good) patterns for privilege levels for core platforms.
• Added _generic_driver_mode to the NetworkDriver classes -- this is a private mode as it should probably be used cautiously -- the idea here is that you can send any strings you want and scrapli will not care about privilege levels at all. See the discussion about this here.
• BUGFIX: fixed asynctelnet issue with control character handling, thank you to @davaeron -- see #147
• BREAKING CHANGE removed the transport.username_prompt and transport.password_prompt attributes of the telnet transports. All authentication has been moved into the channel, so it made no sense to leave these attributes on the transports. This may cause an issue for users that had explicitly set their prompts to something non-standard.
• Finally added logic to auto set port to 23 for telnet :)
• BUGFIX: fixed a rare issue where decoding bytes received from the channel (in the response object) would raise a UnicodedecodEerror; we now catch this exception and decode with ISO-8859-1 encoding which seems to be much less picky about what it decodes. Thanks to Alex Lardschneider for yet another good catch and fix!
• Added interaction_complete_patterns to all "interactive" methods -- this argument accepts a list of strings/patterns; will be re-escape'd if each string does not start with and end with "^" and "$" (line anchors), otherwise will be compiled with the standard scrapli case-insensitive and multiline flags. If the interactive event finds any of these pattenrs during the course of the interacting it will terminate the interactive session. Note that this is entirely optional and is a keyword only argument so no changes are necessary to any existing scrapli programs.

Logo! Deprecate comms_ansi, 3.10 testing, doc fixes, examples, minor bug fixes and more!

mostly another pre-release to keep replay/cfg chugging along toward their eventual first "real" release.

• bump pins
• deprecate comms_ansi
• convert deprecation warning -> FutureWarning so users see it
• update docs, fix some links, redeploy docs
• handle annoying rare paramiko eof at tear down of integration tests
• first swag adding 3.10 testing
  • skip ssh2 on 3.10 for now
  • skip pyfakefs tests on 3.10 for now
• make tests not a package.... remove unnecessary sleep for iosxr
• fix system transport double timeout issue
• add logging and commandeer example
• remove unnecessary call to strip ansi in in channel auth
• remove comms_ansi arg from tests
• handle "parallel" privs correctly (i.e. configuration and configuration_exclusive)
• Logo (#121)
• change creds to just scrapli/scrapli in examples, functional test updates for lab environment
• remove unnecessary ansi strip in async channel telnet auth

Typing Fix for `send_interactive`

Releasing to help un block typing improvements in scrapli-cfg as well.

2021.07.30a1 Pre-Release; commandeer, a sneaky Easter egg, unblock some work on scrapli netconf!

• Added "% Unavailable command" to EOS failed_when_contains
• Moved core platform failed_when_contains to base to not have to duplicate them in sync and async platforms
• Add file_mode to the enable_basic_logging function, can now choose "append" or "write" for logfile
• Add channel_log_mode to the base driver arguments; you can now choose "append" or "write" for this as well!
• Improve reading until prompt methods; no longer use re.search on the entire received byte string, now only checks
  for prompt on the last N chars where N is governed by the base channel args comms_prompt_search_depth attribute..
  . this fixes an issue where scrapli could be wayyyyyy slow for very very large outputs (like full tables show bgp)
• Fix bug (or just terrible initial idea!?) in asynctelnet that reset a timer back to a very small value that was used
  for testing; most people shouldn't have noticed an issue here, but if you had slow devices this could cause
  issues that "looked" like an authentication issue due to scrapli not having responded to all telnet control
  characters before punting to auth
• Added commandeer to driver object; this is used to "commandeer" an existing connection but treat it like the new
  connection object (prompt patterns, methods, etc.) -- generally this would be used for using GenericDriver to
  connect to a console server, then "commandeering" that connection and turning it into an IOSXR/IOSXE/etc.
  connection object so you have all the "normal" behavior of scrapli
• Add missing timeout on the asynctelnet open method
• Add py.typed to hopefully do typing more correctly :P

Ginormous Overhaul

• BREAKING CHANGE PrivilegeLevel import location changed -- this will break things!
• timeout_exit deprecated; will always close connection on timeout now
• All exceptions rationalized/changed -- all exceptions now rooted in ScrapliException and scrapli should not raise any exception that is not rooted in this! It is of course possible that non-scrapli exceptions will get raised at some point, but all "common" exceptions will now follow this pattern.
• Added opinionated logging option -- should be used only for debugging/testing, otherwise use your own logging setup!
• Moved "in channel" auth into channel (for telnet/system ssh authentication)
• Added channel_lock option, defaults to false
• Added channel_log option
• Decorators got reswizzled a little, no more requires open as the transports handle this. There is now a dedicated
  ChannelTimeout and TransportTimeout to keep things simpler.
• All transport plugins are now in scrapli "core"
• All (ok, most...) channel and transport args are now properties of the driver class -- this should remove confusion about where to update what timeout/value
• Response._record_response is now public but only for linting reasons, people generally should ignore this anyway!
• Python 3.6 will now require dataclasses backport
• All driver methods now have only the "main" argument as an allowable positional argument, the rest of the arguments are keyword-only! For example, send_command you can pass a positional argument for command, but strip_prompt and any additional arguments must be keyword arguments!
• BREAKING CHANGE Scrape/AsyncScrape renamed --> Driver/AsyncDriver -- given most folks should not be using these directly there will be no alias for this, just a hard change!
• More improvements to IOSXE tclsh pattern handling; handles tclsh in exec or privilege exec mode now
• read_until_prompt_or_time now supports regex patterns in the channel_outputs list (pass as a string, will be compiled for you)
• Big improvements to Factory for users of IDEs -- factories now have proper typing data so you will have nice auto completion things there/typing will be much happier

Asynctelnet for... science! Weird priv levels for... reasons! Plus also more things!

• Make log messages for textfsm and genie parsers failing to parse consistent as log.warning
• Add factory example
• Add "root" priv level to junos driver -- probably should be considered experimental for now :)
• Fix issue where send_config unified result did not have finish time set
• POSSIBLY BREAKING CHANGE: logger names have changed to be easier to get/make more sense -- the logger for each instance used to look like: "scrapli-channel-{{ HOST }}" which kinda was not really smart :). Loggers now look like: "scrapli.{{ HOST }}:{{ PORT }}.channel" -- can be channel|driver|transport!
• Changes to test environment:
  • Support running devices on localhost w/ nat'd management ports -- in "vrouter" mode (poorly named) -- this is enabled with the SCRAPLI_VROUTER environment variable set to on/true/something
  • Added bootvar into nxos base config -- when missing causes qemu nxosv to boot into loader prompt so thats no good
  • Replace resource settings in vdc in nxos to account for nxos instances with differing resources (memory/cpu)
  • Got rid of static license udi in iosxe config, replaced more certificate stuff so show run comparisons are easier on iosxe
• NEW TRANSPORT asynctelnet transport is built using standard library asyncio, as such it is part of scrapli core
  • Should be considered beta for a while :)
  • Added a bunch of tests mocking streamreader/writer to ensure that this driver is well tested
• Added asynctelnet support in nxos and juniper drivers (to change prompt for those platforms)
• Support asynctelnet in base driver
• auth_bypass for both telnet drivers completely bypasses not only auth (as it did previously) but also the auth validation where we confirm we got logged in successfully -- reason being is that for console servers and such you may not care about that, you may just want to log in and read data.
• Removed unnecessary re-checking/verifying of ssh config file in system transport (was basically duplicated from base transport, so was pointless!)
• Bumped all the default timeout values up as they were probably a bit on the aggressive side
• Added eager argument to send commands/commands from file and config/configs/configs from file methods -- basically this eager mode will not look for a prompt between lines of commands/configs. This means that things have potential to get out of whack because we will just send things as fast as possible. In order to not totally break things we will (whether you like it or not!) wait and find the prompt on the last command/config in the list though -- that way we dont get too out of whack. This now means we can use eager to configure banners and macros and things and we no longer need to do the dirty send interactive workaround.
• Added ScrapliConnectionLost exception and raise it if we get EOF in system transport -- with a message that is more clear than just "EOF" and some obscure line in ptyprocess!
• Added tclsh privilege level for IOSXE
• Fixed a bug that would prevent going to "parallel" privilege levels -- i.e. going from tclsh to configuration or visa versa in IOSXE or from configuration to configuration_exclusive in IOSXR
• If no failed_when_contains is passed to send_interactive network drivers will now use the network drivers failed_when_contains attribute to bring it inline with the normal command/config methods
• Added timeout_ops to send_interactive and wrap those methods with the TimeoutModifier decorator
• Add logic to properly fetch socket address family type so we can handle IPv6 hosts (w/ scrapli-ssh2/scrapli-paramiko)
• Added tclsh privilege level for NXOS, didn't even know that existed before!

House Keeping! Few prompt pattern improvements, remove some unused stuff, add some stuff that should exist!

• Fix a regex that sometimes caused a failed functional IOSXR test
• Add ptyprocess transport options for system transport -- sounds like this may be needed for huawei community platform to be able to set the pty process terminal size -- also added some basic testing for this
• Update scrapli-ssh2 pin to latest version -- now supports keyboard interactive auth; also un-skipped all related EOS tests now that this works
• Fix missing acquire priv in default on_open methods for nxos and eos async version
• Fix incorrect textfsm_platform for iosxr (was cisco_iosxr, now is cisco_xr)
• Remove unnecessary decorator on write operations for systemssh and telnet -- this operation shouldn't block so this was unnecessary; any issue here should raise some exception from the lower level library.
• Playing around w/ adding coverage reports with Codecov

BREAKING: Removed keep alive; TTP parser; Big session lock handling improvements, better logging, housekeeping!

• Improve logging in helper functions - especially around resolving ssh config/known hosts
• Add ttp_parse_output method to Response object; add ttp_parse function in helper
• Load requirements from requirements files and parse them for setup.py -- stop me from forgetting to update in one place or another!
• Slacken the IOSXE configuration prompt pattern -- hostname(ipsec-profile) was not being caught by the pattern as it was expecting the part in parenthesis to start with "conf" - thank you Talha Javaid for bringing this up on ntc slack, and Alex Lardschneider for confirming the "fix" should be good to go!
• Add community pip extra to install scrapli community
• Minor README house keeping!
• Made transport set_timeout saner -- I genuinely don't know what I was doing with that before... this included the base class as well as updating telnet and systemssh... in theory this could be a breaking change if you were just calling set_timeout for some reason without passing an argument... you probably weren't doing that... because why would you? There was some precedent for doing it like this before but it isn't worth caring about now :)
• Did smarter things with imports in helper, added tests to make sure the warnings are correct
• Dramatically simplified session locking... this had just gotten out of hand over time... now only the channel locks. This means that basically all inputs/outputs should go through the channel and/or you should acquire the lock yourself if you wish to read/write directly to the transport. Critically this means that all the external transport plugins AND scrapli-netconf need to be updated as well -- this means that you must update all of these if you are using this release! (requirements are of course pinned to make sure this is the case)
• BREAKING CHANGE: removed ALL keepalive stuff... for now. This will probably get added back, but AFAIK nobody uses it right now and the implementation of it is frankly not very good... keeping it around right now added complexity for little gain. Keepalives will be back and improved hopefully in the next release. If you need them, please just pin to 2020.09.26!