Doc and packaging updates; re-did main doc in sphinx.

README.rst

@@ -24,15 +24,16 @@ From YAML
 
 But you don’t have to start with authoring Python: you can use hikaru to
 parse Kubernetes YAML into these same Python objects, at which point you
-can inspect the created objects, or even have hikaru emit Python source
+can inspect the created objects, modify them and re-generate new YAML,
+or even have hikaru emit Python source
 code that will re- create the same structure but from the Python
 interface.
 
 To YAML, Python, or JSON
 ~~~~~~~~~~~~~~~~~~~~~~~~
 
 hikaru can output a Python Kubernetes object as Python source code,
-YAML, or JSON (going to the other two from JSON is coming), allowing you
+YAML, or JSON (going from JSON to the other two is coming), allowing you
 to shift easily between representational formats for various purposes.
 
 Alternative to templating for customisation
@@ -185,7 +186,7 @@ property like so:
 
    execs = p.find_by_name("exec", following='containers.1.lifecycle.httpGet')
 
-These queries result in a list of ‘CatalogEntry’ objects, which are
+These queries result in a list of ``CatalogEntry`` objects, which are
 named tuples that provide the path to the found element. You can acquire
 the actual element for inspection with the ``object_at_path()`` method:
 

docs/Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = .
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/conf.py

@@ -0,0 +1,66 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# This file only contains a selection of the most common options. For a full
+# list see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+# import os
+# import sys
+# sys.path.insert(0, os.path.abspath('.'))
+
+
+# -- Project information -----------------------------------------------------
+
+project = 'hikaru'
+copyright = '2021, Tom Carroll'
+author = 'Tom Carroll'
+
+# The full version, including alpha/beta/rc tags
+release = '0.1'
+
+
+# -- General configuration ---------------------------------------------------
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+    'sphinx.ext.autodoc'
+]
+
+autodoc_default_options = {
+    'member-order': 'bysource'
+}
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
+
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = 'alabaster'
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+
+# Alabaster theme options
+html_theme_options = {
+    "fixed_sidebar": True,
+}

docs/hikaru-base.rst

@@ -0,0 +1,173 @@
+********************
+The HikaruBase class
+********************
+
+All hikaru model objects are based on the HikaruBase class, and the model objects
+only add data; there are no additional behaviours. All operations that you can do
+on hikaru objects are defined on the HikaruBase class.
+
+Full documentation for the class can be found in the `Reference` section, but some of the 
+key methods are discussed here.
+
+from_yaml() (classmethod)
+*************************
+
+The class method ``from_yaml()`` allows you to create a populated instance instance from a supplied `ruamel.yaml.YAML` instance (this is what is used internally for loading and parsing Kubernetes YAML). So you can use ``from_yaml()`` to manually load a specific hikaru class:
+
+.. code:: python
+
+    from ruamel.yaml import YAML
+    from hikaru import Pod
+    yaml = YAML()
+    f = open("<path to yaml containing a pod>", "r")
+    doc = yaml.load(f)
+    p = Pod.from_yaml(doc)
+    assert isinstance(p, Pod)
+
+While ``load_full_yaml()`` relies on `apiVersion` and `kind` properties in the YAML to
+determine what class to instantiate and populate, ``from_yaml()`` assumes you are invoking
+it on a class that matches the kind of thing you want to load from the YAML. This allows
+you to actually load any hikaru object from YAML, even ones that are fragments of
+larger Kubernetes documents. For instance, if you had a YAML file that only contained
+the definition of a container (no `apiVersion` or `kind`), ``from_yaml()`` would still
+allow you to load it:
+
+.. code:: python
+
+    from ruamel.yaml import YAML
+    from hikaru import Container
+    yaml = YAML()
+    f = open("<path to yaml containing a container>", "r")
+    doc = yaml.load(f)
+    c = Container.from_yaml(doc)
+    assert isinstance(c, Container)
+
+Note that loading fragments in this way requires the fragment to appear to be the
+top-level YAML object in the file; there can be no indentation of the initial lines.
+
+as_python_source()
+*************************
+
+HikaruBase can render itself as Python source with ``as_python_source()`` that will
+re-create the state of the object. The source is unformatted with respect to PEP8, and may
+in fact be quite difficult to read. However, it is legal Python and will execute properly.
+It is better to use the ``get_python_source()`` function for this, as it will also run the
+PEP8 formatter to make the code more readable.
+
+Support for ==
+*************************
+
+Instances of models can be checked for equality using '=='. HikaruBase understands how to
+inspect subclasses and recursivly ensure that all field values, dict keys, list entries, etc
+are the same.
+
+dup()
+*************************
+
+Any HikaruBase instance can generate a duplicate of itself, a deep copy. This is especially
+useful in cases where pre-made components are loaded from a library and a particular
+component is used mutliple times within the same containing object, but where you may wish
+to tweak the values in each use. Since these are all object references, tweaking the values
+in one place will be seen in another unless a full copy is used in each location so the same group of objects are all being operated on from different places.
+
+find_by_name()
+*************************
+
+As HikaruBase objects are populated via processing YAML or by being created with Python
+code, an internal search catalog is created on each object that provides assistance in
+searching through the object hierarchy for specific fields or nested objects. This provides
+significant assistance in constructing automated reviewing tools that can locate and
+highlight specific objects to ensure consistency of usage and compliance to standards.
+
+This catalog is used by the ``find_by_name()`` method, which returns a list of CatalogEntry
+objects (named tuples) that describe all attributes and their location in the model that satisfy the query arguments to the method.
+
+The simplest use of this method is to supply a name to find; in this case, ``find_by_name()``
+will return every attribute called name wherever it is in the model. For example, here is
+the result when querying for the 'name' attribute against a Pod (p) in one of hikaru's test
+cases:
+
+.. code:: python
+
+    >>> for ce in p.find_by_name("name"):
+    ...     print(ce)
+    ... 
+    CatalogEntry(cls='str', attrname='name', path=['metadata', 'name'])
+    CatalogEntry(cls='str', attrname='name', path=['spec', 'containers', 0, 'name'])
+    CatalogEntry(cls='str', attrname='name', path=['spec', 'containers', 1, 'name'])
+    CatalogEntry(cls='str', attrname='name', path=['spec', 'containers', 1, 'lifecycle', 'postStart', 'httpGet', 'httpHeaders', 0, 'name'])
+    CatalogEntry(cls='str', attrname='name', path=['spec', 'containers', 1, 'env', 0, 'name'])
+    CatalogEntry(cls='str', attrname='name', path=['spec', 'containers', 1, 'env', 1, 'name'])
+    CatalogEntry(cls='str', attrname='name', path=['spec', 'containers', 1, 'envFrom', 0, 'configMapRef', 'name'])
+    CatalogEntry(cls='str', attrname='name', path=['spec', 'containers', 1, 'envFrom', 0, 'secretRef', 'name'])
+    CatalogEntry(cls='str', attrname='name', path=['spec', 'containers', 1, 'volumeDevices', 0, 'name'])
+    CatalogEntry(cls='str', attrname='name', path=['spec', 'containers', 1, 'volumeMounts', 0, 'name'])
+    CatalogEntry(cls='str', attrname='name', path=['spec', 'imagePullSecrets', 0, 'name'])
+    CatalogEntry(cls='str', attrname='name', path=['spec', 'imagePullSecrets', 1, 'name'])
+
+As you can see, the field occurs in quite a lot of places at different depths of the object
+hierarchy, and this is only a Pod with two containers, so the result could be a lot more
+voluminous. We can establish a search scope with ``find_by_name()`` by using the ``following``
+keyword argument. This argument tells the function to return CatalogEntries for each instance
+of the named attribute **if** that attribute comes after one or more other attributes in
+the path to attribute we want. For example, we can narrow the search down to only ones where
+'name' comes somewhere within the containers:
+
+.. code:: python
+
+    >>> for ce in p.find_by_name("name", following="containers"):
+    ...     print(ce)
+    ... 
+    CatalogEntry(cls=<class 'str'>, attrname='name', path=['spec', 'containers', 0, 'name'])
+    CatalogEntry(cls=<class 'str'>, attrname='name', path=['spec', 'containers', 1, 'name'])
+    CatalogEntry(cls=<class 'str'>, attrname='name', path=['spec', 'containers', 1, 'lifecycle', 'postStart', 'httpGet', 'httpHeaders', 0, 'name'])
+    CatalogEntry(cls=<class 'str'>, attrname='name', path=['spec', 'containers', 1, 'env', 0, 'name'])
+    CatalogEntry(cls=<class 'str'>, attrname='name', path=['spec', 'containers', 1, 'env', 1, 'name'])
+    CatalogEntry(cls=<class 'str'>, attrname='name', path=['spec', 'containers', 1, 'envFrom', 0, 'configMapRef', 'name'])
+    CatalogEntry(cls=<class 'str'>, attrname='name', path=['spec', 'containers', 1, 'envFrom', 0, 'secretRef', 'name'])
+    CatalogEntry(cls=<class 'str'>, attrname='name', path=['spec', 'containers', 1, 'volumeDevices', 0, 'name'])
+    CatalogEntry(cls=<class 'str'>, attrname='name', path=['spec', 'containers', 1, 'volumeMounts', 0, 'name'])
+
+That gets rid of metadata and imagePullSecrets, but that's still too much. Say we only care about
+the second container, and under that we just want the postStart:
+
+.. code:: python
+
+    >>> for ce in p.find_by_name("name", following="containers.1.postStart"):
+    ...     print(ce)
+    ... 
+    CatalogEntry(cls=<class 'str'>, attrname='name', path=['spec', 'containers', 1, 'lifecycle', 'postStart', 'httpGet', 'httpHeaders', 0, 'name'])
+
+Now we only have one entry in the result. In this case, although we could have used just used 'lifecycle' as the value of ``following``, we want to illustrate a couple of things:
+
+  - First, notice that we can use a series of attributes in the ``following`` expression, separated by '.'.
+  - Second, notice that the attributes don't have to be directly sequential as you tunnel into an object.
+  - Third, note that we can use integers as indexes into a list of objects; we will only search under that index.
+
+The ``following`` expression can either be a '.' separated string, or a list of strings
+and ints.
+
+The attributes of the returned CatalogEntry namedtuples are:
+
+  - cls: the class object for the value of the item that was named
+  - attrname: the name of the attribute found
+  - path: a list of strings that will take you from object where you did the search to the located item
+
+object_at_path()
+*************************
+
+The ``object_at_path()`` method works with the ``path`` attribute of the returned 
+CatalogEntry object. By passing the the path into ``object_at_path()``, you can access
+the actual value of the object stored there. This gives you the means to inspect the 
+object that you've located.
+
+repopulate_catalog()
+*************************
+
+Normally, the catalogs are created automatically when you create an object in Python or when
+you load an instance from YAML. However, once you've loaded the instance, you are free to
+modify the existing entries, add additional ones, or even delete existing pieces. Such
+operations will make the catalog inaccurate if you intend to use ``find_by_name()`` again.
+To bring the catalog up to date, invoke ``repopulate_catalog()``, and all catalogs from
+the object where you invoked the method on down with have their catalogs recomputed and
+made up to date.

docs/index.rst

@@ -0,0 +1,27 @@
+.. hikaru documentation master file, created by
+   sphinx-quickstart on Mon Mar 15 09:35:55 2021.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+hikaru: Usage and Reference
+==================================
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+   introduction
+   installation-quickstart
+   key-functions
+   models
+   hikaru-base
+   issues
+   reference
+
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`

docs/installation-quickstart.rst

@@ -0,0 +1,71 @@
+***************************
+Installation and Quickstart
+***************************
+
+Installation
+############
+
+From PyPI, you can just use the normal ``pip install`` dance:
+
+    ``pip install hikaru``
+
+Or if installing from source,  cd into the project root an then you can install from setup.py:
+
+    ``python setup.py install``
+
+Quickstart
+############
+
+The following are the 'bread and butter' functions of hikaru.
+
+To read Kubernetes YAML documents into hikaru Python objects:
+*************************************************************
+
+For loading Kubernetes YAML documents into live hikaru Python objects, use the
+``load_full_yaml()`` function:
+
+.. code:: python
+
+    from hikaru import load_full_yaml
+    docs = load_full_yaml(path="<path to yaml file>")
+    # 'docs' is a list of different doc 'kinds' such
+    # as Pod, Deployment, etc
+
+The objects in the resultant list will always have *kind* and *apiVersion*
+attributes populat4ed. If any of the input YAML doesn't have these attributes for their
+documents, hikaru can't tell what classes to build. You can then use Kubernetes YAML
+property names to navigate through the Python objects.
+
+To write Kubernetes YAML documents from hikaru Python objects:
+==============================================================
+
+You can print out the equivalent Kubernetes YAML from hikaru Python objects with the
+``get_yaml()`` function:
+
+.. code:: python
+
+    from hikaru import get_yaml
+    # assume that 'p' below is an instance of the Pod class
+    print(get_yaml(p))
+
+The output YAML will start with a 'start of document' marker (---) and then the
+YAML for the hikaru objects will be printed.
+
+To generate hikaru Python source from hikaru Python objects:
+============================================================
+
+If you want to convert your Kubernetes YAML to actual hikaru Python source code, use
+the ``get_python_source()`` function:
+
+.. code:: python
+
+    from hikaru import get_python_source, load_full_yaml
+    docs = load_full_yaml(path="<path to yaml>")
+    p = docs[0]
+    # when rendering the Python source, you can indicate a
+    # variable to assign the created object to:
+    print(get_python_source(p, assign_to='x'))
+
+This will output a PEP8-compliant set of Python. Generation may take a short while
+depending on how many deeply nested Python objects are involved.
+