Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NAU7802 implementation and supporting refactor #33

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

fivesixzero
Copy link

Refactor to support the NAU7802 ADC for weight measurements in addition to the HX711. More details can be found in issue #32.

Goals

  1. Refactor backend to allow for a common Scale interface
  2. Create HX711 driver or wrapper an for existing driver to use the new interface
  3. Create NAU7802 driver or wrapper for an existing driver to use the new interface
  4. Add UI elements to settings interface to allow for selecting which driver to use
  5. Add UI elements to settings interface to allow for configuring lower-level per-driver settings, like HX711 clock/data pin and NAU7802 I2C bus ID
  6. Implement settings migration to allow for automatic automatic setup of legacy HX711-based installation and settings versioning to allow for more easily updating configuration between versions moving forward

Related Open Issues

This will address issue #32 and may also fully or partially address #28, #26, and #18.

Moving to use of a common "scale" interface and settings versioning should also lay the groundwork to allow for addressing #31, #22, and #17.

@fivesixzero
Copy link
Author

The first commit is rough but it should take care of a few of the goals listed above.

  1. A common Scale interface was added and the backend was refactored to allow for this new interface.
    • Some basic changes were made the frontend as well to allow for tare/calibrate operations using this new interface.
  2. A wrapper implementing the Scale interface was created for the hx711 driver, which is a pretty robust and easy to implement driver for the HX711 on a Raspberry Pi.
    • This was tested with a common HX711 breakout with a simple filament scale configuration.
  3. A wrapper implementing the Scale interface was created for the PyNAU7802 driver)
    • This was tested with both Adafruit and Sparkfun's NAU7802 breakouts with a simple filament scale configuration.

I ran out of time to start in earnest on the frontend changes, settings migration, and basic unit/integration testing, so those remain left to implement before this PR can be considered for proper review/merge.

Copy link
Owner

@techman83 techman83 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really cool! I'll ponder the testing side of things. I want to test the various migration paths in particular.

tare=8430152,
reference_unit=-411,
# chips=["hx711", "nau7802", "none"],
chip="nau7802",
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be worth keeping the default as the cheaper chip, that tends to be what most implementations seem to use in all the guides. A default with the other type may confuse beginners.

Suggested change
chip="nau7802",
chip='hx711',

Copy link
Author

@fivesixzero fivesixzero Jun 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely agree there. That default was a debug change that I forgot to change out before commit. 🤦

  • Assure that default chip is hx711, not nau7802

Comment on lines 57 to 75
if chip == "nau7802":
from .scale.nau7802 import NAU7802

bus_id = int(self._settings.get(["nau_bus_id"]))

self._logger.debug("Initializing NAU7802, bus_id: [{}]".format(bus_id))
self.scale = NAU7802(offset, cal_factor, bus_id)
self._logger.debug("Initialized NAU7802, is_ready: [{}]".format(self.scale.is_ready()))
elif chip == "hx711":
from .scale.hx711 import HX711

clock_pin = self._settings.get(["hx_clockpin"])
data_pin = self._settings.get(["hx_datapin"])

self._logger.debug("Initializing HX711, clock_pin: [{}], data_pin: [{}]".format(clock_pin, data_pin))
self.scale = HX711(offset, cal_factor, clock_pin, data_pin)
self._logger.debug("Initialized HX711, is_ready: [{}]".format(self.scale.is_ready()))
else:
self.scale = None
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'll save some boiler plate and give room for different backends in the future (maybe say an external arduino), if we from . import scale at the top of the file, then use getattr and pass the whole config payload into the library. Let each implementation handle parsing it during instantiation. Along with the logging statements. Potentially a try/except for if the backend configured doesn't exist

Suggested change
if chip == "nau7802":
from .scale.nau7802 import NAU7802
bus_id = int(self._settings.get(["nau_bus_id"]))
self._logger.debug("Initializing NAU7802, bus_id: [{}]".format(bus_id))
self.scale = NAU7802(offset, cal_factor, bus_id)
self._logger.debug("Initialized NAU7802, is_ready: [{}]".format(self.scale.is_ready()))
elif chip == "hx711":
from .scale.hx711 import HX711
clock_pin = self._settings.get(["hx_clockpin"])
data_pin = self._settings.get(["hx_datapin"])
self._logger.debug("Initializing HX711, clock_pin: [{}], data_pin: [{}]".format(clock_pin, data_pin))
self.scale = HX711(offset, cal_factor, clock_pin, data_pin)
self._logger.debug("Initialized HX711, is_ready: [{}]".format(self.scale.is_ready()))
else:
self.scale = None
self.scale = getattr(scale, chip)(settings=self._settings)

Copy link
Author

@fivesixzero fivesixzero Jun 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a great idea and I'm all for simplifying this code.

  • Move per-device instantiation details to scale library
  • Implement config parsing in scale library to determine which subclass to init and handle per-subclass variables
  • Use getarr() to init self.scale with the active config using the above implementation during on_startup()

Comment on lines 88 to 102
chip = self._settings.get(["chip"])
if chip == "hx711":
self.scale.enable()

v = self.scale.get_weight()

self._logger.debug("weight: [{}]".format(v))

self._settings.set(["lastknownweight"], v)
self._settings.save()

self._plugin_manager.send_plugin_message(self._identifier, v)
self.hx.power_down()

if chip == "hx711":
self.scale.disable()
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't recall why it's necessary to power up/down here for the hx11. But maybe implementing a initial_check_weight method, that by default calls get_weight and is overriden by the hx711 chip might be a touch cleaner

Something like this in the hx11 class

def initial_check_weight(self) -> float:
    self.enable()
    weight = self.get_weight()
    self.disable()
    return weight

Copy link
Author

@fivesixzero fivesixzero Jun 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was one of the first parts of the code I dug into and this is an example of just trying to keep things as identical as possible to the original implementation before making more changes.

When I was working on the CircuitPython HX711 GPIO bitbang driver I definitely observed some oddities around 'enable' vs 'disable' state management. In that driver's case I basically just abandoned all hope of tracking a 'powered up' vs 'powered down' state and pared the implementation down to the barest of basics, the raw ADC count read. That has its own downside - leaving the clock line low means the first read-in could provide a stale reading - but its easy enough to just read a few dozen counts, throw away the first read (or just remove outliers) before averaging.

Even with that knowledge though there's always going to be a risk of severe timing issues causing odd behavior in general, which is kinda unavoidable when bitbanging from high in the stack like this. Even with the much more lightweight CircuitPython, platforms like nrf and esp32 (which do BLE/WiFi management in background threads) can see some weird timing issues pop up that can totally wreck the driver's reliability. So this kind of mitigation doesn't surprise me or seem out of place, even if I'm not sure precisely why it was included.

Anyway, long story short, I think this "state toggle" should probably stick around (for now, at least). And I agree that it probably belongs in the scale.hx711 wrapper rather than here in the main plugin code. :)

  • Move hx711 state toggle on read out of the check_weight() and into the scale driver wrapper

@fivesixzero
Copy link
Author

Finally got some time set aside to get my dev environment reconfigured and dig into this again. Should have a few solid, multi-hour chunks of time to dive in over the next few days.

Right off the bat I found a few embarassing typos that broke things, which I fixed in e6d858d. These slipped in because my original setup involved some copy/paste from my Pi devbox to my main dev PC (gross!). Now I'm just doing git commits directly from the devbox with my dev PC connecting via the VSCode sshfs plugin for easier editing. No more copy/paste errors now. :)

On the dev system I've got two filament scales attached, one via HX711 on the usual pin 20/21 combo and the other via a NAU7802 on I2C. This way I can easily switch between the two by changing the setting in the Octoprint config.

Not sure what the order of operations is going to be but I've got a few chunks of to-do items.

  1. Refactor scale lib to take Octoprint settings as input and handle per-device instantiation
    • As a side goal, I'd like to make it easier to re-init the scale without restarting the whole app, which would allow for proper settings changes.
  2. Rest API auth
    • If there's a REST API it should require and use an API token, at the very least
  3. Refactor JS and templates
    • It looks like there have been some improvements to Octoprint's plugin API since the original JS/templates were created so it could use a refresh to make it easier to work with
  4. Start investigating settings versioning and migration
  5. Implement proper unit tests for the scale library
    • I'm still new to Pythonic unit testing but I'm eager to learn, so this may actually be a higher priority since I'm a bit more comfortable working with at least partially test-driven dev cycles

@fivesixzero
Copy link
Author

fivesixzero commented Jun 30, 2022

Latest commit tackles a lot more than I like to bundle into a single commit but most of it had to land at the same time, so I'll let it slide. 😎

  • Changed chip setting to scale_type to reflect possible future scale options
    • Future Scale implementations could pull scale data from an external REST API, an MQTT or other message queue, platform-specific GPIO/SPI/I2C connections, or any other source
  • Refactored scale module
    • New get_scale(settings: Dict) simplifies scale init
    • Added deinit() method to Scale interface for releasing resources like GPIOs, I2C busses, etc
    • Changed hx711 to rpi_hx711 to reflect the HX711 driver's platform requirements
    • Moved pre- and post-read-in HX711 enable/disable into the rpi_hx711 driver wrapper
    • Considered changing nau7802 as well but this driver's smbus2 implementation may work fine on a variety of platforms
  • Moved self.scale and UI-updating RepeatingTimer init to helper methods and added deinit helpers as well
  • Added new API endpoints in __init__.on_api_get()
    • spool_weight allows for get/set of spool weight variable
    • scale_type allows for get/set of scale_type setting, including deinit/reinit of self.scale if a change is made
  • Added empty stubs for get_settings_version and on_settings_migrate to help with future settings migration implementation

I've been doing functional testing as I go, and this commit's had at least these functions manually tested.

  • Startup and scale init with all three "scale types"
  • All REST API operations, including the new REST scale_type and spool_weight get and set ops
  • Scale deinit() resource release on HX711 (RPi.GPIO pins) and NAU7802 (smbus2 bus)
  • At least 20 minutes of continuous run time with both HX711 and NAU7802 drivers

This iteration doesn't use getattr(), but it does return a scale given a settings dict. Between that and breaking out the two main operations - scale init and timer init - into helper methods the on_startup() method is heavily simplified, which is good. Let me know if this looks good or if another implementation would be better, I'm still learning the Pythonic way and I'm flexible. :)

Also rearranged a lot of code to group similar things together and satisfy linters. My first commit was a total mess. 😂

These are the linters/formatters I tested with and their basic configs, if different from the project's config files:

  • flake8
    • No changes
  • pylint
    • max-line-length=140 to match project's flake8 config
    • Added init-hook for pylint-venv to handle OctoPrint dev environment venv imports
  • mypy
    • mypy --disallow-untyped-defs --disallow-untyped-calls --disallow-incomplete-defs --check-untyped-defs --disallow-untyped-decorators ./filament_scale_enhanced/scale/
  • black (not in project, yet, but handy)
    • --line-length 140 to match project's flake8 config

If these linter settings are good then I can enshrine most (or all) of them in their respective configs and add them to the GitHub test.yml workflow.

I've used pre-commit on a few other projects, which can help with running linters/tests in a consistent way during ongoing development. I may look at implementing that here if you're cool with it.

Next stuff...

Didn't dive into REST API Auth, JS/templates, settings versioning, or unit test implementation in these changes but I spent a bit of time researching each as I worked through things. Of those, the JS/template work is probably the foggiest in my head at the moment. For the others I have at least some basic implementation ideas/examples in mind.

Let me know if you've got any feedback on the latest state of the code, or if I've missed implementing any changes suggested on the first commit. If all goes well I should have another few chunks of time like this in the coming days.

Copy link
Owner

@techman83 techman83 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to spend a good chunk of time reading this, but I think it's great! I'll try and keep up to date and appreciate the effort (I've lacked spare time lately!)

Comment on lines +203 to +204
if command == "tare":
self.last_api_time = time.time()
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A method I often use here is

getattr(self, command)()

This makes adding new functionality just adding a method, keeps the entry function smaller, and makes unittesting easier.

As it all involves hardware, unit testing is probably hard. But we might be able to write a mock scale class. But I wouldn't consider that level required to merge.

Comment on lines +13 to +17
if scale_type not in SCALE_TYPES:
raise ValueError(f"Invalid scale_type: {scale_type}")

if scale_type == "rpi_hx711":
from .rpi_hx711 import RPI_HX711
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I mentioned an importlib method previously. This keeps the logic out of the main entry point which is good.

@techman83
Copy link
Owner

I've also caught Caught covid, so I'll see how functional I am. It's put me away from where my 3D printer is, but I have setup a VPN now, which makes it easier to get to. VS Code + Remote SSH is pretty nifty. I think I produced a dist and uploaded for most of my testing, it wasn't super efficient. An editable package would have been potentially easier.

@fivesixzero
Copy link
Author

Oof, hope you're feeling better! I ended up getting sidetracked into other tasks over the last few weeks but I'm hoping to have more time to dive back in next week.

@techman83 techman83 mentioned this pull request Sep 28, 2022
@techman83
Copy link
Owner

Oof, hope you're feeling better! I ended up getting sidetracked into other tasks over the last few weeks but I'm hoping to have more time to dive back in next week.

Turns out Covid doesn't mix well with ADHD! Took months for the brain fog to lift, and I'm just starting to get enough energy back to keep up with work. But I have been 3D printing again! So I'm going to try and get through this, and try and come up with ways to make it easier to approve/merge/release changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants