Merge pull request #225 from spotify/psobot/add-midi-note-support

Add support for external instrument plugins.

README.md

@@ -14,7 +14,7 @@
 [![GitHub Repo stars](https://img.shields.io/github/stars/spotify/pedalboard?style=social)](https://github.com/spotify/pedalboard/stargazers)
 
 
-`pedalboard` is a Python library for working with audio: reading, writing, adding effects, and more. It supports most popular audio file formats and a number of common audio effects out of the box, and also allows the use of [VST3®](https://www.steinberg.net/en/company/technologies/vst3.html) and [Audio Unit](https://en.wikipedia.org/wiki/Audio_Units) formats for third-party plugins.
+`pedalboard` is a Python library for working with audio: reading, writing, rendering, adding effects, and more. It supports most popular audio file formats and a number of common audio effects out of the box, and also allows the use of [VST3®](https://www.steinberg.net/en/company/technologies/vst3.html) and [Audio Unit](https://en.wikipedia.org/wiki/Audio_Units) formats for loading third-party software instruments and effects.
 
 `pedalboard` was built by [Spotify's Audio Intelligence Lab](https://research.atspotify.com/audio-intelligence/) to enable using studio-quality audio effects from within Python and TensorFlow. Internally at Spotify, `pedalboard` is used for [data augmentation](https://en.wikipedia.org/wiki/Data_augmentation) to improve machine learning models and to help power features like [Spotify's AI DJ](https://newsroom.spotify.com/2023-02-22/spotify-debuts-a-new-ai-dj-right-in-your-pocket/). `pedalboard` also helps in the process of content creation, making it possible to add effects to audio without using a Digital Audio Workstation.
 
@@ -35,8 +35,8 @@
    - Pitch effects: `PitchShift`
    - Lossy compression: `GSMFullRateCompressor`, `MP3Compressor`
    - Quality reduction: `Resample`, `Bitcrush`
- - Supports VST3® plugins on macOS, Windows, and Linux ([`pedalboard.load_plugin`](https://spotify.github.io/pedalboard/reference/pedalboard.html#pedalboard.load_plugin))
- - Supports Audio Units on macOS
+ - Supports VST3® instrument and effect plugins on macOS, Windows, and Linux ([`pedalboard.load_plugin`](https://spotify.github.io/pedalboard/reference/pedalboard.html#pedalboard.load_plugin))
+ - Supports instrument and effect Audio Units on macOS
  - Strong thread-safety, memory usage, and speed guarantees
    - Releases Python's Global Interpreter Lock (GIL) to allow use of multiple CPU cores
      - No need to use `multiprocessing`!
@@ -146,16 +146,18 @@ with AudioFile('processed-output.wav', 'w', samplerate, effected.shape[0]) as f:
   f.write(effected)
 ```
 
-### Using VST3® or Audio Unit plugins
+### Using VST3® or Audio Unit instrument and effect plugins
 
 ```python
 from pedalboard import Pedalboard, Reverb, load_plugin
 from pedalboard.io import AudioFile
+from mido import Message # not part of Pedalboard, but convenient!
 
 # Load a VST3 or Audio Unit plugin from a known path on disk:
-vst = load_plugin("./VSTs/RoughRider3.vst3")
+instrument = load_plugin("./VSTs/Magical8BitPlug2.vst3")
+effect = load_plugin("./VSTs/RoughRider3.vst3")
 
-print(vst.parameters.keys())
+print(effect.parameters.keys())
 # dict_keys([
 #   'sc_hpf_hz', 'input_lvl_db', 'sensitivity_db',
 #   'ratio', 'attack_ms', 'release_ms', 'makeup_db',
@@ -164,16 +166,21 @@ print(vst.parameters.keys())
 # ])
 
 # Set the "ratio" parameter to 15
-vst.ratio = 15
+effect.ratio = 15
 
-# Use this VST to process some audio:
-with AudioFile('some-file.wav', 'r') as f:
-  audio = f.read(f.frames)
-  samplerate = f.samplerate
-effected = vst(audio, samplerate)
+# Render some audio by passing MIDI to an instrument:
+samplerate = 44100
+audio = instrument(
+  [Message("note_on", note=60), Message("note_off", note=60, time=5)],
+  samplerate,
+  duration=5, # seconds
+)
+
+# Apply effects to this audio:
+effected = effect(audio, samplerate)
 
-# ...or put this VST into a chain with other plugins:
-board = Pedalboard([vst, Reverb()])
+# ...or put the effect into a chain with other plugins:
+board = Pedalboard([effect, Reverb()])
 # ...and run that pedalboard with the same VST instance!
 effected = board(audio, samplerate)
 ```

dev-requirements.txt

@@ -6,6 +6,6 @@ bumpversion==0.5.3
 ipython
 ipdb
 pip>=20
-pybind11>=2.6.0
+pybind11>=2.10.4
 setuptools>=42
 wheel

docs/_sources/examples.rst.txt

@@ -2,5 +2,5 @@ Examples
 ========
 
 .. mdinclude:: ../../README.md
-   :start-line: 73
-   :end-line: 252
+   :start-line: 75
+   :end-line: 260

docs/_sources/faq.rst.txt

@@ -4,16 +4,9 @@ Frequently Asked Questions
 Can Pedalboard be used with live (real-time) audio?
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Technically, yes, Pedalboard could be used with live audio input/output. See:
+As of version 0.7.0, yes! See the :class:`pedalboard.io.AudioStream` class for more details.
 
 
-* `Pull request #98 <https://github.com/spotify/pedalboard/pull/98>`_\ , which contains an experimental live audio interface written in C++.
-* `@stefanobazzi <https://github.com/stefanobazzi>`_\ 's `guitarboard <https://github.com/stefanobazzi/guitarboard>`_ project for an example that uses the ``python-sounddevice`` library.
-
-However, there are a couple big caveats when talking about using Pedalboard in a live context. Python, as a language, is `garbage-collected <https://devguide.python.org/garbage_collector/>`_\ , meaning that your code randomly pauses on a regular interval to clean up unused objects. In most programs, this is not an issue at all. However, for live audio, garbage collection can result in random pops, clicks, or audio drop-outs that are very difficult to prevent. Python's `Global Interpreter Lock <https://wiki.python.org/moin/GlobalInterpreterLock>`_ also adds potentially-unbounded delays when switching threads, and most operating systems use a separate high-priority thread for audio processing; meaning that Python could block this thread and cause stuttering if *any* Python objects are accessed or mutated in the audio thread.
-
-Note that if your application processes audio in a streaming fashion, but allows for large buffer sizes (multiple seconds of audio) or soft real-time requirements, Pedalboard can be used there without issue. Examples of this use case include streaming audio processing over the network, or processing data offline but chunk-by-chunk.
-
 Does Pedalboard support changing a plugin's parameters over time?
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -64,9 +57,10 @@ With this technique, it's possible to automate any parameter. Usually, using a s
 Can Pedalboard be used with VST instruments, instead of effects?
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Not yet! The underlying framework (JUCE) supports VST and AU instruments just fine, but Pedalboard itself would have to be modified to support instruments.
+As of version 0.7.4, yes! See the :class:`pedalboard.VST3Plugin` and :class:`pedalboard.AudioUnitPlugin` classed for more details.
 
 Can Pedalboard plugins accept MIDI?
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Not yet, either - although the underlying framework (JUCE) supports passing MIDI to plugins, so this would also be possible to add.
+As of version 0.7.4, both :class:`pedalboard.VST3Plugin` and :class:`pedalboard.AudioUnitPlugin` support passing MIDI
+messages to instrument plugins for audio rendering. However, effect plugins cannot yet be passed MIDI data.

docs/_sources/index.rst.txt

@@ -2,15 +2,15 @@
 :og:description: 🎛 🔊 Documentation for Pedalboard: A Python library for working with audio.
 
 .. mdinclude:: ../../README.md
-   :end-line: 72
+   :end-line: 74
 
 Reference
 ---------
 
 .. toctree::
    :maxdepth: 1
 
-   API Reference (Audio Effects) <reference/pedalboard>
+   API Reference (Audio Plugins) <reference/pedalboard>
    API Reference (Audio I/O) <reference/pedalboard.io>
 
 

docs/_sources/license.rst.txt

@@ -2,5 +2,5 @@ License
 =======
 
 .. mdinclude:: ../../README.md
-   :start-line: 256
-   :end-line: 267
+   :start-line: 283
+   :end-line: 295

docs/_static/documentation_options.js

@@ -1,6 +1,6 @@
 var DOCUMENTATION_OPTIONS = {
     URL_ROOT: document.getElementById("documentation_options").getAttribute('data-url_root'),
-    VERSION: 'v0.7.3',
+    VERSION: 'v0.7.4',
     LANGUAGE: 'en',
     COLLAPSE_INDEX: false,
     BUILDER: 'html',

docs/compatibility.html

@@ -6,15 +6,15 @@
 <meta property="og:title" content="Plugin Compatibility" />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://spotify.github.io/pedalboard/compatibility.html" />
-<meta property="og:site_name" content="Pedalboard v0.7.3 Documentation" />
+<meta property="og:site_name" content="Pedalboard v0.7.4 Documentation" />
 <meta property="og:description" content="pedalboard allows loading VST3® and Audio Unit plugins, which could contain any code. Most plugins that have been tested work just fine with pedalboard, but some plugins may not work with pedalboar..." />
 <meta property="og:image" content="https://repository-images.githubusercontent.com/383471193/0f79b949-9b91-4436-bfc4-05d30ef384cc" />
 <meta property="og:image:alt" content="🎛 🔊 Documentation for Pedalboard: A Python library for working with audio." />
 <meta name="description" content="pedalboard allows loading VST3® and Audio Unit plugins, which could contain any code. Most plugins that have been tested work just fine with pedalboard, but some plugins may not work with pedalboar..." />
 <link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="next" title="Pedalboard Internals" href="internals.html" /><link rel="prev" title="Frequently Asked Questions" href="faq.html" />
 
     <link rel="shortcut icon" href="_static/favicon.ico"/><meta name="generator" content="sphinx-5.3.0, furo 2022.09.29"/>
-        <title>Plugin Compatibility - Pedalboard v0.7.3 Documentation</title>
+        <title>Plugin Compatibility - Pedalboard v0.7.4 Documentation</title>
       <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
     <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=d81277517bee4d6b0349d71bb2661d4890b5617c" />
     <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
@@ -132,7 +132,7 @@
       </label>
     </div>
     <div class="header-center">
-      <a href="index.html"><div class="brand">Pedalboard v0.7.3 Documentation</div></a>
+      <a href="index.html"><div class="brand">Pedalboard v0.7.4 Documentation</div></a>
     </div>
     <div class="header-right">
       <div class="theme-toggle-container theme-toggle-header">
@@ -158,7 +158,7 @@
     <img class="sidebar-logo" src="_static/pedalboard_logo_small.png" alt="Logo"/>
   </div>
 
-  <span class="sidebar-brand-text">Pedalboard v0.7.3 Documentation</span>
+  <span class="sidebar-brand-text">Pedalboard v0.7.4 Documentation</span>
 
 </a><form class="sidebar-search-container" method="get" action="search.html" role="search">
   <input class="sidebar-search" placeholder=Search name="q" aria-label="Search">
@@ -167,7 +167,7 @@
 </form>
 <div id="searchbox"></div><div class="sidebar-scroll"><div class="sidebar-tree">
   <ul>
-<li class="toctree-l1"><a class="reference internal" href="reference/pedalboard.html">API Reference (Audio Effects)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="reference/pedalboard.html">API Reference (Audio Plugins)</a></li>
 <li class="toctree-l1"><a class="reference internal" href="reference/pedalboard.io.html">API Reference (Audio I/O)</a></li>
 </ul>
 <ul class="current">