Skip to content

Commit

Permalink
Adding usage docs.
Browse files Browse the repository at this point in the history
  • Loading branch information
blackears committed Apr 19, 2024
1 parent 5ba1dae commit 9f5fbf9
Show file tree
Hide file tree
Showing 3 changed files with 47 additions and 3 deletions.
43 changes: 43 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,49 @@ You can install the addon by:

Once installed, make sure that you check the box to enable it. A new tab will appear in the viewport right hand tab menu.

## Usage

Once installed, click on the Parrot Lipsync panel to access Parrot's controls.

[Parrot running in Blender 4.1](doc/parrot_in_blender.jpg)

You can then configure various parameters:

### General Parameters

* Whisper library - Language library Whisper uses for AI translation
* Phoneme Table - Table used to map phonemes to mouth poses
* Key interpolation - Way to interpolate between mouth poses. It is recommended that you use `bezier` for 3D and `constant` for keyframe animation.
* Silence cutoff - During processing, a frame is considered to be silent if the audio track is below this value for the entire frame. Used to positioning of words.
* Word pad frames - The number of frames to pad at the begining and end of words before placing a `rest` pose.
* Strength multiplier - Pose stength is multiplied buy this value. Set to less than 1 to soften the animation.
* Track volume multiplier - If checked, the volume of the track at a particular frame will be used to adjust the strength of this frame. Can be used to make the mouth poses more extreme when the speaker is being loud and less extreme when the voice is quiet.
* Auto detect language - Automatically determine the language being used by examining the audio data
* Language code - You can specify the language code to be used here if `auto detect language` is not checked

### Lipsync Action Paramaters

This generates an action based on audio data. This action is added to the project's actions and is not assigned to any particular object. You must have added an audio track to the `Video Sequencer` and selected it as the active object.

* Lipsync Action - Action object that will have it's data replaced with the data generated for this audio track. If blank, a new action will be created.

Press the `Render Lipsync to Action` button to generate the action.

### NLA Action Paramaters

This uses Blender's NLA editor to generate the animation action and then add it to the target object's NLA tracks. NLA tracks will be generatd for all selected tracks in the `Video Sequencer`.

* Target object - Object action will be created for
* Action name suffix - This string will be appended to the names of any actions generated to help you keep track of them

Press the `Render Lipsync to Action` button to generate the NLA tracks and actions.

### Phoneme Groups

This is a list generated by reading the phoneme table. It lists every Phoneme group deacribed in the table and has a `Pose Action` field for each group that lets you define the mouth position for each group. You'll need to provide poses for each entery in the table for Parrot to generate lipsync tracks.

If the Phoneme Groups are not showing, or if you want to load a different table, click the `Reload Phoneme Table` to rebuild the table.

## Running in headless mode

A script has been included in `/examples/headless_example/run_headless.py` that demonstrates running Parrot from the command line without having to launch Blender first.
Expand Down
Binary file added doc/parrot_in_blender.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
7 changes: 4 additions & 3 deletions source/parrot_lipsync.py
Original file line number Diff line number Diff line change
Expand Up @@ -166,9 +166,8 @@ class ParrotLipsyncProps(bpy.types.PropertyGroup):
name="Key Interpolation",
items= [('constant', 'constant', 'constant interpolation'),
('linear', 'linear', 'linaer interpolation'),
('bezier', 'bezier', 'bezier interpolation'),
('default', 'default', 'use the interpolation type set on the key'),],
default='default'
('bezier', 'bezier', 'bezier interpolation'),],
default='bezier'
)
# override_espeak_language_code: bpy.props.BoolProperty(
# name="Specify Espeak Language Code",
Expand Down Expand Up @@ -205,6 +204,8 @@ class ParrotLipsyncProps(bpy.types.PropertyGroup):
name="Word pad frames",
description="Number of frames surrounding a word to place rests",
default=2,
min=1,
soft_max=10
)
rig_action_suffix: bpy.props.StringProperty(
name="Action Name Suffix",
Expand Down

0 comments on commit 9f5fbf9

Please sign in to comment.