*Used Posenet to OSC
This repository contains a modified version of PoseNet by Dan Oved that sends detected keypoints as OSC encoded datagram packets. This enables tracked keypoints to be used within any OSC compatible environment (Max/MSP, Python, PD, C++, Processing, you name it). This is effectively PoseNet + osc-js.
PoseNet is a wabcam-based real-time motion tracking system that runs in a browser using TensorFlow.
PoseNet can be used to estimate either a single pose or multiple poses, meaning there is a version of the algorithm that can detect only one person in an image/video and one version that can detect multiple persons in an image/video.
Refer to this blog post for a high-level description of PoseNet running on Tensorflow.js.
To get this running stand alone on your computer, clone this repo and follow these steps:
Install node (using homebrew):
brew install nodeInstall yarn:
brew install yarnInstall dependencies and prepare the build directory:
yarnWatch files for changes, and launch a dev server:
yarn watchIn another terminal window run the bridge node application:
node bridge.jsIn Firefox or Chrome, browse to http://localhost:1234
Now keypoints can be received as OSC messages on port 9876.
OSC messages will have an address pattern /pose/n, where n is the index of the pose (person) found in a frame, and all keypoints for that pose will follow as arguments in the order part, x, y where part is the keypoint body part (string) and x and y are the coordinates of the part (floats).
All keypoints correspond to a body part. The parts are:
| Part |
|---|
| nose |
| leftEye |
| rightEye |
| leftEar |
| rightEar |
| leftShoulder |
| rightShoulder |
| leftElbow |
| rightElbow |
| leftWrist |
| rightWrist |
| leftHip |
| rightHip |
| leftKnee |
| rightKnee |
| leftAnkle |
| rightAnkle |