Finding approximate distance between facial landmarks

[Jeandcc]

Hi all. Could someone please point me towards an example of how I could determine the approximate distance between facial landmarks? I went through all the wiki pages but couldn't find anything.

One example need I have is to get an approximate distance (in millimeters) from left iris to right iris.

I was doing something similar to the below, but I then realized I was looking at the wrong distance method.

human.detect(input, userConfig)
      .then((result) => {
        const annotations = result.face?.[0]?.annotations;
        console.log(`DISTANCES`, {
          ipd: human.distance(
            annotations.leftEyeIris?.[0],
            annotations.rightEyeIris?.[0],
          )
        })
        [...]

[vladmandic]

example of a simplified code how human measures distance of user from camera extrapolated from iris size
you could use similar math to approximate distance from left to right eye.

// lets look at both left iris as an example
const irisSize = Math.abs(faces[i].annotations.leftEyeIris[3][0] - faces[i].annotations.leftEyeIris[1][0]), // size of left eye iris in pixels
// now we can extrapolate how far is face from camera
// constant for average webcam camera field-of-view, may need to be adjusted for other camera types if they are ultra-wide or tele-photo
const fovConstant = 500;
// average human iris size is relatively constant at 11.7mm regardless of age/gender
const distance = Math.trunc(fovConstant / irisSize / 11.7) / 100;

ideally, you should look at both iris sizes and pick one that is more visible (closer to camera) as extrapolation base, but this is just an example

btw, human.distance method is something completely different, it measures facial similarity/differences between two different inputs and its used for facial recognition

[Jeandcc]

Hi @vladmandic . Thanks for the help!

I used what you provided and tried to evolve it to a point where I could obtain the distance between irises, but I'm falling short of obtaining a reliable result:

           // constant for average webcam camera field-of-view, may need to be adjusted for other camera types if they are ultra-wide or tele-photo
           const FOV_CONSTANT = 500;

           // average human iris size is relatively constant at 11.7mm regardless of age/gender
           const IRIS_SIZE_CONSTANT = 11.7;

           const annotations = result.face?.[0]?.annotations;

            const irisPxSize = (annotations?.leftEyeIris?.[0] && annotations?.rightEyeIris?.[0]
              && (annotations.leftEyeIris.length > 0) && (annotations.rightEyeIris.length > 0)
              && (annotations.leftEyeIris[0] !== null) && (annotations.rightEyeIris[0] !== null))
              ? Math.max(Math.abs(annotations.leftEyeIris[3][0] - annotations.leftEyeIris[1][0]), Math.abs(annotations.rightEyeIris[4][0] - annotations.rightEyeIris[2][0]))
              : 0;

            // now we can extrapolate how far is face from camera
            const irisSize = irisPxSize / human.process.tensor.shape[2]
            const mmDistanceOfIrisToCamera = Math.trunc(FOV_CONSTANT / irisSize / IRIS_SIZE_CONSTANT);

            const centerXofLeftIris = (annotations.leftEyeIris[3][0] + annotations.leftEyeIris[1][0]) / 2
            const centerXofRightIris = (annotations.rightEyeIris[4][0] + annotations.rightEyeIris[2][0]) / 2
            const pxDistBetweenIrises = Math.abs(centerXofLeftIris - centerXofRightIris)

            // NEED HELP ON LINES BELOW
            const growthFactorBasedOnAvgIrisSize = irisPxSize / IRIS_SIZE_CONSTANT;
            const mmDistBetweenIrises = pxDistBetweenIrises * growthFactorBasedOnAvgIrisSize;

I feel like there's some essential piece of knowledge regarding Computer Vision that I'm missing...

Anyhow, would you be able to help me get this over the finish line? I feel like it's something simple that I'm missing out...

Thanks

[vladmandic]

actually, forget about FOV, i no longer think its needed and will change math in human.
anyhow, is just tested this and seems to be semi-decent. as expected, it does show a lot of varying since iris size itself is small and difficult to predict accurately, but its the only fixed point that math can be based on.

  // iris points are array of [center, left, top, right, bottom] each with [x,y,z] values
  // average size of human iris is 11.7mm - fairly constant for all ages/genders/races
  // average distance between eyes is 65mm - fairly constant for typical adult male, but varies otherwise
  // get annotated keypoints so we dont have to search for them manually
  const face = human.result.face[0]?.annotations;
  // we dont have face and iris details
  if (!face?.leftEyeIris) return;
  // get size of left and right iris in pixels, pick larger one as its likely to be more accurate and normalize to 0..1 range instead of pixels
  const irisSize = Math.max(Math.abs(face.leftEyeIris[3][0] - face.leftEyeIris[1][0]), Math.abs(face.rightEyeIris[3][0] - face.rightEyeIris[1][0])) / width;
  // pixel x and y distance of centers of left and right iris, you can use edges instead
  const irisDistanceXY = [face.leftEyeIris[0][0] - face.rightEyeIris[0][0], face.leftEyeIris[0][1] - face.rightEyeIris[0][1]];
  // absolute distance bewtween eyes in 0..1 range to account for head pitch (we can ignore yaw)
  const irisDistance = Math.sqrt((irisDistanceXY[0] * irisDistanceXY[0]) + (irisDistanceXY[1] * irisDistanceXY[1])) / width;
  // distance of eye from camera in meters
  const cameraDistance = 1.17 / irisSize / 100;
  // distance between eyes in meters
  const eyesDistance = 1.17 * irisDistance / irisSize / 100;
  console.log(cameraDistance, eyesDistance);

[Jeandcc]

Hi @vladmandic .

Thanks for updating the response.

I'm just not sure: Where is this width variable used in the irisSize calculation coming from?

[vladmandic]

its just the width of input frame, e.g. video.videoWidth or canvas.width or like you used above human.process.tensor.shape[2]
ideally, it should also account for height, but since this math looks only at iris left and right edges and not top and bottom, height doesnt matter. and left/right is always going to be more accurate than top/bottom because of the shape of the eye.