Skip to content

Gesture

Jump to bottom
Brice Donval edited this page Jan 13, 2020 · 7 revisions

Table of Contents

Gestures 

#allow(WIKI_CREATE)

In GretaModular, gestures are the movements executed with the arms.

These gestures follow Kendon description of gestures when describing their phases. When executing a gesture, we have a preparation phase, a stroke and a retraction phase. 

#allow(WIKI_CREATE)

The gestures are defined in the file gestuary.xml.

New gestures can be defined in this file using an XML description of the gestures.

The content of this file can also be edited using the GestureEditor tool, a graphical interface for gesture edition, provided with GretaModular.

Naming Convention

In GretaModular, a naming convention is followed when creating gestures. A gesture is referenced by its category and its name/id.

The category is a single word, like performative or iconic for instance. The name contains several informations :

  • The actual name of the gesture
  • The type if it uses one (a position) or several (a complete gesture) phases
  • If it uses one or both hands
  • An (optional) adjective to differentiate similar gestures
<gesture category="<categoryName>" id="<actualName>_<Ges/Pos>_<B/R/L>%_<adjective>%">
</gesture>

Examples: 

<gesture category="performative" id="accept_Ges_L">
</gesture>
<gesture category="deictic" id="you_Ges_R_High">
</gesture>

Naming Rules

Category

The name of the category is always in *LOWER* letters

Name of gesture

nameofgesture*_Pos_Hand_Adjective

So for example, the gesture ‘*You*’ made with the right hand has 2 phases (ie it is a *gesture* not a position), and a rather strong movement quality; it is named:

You_*Ges*_R_Strong

Allright with 2 hands (just one phase, ie it is a *position*) with a middle height for the position of the wrist is named;

Allright_*Pos*_B_Medium

OLD gestures have been renamed with *zz* at the beginning of their category name. So they appear at the end of the list in the GestureEditor.

Phases

A gesture is composed of one or many phases. These phases describe the stroke of the gesture. If a gesture is composed of only one phase, it is considered as a position. A phase is referenced with a type :

  • STROKE-START if it is the first or the only phase of the gesture
  • STROKE-END if it is the last phase of the gesture
  • STROKE for every phase between STROKE_START and STROKE-END
Examples : 
<gesture category="deictic" id="you_Ges_R_High">
	<phase type="STROKE-START">
	</phase>
	<phase type="STROKE-END">
	</phase>
</gesture>
<gesture category="iconic" id="count_Ges_R">
	<phase type="STROKE-START">
	</phase>
	<phase type="STROKE">
	</phase>
	<phase type="STROKE">
	</phase>
	<phase type="STROKE">
	</phase>
	<phase type="STROKE-END">
	</phase>
</gesture>

Hand configurations

In each phase, a hand configuration has to be described. Both hands (left and right) can be described or only one. Hand configurations are composed of the actual position of the wrist in the space and the orientation of the hand. For now the wrist positions can be described using two different kinds of position:

  • The SymbolicPosition
  • The TouchPosition
The hand orientation is described by defining the orientation of the palm and the fingers.

Examples : 

<phase type="STROKE">
	<hand distanceFixed="true" fingersOrientationRatio="1.0"
		handShapeFixed="true" horizontalFixed="true"
		palmOrientationRatio="0.0" side="Right" verticalFixed="true">
		<verticalLocation>YUpperC</verticalLocation>
		<horizontalLocation>XC</horizontalLocation>
		<locationDistance>ZNear</locationDistance>
		<handShape>form_fist</handShape>
		<palmOrientation>INWARD</palmOrientation>
		<fingersOrientation>AWAY</fingersOrientation>
		<palmOrientationSupplementary>DOWN</palmOrientationSupplementary>
		<fingersOrientationSupplementary>AWAY</fingersOrientationSupplementary>
	</hand>
</phase>

<phase type="STROKE-START">
	<hand distanceFixed="true" fingersOrientationRatio="0.1"
		handShapeFixed="true" horizontalFixed="true"
		palmOrientationRatio="0.0" side="LEFT" verticalFixed="true">
		<verticalLocation>YUpperP</verticalLocation>
		<horizontalLocation>XC</horizontalLocation>
		<locationDistance>ZNear</locationDistance>
		<handShape>form_fist</handShape>
		<palmOrientation>INWARD</palmOrientation>
		<fingersOrientation>UP</fingersOrientation>
		<palmOrientationSupplementary>DOWN</palmOrientationSupplementary>
		<fingersOrientationSupplementary>AWAY</fingersOrientationSupplementary>
	</hand>
	<hand distanceFixed="true" fingersOrientationRatio="0.0"
		handShapeFixed="true" horizontalFixed="true"
		palmOrientationRatio="0.0" side="Right" verticalFixed="true">
		<verticalLocation>YUpperP</verticalLocation>
		<horizontalLocation>XC</horizontalLocation>
		<locationDistance>ZNear</locationDistance>
		<handShape>form_fist</handShape>
		<palmOrientation>INWARD</palmOrientation>
		<fingersOrientation>UP</fingersOrientation>
		<palmOrientationSupplementary>DOWN</palmOrientationSupplementary>
		<fingersOrientationSupplementary>AWAY</fingersOrientationSupplementary>
	</hand>
</phase>

<phase type="STROKE-START">
	<hand fingersOrientationRatio="0.5" handShapeFixed="true"
		palmOrientationRatio="0.5" side="Left">
		<touchPosition>belly</touchPosition>
		<handShape>symbol_3_open</handShape>
		<palmOrientation>TOWARD</palmOrientation>
		<fingersOrientation>INWARD</fingersOrientation>
		<palmOrientationSupplementary>TOWARD</palmOrientationSupplementary>
		<fingersOrientationSupplementary>INWARD</fingersOrientationSupplementary>
	</hand>
</phase>

New gestures can be added or edited directly by changing the gestuary.xml file, or by using the GestureEditor tool. The GestureEditor actually edit the same gestuary.xml file.

Examples of complete gestures : 

<gesture category="adjectival" id="large_Ges_B">
	<phase type="STROKE-START">
		<hand distanceFixed="true" fingersOrientationRatio="0.5"
			handShapeFixed="true" horizontalFixed="true"
			palmOrientationRatio="0.34" side="LEFT" verticalFixed="true">
			<verticalLocation>YUpperC</verticalLocation>
			<horizontalLocation>XRF</horizontalLocation>
			<locationDistance>ZMiddle</locationDistance>
			<handShape>form_open</handShape>
			<palmOrientation>INWARD</palmOrientation>
			<fingersOrientation>AWAY</fingersOrientation>
			<palmOrientationSupplementary>TOWARD</palmOrientationSupplementary>
			<fingersOrientationSupplementary>UP</fingersOrientationSupplementary>
		</hand>
		<hand distanceFixed="true" fingersOrientationRatio="0.53"
			handShapeFixed="true" horizontalFixed="true"
			palmOrientationRatio="0.34" side="Right" verticalFixed="true">
			<verticalLocation>YUpperC</verticalLocation>
			<horizontalLocation>XRF</horizontalLocation>
			<locationDistance>ZMiddle</locationDistance>
			<handShape>form_open</handShape>
			<palmOrientation>INWARD</palmOrientation>
			<fingersOrientation>AWAY</fingersOrientation>
			<palmOrientationSupplementary>TOWARD</palmOrientationSupplementary>
			<fingersOrientationSupplementary>UP</fingersOrientationSupplementary>
		</hand>
	</phase>
	<phase type="STROKE-END">
		<hand distanceFixed="true" fingersOrientationRatio="0.52"
			handShapeFixed="true" horizontalFixed="true"
			palmOrientationRatio="0.28" side="LEFT" verticalFixed="true">
			<verticalLocation>YUpperC</verticalLocation>
			<horizontalLocation>XP</horizontalLocation>
			<locationDistance>ZMiddle</locationDistance>
			<handShape>form_open</handShape>
			<palmOrientation>INWARD</palmOrientation>
			<fingersOrientation>UP</fingersOrientation>
			<palmOrientationSupplementary>AWAY</palmOrientationSupplementary>
			<fingersOrientationSupplementary>AWAY</fingersOrientationSupplementary>
		</hand>
		<hand distanceFixed="true" fingersOrientationRatio="0.57"
			handShapeFixed="true" horizontalFixed="true"
			palmOrientationRatio="0.33" side="Right" verticalFixed="true">
			<verticalLocation>YUpperC</verticalLocation>
			<horizontalLocation>XP</horizontalLocation>
			<locationDistance>ZMiddle</locationDistance>
			<handShape>form_open</handShape>
			<palmOrientation>INWARD</palmOrientation>
			<fingersOrientation>UP</fingersOrientation>
			<palmOrientationSupplementary>AWAY</palmOrientationSupplementary>
			<fingersOrientationSupplementary>AWAY</fingersOrientationSupplementary>
		</hand>
	</phase>
</gesture>

TouchPosition

When describing gestures, the wrist position can be specified using a SymbolicPosition or a TouchPosition.

The TouchPosition allows to reference a particular point described in the touchpoint additional file of a character.

In the .ini file of a character, the following line can be added : 

TOUCHPOINT=./BehaviorRealizer/Skeleton/poppy_touchpoint.xml

This will allow the character to use the touch points described in poppy_touchpoint.xml.

A touchpoint is composed of the following :

  • An id that will be used to reference it in the gestures
  • A type to identify what it is referencing
  • the id of the reference point
  • A position offset from the reference point
  • A rotation offset that is used as the normal vector of the wrist for the gestures
Example :

In the following example, the touch point is a point situated on the left-bottom part of the head. It references the skullbase bone from the skeleton. 

<touchpoints>
	<touchpoint id="skullbase_L" type="bone" reference="skullbase">
		<posOffset x="3" y="-3" z="4"/>
		<rotOffset x="0" y="0" z="-1"/>
	</touchpoint>
</touchpoints>

This point can then be used in the description of a gesture : 

<gesture category="adaptor" id="head_Pos_L">
	<phase type="STROKE-START">
		<hand fingersOrientationRatio="0.5" handShapeFixed="true"
			palmOrientationRatio="0.5" side="Left">
			<touchPosition>skullbase_L</touchPosition>
			<handShape>form_open</handShape>
			<palmOrientation>TOWARD</palmOrientation>
			<fingersOrientation>TOWARD</fingersOrientation>
			<palmOrientationSupplementary>TOWARD</palmOrientationSupplementary>
			<fingersOrientationSupplementary>TOWARD</fingersOrientationSupplementary>
		</hand>
	</phase>
</gesture>

Symbolic Positions

To keep the gesture description in abstraction level, the possible values of gesture attributes are symbolically setup. For instance the symbolical values for wrist locations (vertical, horizontal, frontal) are adopted from the concentric gestural space of McNeill (2002). The abbreviations are used like C as Center, CC as Center-Center, P as Periphery, EP as Extreme Periphery,and Opp as Opposite Side.

Abstract arm positions (thus called since they operate on the angles of the arm, although they specify the location of the wrist in space) are specified using the following enumeration types for their relative X,Y, and Z coordinates:

  • enum ArmX {XEP=0, XP, XC, XCC, XOppC, XDefault};
  • enum ArmY {YUpperEP=0, YUpperP, YUpperC, YCC, YLowerC, YLowerP, YlowerEP, YDefault};
  • enum ArmZ {ZNear=0, ZMiddle, ZFar, ZDefault};
Thus, a valid hand shape definition within a gesture definition file is of the following forms: 
 <code>ARM <ArmX> <ArmY> <ArmZ></code>

No axis type may be omitted - the definition always needs a complete (x,y,z) value triple. The terms are derived from McNeill's definition of the human gesture space (McNeill, 1992). Following abbreviations are used:

  • C = Center
  • CC = Center-Center
  • P = Periphery
  • EP = Extreme Periphery
  • Opp = Opposite Side

Home

Getting started with Greta

Greta Architecture

Quick start

Advanced

Functionalities

Core functionality

Auxiliary functionalities

Preview functionality

Nothing to show here

Previous functionality (possibly it still works, but not supported anymore)

Clone this wiki locally