Skip to content

Commit

Permalink
Add sending audio failure and recovery events (#2390)
Browse files Browse the repository at this point in the history 
Co-authored-by: Venkatesh Devale <[email protected]>
  • Loading branch information
@avinashmidathada @devalevenkatesh
avinashmidathada and devalevenkatesh authored Aug 17, 2022
1 parent bce872c commit 6037e5c
Show file tree
Hide file tree
Showing 40 changed files with 4,318 additions and 2,939 deletions.
6 changes: 6 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added

- Add support in `MessagingSession` to allow websocket connection for messaging service to select sort by for Prefetch.
- Added two new meeting events: `sendingAudioFailed` to indicate a failure in sending audio packets out and `sendingAudioRecovered` to indicate recovery after a `sendingAudioFailed` event.
- Added a new `ConnectionHealthPolicy` named `SendingAudioFailureConnectionHealthPolicy` to evaluate and trigger the above events. Also, added `sendingAudioFailureSamplesToConsider` and `sendingAudioFailureInitialWaitTimeMs` to `ConnectionHealthPolicyConfiguration` fields for configuring the new health policy.
- Added a new field `consecutiveStatsWithNoAudioPacketsSent` in `ConnectionHealthData` for evaluating the `SendingAudioFailureConnectionHealthPolicy`.

### Removed

### Changed

- Update package.json to include Node 18.
- Refactored `connectionHealthDidChange()` in `MonitorTask` by creating a new `applyHealthPolicy()` method to reduce redundancy in the health policy evaluation.
- Modified `SignalingAndMetricsConnectionMonitor` to update `consecutiveStatsWithNoAudioPacketsSent`.
- Reset `connectionHealthData` on every connect (as opposed to doing it only when `signalingClient` is not initialized). This allows us to honor the cool-down time in both `SendingAudioFailureConnectionHealthPolicy` and `UnusableAudioWarningConnectionHealthPolicy` in case when the connection is failing due to an error like "no ice candidates were gathered".

### Fixed

Expand Down
17 changes: 17 additions & 0 deletions demos/browser/app/meetingV2/meetingV2.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1640,6 +1640,8 @@ export class DemoMeetingApp
case 'meetingReconnected':
case 'receivingAudioDropped':
case 'signalingDropped':
case 'sendingAudioFailed':
case 'sendingAudioRecovered':
case 'attendeePresenceReceived': {
// Exclude the "meetingHistory" attribute for successful -> published events.
this.meetingEventPOSTLogger?.info(
Expand Down Expand Up @@ -2523,6 +2525,12 @@ export class DemoMeetingApp
const additionalStereoTestDevices = ['L-500Hz R-1000Hz', 'Prerecorded Speech (Stereo)'];
const additionalToggles = [];

if (!this.defaultBrowserBehavior.hasFirefoxWebRTC()) {
// We don't add this in Firefox because there is no known mechanism, using MediaStream or WebAudio APIs,
// to *not* generate audio in Firefox. By default, everything generates silent audio packets in Firefox.
additionalDevices.push('No Audio');
}

// This can't work unless Web Audio is enabled.
if (this.enableWebAudio && this.supportsVoiceFocus) {
additionalToggles.push({
Expand Down Expand Up @@ -2920,7 +2928,16 @@ export class DemoMeetingApp
}
}

if (value === 'No Audio') {
// An empty media stream destination without any source connected to it, so it doesn't generate any audio.
// This is currently only used for integration testing of 'sendingAudioFailed' and 'sendingAudioRecovered' events.
// Note: It's currently not possible to emulate 'No Audio' in Firefox, so we don't provide it
// as an option in the audio inputs list.
return DefaultDeviceController.getAudioContext().createMediaStreamDestination().stream;
}

if (value === 'None' || value === '') {
// When the device is passed in as null, the SDK will synthesize an empty audio device that generates silence.
return null;
}

Expand Down
5,745 changes: 2,916 additions & 2,829 deletions docs/assets/js/search.js
View file

Large diffs are not rendered by default.

48 changes: 37 additions & 11 deletions docs/classes/baseconnectionhealthpolicy.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,9 @@ <h3>Hierarchy</h3>
<li>
<a href="reconnectionhealthpolicy.html" class="tsd-signature-type" data-tsd-kind="Class">ReconnectionHealthPolicy</a>
</li>
<li>
<a href="sendingaudiofailureconnectionhealthpolicy.html" class="tsd-signature-type" data-tsd-kind="Class">SendingAudioFailureConnectionHealthPolicy</a>
</li>
<li>
<a href="unusableaudiowarningconnectionhealthpolicy.html" class="tsd-signature-type" data-tsd-kind="Class">UnusableAudioWarningConnectionHealthPolicy</a>
</li>
Expand All @@ -97,13 +100,14 @@ <h3>Constructors</h3>
<li class="tsd-kind-constructor tsd-parent-kind-class"><a href="baseconnectionhealthpolicy.html#constructor" class="tsd-kind-icon">constructor</a></li>
</ul>
</section>
<section class="tsd-index-section tsd-is-private-protected">
<section class="tsd-index-section ">
<h3>Properties</h3>
<ul class="tsd-index-list">
<li class="tsd-kind-property tsd-parent-kind-class tsd-is-protected"><a href="baseconnectionhealthpolicy.html#currentdata" class="tsd-kind-icon">current<wbr>Data</a></li>
<li class="tsd-kind-property tsd-parent-kind-class tsd-is-protected"><a href="baseconnectionhealthpolicy.html#currenthealth" class="tsd-kind-icon">current<wbr>Health</a></li>
<li class="tsd-kind-property tsd-parent-kind-class tsd-is-protected"><a href="baseconnectionhealthpolicy.html#maxhealth" class="tsd-kind-icon">max<wbr>Health</a></li>
<li class="tsd-kind-property tsd-parent-kind-class tsd-is-protected"><a href="baseconnectionhealthpolicy.html#minhealth" class="tsd-kind-icon">min<wbr>Health</a></li>
<li class="tsd-kind-property tsd-parent-kind-class"><a href="baseconnectionhealthpolicy.html#name" class="tsd-kind-icon">name</a></li>
</ul>
</section>
<section class="tsd-index-section ">
Expand All @@ -127,13 +131,13 @@ <h2>Constructors</h2>
<a name="constructor" class="tsd-anchor"></a>
<h3>constructor</h3>
<ul class="tsd-signatures tsd-kind-constructor tsd-parent-kind-class">
<li class="tsd-signature tsd-kind-icon">new <wbr>Base<wbr>Connection<wbr>Health<wbr>Policy<span class="tsd-signature-symbol">(</span>configuration<span class="tsd-signature-symbol">: </span><a href="connectionhealthpolicyconfiguration.html" class="tsd-signature-type" data-tsd-kind="Class">ConnectionHealthPolicyConfiguration</a>, data<span class="tsd-signature-symbol">: </span><a href="connectionhealthdata.html" class="tsd-signature-type" data-tsd-kind="Class">ConnectionHealthData</a><span class="tsd-signature-symbol">)</span><span class="tsd-signature-symbol">: </span><a href="baseconnectionhealthpolicy.html" class="tsd-signature-type" data-tsd-kind="Class">BaseConnectionHealthPolicy</a></li>
<li class="tsd-signature tsd-kind-icon">new <wbr>Base<wbr>Connection<wbr>Health<wbr>Policy<span class="tsd-signature-symbol">(</span>configuration<span class="tsd-signature-symbol">: </span><a href="connectionhealthpolicyconfiguration.html" class="tsd-signature-type" data-tsd-kind="Class">ConnectionHealthPolicyConfiguration</a>, data<span class="tsd-signature-symbol">: </span><a href="connectionhealthdata.html" class="tsd-signature-type" data-tsd-kind="Class">ConnectionHealthData</a>, name<span class="tsd-signature-symbol">?: </span><span class="tsd-signature-type">string</span><span class="tsd-signature-symbol">)</span><span class="tsd-signature-symbol">: </span><a href="baseconnectionhealthpolicy.html" class="tsd-signature-type" data-tsd-kind="Class">BaseConnectionHealthPolicy</a></li>
</ul>
<ul class="tsd-descriptions">
<li class="tsd-description">
<aside class="tsd-sources">
<ul>
<li>Defined in <a href="https://github.com/aws/amazon-chime-sdk-js/blob/main/src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts#L12">src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts:12</a></li>
<li>Defined in <a href="https://github.com/aws/amazon-chime-sdk-js/blob/main/src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts#L13">src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts:13</a></li>
</ul>
</aside>
<h4 class="tsd-parameters-title">Parameters</h4>
Expand All @@ -144,13 +148,16 @@ <h5>configuration: <a href="connectionhealthpolicyconfiguration.html" class="tsd
<li>
<h5>data: <a href="connectionhealthdata.html" class="tsd-signature-type" data-tsd-kind="Class">ConnectionHealthData</a></h5>
</li>
<li>
<h5><span class="tsd-flag ts-flagOptional">Optional</span> name: <span class="tsd-signature-type">string</span></h5>
</li>
</ul>
<h4 class="tsd-returns-title">Returns <a href="baseconnectionhealthpolicy.html" class="tsd-signature-type" data-tsd-kind="Class">BaseConnectionHealthPolicy</a></h4>
</li>
</ul>
</section>
</section>
<section class="tsd-panel-group tsd-member-group tsd-is-private-protected">
<section class="tsd-panel-group tsd-member-group ">
<h2>Properties</h2>
<section class="tsd-panel tsd-member tsd-kind-property tsd-parent-kind-class tsd-is-protected">
<a name="currentdata" class="tsd-anchor"></a>
Expand Down Expand Up @@ -192,6 +199,22 @@ <h3><span class="tsd-flag ts-flagProtected">Protected</span> min<wbr>Health</h3>
</ul>
</aside>
</section>
<section class="tsd-panel tsd-member tsd-kind-property tsd-parent-kind-class">
<a name="name" class="tsd-anchor"></a>
<h3><span class="tsd-flag ts-flagOptional">Optional</span> <span class="tsd-flag ts-flagReadonly">Readonly</span> name</h3>
<div class="tsd-signature tsd-kind-icon">name<span class="tsd-signature-symbol">:</span> <span class="tsd-signature-type">string</span></div>
<aside class="tsd-sources">
<p>Implementation of <a href="../interfaces/connectionhealthpolicy.html">ConnectionHealthPolicy</a>.<a href="../interfaces/connectionhealthpolicy.html#name">name</a></p>
<ul>
<li>Defined in <a href="https://github.com/aws/amazon-chime-sdk-js/blob/main/src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts#L13">src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts:13</a></li>
</ul>
</aside>
<div class="tsd-comment tsd-typography">
<div class="lead">
<p>A unique readable name for the purpose of logging</p>
</div>
</div>
</section>
</section>
<section class="tsd-panel-group tsd-member-group ">
<h2>Methods</h2>
Expand All @@ -205,7 +228,7 @@ <h3>get<wbr>Connection<wbr>Health<wbr>Data</h3>
<li class="tsd-description">
<aside class="tsd-sources">
<ul>
<li>Defined in <a href="https://github.com/aws/amazon-chime-sdk-js/blob/main/src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts#L37">src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts:37</a></li>
<li>Defined in <a href="https://github.com/aws/amazon-chime-sdk-js/blob/main/src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts#L43">src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts:43</a></li>
</ul>
</aside>
<h4 class="tsd-returns-title">Returns <a href="connectionhealthdata.html" class="tsd-signature-type" data-tsd-kind="Class">ConnectionHealthData</a></h4>
Expand All @@ -223,7 +246,7 @@ <h3>health</h3>
<aside class="tsd-sources">
<p>Implementation of <a href="../interfaces/connectionhealthpolicy.html">ConnectionHealthPolicy</a>.<a href="../interfaces/connectionhealthpolicy.html#health">health</a></p>
<ul>
<li>Defined in <a href="https://github.com/aws/amazon-chime-sdk-js/blob/main/src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts#L29">src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts:29</a></li>
<li>Defined in <a href="https://github.com/aws/amazon-chime-sdk-js/blob/main/src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts#L35">src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts:35</a></li>
</ul>
</aside>
<div class="tsd-comment tsd-typography">
Expand All @@ -246,7 +269,7 @@ <h3>health<wbr>IfChanged</h3>
<aside class="tsd-sources">
<p>Implementation of <a href="../interfaces/connectionhealthpolicy.html">ConnectionHealthPolicy</a>.<a href="../interfaces/connectionhealthpolicy.html#healthifchanged">healthIfChanged</a></p>
<ul>
<li>Defined in <a href="https://github.com/aws/amazon-chime-sdk-js/blob/main/src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts#L45">src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts:45</a></li>
<li>Defined in <a href="https://github.com/aws/amazon-chime-sdk-js/blob/main/src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts#L51">src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts:51</a></li>
</ul>
</aside>
<div class="tsd-comment tsd-typography">
Expand All @@ -269,7 +292,7 @@ <h3>healthy</h3>
<aside class="tsd-sources">
<p>Implementation of <a href="../interfaces/connectionhealthpolicy.html">ConnectionHealthPolicy</a>.<a href="../interfaces/connectionhealthpolicy.html#healthy">healthy</a></p>
<ul>
<li>Defined in <a href="https://github.com/aws/amazon-chime-sdk-js/blob/main/src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts#L41">src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts:41</a></li>
<li>Defined in <a href="https://github.com/aws/amazon-chime-sdk-js/blob/main/src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts#L47">src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts:47</a></li>
</ul>
</aside>
<div class="tsd-comment tsd-typography">
Expand All @@ -292,7 +315,7 @@ <h3>maximum<wbr>Health</h3>
<aside class="tsd-sources">
<p>Implementation of <a href="../interfaces/connectionhealthpolicy.html">ConnectionHealthPolicy</a>.<a href="../interfaces/connectionhealthpolicy.html#maximumhealth">maximumHealth</a></p>
<ul>
<li>Defined in <a href="https://github.com/aws/amazon-chime-sdk-js/blob/main/src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts#L25">src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts:25</a></li>
<li>Defined in <a href="https://github.com/aws/amazon-chime-sdk-js/blob/main/src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts#L31">src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts:31</a></li>
</ul>
</aside>
<div class="tsd-comment tsd-typography">
Expand All @@ -315,7 +338,7 @@ <h3>minimum<wbr>Health</h3>
<aside class="tsd-sources">
<p>Implementation of <a href="../interfaces/connectionhealthpolicy.html">ConnectionHealthPolicy</a>.<a href="../interfaces/connectionhealthpolicy.html#minimumhealth">minimumHealth</a></p>
<ul>
<li>Defined in <a href="https://github.com/aws/amazon-chime-sdk-js/blob/main/src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts#L21">src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts:21</a></li>
<li>Defined in <a href="https://github.com/aws/amazon-chime-sdk-js/blob/main/src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts#L27">src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts:27</a></li>
</ul>
</aside>
<div class="tsd-comment tsd-typography">
Expand All @@ -338,7 +361,7 @@ <h3>update</h3>
<aside class="tsd-sources">
<p>Implementation of <a href="../interfaces/connectionhealthpolicy.html">ConnectionHealthPolicy</a>.<a href="../interfaces/connectionhealthpolicy.html#update">update</a></p>
<ul>
<li>Defined in <a href="https://github.com/aws/amazon-chime-sdk-js/blob/main/src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts#L33">src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts:33</a></li>
<li>Defined in <a href="https://github.com/aws/amazon-chime-sdk-js/blob/main/src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts#L39">src/connectionhealthpolicy/BaseConnectionHealthPolicy.ts:39</a></li>
</ul>
</aside>
<div class="tsd-comment tsd-typography">
Expand Down Expand Up @@ -388,6 +411,9 @@ <h4 class="tsd-returns-title">Returns <span class="tsd-signature-type">void</spa
<li class=" tsd-kind-property tsd-parent-kind-class tsd-is-protected">
<a href="baseconnectionhealthpolicy.html#minhealth" class="tsd-kind-icon">min<wbr>Health</a>
</li>
<li class=" tsd-kind-property tsd-parent-kind-class">
<a href="baseconnectionhealthpolicy.html#name" class="tsd-kind-icon">name</a>
</li>
<li class=" tsd-kind-method tsd-parent-kind-class">
<a href="baseconnectionhealthpolicy.html#getconnectionhealthdata" class="tsd-kind-icon">get<wbr>Connection<wbr>Health<wbr>Data</a>
</li>
Expand Down
Loading

0 comments on commit 6037e5c

Please sign in to comment.