Skip to content
Frank Seide edited this page Aug 27, 2016 · 3 revisions

You might be wondering if Dropout applied to a Constant is dynamically evaluated. It is! Therefore Zoneout is as simple as

Zoneout (x, xprev) =
{
    mask = Dropout (BS.Constants.OnesLike (x))
    res = BS.Boolean.If (mask, xprev, x)
}.res
Clone this wiki locally