Skip to content

Full Function Reference

Jump to bottom Edit New page
Frank Seide edited this page Aug 19, 2016 · 76 revisions

This section provides information on BrainScript built-in functions.

The declarations of all built-in functions can be found in the CNTK.core.bs located next to the CNTK binary.

The primitive operations and layers are declared in the global namespace. Additional operations are declared in namespaces, and will be given with the respective prefix (e.g. BS.RNN.LSTMP).

Layers (note: requires latest master build, or 1.7 binaries available mid August)

  • DenseLayer {outDim, bias= true, activation=Identity, init='uniform', initValueScale=1}
  • ConvolutionalLayer {numOutputChannels, filterShape, activation = Identity,
    init = "uniform", initValueScale = 1,
    stride = 1, pad = false, lowerPad = 0, upperPad = 0, bias=true, maxTempMemSizeInSamples = 0}`
  • MaxPoolingLayer {filterShape, stride = 1, pad = false, lowerPad = 0, upperPad = 0}
  • AveragePoolingLayer {filterShape, stride = 1, pad = false, lowerPad = 0, upperPad = 0}
  • EmbeddingLayer {outDim, embeddingPath = '', transpose = false}
  • RecurrentLSTMLayer {outputDim, cellShape = None, goBackwards = false, enableSelfStabilization = false}
  • DelayLayer {T=1, defaultHiddenActivation=0}
  • Dropout
  • BatchNormalizationLayer {spatialRank = 0, initialScale = 1, normalizationTimeConstant = 0, blendTimeConstant = 0, epsilon = 0.00001, useCntkEngine = true}
  • LayerNormalizationLayer {dim = None, initScale = 1, initBias = 0}
  • StabilizerLayer{}
  • FeatureMVNLayer{}

Activation functions

Elementwise operations, unary

Elementwise operations, binary

Elementwise operations, ternary

  • If (condition, thenVal, elseVal)

Matrix product and convolution

  • Times(A, B, outputRank=1)
    A * B

  • TransposeTimes(A, B, outputRank=1)

  • Convolution(weights, x, kernelShape, mapCount=(0), stride=(1), sharing=(true), autoPadding=(true), lowerPadding=(0), upperPadding=(0), imageLayout='CHW', maxTempMemSizeInSamples=0)

  • Pooling(x, poolKind/*'max'|'average'*/, kernelShape, stride=(1), autoPadding=(true), lowerPadding=(0), upperPadding=(0), imageLayout='CHW')

Learnable parameters and constants

  • ParameterTensor {shape, learningRateMultiplier=1.0, init='uniform'/*|gaussian*/, initValueScale=1.0, initValue=0.0, randomSeed=-1, initFromFilePath=''}

  • Constant {scalarValue, rows = 1, cols = 1}

  • BS.Constants.Zero, BS.Constants.One
    BS.Constants.True, BS.Constants.False, BS.Constants.None

  • BS.Constants.OnesTensor (shape)

  • BS.Constants.ZeroSequenceLike (x)

Inputs

  • Input (shape, dynamicAxis='', sparse=false, tag='feature')

  • DynamicAxis{}

  • EnvironmentInput (propertyName)

    Mean (x), InvStdDev (x)

Loss functions and metrics

  • CrossEntropyWithSoftmax (targetDistribution, nonNormalizedLogClassPosteriors)
    CrossEntropy (targetDistribution, classPosteriors)
  • Logistic (label, probability)
    [WeightedLogistic] (./Loss-Functions-and-Metrics#logistic-weightedlogistic)(label, probability, instanceWeight)
  • ErrorPrediction (labels, nonNormalizedLogClassPosteriors)
  • MatrixL1Reg(matrix)
  • MatrixL2Reg(matrix)
  • SquareError (x, y)

Reductions

  • ReduceSum (z, axis=0)
    ReduceLogSum (z)

  • Hardmax (z)

  • CosDistance (x, y)

  • SumElements (z)

Recurrence

  • BS.Loop.Previous (x, timeStep=1, defaultHiddenActivation=0)
    PastValue (shape, x, defaultHiddenActivation=0.1, ...) = BS.Loop.Previous (0, shape, ...)

  • BS.Loop.Next (x, timeStep=1, defaultHiddenActivation=0)
    FutureValue (shape, x, defaultHiddenActivation=0.1, ...) = BS.Loop.Next (0, shape, ...)

  • LSTMP (outputDim, cellDim=outputDim, x, inputDim=x.shape, aux=BS.Constants.None, auxDim=aux.shape, prevState, enableSelfStabilization=false)

  • BS.Boolean.Toggle (clk, initialValue=BS.Constants.False)

Training operations

Reshaping operations

  • CNTK2.Reshape (x, shape, beginAxis=0, endAxis=0)
    ReshapeDimension (x, axis, shape) = CNTK2.Reshape (x, shape, beginAxis=axis, endAxis=axis + 1)
    FlattenDimensions (x, axis, num) = CNTK2.Reshape (x, 0, beginAxis=axis, endAxis=axis + num)
    SplitDimension (x, axis, N) = ReshapeDimension (x, axis, 0:N)

  • Slice (beginIndex, endIndex, input, axis=1)
    BS.Sequences.First (x) = Slice (0, 1, x, axis=-1)
    BS.Sequences.Last (x) = Slice (-1, 0, x, axis=-1)

  • Splice (inputs, axis=1)

  • TransposeDimensions (x, axis1, axis2)
    Transpose (x) = TransposeDimensions (x, 1, 2)

  • BS.Sequences.BroadcastSequenceAs (type, data1)

  • BS.Sequences.Gather (where, x)
    BS.Sequences.Scatter (where, y)
    Where (cond)
    IsFirst (x)
    IsLast (x)

Recurrence

  • BS.RNNs.RecurrentLSTMP (outputDim, cellDim=outputDim, x, inputDim=x.shape, previousHook=BS.RNNs.PreviousHC, augmentInputHook=NoAuxInputHook, augmentInputDim=0, layerIndex=0, enableSelfStabilization=false)

  • BS.RNNs.RecurrentLSTMPStack (layerShapes, cellDims=layerShapes, input, inputShape=input.shape, previousHook=PreviousHC, augmentInputHook=NoAuxInputHook, augmentInputShape=0, enableSelfStabilization=false)

  • BS.RNNs.RecurrentBirectionalLSTMPStack (layerShapes, cellDims=layerShapes, input, inputShape=input.dim, previousHook=PreviousHC, nextHook=NextHC, enableSelfStabilization=false)

Sequence-to-sequence support

  • BS.Seq2Seq.CreateAugmentWithFixedWindowAttentionHook (attentionDim, attentionSpan, decoderDynamicAxis, encoderOutput, enableSelfStabilization=false)
  • BS.Seq2Seq.GreedySequenceDecoderFrom (modelAsTrained)
  • BS.Seq2Seq.BeamSearchSequenceDecoderFrom (modelAsTrained, beamDepth)

Special-purpose operations

  • ClassBasedCrossEntropyWithSoftmax (labelClassDescriptorVectorSequence, mainInputInfo, mainWeight, classLogProbsBeforeSoftmax)

Model editing

Other

  • Fail (what)
  • IsSameObject (a, b)
  • Trace (node, say='', logFrequency=traceFrequency, logFirst=10, logGradientToo=false, onlyUpToRow=100000000, onlyUpToT=100000000, format=[])

Deprecated

  • ColumnElementTimes (...) = ElementTimes (...)
  • DiagTimes (...) = ElementTimes (...)
  • LearnableParameter(...) = Parameter(...)
  • LookupTable (embeddingMatrix, inputTensor)
  • RowRepeat (input, numRepeats)
  • RowSlice (beginIndex, numRows, input) = Slice(beginIndex, beginIndex + numRows, input, axis = 1)
  • RowStack (inputs)
  • RowElementTimes (...) = ElementTimes (...)
  • Scale (...) = ElementTimes (...)
  • ConstantTensor (scalarVal, shape)
    Parameter (outputDim, inputDim, ...) = ParameterTensor ((outputDim:input), ...)
    WeightParam (outputDim, inputDim) = Parameter (outputDim, inputDim, init='uniform', initValueScale=1, initOnCPUOnly=true, randomSeed=1)
    DiagWeightParam (outputDim) = ParameterTensor ((outputDim), init='uniform', initValueScale=1, initOnCPUOnly=true, randomSeed=1)
    BiasParam (dim) = ParameterTensor ((dim), init='fixedValue', value=0.0)
    ScalarParam() = BiasParam (1)
  • SparseInput (shape, dynamicAxis='', tag='feature')
    ImageInput (imageWidth, imageHeight, imageChannels, imageLayout='CHW', dynamicAxis='', tag='feature')
    SparseImageInput (imageWidth, imageHeight, imageChannels, imageLayout='CHW', dynamicAxis='', tag='feature')
  • MeanVarNorm(feat) = PerDimMeanVarNormalization(feat, Mean (feat), InvStdDev (feat))
    PerDimMeanVarNormalization (x, mean, invStdDev),
    PerDimMeanVarDeNormalization (x, mean, invStdDev)
  • ReconcileDynamicAxis (dataInput, layoutInput)
Add a custom footer

Getting Started

Additional Documentation

How to use CNTK

Using CNTK Models in Your Code

Advanced topics

Licenses

Source Code & Development

Clone this wiki locally