BS Functions

Macros are a combination of multiple Functions combined in a block. They can be defined in two ways:

Single line nested Macros

Macros defined in a single-line nested fashion look like this:

RFF(x1, w1, b1)=RectifiedLinear(Plus(Times(w1,x1),b1))

Functions will be called and evaluated from the innermost to the outermost. That is, in the example above Function Times(w1,x1) will be called first followed by the call of Function Plus() with the outcome of Times(w1,x1) call as the first parameter. Finally RectifiedLinear() is called with the outcome of Plus() as the input parameter. Macro is resolved during the first call and the result is stored with the user assigned variable. For instance:

RFFResult=RFF(x1, w1, b1)

Programming block Macros

Another method of defining Macros uses a “programming block” style:

FF(X1, W1, B1)
{
	T=Times(W1,X1);
    FF=Plus(T, B1);
}

In this case intermediate variables, which are local to the Macro, can be accessed from the outside using the dot syntax for variables.

Note the following about Macros:

• Parameters used in Macros are local to each Macro
• Return Value of a Macro is returned via the local Macro variable that has the same name as the Macro. If no such variable found, the last variable in the Macro will be returned
• Parameters can be declared within Macros
• A Macro can call another Macro, but recursion is not supported
• # declares a comment line

Examples

This Macro

RFF(x1, w1, b1) = RectifiedLinear(Plus(Times(w1,x1),b1))

is equivalent to the following three lines

Times1=Times(W0, features)    
Plus1=Plus(Times1, B0)    
RFF = RectifiedLinear(Plus1)

---

This Macro is a feed forward computation without the energy function. It shows the alternative format of Macros. Semicolons are optional. As specified above the Return Value of a Macro is returned via the local Macro variable that has the same name as the Macro. In this case the FF() Macro return value will be stored in FF local variable.

FF(X1, W1, B1)
{
    T=Times(W1,X1);
    FF=Plus(T, B1);
}

---

This Macro shows how parameters can be declared within a Macro. It also shows that # as the first character in line specifies a comment line. Finally this example shows that a Macro may call another Macro (note that recursion is not supported).

# Base Feed Forward network, includes Bias and weight parameters
BFF(in, rows, cols)
{
    B=Parameter(rows)
    W=Parameter(rows, cols)
    BFF = FF(in, w, b)
}

---

This Macro calls the previous Macro adding the RectifiedLinear() energy function for a complete layer.

RBFF(input,rowCount,colCount)
{
    F = BFF(input, rowCount, colCount);
    RBFF = RectifiedLinear(F);
}

---

This Macro defines a full Top layer, also using the BFF as the other full layer Macro. In this case no variables match the name of the Macro, so the SM variable will be used as the return value, since it is the last value defined in the Macro.

SMBFF(x,r,c, labels)
{
    F = BFF(x,r,c);  
    SM = CrossEntropyWithSoftmax(labels, F)
}

---

The example below uses all the Macros defined above. The network definition is equivalent to the network presented in NDL Basic concepts but using Macros.

Note how to manage access to Macro variables. ErrorPrediction() needs accessing the result of the FeedForward result before the CrossEntropyWithSoftmax() is applied to it. However the needed variable is local to the Macro, but can still be accessed via “dot” syntax. The return value of the Macro is CE variable, so to access the local F variable defined in the Macro itself CE.F syntax can be used. Note, that in the single line version of Macros this syntax can not be used, because single line Macros do not support user defined variable names.

# constants defined
# Sample, Hidden, and Label dimensions
SDim=28*28
HDim=256
LDim=10

features=Input(SDim)
labels=Input(LDim)

# Layer operations
L1 = RBFF(features, HDim, SDim)
CE = SMBFF(L1, LDim, HDim, labels)
Err=ErrorPrediction(labels, CE.F)