Name Conventions for Paradigm Methods and Parameters

by Martijn Meeter & Jaap Murre

It would be highly beneficial to Nutshell users if methods and parameters that have the same name in different paradigms mean the same thing in those paradigms. If one has learned to use a certain method in one paradigm, one should not have to look up in a manual whether a method with the same name in a different paradigm does the same thing in both paradigms. Therefore a set of rules governs name-giving in Nutshell paradigms. For an understanding of the terms used in this text (e.g., "tract", or "layer"), see the Glossary of definitions used in Paradigm Methods and Parameters document.

Below, in this document:

• Methods
  
  • Learning, Cycling, Steps, "Cycle steps"
  • Clamp and deactivate
  • "Insert Layer", "Resize Layer", "Delete Layer", "Insert Tract", "Delete Tract"
  • Resetting layers and tracts
• Parameters of Nodes and Layers
  
  • Nodes: Clamped and Deactivated
  • Nodes: Act
  • Layer: K and hard kWTA
• Parameters of Connections and Tracts
  
  • Connections: Weight and Exists
  • Tract: "Weight Max", "Weight Sum", and "Normalized"

Top of this document.

Methods

[Back to contents]

• Learning, Cycling, Steps, "Cycle steps"
• Clamp and deactivate
• "Insert Layer", "Resize Layer", "Delete Layer", "Insert Tract", "Delete Tract"
• Resetting layers and tracts

Learning, Cycling, Steps, "Cycle steps"

Most paradigms have two different types of cycles: learning and activation cycles, in which respectively the weights and the state of the nodes are updated. This can occur for one step (iteration), but it is often more efficient to let one command result in a number of cycles. In Nutshell the global parameter name "Cycle Steps" is reserved for the number of cycle steps that one cycle-command will result in. A number of methods in this realm have set definitions:

• "Act Cycle" means updating node states (either of one node per step as in Hopfield or of all nodes simultaneously) for a "Cycle Steps" number of times. Weights must be left alone during "Act Cycle".
• "Act Step" means updating node states (either of one node or of all nodes simultaneously) just once.
• "Learn" means updating the weights once. Node states must be left alone during "Learn". It often does not make sense to "learn" for more than one step at a time, since without intermediate updating of activities this would amount to learning with a higher learning parameter. In paradigms where a difference would occur, "Learn Step" would mean updating the weights just once, and "Learn Cycle" would mean updating for a "Cycle Steps" number of times.
• "LearnAct Cycle" means first doing one "Learn Step", then one "Act Step". This combination occurs a "Cycle Steps" number of times (or once if the method is: "LearnAct Step").
• "ActLearn Cycle" means first doing one "Act Step", then one "Learn Step". This combination is repeated a "Cycle Steps" number of times (or once if the method is: "ActLearn Step" as in Kohonen).

Anything else, anything that updates weights and node variables at the same time, should have a different name than those mentioned. Anything that combines learning and activation updating in other ways that doing them sequentially should be called otherwise. If there is one dominant way in which the network iterates, but it is not "Act Cycle", not "Learn Cycle" because weights and node states are both updated, then the method name "Cycle" can be used for this iteration way. (e.g., updating of weights and activations together occurs in the MacGregor paradigm in intermixed ways, and is therefore called "Cycle").

• "Cycle" means updating weights and node states at once for a "Cycle Steps" number of times.
• "Step" means updating weights and node states at once just one time.

Since "Cycle Steps" may influence all cycling methods or none at all, paradigm authors should specify in the comment line in the parameter definition to what methods "Cycle Steps" applies.

Clamp and deactivate

Clamping a layer means making all the nodes clamped, and the same holds for deactivate, (see definitions). It is unnecessary to have methods to clamp or deactivate layers: in the browser one can easily clamp whole layers by using the mouse, while in scripting one can use the notation "layer.clamped=true/false" or "layer.deactivated=true/false". If one would wish to program methods anyway, there would be two choices for a correct nomenclature: either implement both "Clamp Layer" and "Unclamp Layer", or implement "Toggle Clamping" (of course they can also be both implemented). The same goes for deactivation.

• "Clamp Layer" makes all nodes in the selected layer clamped, independent of the state they are in (deactivated nodes are not clamped). If this method is implemented, there should be an "Unclamp Layer" command as well.
• "Unclamp Layer" makes all nodes in the selected layer unclamped, independent of their current state.
• "Toggle Clamping" may be implemented on the layer level, or on the layer and node level. On the node level it makes the selected node unclamped when it is clamped, and clamped when it is unclamped. On the layer level, it either clamps or unclamps all nodes in the selected layer. If all nodes are not in the same clamping state, whether they are clamped or not should depend on the state of the first layer node. If the first node of the layer is unclamped, then the layer gets clamped, and the other way around (this is arbitrary, but one should strive for consistency in arbitrariness).
• "Deactivate Layer" makes all nodes in the selected layer deactivated, independent of the state they are in. If this method is implemented, there should be an "Reactivate Layer" command as well.
• "Reactivate Layer" reactivates (=un-deactivates) all nodes in the selected layer, independent of their current state.
• "Toggle Deactivation" may be implemented on the layer level, or on the layer and node level. On the node level it deactivates the selected node if it is not deactivated, and reactivates it if it is deactivated. On the layer level, it either deactivates or reactivates all nodes in the selected layer depending on the state of the first layer node. If the first node of the layer is deactivated, then the layer gets un-deactivated, and the other way around.

"Insert Layer", "Resize Layer", "Delete Layer", "Insert Tract", "Delete Tract"

These are all necessary methods. They should perhaps preferably be the first methods, and appear in the above order. All have obvious meanings. Less obvious but nevertheless sensible: both insertion and resizing should set the layer or tract to a "reset" state, with layer or tract parameters at suitable default values.

Resetting layers and tracts

To resolve ambiguity as to what is reset, several resetting methods may be implemented:

• "Reset Layer Nodes" should reset node parameters and leave layer parameters alone ("Reset Layer Nodes" should NOT reset clamping or deactivation).
• "Reset Tract Conn" should reset connection parameters and leave tract parameters alone.
• "Reset Layer Pars" should reset layer parameters and leave node parameters alone.
• "Reset Tract Pars" should reset tract parameters and leave connection parameters alone.

Parameters should be reset to a sensible default; what those defaults are is up to the paradigm. Paradigm authors should make them clear in their documentation.

Parameters of Nodes and Layers

[Back to contents]

• Nodes: Clamped and Deactivated
• Nodes: Act
• Layer: K and hard kWTA

Nodes: Clamped and Deactivated

Paradigm builders must implement clamping and deactivation on the node level (see definitions).

Nodes: Act

• "Act" is the activation state of the node. One should only call an activation parameter "Act" if it is at the same time the output of the node. If the activation is not the output variable, both must get a different name. E.g., membrane potential in MacGregor nodes is not the output of a node, firing is; it would confuse users who have used other paradigms if membrane potential was nevertheless called "Act" (instead of "Membrane").
• "Spike" should be used for the output state if it is different from the activation state (and thus cannot be called "Act"), and if the output is binary (e.g., true or false, 0 or 1). If the activation and output state differ from the above in other ways, new terms should be used.

Layer: K and hard kWTA

• "K" should mean: number of nodes in the layer that should be active (either through hard or soft "k Winner Take All").
• "Hard kWTA" should be a Boolean parameter of a layer that determines if a layer has hard kWTA dynamics or not.

Parameters of Connections and Tracts

[Back to contents]

• Connections: Weight and Exists
• Tract: "Weight Max", "Weight Sum", and "Normalized"

Connections: Weight and Exists

"Weight" should be a factor by which the output state of the sending neuron is multiplied. Anything else (e.g., nonlinear calculation factors) should not be called "Weight".
Sometimes, a modeler may not want all nodes in two layers to be connected, or may wish to lesion weights. A weight of 0 is then often used to indicate that a weight does not exist. Though this is efficient, weights can also become 0 through learning, and it may not be the intention of the modeler to let the weight "die" by making it 0. Therefore, existence of a weight should be coded by making a second, Boolean, parameter for connections: "Exists". This parameter should be implemented whenever the paradigm author suspects that users of the paradigm may want to lesion or leave uncreated certain weights within a tract.

Tract: "Weight Max", "Weight Sum", and "Normalized"

"Weight Max" is the maximum value a weight can attain.
If there is a choice within the paradigm whether weights are normalized or not, "Normalized" must be a Boolean parameter of a tract that determines whether a tract has normalizing weights or not. The default normalizing method should be the simplest: incoming, linear normalization, where all weights are multiplied with the desired weight sum divided by the actual weight sum.
"Weight Sum" is that desired weight sum. It has no meaning if the tract is non-normalizing.