Parameter and Block Naming

In gluon, each Parameter or Block has a name. Parameter names and Block names can be automatically created.

In this tutorial we talk about the best practices on naming. First, let’s import MXNet and Gluon:

from __future__ import print_function
import mxnet as mx
from mxnet import gluon

Naming Blocks

When creating a block, you can simply do as follows:

mydense = gluon.nn.Dense(100)

When you create more Blocks of the same kind, they will be named with incrementing suffixes to avoid collision:

dense1 = gluon.nn.Dense(100)

Naming Parameters

Parameters will be named automatically by a unique name in the format of param_{uuid4}_{name}:

param = gluon.Parameter(name = 'bias')
bias is used as the name of a parameter’s symbol representation. And it can not be changed once the parameter is created.

When getting parameters within a Block, you should use the structure based name as the key:

{'weight': Parameter (shape=(100, -1), dtype=float32), 'bias': Parameter (shape=(100,), dtype=float32)}

Nested Blocks

In MXNet 2, we don’t have to define children blocks within a name_scope any more. Let’s demonstrate this by defining and initiating a simple neural net:

class Model(gluon.HybridBlock):
    def __init__(self):
        super(Model, self).__init__()
        self.dense0 = gluon.nn.Dense(20)
        self.dense1 = gluon.nn.Dense(20)
        self.mydense = gluon.nn.Dense(20)

    def forward(self, x):
        x = mx.npx.relu(self.dense0(x))
        x = mx.npx.relu(self.dense1(x))
        return mx.npx.relu(self.mydense(x))

model0 = Model()
model0(, 20)))
[04:46:00] /work/mxnet/src/storage/ Using Pooled (Naive) StorageManager for CPU
array([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0.]])

The same principle also applies to container blocks like Sequential. We can simply do as follows:

net = gluon.nn.Sequential()

Saving and loading

For HybridBlock, we use save_parameters/load_parameters, which uses model structure, instead of parameter name, to match parameters.

model1 = Model()
dict_keys(['dense0.bias', 'dense0.weight', 'dense1.bias', 'dense1.weight', 'mydense.bias', 'mydense.weight'])

For SymbolBlock.imports, we use export, which uses parameter name, to save parameters.

model2 = gluon.SymbolBlock.imports('model0-symbol.json', ['data'], 'model0-0000.params')
/work/mxnet/python/mxnet/gluon/ UserWarning: Cannot decide type for the following arguments. Consider providing them as input:
        data: None
  input_sym_arg_type = in_param.infer_type()[0]

Replacing Blocks from networks and fine-tuning

Sometimes you may want to load a pretrained model, and replace certain Blocks in it for fine-tuning.

For example, the alexnet in model zoo has 1000 output dimensions, but maybe you only have 100 classes in your application.

To see how to do this, we first load a pretrained ResNet.

  • In Gluon model zoo, all image classification models follow the format where the feature extraction layers are named features while the output layer is named output.

  • Note that the output layer is a dense block with 1000 dimension outputs.

resnet =
Dense(2048 -> 1000, linear)

To change the output to 100 dimension, we replace it with a new block.

resnet.output = gluon.nn.Dense(100)