Developer Guide

Debugging and performance optimization tips

The general workflow when defining your network with Gluon API is either:

  • build sequentially using nn.Sequential or nn.HybridSequential

  • inherit from nn.Block or nn.HybridBlock

Debugging

When debugging your MXNet code, remember the following:

Do NOT hybridize for debugging

The difference between imperative style (Gluon non-hybridized) and symbolic style (Gluon hybridized) is:

  • imperative style is define-by-run
  • symbolic style is define-then-run

Basically, that means the execution path changes when calling hybridize on your network inherited from HybridBlock or HybridSequential (note that inheriting directly from Block is the same as not hybridizing your network). For efficiency, symbolic code does not keep the intermediate results and so it would be hard to debug and examine the intermediate outputs. Therefore, if you want to examine the intermediate results for debugging, do NOT hybridize. Once everything is working as expected, then you can hybridize and enjoy the speed up.

Please checkout the d2l for more details about the hybrid-programming model.

Use naive engine

It is also useful to set the environment variable MXNET_ENGINE_TYPE='NaiveEngine' prior to running your (end-to-end) code. This setting disables multi-threading and the execution engine will be synchronous, so you can examine the backtrace more easily. Remember to change it back to either the default 'ThreadedEnginePerDevice' or 'ThreadedEngine'.

For more details, here is a comprehensive tutorial on interactive debugging on YouTube.

Performance optimization

Following up on using the environment variable MXNET_ENGINE_TYPE for debugging, here are the available environment variables that affect the performance of your code.

Please refer to this presentation for more information on debugging and performance optimization.