Collaboration diagram for Batch Normalization:

Functions
mkldnn_status_t MKLDNN_API	mkldnn_batch_normalization_forward_desc_init (mkldnn_batch_normalization_desc_t bnrm_desc, mkldnn_prop_kind_t prop_kind, const mkldnn_memory_desc_t data_desc, float epsilon, unsigned flags)

mkldnn_status_t MKLDNN_API	mkldnn_batch_normalization_backward_desc_init (mkldnn_batch_normalization_desc_t bnrm_desc, mkldnn_prop_kind_t prop_kind, const mkldnn_memory_desc_t diff_data_desc, const mkldnn_memory_desc_t *data_desc, float epsilon, unsigned flags)

Detailed Description

A primitive to perform batch normalization.

$dst[n][c][h][w] = \gamma[c] \frac{src[n][c][h][w] - \mu[c]} {\sqrt{\sigma[c] + eps}} + \beta[c],$

where $\gamma[c], \beta[c]$ are weights and bias for a channel and,

$\mu[c] = \frac{1}{NHW} \sum\limits_{whn} src[n][c][h][w]$ , $\sigma[c] = \frac{1}{NHW} \sum\limits_{whn} (src[n][c][h][w] - \mu[c])^2$ ,

and eps is a constant to improve numerical stability.

Both forward and backward passes support in-place operation; that is, src and dst point to the same memory for forward pass, and diff_dst and diff_src point to the same memory for backward pass.

Batch normalization supports different flavors controlled by mkldnn_batch_normalization_desc_t. For example, batch normalization can compute the mean and variance on its own or take them as inputs. It can either perform scaling and shifting using gamma and beta parameters or not. Optionally it can also perform a fused ReLU, which in case of training would also require a workspace.

See also: mkldnn_batch_normalization_desc_t

Function Documentation

mkldnn_status_t MKLDNN_API mkldnn_batch_normalization_backward_desc_init	(	mkldnn_batch_normalization_desc_t *	bnrm_desc,
		mkldnn_prop_kind_t	prop_kind,
		const mkldnn_memory_desc_t *	diff_data_desc,
		const mkldnn_memory_desc_t *	data_desc,
		float	epsilon,
		unsigned	flags
	)

Initializes a batch normalization descriptor bnrm_desc for backward propagation with respect to data and scale-shift parameters using memory descriptors data_desc and diff_data_desc, normalization parameter epsilon, and flags set using bit flags of type mkldnn_batch_normalization_desc_t.

Order of inputs:

src (mkldnn_query_src_pd, 0)
mean (mkldnn_query_src_pd, 1)
variance (mkldnn_query_src_pd, 2)
diff_dst (mkldnn_query_diff_dst_pd, 0)
scale_and_shift (mkldnn_query_weights_pd, 0), if mkldnn_use_scaleshift bit-flags is set in flags
workspace (mkldnn_query_workspace_pd, 0), if mkldnn_fuse_bn_relu bit-flags is set in flags

Order of outputs:

diff_src (mkldnn_query_diff_src_pd, 0)
diff_scale_and_shift (mkldnn_query_diff_weights_pd, 0), if mkldnn_use_scaleshift bit-flags is set in flags and prop_kind = mkldnn_backward

Note: in-place operation is supported, i.e. diff_src points to the same memory as diff_dst.

See also: mkldnn_batch_normalization_desc_t

mkldnn_status_t MKLDNN_API mkldnn_batch_normalization_forward_desc_init	(	mkldnn_batch_normalization_desc_t *	bnrm_desc,
		mkldnn_prop_kind_t	prop_kind,
		const mkldnn_memory_desc_t *	data_desc,
		float	epsilon,
		unsigned	flags
	)

Initializes a batch normalization descriptor bnrm_desc for forward propagation using prop_kind (possible values are mkldnn_forward_training and mkldnn_forward_inference), memory descriptor data_desc, normalization parameter epsilon, and flags set using bit flags of type mkldnn_batch_normalization_desc_t.

Order of inputs:

src (mkldnn_query_src_pd, 0)
mean (mkldnn_query_src_pd, 1), if mkldnn_use_global_stats bit-flags is set in flags
variance (mkldnn_query_src_pd, 2), if mkldnn_use_global_stats bit-flags is set in flags
scale_and_shift (mkldnn_query_weights_pd, 0), if mkldnn_use_scaleshift bit-flags is set in flags

Order of outputs:

dst (mkldnn_query_dst_pd, 0)
mean (mkldnn_query_dst_pd, 1), if mkldnn_use_global_stats bit-flags is not set in flags prop_kind = mkldnn_forward_training
variance (mkldnn_query_dst_pd, 2), if mkldnn_use_global_stats bit-flags is not set in flags and prop_kind = mkldnn_forward_training
workspace (mkldnn_query_workspace_pd, 0), if mkldnn_fuse_bn_relu bit-flags is set in flags and prop_kind = mkldnn_forward_training

Note: In-place operation is supported; that is, dst points to the same memory as src.

See also: mkldnn_batch_normalization_desc_t

Functions

Detailed Description

Function Documentation