API Reference¶

Tools for testing.

mxnet.test_utils.default_context()[source]¶

Get default context for regression test.

mxnet.test_utils.set_default_context(ctx)[source]¶

Set default context.

mxnet.test_utils.default_dtype()[source]¶

Get default data type for regression test.

mxnet.test_utils.get_atol(atol=None)[source]¶

Get default numerical threshold for regression test.

mxnet.test_utils.get_rtol(rtol=None)[source]¶

Get default numerical threshold for regression test.

mxnet.test_utils.get_etol(etol=None)[source]¶

Get default numerical threshold for regression test.

mxnet.test_utils.random_arrays(*shapes)[source]¶

Generate some random numpy arrays.

mxnet.test_utils.random_sample(population, k)[source]¶

Return a k length list of the elements chosen from the population sequence.

mxnet.test_utils.shuffle_csr_column_indices(csr)[source]¶

Shuffle CSR column indices per row This allows validation of unordered column indices, which is not a requirement for a valid CSR matrix

mxnet.test_utils.assign_each(the_input, function)[source]¶

Return ndarray composed of passing each array value through some function

mxnet.test_utils.assign_each2(input1, input2, function)[source]¶

Return ndarray composed of passing two array values through some function

mxnet.test_utils.rand_sparse_ndarray(shape, stype, density=None, dtype=None, distribution=None, data_init=None, rsp_indices=None, modifier_func=None, shuffle_csr_indices=False, ctx=None)[source]¶

Generate a random sparse ndarray. Returns the ndarray, value(np) and indices(np)

Parameters:	shape (list or tuple) – stype (str) – valid values: “csr” or “row_sparse” density (float, optional) – should be between 0 and 1 distribution (str, optional) – valid values: “uniform” or “powerlaw” dtype (numpy.dtype, optional) – default value is None
Returns:
Return type:	Result of type CSRNDArray or RowSparseNDArray

Examples

Below is an example of the powerlaw distribution with csr as the stype. It calculates the nnz using the shape and density. It fills up the ndarray with exponentially increasing number of elements. If there are enough unused_nnzs, n+1th row will have twice more nnzs compared to nth row. else, remaining unused_nnzs will be used in n+1th row If number of cols is too small and we have already reached column size it will fill up all following columns in all followings rows until we reach the required density.

>>> csr_arr, _ = rand_sparse_ndarray(shape=(5, 16), stype="csr",
                                     density=0.50, distribution="powerlaw")
>>> indptr = csr_arr.indptr.asnumpy()
>>> indices = csr_arr.indices.asnumpy()
>>> data = csr_arr.data.asnumpy()
>>> row2nnz = len(data[indptr[1]:indptr[2]])
>>> row3nnz = len(data[indptr[2]:indptr[3]])
>>> assert(row3nnz == 2*row2nnz)
>>> row4nnz = len(data[indptr[3]:indptr[4]])
>>> assert(row4nnz == 2*row3nnz)

mxnet.test_utils.rand_ndarray(shape, stype='default', density=None, dtype=None, modifier_func=None, shuffle_csr_indices=False, distribution=None, ctx=None)[source]¶

Generate a random sparse ndarray. Returns the generated ndarray.

mxnet.test_utils.create_sparse_array(shape, stype, data_init=None, rsp_indices=None, dtype=None, modifier_func=None, density=0.5, shuffle_csr_indices=False)[source]¶

Create a sparse array, For Rsp, assure indices are in a canonical format

mxnet.test_utils.create_sparse_array_zd(shape, stype, density, data_init=None, rsp_indices=None, dtype=None, modifier_func=None, shuffle_csr_indices=False)[source]¶

Create sparse array, using only rsp_indices to determine density

mxnet.test_utils.np_reduce(dat, axis, keepdims, numpy_reduce_func)[source]¶

Compatible reduce for old version of NumPy.

Parameters:	dat (np.ndarray) – Same as NumPy. axis (None or int or list-like) – Same as NumPy. keepdims (bool) – Same as NumPy. numpy_reduce_func (function) – A NumPy reducing function like np.sum or np.max.

mxnet.test_utils.find_max_violation(a, b, rtol=None, atol=None)[source]¶

Finds and returns the location of maximum violation.

mxnet.test_utils.same(a, b)[source]¶

Test if two NumPy arrays are the same.

Parameters:	a (np.ndarray) – b (np.ndarray) –

mxnet.test_utils.almost_equal(a, b, rtol=None, atol=None, equal_nan=False)[source]¶

Test if two numpy arrays are almost equal.

mxnet.test_utils.assert_almost_equal(a, b, rtol=None, atol=None, names=('a', 'b'), equal_nan=False)[source]¶

Test that two numpy arrays are almost equal. Raise exception message if not.

Parameters:	a (np.ndarray) – b (np.ndarray) – threshold (None or float) – The checking threshold. Default threshold will be used if set to None.

mxnet.test_utils.assert_almost_equal_with_err(a, b, rtol=None, atol=None, etol=None, names=('a', 'b'), equal_nan=False)[source]¶

Test that two numpy arrays are almost equal within given error rate. Raise exception message if not.

Parameters:	a (np.ndarray) – b (np.ndarray) – threshold (None or float) – The checking threshold. Default threshold will be used if set to None. etol (None or float) – The error rate threshold. If etol is float, return true if error_rate < etol even if any error is found.

mxnet.test_utils.almost_equal_ignore_nan(a, b, rtol=None, atol=None)[source]¶

Test that two NumPy arrays are almost equal (ignoring NaN in either array). Combines a relative and absolute measure of approximate eqality. If either the relative or absolute check passes, the arrays are considered equal. Including an absolute check resolves issues with the relative check where all array values are close to zero.

Parameters:	a (np.ndarray) – b (np.ndarray) – rtol (None or float) – The relative threshold. Default threshold will be used if set to None. atol (None or float) – The absolute threshold. Default threshold will be used if set to None.

mxnet.test_utils.assert_almost_equal_ignore_nan(a, b, rtol=None, atol=None, names=('a', 'b'))[source]¶

Test that two NumPy arrays are almost equal (ignoring NaN in either array). Combines a relative and absolute measure of approximate eqality. If either the relative or absolute check passes, the arrays are considered equal. Including an absolute check resolves issues with the relative check where all array values are close to zero.

Parameters:	a (np.ndarray) – b (np.ndarray) – rtol (None or float) – The relative threshold. Default threshold will be used if set to None. atol (None or float) – The absolute threshold. Default threshold will be used if set to None.

mxnet.test_utils.assert_exception(f, exception_type, *args, **kwargs)[source]¶

Test that function f will throw an exception of type given by exception_type

mxnet.test_utils.retry(n)[source]¶

Retry n times before failing for stochastic test cases.

mxnet.test_utils.simple_forward(sym, ctx=None, is_train=False, **inputs)[source]¶

A simple forward function for a symbol.

Primarily used in doctest to test the functionality of a symbol. Takes NumPy arrays as inputs and outputs are also converted to NumPy arrays.

Parameters:	ctx (Context) – If None, will take the default context. inputs (keyword arguments) – Mapping each input name to a NumPy array.
Returns:	The result as a numpy array. Multiple results will be returned as a list of NumPy arrays.

mxnet.test_utils.numeric_grad(executor, location, aux_states=None, eps=0.0001, use_forward_train=True, dtype=)[source]¶

Calculates a numeric gradient via finite difference method.

Class based on Theano’s theano.gradient.numeric_grad [1]

Parameters:

executor (Executor) – Executor that computes the forward pass. location (list of numpy.ndarray or dict of str to numpy.ndarray) – Argument values used as location to compute gradient Maps the name of arguments to the corresponding numpy.ndarray. Value of all the arguments must be provided. aux_states (None or list of numpy.ndarray or dict of str to numpy.ndarray, optional) – Auxiliary states values used as location to compute gradient Maps the name of aux_states to the corresponding numpy.ndarray. Value of all the auxiliary arguments must be provided. eps (float, optional) – Epsilon for the finite-difference method. use_forward_train (bool, optional) – Whether to use is_train=True in testing. dtype (np.float16 or np.float32 or np.float64) – Datatype for mx.nd.array.

References

..[1] https://github.com/Theano/Theano/blob/master/theano/gradient.py

mxnet.test_utils.check_numeric_gradient(sym, location, aux_states=None, numeric_eps=0.001, rtol=0.01, atol=None, grad_nodes=None, use_forward_train=True, ctx=None, grad_stype_dict=None, dtype=)[source]¶

Verify an operation by checking backward pass via finite difference method.

Based on Theano’s theano.gradient.verify_grad [1]

Parameters:

sym (Symbol) – Symbol containing op to test location (list or tuple or dict) – Argument values used as location to compute gradient if type is list of numpy.ndarray, inner elements should have the same order as mxnet.sym.list_arguments(). if type is dict of str -> numpy.ndarray, maps the name of arguments to the corresponding numpy.ndarray. In either case, value of all the arguments must be provided. aux_states (list or tuple or dict, optional) – The auxiliary states required when generating the executor for the symbol. numeric_eps (float, optional) – Delta for the finite difference method that approximates the gradient. check_eps (float, optional) – relative error eps used when comparing numeric grad to symbolic grad. grad_nodes (None or list or tuple or dict, optional) – Names of the nodes to check gradient on use_forward_train (bool) – Whether to use is_train=True when computing the finite-difference. ctx (Context, optional) – Check the gradient computation on the specified device. grad_stype_dict (dict of str->str, optional) – Storage type dictionary for gradient ndarrays. dtype (np.float16 or np.float32 or np.float64) – Datatype for mx.nd.array.

References

[1] https://github.com/Theano/Theano/blob/master/theano/gradient.py

mxnet.test_utils.check_symbolic_forward(sym, location, expected, rtol=0.0001, atol=None, aux_states=None, ctx=None, equal_nan=False, dtype=)[source]¶

Compares a symbol’s forward results with the expected ones. Prints error messages if the forward results are not the same as the expected ones.

Parameters:

sym (Symbol) – output symbol location (list of np.ndarray or dict of str to np.ndarray) – The evaluation point if type is list of np.ndarray Contains all the numpy arrays corresponding to sym.list_arguments(). if type is dict of str to np.ndarray Contains the mapping between argument names and their values. expected (list of np.ndarray or dict of str to np.ndarray) – The expected output value if type is list of np.ndarray Contains arrays corresponding to exe.outputs. if type is dict of str to np.ndarray Contains mapping between sym.list_output() and exe.outputs. check_eps (float, optional) – Relative error to check to. aux_states (list of np.ndarray of dict, optional) – if type is list of np.ndarray Contains all the NumPy arrays corresponding to sym.list_auxiliary_states if type is dict of str to np.ndarray Contains the mapping between names of auxiliary states and their values. ctx (Context, optional) – running context dtype ("asnumpy" or np.float16 or np.float32 or np.float64) – If dtype is “asnumpy” then the mx.nd.array created will have the same type as th numpy array from which it is copied. Otherwise, dtype is the explicit datatype for all mx.nd.array objects created in this function. equal_nan (Boolean) – if True, nan is a valid value for checking equivalency (ie nan == nan)

Example

>>> shape = (2, 2)
>>> lhs = mx.symbol.Variable('lhs')
>>> rhs = mx.symbol.Variable('rhs')
>>> sym_dot = mx.symbol.dot(lhs, rhs)
>>> mat1 = np.array([[1, 2], [3, 4]])
>>> mat2 = np.array([[5, 6], [7, 8]])
>>> ret_expected = np.array([[19, 22], [43, 50]])
>>> check_symbolic_forward(sym_dot, [mat1, mat2], [ret_expected])

mxnet.test_utils.check_symbolic_backward(sym, location, out_grads, expected, rtol=1e-05, atol=None, aux_states=None, grad_req='write', ctx=None, grad_stypes=None, equal_nan=False, dtype=)[source]¶

Compares a symbol’s backward results with the expected ones. Prints error messages if the backward results are not the same as the expected results.

Parameters:

sym (Symbol) – output symbol location (list of np.ndarray or dict of str to np.ndarray) – The evaluation point if type is list of np.ndarray Contains all the NumPy arrays corresponding to mx.sym.list_arguments. if type is dict of str to np.ndarray Contains the mapping between argument names and their values. out_grads (None or list of np.ndarray or dict of str to np.ndarray) – NumPys arrays corresponding to sym.outputs for incomming gradient. if type is list of np.ndarray Contains arrays corresponding to exe.outputs. if type is dict of str to np.ndarray contains mapping between mxnet.sym.list_output() and Executor.outputs expected (list of np.ndarray or dict of str to np.ndarray) – expected gradient values if type is list of np.ndarray Contains arrays corresponding to exe.grad_arrays if type is dict of str to np.ndarray Contains mapping between sym.list_arguments() and exe.outputs. check_eps (float, optional) – Relative error to check to. aux_states (list of np.ndarray or dict of str to np.ndarray) – grad_req (str or list of str or dict of str to str, optional) – Gradient requirements. ‘write’, ‘add’ or ‘null’. ctx (Context, optional) – Running context. grad_stypes (dict of str->str) – dictionary of mapping argument name to stype for the gradient equal_nan (Boolean) – if True, nan is a valid value for checking equivalency (ie nan == nan) dtype (np.float16 or np.float32 or np.float64) – Datatype for mx.nd.array.

Example

>>> lhs = mx.symbol.Variable('lhs')
>>> rhs = mx.symbol.Variable('rhs')
>>> sym_add = mx.symbol.elemwise_add(lhs, rhs)
>>> mat1 = np.array([[1, 2], [3, 4]])
>>> mat2 = np.array([[5, 6], [7, 8]])
>>> grad1 = mx.nd.zeros(shape)
>>> grad2 = mx.nd.zeros(shape)
>>> exec_add = sym_add.bind(default_context(), args={'lhs': mat1, 'rhs': mat2},
... args_grad={'lhs': grad1, 'rhs': grad2}, grad_req={'lhs': 'write', 'rhs': 'write'})
>>> exec_add.forward(is_train=True)
>>> ograd = mx.nd.ones(shape)
>>> grad_expected = ograd.copy().asnumpy()
>>> check_symbolic_backward(sym_add, [mat1, mat2], [ograd], [grad_expected, grad_expected])

mxnet.test_utils.check_speed(sym, location=None, ctx=None, N=20, grad_req=None, typ='whole', **kwargs)[source]¶

Check the running speed of a symbol.

Parameters:

sym (Symbol) – Symbol to run the speed test. location (none or dict of str to np.ndarray) – Location to evaluate the inner executor. ctx (Context) – Running context. N (int, optional) – Repeat times. grad_req (None or str or list of str or dict of str to str, optional) – Gradient requirements. typ (str, optional) – “whole” or “forward” “whole” Test the forward_backward speed. “forward” Only test the forward speed.

mxnet.test_utils.check_consistency(sym, ctx_list, scale=1.0, grad_req='write', arg_params=None, aux_params=None, tol=None, raise_on_err=True, ground_truth=None, equal_nan=False, use_uniform=False, rand_type=)[source]¶

Check symbol gives the same output for different running context

Parameters:

sym (Symbol or list of Symbols) – Symbol(s) to run the consistency test. ctx_list (list) – Running context. See example for more detail. scale (float, optional) – Standard deviation of the inner normal distribution. Used in initialization. grad_req (str or list of str or dict of str to str) – Gradient requirement. use_unifrom (bool) – Optional, When flag set to true, random input data generated follows uniform distribution, not normal distribution rand_type (np.dtype) – casts the randomly generated data to this type Optional, when input data is passed via arg_params, defaults to np.float64 (numpy float default)

Examples

>>> # create the symbol
>>> sym = mx.sym.Convolution(num_filter=3, kernel=(3,3), name='conv')
>>> # initialize the running context
>>> ctx_list =[{'ctx': mx.gpu(0), 'conv_data': (2, 2, 10, 10), 'type_dict': {'conv_data': np.float64}}, {'ctx': mx.gpu(0), 'conv_data': (2, 2, 10, 10), 'type_dict': {'conv_data': np.float32}}, {'ctx': mx.gpu(0), 'conv_data': (2, 2, 10, 10), 'type_dict': {'conv_data': np.float16}}, {'ctx': mx.cpu(0), 'conv_data': (2, 2, 10, 10), 'type_dict': {'conv_data': np.float64}}, {'ctx': mx.cpu(0), 'conv_data': (2, 2, 10, 10), 'type_dict': {'conv_data': np.float32}}]
>>> check_consistency(sym, ctx_list)
>>> sym = mx.sym.Concat(name='concat', num_args=2)
>>> ctx_list = [{'ctx': mx.gpu(0), 'concat_arg1': (2, 10), 'concat_arg0': (2, 10),  'type_dict': {'concat_arg0': np.float64, 'concat_arg1': np.float64}}, {'ctx': mx.gpu(0), 'concat_arg1': (2, 10), 'concat_arg0': (2, 10),  'type_dict': {'concat_arg0': np.float32, 'concat_arg1': np.float32}}, {'ctx': mx.gpu(0), 'concat_arg1': (2, 10), 'concat_arg0': (2, 10),  'type_dict': {'concat_arg0': np.float16, 'concat_arg1': np.float16}}, {'ctx': mx.cpu(0), 'concat_arg1': (2, 10), 'concat_arg0': (2, 10),  'type_dict': {'concat_arg0': np.float64, 'concat_arg1': np.float64}}, {'ctx': mx.cpu(0), 'concat_arg1': (2, 10), 'concat_arg0': (2, 10),  'type_dict': {'concat_arg0': np.float32, 'concat_arg1': np.float32}}]
>>> check_consistency(sym, ctx_list)

mxnet.test_utils.list_gpus()[source]¶

Return a list of GPUs

Returns:	If there are n GPUs, then return a list [0,1,...,n-1]. Otherwise returns [].
Return type:	list of int

mxnet.test_utils.download(url, fname=None, dirname=None, overwrite=False, retries=5)[source]¶

Download an given URL

Parameters:	url (str) – URL to download fname (str, optional) – filename of the downloaded file. If None, then will guess a filename from url. dirname (str, optional) – output directory name. If None, then guess from fname or use the current directory overwrite (bool, optional) – Default is false, which means skipping download if the local file exists. If true, then download the url to overwrite the local file if exists. retries (integer, default 5) – The number of times to attempt the download in case of failure or non 200 return codes
Returns:	The filename of the downloaded file
Return type:	str

mxnet.test_utils.get_mnist()[source]¶

Download and load the MNIST dataset

Returns:	A dict containing the data
Return type:	dict

mxnet.test_utils.get_mnist_pkl()[source]¶

Downloads MNIST dataset as a pkl.gz into a directory in the current directory with the name data

mxnet.test_utils.get_mnist_ubyte()[source]¶

Downloads ubyte version of the MNIST dataset into a directory in the current directory with the name data and extracts all files in the zip archive to this directory.

mxnet.test_utils.get_cifar10()[source]¶

Downloads CIFAR10 dataset into a directory in the current directory with the name data, and then extracts all files into the directory data/cifar.

mxnet.test_utils.get_mnist_iterator(batch_size, input_shape, num_parts=1, part_index=0)[source]¶

Returns training and validation iterators for MNIST dataset

mxnet.test_utils.get_zip_data(data_dir, url, data_origin_name)[source]¶

Download and extract zip data.

Parameters:	data_dir (str) – Absolute or relative path of the directory name to store zip files url (str) – URL to download data from data_origin_name (str) – Name of the downloaded zip file

Examples

>>> get_zip_data("data_dir",
                 "http://files.grouplens.org/datasets/movielens/ml-10m.zip",
                 "ml-10m.zip")

mxnet.test_utils.get_bz2_data(data_dir, data_name, url, data_origin_name)[source]¶

Download and extract bz2 data.

Parameters:	data_dir (str) – Absolute or relative path of the directory name to store bz2 files data_name (str) – Name of the output file in which bz2 contents will be extracted url (str) – URL to download data from data_origin_name (str) – Name of the downloaded b2 file

Examples

>>> get_bz2_data("data_dir", "kdda.t",
                 "https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary/kdda.t.bz2",
                 "kdda.t.bz2")

mxnet.test_utils.set_env_var(key, val, default_val='')[source]¶

Set environment variable

Parameters:	key (str) – Env var to set val (str) – New value assigned to the env var default_val (str, optional) – Default value returned if the env var doesn’t exist
Returns:	The value of env var before it is set to the new value
Return type:	str

mxnet.test_utils.same_array(array1, array2)[source]¶

Check whether two NDArrays sharing the same memory block

Parameters:	array1 (NDArray) – First NDArray to be checked array2 (NDArray) – Second NDArray to be checked
Returns:	Whether two NDArrays share the same memory
Return type:	bool

mxnet.test_utils.discard_stderr(*args, **kwds)[source]¶

Discards error output of a routine if invoked as:

with discard_stderr():

...

class mxnet.test_utils.DummyIter(real_iter)[source]¶

A dummy iterator that always returns the same batch of data (the first data batch of the real data iter). This is usually used for speed testing.

Parameters:	real_iter (mx.io.DataIter) – The real data iterator where the first batch of data comes from

next()[source]¶

Get a data batch from iterator. The first data batch of real iter is always returned. StopIteration will never be raised.

Returns:	The data of next batch.
Return type:	DataBatch

mxnet.test_utils.gen_buckets_probs_with_ppf(ppf, nbuckets)[source]¶

Generate the buckets and probabilities for chi_square test when the ppf (Quantile function)

is specified.

Parameters:	ppf (function) – The Quantile function that takes a probability and maps it back to a value. It’s the inverse of the cdf function nbuckets (int) – size of the buckets
Returns:	buckets (list of tuple) – The generated buckets probs (list) – The generate probabilities

mxnet.test_utils.mean_check(generator, mu, sigma, nsamples=1000000)[source]¶

Test the generator by matching the mean.

We test the sample mean by checking if it falls inside the range

(mu - 3 * sigma / sqrt(n), mu + 3 * sigma / sqrt(n))

References:

@incollection{goucher2009beautiful,
      title={Beautiful Testing: Leading Professionals Reveal How They Improve Software},
      author={Goucher, Adam and Riley, Tim},
      year={2009},
      chapter=10
}

Examples:

generator = lambda x: np.random.normal(0, 1.0, size=x)
mean_check_ret = mean_check(generator, 0, 1.0)

Parameters:	generator (function) – The generator function. It’s expected to generate N i.i.d samples by calling generator(N). mu (float) – sigma (float) – nsamples (int) –
Returns:	ret – Whether the mean test succeeds
Return type:	bool

mxnet.test_utils.get_im2rec_path(home_env='MXNET_HOME')[source]¶

Get path to the im2rec.py tool

Parameters:	home_env (str) – Env variable that holds the path to the MXNET folder
Returns:	The path to im2rec.py
Return type:	str

mxnet.test_utils.var_check(generator, sigma, nsamples=1000000)[source]¶

Test the generator by matching the variance. It will need a large number of samples and is not recommended to use

We test the sample variance by checking if it falls inside the range

(sigma^2 - 3 * sqrt(2 * sigma^4 / (n-1)), sigma^2 + 3 * sqrt(2 * sigma^4 / (n-1)))

References:

@incollection{goucher2009beautiful,
      title={Beautiful Testing: Leading Professionals Reveal How They Improve Software},
      author={Goucher, Adam and Riley, Tim},
      year={2009},
      chapter=10
}

Examples:

generator = lambda x: np.random.normal(0, 1.0, size=x)
var_check_ret = var_check(generator, 0, 1.0)

Parameters:	generator (function) – The generator function. It’s expected to generate N i.i.d samples by calling generator(N). sigma (float) – nsamples (int) –
Returns:	ret – Whether the variance test succeeds
Return type:	bool

mxnet.test_utils.chi_square_check(generator, buckets, probs, nsamples=1000000)[source]¶

Run the chi-square test for the generator. The generator can be both continuous and discrete.

If the generator is continuous, the buckets should contain tuples of (range_min, range_max) and the probs should be the corresponding ideal probability within the specific ranges. Otherwise, the buckets should contain all the possible values generated over the discrete distribution and the probs should be groud-truth probability.

Usually the user is required to specify the probs parameter.

After obtaining the p value, we could further use the standard p > 0.05 (alpha) threshold to get the final result.

Examples:

buckets, probs = gen_buckets_probs_with_ppf(lambda x: ss.norm.ppf(x, 0, 1), 5)
generator = lambda x: np.random.normal(0, 1.0, size=x)
p = chi_square_check(generator=generator, buckets=buckets, probs=probs)
assert(p > 0.05)

Parameters:

generator (function) – A function that is assumed to generate i.i.d samples from a specific distribution. generator(N) should generate N random samples. buckets (list of tuple or list of number) – The buckets to run the chi-square the test. Make sure that the buckets cover the whole range of the distribution. Also, the buckets must be in ascending order and have no intersection probs (list or tuple) – The ground-truth probability of the random value fall in a specific bucket. nsamples (int) – The number of samples to generate for the testing

Returns:

p (float) – p value that the generator has the expected distribution. A higher value indicates a larger confidence obs_freq (list) – Observed frequency of buckets expected_freq (list) – The expected (ground-truth) frequency of the buckets

mxnet.test_utils.verify_generator(generator, buckets, probs, nsamples=1000000, nrepeat=5, success_rate=0.2, alpha=0.05)[source]¶

Verify whether the generator is correct using chi-square testing.

The test is repeated for “nrepeat” times and we check if the success rate is

above the threshold (25% by default).

Parameters:	generator (function) – A function that is assumed to generate i.i.d samples from a specific distribution. generator(N) should generate N random samples. buckets (list of tuple or list of number) – The buckets to run the chi-square the test. Make sure that the buckets cover the whole range of the distribution. Also, the buckets must be in ascending order and have no intersection probs (list or tuple) – The ground-truth probability of the random value fall in a specific bucket. nsamples (int) – The number of samples to generate for the testing nrepeat (int) – The times to repeat the test success_rate (float) – The desired success rate alpha (float) – The desired threshold for type-I error i.e. when a true null hypothesis is rejected
Returns:	cs_ret_l – The p values of the chi-square test.
Return type:	list

mxnet.test_utils.compare_ndarray_tuple(t1, t2, rtol=None, atol=None)[source]¶

Compare ndarray tuple.

mxnet.test_utils.compare_optimizer(opt1, opt2, shape, dtype, w_stype='default', g_stype='default', rtol=0.0001, atol=1e-05, compare_states=True)[source]¶

Compare opt1 and opt2.

class mxnet.test_utils.EnvManager(key, val)[source]¶

Environment variable setter and unsetter via with idiom