org.apache.clojure-mxnet.io
batch-data
(batch-data batch)
Returns the vector of ndarrays that represents the data
batch-index
(batch-index batch)
Returns the vector of ints that represents the index
batch-label
(batch-label batch)
Returns the vector of ndarrays that represents the label
batch-pad
(batch-pad batch)
Returns the pad of the batch
batches
(batches data-pack)
Convert the data-pack to a batch seq
create-iterator
(create-iterator iter-name kwargs-map)
create-mx-data-pack
(create-mx-data-pack pack-name kwargs-map)
csv-iter
(csv-iter kwargs)
csv-pack
(csv-pack kwargs)
data-batch
(data-batch {:keys [data label index pad bucket-key provided-data provided-label], :as info, :or {data [], label [], index [], pad 0}})
data-desc
(data-desc {:keys [name shape dtype layout], :as opts, :or {dtype base/MX_REAL_TYPE, layout layout/UNDEFINED}})(data-desc name shape)
data-desc->map
(data-desc->map dd)
dispose
(dispose iterator)
do-batches
macro
(do-batches iter f)
for-batches
macro
(for-batches iter f)
Takes an iterator and a function of one argument. The iterator will be reset and run thhrough all the batches with the batch passed to the function argument. The result of the function will be conjed to a vector result of all the batches and returned at the end.
has-next?
(has-next? iterator)
image-recode-pack
(image-recode-pack kwargs)
image-record-iter
(image-record-iter kwargs)
iter-data
(iter-data iterator)
iter-init-label
(iter-init-label iterator)
iter-label
(iter-label iterator)
iterator
(iterator data-pack)
mnist-iter
(mnist-iter kwargs)
mnist-pack
(mnist-pack kwargs)
ndarray-iter
(ndarray-iter data {:keys [label data-batch-size shuffle last-batch-handle data-name label-name], :as opts, :or {label nil, data-batch-size 1, shuffle false, last-batch-handle "pad", data-name "data", label-name "label"}})(ndarray-iter data)
* NDArrayIter object in mxnet. Taking NDArray to get dataiter.
*
* @param data vector of iter - Can either by in the form for [ndarray..] or
* {data-desc0 ndarray0 data-desc2 ndarray2 ...}
* @opts map of:
* :label Same as data, but is not fed to the model during testing.
* :data-batch-size Batch Size (default 1)
* :shuffle Whether to shuffle the data (default false)
* :last-batch-handle = pad, discard, or rollover. (default pad)
* :data-name String of data name (default data)
* :label-name String of label name (default label)
* How to handle the last batch
* This iterator will pad, discard or roll over the last batch if
* the size of data does not match batch-size. Roll over is intended
* for training and can cause problems if used for prediction.prefetching-iter
(prefetching-iter iters data-names label-names)
Takes one or more data iterators and combines them with pre-fetching
provide-data
(provide-data pack-iterator)
Provides the description of the data iterator in the form of
[{:name name :shape shape-vec}]provide-data-desc
(provide-data-desc pack-iterator)
Provides the Data Desc of the data iterator in the form of
[{:name name :shape shape-vec :dtype dtype :layout layout}]provide-label
(provide-label pack-iterator)
Provides the description of the label iterator in the form of
[{:name name :shape shape-vec}]provide-label-desc
(provide-label-desc pack-iterator)
Provides the Data Desc of the label iterator in the form of
[{:name name :shape shape-vec :dtype dtype :layout layout}]rand-iter
(rand-iter shape-vec {:keys [label data-batch-size shuffle last-batch-handle data-name label-name], :as opts, :or {label nil, data-batch-size 1, shuffle false, last-batch-handle "pad", data-name "rand", label-name "label"}})(rand-iter shape-vec)
A implementation of a random noise iterator
Instead of data pass in the shape vector of the noise shape
reduce-batches
macro
(reduce-batches iter f initial-val)(reduce-batches iter f)
Takes an iterator and a function of two arguments. The iterator will be reset and run thhrough all the batches with the batch passed to the function argument. The result of the function will the result of the reduce function
resize-iter
(resize-iter data-iter resize reset-iternal)
* Resize a data iterator to given number of batches per epoch.
* May produce incomplete batch in the middle of an epoch due
* to padding from internal iterator.
*
* @param data-iter Internal data iterator.
* @param resize number of batches per epoch to resize to.
* @param reset-internal whether to reset internal iterator with reset