parser interface that parses input data used to load dmlc data format into your own data format Difference between RowBlockIter and Parser: RowBlockIter caches the data internally that can be used to iterate the dataset multiple times, Parser holds very limited internal state and was usually used to read data only once
More...
#include <data.h>
|
typedef Parser< IndexType, DType > *(* | Factory) (const std::string &path, const std::map< std::string, std::string > &args, unsigned part_index, unsigned num_parts) |
| Factory type of the parser. More...
|
|
|
static Parser< IndexType, DType > * | Create (const char *uri_, unsigned part_index, unsigned num_parts, const char *type) |
| create a new instance of parser based on the "type" More...
|
|
template<typename IndexType, typename DType = real_t>
class dmlc::Parser< IndexType, DType >
parser interface that parses input data used to load dmlc data format into your own data format Difference between RowBlockIter and Parser: RowBlockIter caches the data internally that can be used to iterate the dataset multiple times, Parser holds very limited internal state and was usually used to read data only once
- See also
- RowBlockIter
- Template Parameters
-
IndexType | type of index in RowBlock |
DType | type of label and value in RowBlock Create function was only implemented for IndexType uint64_t and uint32_t and DType real_t and int |
template<typename IndexType, typename DType = real_t>
typedef Parser<IndexType, DType>*(* dmlc::Parser< IndexType, DType >::Factory) (const std::string &path, const std::map< std::string, std::string > &args, unsigned part_index, unsigned num_parts) |
Factory type of the parser.
template<typename IndexType, typename DType = real_t>
virtual size_t dmlc::Parser< IndexType, DType >::BytesRead |
( |
void |
| ) |
const |
|
pure virtual |
- Returns
- size of bytes read so far
template<typename IndexType, typename DType = real_t>
static Parser<IndexType, DType>* dmlc::Parser< IndexType, DType >::Create |
( |
const char * |
uri_, |
|
|
unsigned |
part_index, |
|
|
unsigned |
num_parts, |
|
|
const char * |
type |
|
) |
| |
|
static |
create a new instance of parser based on the "type"
- Parameters
-
uri_ | the uri of the input, can contain hdfs prefix |
part_index | the part id of current input |
num_parts | total number of splits |
type | type of dataset can be: "libsvm", "auto", ... |
When "auto" is passed, the type is decided by format argument string in URI.
- Returns
- the created parser
The documentation for this class was generated from the following file:
- /work/mxnet/3rdparty/dmlc-core/include/dmlc/data.h