mxnet.recordio¶

Read and write for the RecordIO data format.

Classes

`IRHeader`	An alias for HEADER.
`MXIndexedRecordIO`(idx_path, uri, flag[, …])	Reads/writes RecordIO data format, supporting random access.
`MXRecordIO`(uri, flag)	Reads/writes RecordIO data format, supporting sequential read and write.

Functions

`pack`(header, s)	Pack a string into MXImageRecord.
`pack_img`(header, img[, quality, img_fmt])	Pack an image into `MXImageRecord`.
`unpack`(s)	Unpack a MXImageRecord to string.
`unpack_img`(s[, iscolor])	Unpack a MXImageRecord to image.

mxnet.recordio.IRHeader¶

An alias for HEADER. Used to store metadata (e.g. labels) accompanying a record. See mxnet.recordio.pack and mxnet.recordio.pack_img for example uses.

Parameters

flag (int) – Available for convenience, can be set arbitrarily.
label (float or an array of float) – Typically used to store label(s) for a record.
id (int) – Usually a unique id representing record.
id2 (int) – Higher order bits of the unique id, should be set to 0 (in most cases).

Attributes

`flag`	Alias for field number 0
`id`	Alias for field number 2
`id2`	Alias for field number 3
`label`	Alias for field number 1

alias of mxnet.recordio.HEADER

class mxnet.recordio.MXIndexedRecordIO(idx_path, uri, flag, key_type=<class 'int'>)[source]¶

Bases: mxnet.recordio.MXRecordIO

Reads/writes RecordIO data format, supporting random access.

Examples

Methods

`close`()	Closes the record file.
`open`()	Opens the record file.
`read_idx`(idx)	Returns the record at given index.
`seek`(idx)	Sets the current read pointer position.
`tell`()	Returns the current position of write head.
`write_idx`(idx, buf)	Inserts input record at given index.

>>> for i in range(5):
...     record.write_idx(i, 'record_%d'%i)
>>> record.close()
>>> record = mx.recordio.MXIndexedRecordIO('tmp.idx', 'tmp.rec', 'r')
>>> record.read_idx(3)
record_3

Parameters

idx_path (str) – Path to the index file.
uri (str) – Path to the record file. Only supports seekable file types.
flag (str) – ‘w’ for write or ‘r’ for read.
key_type (type) – Data type for keys.

close()[source]¶: Closes the record file.

open()[source]¶: Opens the record file.

read_idx(idx)[source]¶

Returns the record at given index.

Examples

>>> record = mx.recordio.MXIndexedRecordIO('tmp.idx', 'tmp.rec', 'w')
>>> for i in range(5):
...     record.write_idx(i, 'record_%d'%i)
>>> record.close()
>>> record = mx.recordio.MXIndexedRecordIO('tmp.idx', 'tmp.rec', 'r')
>>> record.read_idx(3)
record_3

seek(idx)[source]¶

Sets the current read pointer position.

This function is internally called by read_idx(idx) to find the current reader pointer position. It doesn’t return anything.

tell()[source]¶

Returns the current position of write head.

Examples

>>> record = mx.recordio.MXIndexedRecordIO('tmp.idx', 'tmp.rec', 'w')
>>> print(record.tell())
0
>>> for i in range(5):
...     record.write_idx(i, 'record_%d'%i)
...     print(record.tell())
16
32
48
64
80

write_idx(idx, buf)[source]¶

Inserts input record at given index.

Examples

>>> for i in range(5):
...     record.write_idx(i, 'record_%d'%i)
>>> record.close()

Parameters

idx (int) – Index of a file.
buf – Record to write.

class mxnet.recordio.MXRecordIO(uri, flag)[source]¶

Bases: object

Reads/writes RecordIO data format, supporting sequential read and write.

Examples

Methods

`close`()	Closes the record file.
`open`()	Opens the record file.
`read`()	Returns record as a string.
`reset`()	Resets the pointer to first item.
`write`(buf)	Inserts a string buffer as a record.

>>> record = mx.recordio.MXRecordIO('tmp.rec', 'w')
<mxnet.recordio.MXRecordIO object at 0x10ef40ed0>
>>> for i in range(5):
...    record.write('record_%d'%i)
>>> record.close()
>>> record = mx.recordio.MXRecordIO('tmp.rec', 'r')
>>> for i in range(5):
...    item = record.read()
...    print(item)
record_0
record_1
record_2
record_3
record_4
>>> record.close()

Parameters

uri (string) – Path to the record file.
flag (string) – ‘w’ for write or ‘r’ for read.

close()[source]¶: Closes the record file.

open()[source]¶: Opens the record file.

read()[source]¶

Returns record as a string.

Examples

>>> record = mx.recordio.MXRecordIO('tmp.rec', 'r')
>>> for i in range(5):
...    item = record.read()
...    print(item)
record_0
record_1
record_2
record_3
record_4
>>> record.close()

Returns: buf – Buffer read.
Return type: string

reset()[source]¶

Resets the pointer to first item.

If the record is opened with ‘w’, this function will truncate the file to empty.

Examples

>>> record = mx.recordio.MXRecordIO('tmp.rec', 'r')
>>> for i in range(2):
...    item = record.read()
...    print(item)
record_0
record_1
>>> record.reset()  # Pointer is reset.
>>> print(record.read()) # Started reading from start again.
record_0
>>> record.close()

write(buf)[source]¶

Inserts a string buffer as a record.

Examples

>>> record = mx.recordio.MXRecordIO('tmp.rec', 'w')
>>> for i in range(5):
...    record.write('record_%d'%i)
>>> record.close()

Parameters: buf (string (python2), bytes (python3)) – Buffer to write.

mxnet.recordio.pack(header, s)[source]¶

Pack a string into MXImageRecord.

Parameters

header (IRHeader) – Header of the image record. header.label can be a number or an array. See more detail in IRHeader.
s (str) – Raw image string to be packed.

Returns

s – The packed string.

Return type

str

Examples

>>> label = 4 # label can also be a 1-D array, for example: label = [1,2,3]
>>> id = 2574
>>> header = mx.recordio.IRHeader(0, label, id, 0)
>>> with open(path, 'r') as file:
...     s = file.read()
>>> packed_s = mx.recordio.pack(header, s)

mxnet.recordio.pack_img(header, img, quality=95, img_fmt='.jpg')[source]¶

Pack an image into MXImageRecord.

Parameters

header (IRHeader) – Header of the image record. header.label can be a number or an array. See more detail in IRHeader.
img (numpy.ndarray) – Image to be packed.
quality (int) – Quality for JPEG encoding in range 1-100, or compression for PNG encoding in range 1-9.
img_fmt (str) – Encoding of the image (.jpg for JPEG, .png for PNG).

Returns

s – The packed string.

Return type

str

Examples

>>> label = 4 # label can also be a 1-D array, for example: label = [1,2,3]
>>> id = 2574
>>> header = mx.recordio.IRHeader(0, label, id, 0)
>>> img = cv2.imread('test.jpg')
>>> packed_s = mx.recordio.pack_img(header, img)

mxnet.recordio.unpack(s)[source]¶

Unpack a MXImageRecord to string.

Parameters

s (str) – String buffer from MXRecordIO.read.

Returns

header (IRHeader) – Header of the image record.
s (str) – Unpacked string.

Examples

>>> record = mx.recordio.MXRecordIO('test.rec', 'r')
>>> item = record.read()
>>> header, s = mx.recordio.unpack(item)
>>> header
HEADER(flag=0, label=14.0, id=20129312, id2=0)

mxnet.recordio.unpack_img(s, iscolor=-1)[source]¶

Unpack a MXImageRecord to image.

Parameters

s (str) – String buffer from MXRecordIO.read.
iscolor (int) – Image format option for cv2.imdecode.

Returns

header (IRHeader) – Header of the image record.
img (numpy.ndarray) – Unpacked image.

Examples

>>> record = mx.recordio.MXRecordIO('test.rec', 'r')
>>> item = record.read()
>>> header, img = mx.recordio.unpack_img(item)
>>> header
HEADER(flag=0, label=14.0, id=20129312, id2=0)
>>> img
array([[[ 23,  27,  45],
        [ 28,  32,  50],
        ...,
        [ 36,  40,  59],
        [ 35,  39,  58]],
       ...,
       [[ 91,  92, 113],
        [ 97,  98, 119],
        ...,
        [168, 169, 167],
        [166, 167, 165]]], dtype=uint8)

Did this page help you?

Yes

No

Thanks for your feedback!