easycv.datasets.detection.data_sources package

class easycv.datasets.detection.data_sources.DetSourceCoco(ann_file, img_prefix, pipeline, test_mode=False, filter_empty_gt=False, classes=None, iscrowd=False)[source]

Bases: object

coco data source

__init__(ann_file, img_prefix, pipeline, test_mode=False, filter_empty_gt=False, classes=None, iscrowd=False)[source]
Parameters

  • ann_file – Path of annotation file.

  • img_prefix – coco path prefix

  • test_mode (bool, optional) – If set True, self._filter_imgs will not works.

  • filter_empty_gt (bool, optional) – If set true, images without bounding boxes of the dataset’s classes will be filtered out. This option only works when test_mode=False, i.e., we never filter images during tests.

  • iscrowd – when traing setted as False, when val setted as True

load_annotations(ann_file)[source]

Load annotation from COCO style annotation file. :param ann_file: Path of annotation file. :type ann_file: str

Returns

Annotation info from COCO api.

Return type

list[dict]

get_ann_info(idx)[source]

Get COCO annotation by index. :param idx: Index of data. :type idx: int

Returns

Annotation info of specified index.

Return type

dict

get_cat_ids(idx)[source]

Get COCO category ids by index. :param idx: Index of data. :type idx: int

Returns

All categories in the image of specified index.

Return type

list[int]

xyxy2xywh(bbox)[source]

Convert xyxy style bounding boxes to xywh style for COCO evaluation. :param bbox: The bounding boxes, shape (4, ), in

xyxy order.

Returns

The converted bounding boxes, in xywh order.

Return type

list[float]

pre_pipeline(results)[source]

Prepare results dict for pipeline.

prepare_train_img(idx)[source]

Get training data and annotations after pipeline. :param idx: Index of data. :type idx: int

Returns

Training data and annotation after pipeline with new keys introduced by pipeline.

Return type

dict

class easycv.datasets.detection.data_sources.DetSourceCocoPanoptic(ann_file, pan_ann_file, img_prefix, seg_prefix, pipeline, outfile_prefix='test/test_pan', test_mode=False, filter_empty_gt=False, thing_classes=None, stuff_classes=None, iscrowd=False)[source]

Bases: easycv.datasets.detection.data_sources.coco.DetSourceCoco

cocopanoptic data source

__init__(ann_file, pan_ann_file, img_prefix, seg_prefix, pipeline, outfile_prefix='test/test_pan', test_mode=False, filter_empty_gt=False, thing_classes=None, stuff_classes=None, iscrowd=False)[source]
Parameters

  • ann_file (str) – Path of coco detection annotation file

  • pan_ann_file (str) – Path of coco panoptic annotation file

  • img_prefix (str) – Path of image file

  • seg_prefix (str) – Path of semantic image file

  • pipeline (list[dict]) – list of data augmentatin operation

  • outfile_prefix (str, optional) – The filename prefix of the output files. If the prefix is “somepath/xxx”, the json files will be named “somepath/xxx.panoptic.json”, “somepath/xxx.bbox.json”, “somepath/xxx.segm.json”

  • test_mode (bool, optional) – If set True, self._filter_imgs will not works.

  • filter_empty_gt (bool, optional) – If set true, images without bounding boxes of the dataset’s classes will be filtered out. This option only works when test_mode=False, i.e., we never filter images during tests.

  • thing_classes (list[str], optional) – list of thing classes. Defaults to None.

  • stuff_classes (list[str], optional) – list of thing classes. Defaults to None.

  • iscrowd (bool, optional) – when traing setted as False, when val setted as True. Defaults to False.

load_annotations_pan(ann_file)[source]

Load annotation from COCO Panoptic style annotation file.

Parameters

ann_file (str) – Path of annotation file.

Returns

Annotation info from COCO api.

Return type

list[dict]

get_ann_info_pan(idx)[source]

Get COCO annotation by index.

Parameters

idx (int) – Index of data.

Returns

Annotation info of specified index.

Return type

dict

pre_pipeline(results)[source]

Prepare results dict for pipeline.

prepare_train_img(idx)[source]

Get training data and annotations after pipeline.

Parameters

idx (int) – Index of data.

Returns

Training data and annotation after pipeline with new keys introduced by pipeline.

Return type

dict

results2json(results)[source]

Dump the results to a COCO style json file.

There are 4 types of results: proposals, bbox predictions, mask predictions, panoptic segmentation predictions, and they have different data types. This method will automatically recognize the type, and dump them to json files.

[
    {
        'pan_results': np.array, # shape (h, w)
        # ins_results which includes bboxes and RLE encoded masks
        # is optional.
        'ins_results': (list[np.array], list[list[str]])
    },
    ...
]
Parameters

results (list[dict]) – Testing results of the dataset.

Returns

str]: Possible keys are “panoptic”, “bbox”, “segm”, “proposal”, and values are corresponding filenames.

Return type

dict[str

get_gt_json(result_files)[source]

get input for coco panptic evaluation

Parameters

result_files (dict) – path of predict result

Returns

gt label gt_folder (str): path of gt file pred_json(dict): predict result pred_folder(str): path of pred file categories(dict): panoptic categories

Return type

gt_json (dict)

class easycv.datasets.detection.data_sources.DetSourceObjects365(ann_file, img_prefix, pipeline, test_mode=False, filter_empty_gt=False, classes=[], iscrowd=False)[source]

Bases: easycv.datasets.detection.data_sources.coco.DetSourceCoco

objects365 data source. The form of the objects365 dataset folder build:

|- objects365
|- annotation

|- zhiyuan_objv2_train.json |- zhiyuan_objv2_val.json

|- train
|- patch0

|- *****(imageID)

|- patch1

|- *****(imageID)

… |- patch50

|- *****(imageID)

|- val
|- patch0

|- *****(imageID)

|- patch1

|- *****(imageID)

… |- patch43

|- *****(imageID)

__init__(ann_file, img_prefix, pipeline, test_mode=False, filter_empty_gt=False, classes=[], iscrowd=False)[source]
Parameters

  • ann_file – Path of annotation file.

  • img_prefix – coco path prefix

  • test_mode (bool, optional) – If set True, self._filter_imgs will not works.

  • filter_empty_gt (bool, optional) – If set true, images without bounding boxes of the dataset’s classes will be filtered out. This option only works when test_mode=False, i.e., we never filter images during tests.

  • iscrowd – when traing setted as False, when val setted as True

load_annotations(ann_file)[source]

Load annotation from COCO style annotation file. :param ann_file: Path of annotation file. :type ann_file: str

Returns

Annotation info from COCO api.

Return type

list[dict]

class easycv.datasets.detection.data_sources.DetSourcePAI(path, classes=[], cache_at_init=False, cache_on_the_fly=False, parse_fn=<function parser_manifest_row_str>, num_processes=1, **kwargs)[source]

Bases: easycv.datasets.detection.data_sources.base.DetSourceBase

data format please refer to: https://help.aliyun.com/document_detail/311173.html

__init__(path, classes=[], cache_at_init=False, cache_on_the_fly=False, parse_fn=<function parser_manifest_row_str>, num_processes=1, **kwargs)[source]
Parameters

  • path – Path of manifest path with pai label format

  • classes – classes list

  • cache_at_init – if set True, will cache in memory in __init__ for faster training

  • cache_on_the_fly – if set True, will cache in memroy during training

  • parse_fn – parse function to parse item of source iterator

  • num_processes – number of processes to parse samples

get_source_iterator()[source]

Return data list iterator, source iterator will be passed to parse_fn, and parse_fn will receive params of item of source iter and classes for parsing. What does parse_fn need, what does source iterator returns.

class easycv.datasets.detection.data_sources.DetSourceRaw(img_root_path, label_root_path, classes=[], cache_at_init=False, cache_on_the_fly=False, delimeter=' ', parse_fn=<function parse_raw>, num_processes=1, **kwargs)[source]

Bases: easycv.datasets.detection.data_sources.base.DetSourceBase

data dir is as follows: ``` |- data_dir

|-images

|-1.jpg |-…

|-labels

|-1.txt |-…

` Label txt file is as follows: The first column is the label id, and columns 2 to 5 are coordinates relative to the image width and height [x_center, y_center, bbox_w, bbox_h]. ` 15 0.519398 0.544087 0.476359 0.572061 2 0.501859 0.820726 0.996281 0.332178 … ``` .. rubric:: Example

data_source = DetSourceRaw(

img_root_path=’/your/data_dir/images’, label_root_path=’/your/data_dir/labels’,

)

__init__(img_root_path, label_root_path, classes=[], cache_at_init=False, cache_on_the_fly=False, delimeter=' ', parse_fn=<function parse_raw>, num_processes=1, **kwargs)[source]
Parameters

  • img_root_path – images dir path

  • label_root_path – labels dir path

  • classes (list, optional) – classes list

  • cache_at_init – if set True, will cache in memory in __init__ for faster training

  • cache_on_the_fly – if set True, will cache in memroy during training

  • delimeter – delimeter of txt file

  • parse_fn – parse function to parse item of source iterator

  • num_processes – number of processes to parse samples

get_source_iterator()[source]

Return data list iterator, source iterator will be passed to parse_fn, and parse_fn will receive params of item of source iter and classes for parsing. What does parse_fn need, what does source iterator returns.

post_process_fn(result_dict)[source]
get_ann_info(idx)[source]

Get raw annotation info, include bounding boxes, labels and so on. bboxes format is as [x1, y1, x2, y2] without normalization.

class easycv.datasets.detection.data_sources.DetSourceVOC(path, classes=[], img_root_path=None, label_root_path=None, cache_at_init=False, cache_on_the_fly=False, img_suffix='.jpg', label_suffix='.xml', parse_fn=<function parse_xml>, num_processes=1, **kwargs)[source]

Bases: easycv.datasets.detection.data_sources.base.DetSourceBase

data dir is as follows: ``` |- voc_data

|-ImageSets
|-Main

|-train.txt |-…

|-JPEGImages

|-00001.jpg |-…

|-Annotations

|-00001.xml |-…

``` Example1:

data_source = DetSourceVOC(

path=’/your/voc_data/ImageSets/Main/train.txt’, classes=${VOC_CLASSES},

)

Example1:
data_source = DetSourceVOC(

path=’/your/voc_data/train.txt’, classes=${VOC_CLASSES}, img_root_path=’/your/voc_data/images’, img_root_path=’/your/voc_data/annotations’

)

__init__(path, classes=[], img_root_path=None, label_root_path=None, cache_at_init=False, cache_on_the_fly=False, img_suffix='.jpg', label_suffix='.xml', parse_fn=<function parse_xml>, num_processes=1, **kwargs)[source]
Parameters

  • path – path of img id list file in ImageSets/Main/

  • classes – classes list

  • img_root_path – image dir path, if None, default to detect the image dir by the relative path of the path according to the VOC data format.

  • label_root_path – label dir path, if None, default to detect the label dir by the relative path of the path according to the VOC data format.

  • cache_at_init – if set True, will cache in memory in __init__ for faster training

  • cache_on_the_fly – if set True, will cache in memroy during training

  • img_suffix – suffix of image file

  • label_suffix – suffix of label file

  • parse_fn – parse function to parse item of source iterator

  • num_processes – number of processes to parse samples

get_source_iterator()[source]

Return data list iterator, source iterator will be passed to parse_fn, and parse_fn will receive params of item of source iter and classes for parsing. What does parse_fn need, what does source iterator returns.

class easycv.datasets.detection.data_sources.DetSourceVOC2007(path=None, download=True, split='train', classes=[], img_root_path=None, label_root_path=None, cache_at_init=False, cache_on_the_fly=False, img_suffix='.jpg', label_suffix='.xml', parse_fn=<function parse_xml>, num_processes=1, **kwargs)[source]

Bases: easycv.datasets.detection.data_sources.voc.DetSourceVOC

__init__(path=None, download=True, split='train', classes=[], img_root_path=None, label_root_path=None, cache_at_init=False, cache_on_the_fly=False, img_suffix='.jpg', label_suffix='.xml', parse_fn=<function parse_xml>, num_processes=1, **kwargs)[source]
Parameters

  • path – This parameter is optional. If download is True and path is not provided, a temporary directory is automatically created for downloading

  • download – If the value is True, the file is automatically downloaded to the path directory. If False, automatic download is not supported and data in the path is used

  • split – train or val

  • classes – classes list

  • img_root_path – image dir path, if None, default to detect the image dir by the relative path of the path according to the VOC data format.

  • label_root_path – label dir path, if None, default to detect the label dir by the relative path of the path according to the VOC data format.

  • cache_at_init – if set True, will cache in memory in __init__ for faster training

  • cache_on_the_fly – if set True, will cache in memroy during training

  • img_suffix – suffix of image file

  • label_suffix – suffix of label file

  • parse_fn – parse function to parse item of source iterator

  • num_processes – number of processes to parse samples

class easycv.datasets.detection.data_sources.DetSourceVOC2012(path=None, download=True, split='train', classes=[], img_root_path=None, label_root_path=None, cache_at_init=False, cache_on_the_fly=False, img_suffix='.jpg', label_suffix='.xml', parse_fn=<function parse_xml>, num_processes=1, **kwargs)[source]

Bases: easycv.datasets.detection.data_sources.voc.DetSourceVOC

__init__(path=None, download=True, split='train', classes=[], img_root_path=None, label_root_path=None, cache_at_init=False, cache_on_the_fly=False, img_suffix='.jpg', label_suffix='.xml', parse_fn=<function parse_xml>, num_processes=1, **kwargs)[source]
Parameters

  • path – This parameter is optional. If download is True and path is not provided, a temporary directory is automatically created for downloading

  • download – If the value is True, the file is automatically downloaded to the path directory. If False, automatic download is not supported and data in the path is used

  • split – train or val

  • classes – classes list

  • img_root_path – image dir path, if None, default to detect the image dir by the relative path of the path according to the VOC data format.

  • label_root_path – label dir path, if None, default to detect the label dir by the relative path of the path according to the VOC data format.

  • cache_at_init – if set True, will cache in memory in __init__ for faster training

  • cache_on_the_fly – if set True, will cache in memroy during training

  • img_suffix – suffix of image file

  • label_suffix – suffix of label file

  • parse_fn – parse function to parse item of source iterator

  • num_processes – number of processes to parse samples

class easycv.datasets.detection.data_sources.DetSourceCoco2017(pipeline, path=None, download=True, split='train', test_mode=False, filter_empty_gt=False, classes=None, iscrowd=False)[source]

Bases: easycv.datasets.detection.data_sources.coco.DetSourceCoco

coco2017 data source

__init__(pipeline, path=None, download=True, split='train', test_mode=False, filter_empty_gt=False, classes=None, iscrowd=False)[source]
Parameters

  • path – This parameter is optional. If download is True and path is not provided, a temporary directory is automatically created for downloading

  • download – If the value is True, the file is automatically downloaded to the path directory. If False, automatic download is not supported and data in the path is used

  • split – train or val

  • test_mode (bool, optional) – If set True, self._filter_imgs will not works.

  • filter_empty_gt (bool, optional) – If set true, images without bounding boxes of the dataset’s classes will be filtered out. This option only works when test_mode=False, i.e., we never filter images during tests.

  • iscrowd – when traing setted as False, when val setted as True

class easycv.datasets.detection.data_sources.DetSourceLvis(pipeline, path=None, download=True, split='train', test_mode=False, filter_empty_gt=False, classes=None, iscrowd=False, **kwargs)[source]

Bases: easycv.datasets.detection.data_sources.coco.DetSourceCoco

lvis data source

__init__(pipeline, path=None, download=True, split='train', test_mode=False, filter_empty_gt=False, classes=None, iscrowd=False, **kwargs)[source]
Parameters

  • path – This parameter is optional. If download is True and path is not provided, a temporary directory is automatically created for downloading

  • download – If the value is True, the file is automatically downloaded to the path directory. If False, automatic download is not supported and data in the path is used

  • split – train or val

  • test_mode (bool, optional) – If set True, self._filter_imgs will not works.

  • filter_empty_gt (bool, optional) – If set true, images without bounding boxes of the dataset’s classes will be filtered out. This option only works when test_mode=False, i.e., we never filter images during tests.

  • iscrowd – when traing setted as False, when val setted as True

cfg = {'dataset': 'images', 'links': ['https://s3-us-west-2.amazonaws.com/dl.fbaipublicfiles.com/LVIS/lvis_v1_train.json.zip', 'https://s3-us-west-2.amazonaws.com/dl.fbaipublicfiles.com/LVIS/lvis_v1_val.json.zip', 'http://images.cocodataset.org/zips/train2017.zip', 'http://images.cocodataset.org/zips/val2017.zip'], 'train': 'lvis_v1_train.json', 'val': 'lvis_v1_val.json'}
load_annotations(ann_file)[source]

Load annotation from COCO style annotation file. :param ann_file: Path of annotation file. :type ann_file: str

Returns

Annotation info from COCO api.

Return type

list[dict]

download()[source]
merge_images_folder()[source]
class easycv.datasets.detection.data_sources.DetSourceWiderPerson(path, classes=['pedestrians', 'riders', 'partially-visible persons', 'ignore regions', 'crowd'], img_root_path=None, label_root_path=None, cache_at_init=False, cache_on_the_fly=False, img_suffix='.jpg', label_suffix='.txt', parse_fn=<function parse_txt>, num_processes=1, **kwargs)[source]

Bases: easycv.datasets.detection.data_sources.base.DetSourceBase

CLASSES = ['pedestrians', 'riders', 'partially-visible persons', 'ignore regions', 'crowd']

dataset_name=’Wider Person’, paper_info=@article{zhang2019widerperson, Author = {Zhang, Shifeng and Xie, Yiliang and Wan, Jun and Xia, Hansheng and Li, Stan Z. and Guo, Guodong}, journal = {IEEE Transactions on Multimedia (TMM)}, Title = {WiderPerson: A Diverse Dataset for Dense Pedestrian Detection in the Wild}, Year = {2019}}

__init__(path, classes=['pedestrians', 'riders', 'partially-visible persons', 'ignore regions', 'crowd'], img_root_path=None, label_root_path=None, cache_at_init=False, cache_on_the_fly=False, img_suffix='.jpg', label_suffix='.txt', parse_fn=<function parse_txt>, num_processes=1, **kwargs) → None[source]
Parameters

  • path – path of img id list file in root

  • classes – classes list

  • img_root_path – image dir path, if None, default to detect the image dir by the relative path of the path according to the WiderPerso data format.

  • label_root_path – label dir path, if None, default to detect the label dir by the relative path of the path according to the WiderPerso data format.

  • cache_at_init – if set True, will cache in memory in __init__ for faster training

  • cache_on_the_fly – if set True, will cache in memroy during training

  • img_suffix – suffix of image file

  • label_suffix – suffix of label file

  • parse_fn – parse function to parse item of source iterator

  • num_processes – number of processes to parse samples

get_source_iterator()[source]

Return data list iterator, source iterator will be passed to parse_fn, and parse_fn will receive params of item of source iter and classes for parsing. What does parse_fn need, what does source iterator returns.

class easycv.datasets.detection.data_sources.DetSourceAfricanWildlife(path, classes=['buffalo', 'elephant', 'rhino', 'zebra'], cache_at_init=False, cache_on_the_fly=False, img_suffix='.jpg', label_suffix='.txt', parse_fn=<function parse_txt>, num_processes=1, **kwargs)[source]

Bases: easycv.datasets.detection.data_sources.base.DetSourceBase

data dir is as follows: ``` |- data

|-buffalo

|-001.jpg |-001.txt |-…

|-elephant

|-001.jpg |-001.txt |-…

|-rhino

|-001.jpg |-001.txt

|-…

``` Example1:

data_source = DetSourceAfricanWildlife(

path=’/your/data/’, classes=${CLASSES},

)

CLASSES = ['buffalo', 'elephant', 'rhino', 'zebra']
__init__(path, classes=['buffalo', 'elephant', 'rhino', 'zebra'], cache_at_init=False, cache_on_the_fly=False, img_suffix='.jpg', label_suffix='.txt', parse_fn=<function parse_txt>, num_processes=1, **kwargs) → None[source]
Parameters

  • path – path of img id list file in root

  • classes – classes list

  • cache_at_init – if set True, will cache in memory in __init__ for faster training

  • cache_on_the_fly – if set True, will cache in memroy during training

  • img_suffix – suffix of image file

  • label_suffix – suffix of label file

  • parse_fn – parse function to parse item of source iterator

  • num_processes – number of processes to parse samples

get_source_iterator()[source]

Return data list iterator, source iterator will be passed to parse_fn, and parse_fn will receive params of item of source iter and classes for parsing. What does parse_fn need, what does source iterator returns.

class easycv.datasets.detection.data_sources.DetSourcePet(path, classes_id=1, img_root_path=None, label_root_path=None, cache_at_init=False, cache_on_the_fly=False, img_suffix='.jpg', label_suffix='.xml', parse_fn=<function parse_xml>, num_processes=1, **kwargs)[source]

Bases: easycv.datasets.detection.data_sources.base.DetSourceBase

data dir is as follows: ``` |- data

|-annotations
|-annotations

|-list.txt |-test.txt |-trainval.txt |-xmls

|-Abyssinian_6.xml |-…

|-images
|-images

|-Abyssinian_6.jpg |-…

``` Example0:

data_source = DetSourcePet(

path=’/your/data/annotations/annotations/trainval.txt’, classes_id=1 or 2 or 3,

Example1:
data_source = DetSourcePet(

path=’/your/data/annotations/annotations/trainval.txt’, classes_id=1 or 2 or 3, img_root_path=’/your/data//images’, img_root_path=’/your/data/annotations/annotations/xmls’

)

CLASSES_CFG = {1: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37], 2: [1, 2], 3: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25]}
__init__(path, classes_id=1, img_root_path=None, label_root_path=None, cache_at_init=False, cache_on_the_fly=False, img_suffix='.jpg', label_suffix='.xml', parse_fn=<function parse_xml>, num_processes=1, **kwargs)[source]
Parameters

  • path – path of img id list file in pet format

  • classes_id – 1= 1:37 Class ids, 2 = 1:Cat 2:Dog, 3 = 1-25:Cat 1:12:Dog

  • cache_at_init – if set True, will cache in memory in __init__ for faster training

  • cache_on_the_fly – if set True, will cache in memroy during training

  • img_suffix – suffix of image file

  • label_suffix – suffix of label file

  • parse_fn – parse function to parse item of source iterator

  • num_processes – number of processes to parse samples

get_source_iterator()[source]

Return data list iterator, source iterator will be passed to parse_fn, and parse_fn will receive params of item of source iter and classes for parsing. What does parse_fn need, what does source iterator returns.

class easycv.datasets.detection.data_sources.DetSourceWiderFace(ann_file, img_prefix, classes='blur', cache_at_init=False, cache_on_the_fly=False, parse_fn=<function parse_load>, num_processes=1, **kwargs)[source]

Bases: easycv.datasets.detection.data_sources.base.DetSourceBase

CLASSES = {'blur': ['clear', 'normal blur', 'heavy blur'], 'expression': ['typical expression', 'exaggerate expression'], 'illumination': ['normal illumination', 'extreme illumination'], 'invalid': ['false valid image)', 'true (invalid image)'], 'occlusion': ['no occlusion', 'partial occlusion', 'heavy occlusion'], 'pose': ['typical pose', 'atypical pose']}

Citation: @inproceedings{yang2016wider, Author = {Yang, Shuo and Luo, Ping and Loy, Chen Change and Tang, Xiaoou}, Booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, Title = {WIDER FACE: A Face Detection Benchmark}, Year = {2016}}

__init__(ann_file, img_prefix, classes='blur', cache_at_init=False, cache_on_the_fly=False, parse_fn=<function parse_load>, num_processes=1, **kwargs) → None[source]
Parameters

  • ann_file (str) – Path to the annotation file.

  • img_prefix (str) – Path to a directory where images are held.

  • classes (str) – classes defalut=’blur’

  • cache_at_init – if set True, will cache in memory in __init__ for faster training

  • cache_on_the_fly – if set True, will cache in memroy during training

  • parse_fn – parse function to parse item of source iterator

  • num_processes – number of processes to parse samples

get_source_iterator()[source]

Return data list iterator, source iterator will be passed to parse_fn, and parse_fn will receive params of item of source iter and classes for parsing. What does parse_fn need, what does source iterator returns.

class easycv.datasets.detection.data_sources.DetSourceCrowdHuman(ann_file, img_prefix, gt_op='vbox', classes=['mask', 'person'], cache_at_init=False, cache_on_the_fly=False, parse_fn=<function parse_load>, num_processes=1, **kwargs)[source]

Bases: easycv.datasets.detection.data_sources.base.DetSourceBase

CLASSES = ['mask', 'person']
Citation:

@article{shao2018crowdhuman, title={CrowdHuman: A Benchmark for Detecting Human in a Crowd}, author={Shao, Shuai and Zhao, Zijian and Li, Boxun and Xiao, Tete and Yu, Gang and Zhang, Xiangyu and Sun, Jian}, journal={arXiv preprint arXiv:1805.00123}, year={2018}

}

__init__(ann_file, img_prefix, gt_op='vbox', classes=['mask', 'person'], cache_at_init=False, cache_on_the_fly=False, parse_fn=<function parse_load>, num_processes=1, **kwargs) → None[source]
Parameters

  • ann_file (str) – Path to the annotation file.

  • img_prefix (str) – Path to a directory where images are held.

  • gt_op (str) – vbox(visible box), fbox(full box), hbox(head box), defalut vbox

  • classes (list) – classes defalut=[‘mask’, ‘person’]

  • cache_at_init – if set True, will cache in memory in __init__ for faster training

  • cache_on_the_fly – if set True, will cache in memroy during training

  • parse_fn – parse function to parse item of source iterator

  • num_processes – number of processes to parse samples

get_source_iterator()[source]

Return data list iterator, source iterator will be passed to parse_fn, and parse_fn will receive params of item of source iter and classes for parsing. What does parse_fn need, what does source iterator returns.

Submodules

easycv.datasets.detection.data_sources.coco module

class easycv.datasets.detection.data_sources.coco.DetSourceCoco(ann_file, img_prefix, pipeline, test_mode=False, filter_empty_gt=False, classes=None, iscrowd=False)[source]

Bases: object

coco data source

__init__(ann_file, img_prefix, pipeline, test_mode=False, filter_empty_gt=False, classes=None, iscrowd=False)[source]
Parameters

  • ann_file – Path of annotation file.

  • img_prefix – coco path prefix

  • test_mode (bool, optional) – If set True, self._filter_imgs will not works.

  • filter_empty_gt (bool, optional) – If set true, images without bounding boxes of the dataset’s classes will be filtered out. This option only works when test_mode=False, i.e., we never filter images during tests.

  • iscrowd – when traing setted as False, when val setted as True

load_annotations(ann_file)[source]

Load annotation from COCO style annotation file. :param ann_file: Path of annotation file. :type ann_file: str

Returns

Annotation info from COCO api.

Return type

list[dict]

get_ann_info(idx)[source]

Get COCO annotation by index. :param idx: Index of data. :type idx: int

Returns

Annotation info of specified index.

Return type

dict

get_cat_ids(idx)[source]

Get COCO category ids by index. :param idx: Index of data. :type idx: int

Returns

All categories in the image of specified index.

Return type

list[int]

xyxy2xywh(bbox)[source]

Convert xyxy style bounding boxes to xywh style for COCO evaluation. :param bbox: The bounding boxes, shape (4, ), in

xyxy order.

Returns

The converted bounding boxes, in xywh order.

Return type

list[float]

pre_pipeline(results)[source]

Prepare results dict for pipeline.

prepare_train_img(idx)[source]

Get training data and annotations after pipeline. :param idx: Index of data. :type idx: int

Returns

Training data and annotation after pipeline with new keys introduced by pipeline.

Return type

dict

class easycv.datasets.detection.data_sources.coco.DetSourceCoco2017(pipeline, path=None, download=True, split='train', test_mode=False, filter_empty_gt=False, classes=None, iscrowd=False)[source]

Bases: easycv.datasets.detection.data_sources.coco.DetSourceCoco

coco2017 data source

__init__(pipeline, path=None, download=True, split='train', test_mode=False, filter_empty_gt=False, classes=None, iscrowd=False)[source]
Parameters

  • path – This parameter is optional. If download is True and path is not provided, a temporary directory is automatically created for downloading

  • download – If the value is True, the file is automatically downloaded to the path directory. If False, automatic download is not supported and data in the path is used

  • split – train or val

  • test_mode (bool, optional) – If set True, self._filter_imgs will not works.

  • filter_empty_gt (bool, optional) – If set true, images without bounding boxes of the dataset’s classes will be filtered out. This option only works when test_mode=False, i.e., we never filter images during tests.

  • iscrowd – when traing setted as False, when val setted as True

class easycv.datasets.detection.data_sources.coco.DetSourceTinyPerson(ann_file, img_prefix, pipeline, test_mode=False, filter_empty_gt=False, classes=['sea_person', 'earth_person'], iscrowd=False)[source]

Bases: easycv.datasets.detection.data_sources.coco.DetSourceCoco

TINY PERSON data source

CLASSES = ['sea_person', 'earth_person']
__init__(ann_file, img_prefix, pipeline, test_mode=False, filter_empty_gt=False, classes=['sea_person', 'earth_person'], iscrowd=False)[source]
Parameters

  • ann_file – Path of annotation file.

  • img_prefix – coco path prefix

  • test_mode (bool, optional) – If set True, self._filter_imgs will not works.

  • filter_empty_gt (bool, optional) – If set true, images without bounding boxes of the dataset’s classes will be filtered out. This option only works when test_mode=False, i.e., we never filter images during tests.

  • iscrowd – when traing setted as False, when val setted as True

easycv.datasets.detection.data_sources.pai_format module

easycv.datasets.detection.data_sources.pai_format.get_prior_task_id(keys)[source]

“The task id ends with check is the highest priority.

easycv.datasets.detection.data_sources.pai_format.is_itag_v2(row)[source]

The keyword of the data source is picUrl in v1, but is source in v2

easycv.datasets.detection.data_sources.pai_format.parser_manifest_row_str(row_str, classes)[source]
class easycv.datasets.detection.data_sources.pai_format.DetSourcePAI(path, classes=[], cache_at_init=False, cache_on_the_fly=False, parse_fn=<function parser_manifest_row_str>, num_processes=1, **kwargs)[source]

Bases: easycv.datasets.detection.data_sources.base.DetSourceBase

data format please refer to: https://help.aliyun.com/document_detail/311173.html

__init__(path, classes=[], cache_at_init=False, cache_on_the_fly=False, parse_fn=<function parser_manifest_row_str>, num_processes=1, **kwargs)[source]
Parameters

  • path – Path of manifest path with pai label format

  • classes – classes list

  • cache_at_init – if set True, will cache in memory in __init__ for faster training

  • cache_on_the_fly – if set True, will cache in memroy during training

  • parse_fn – parse function to parse item of source iterator

  • num_processes – number of processes to parse samples

get_source_iterator()[source]

Return data list iterator, source iterator will be passed to parse_fn, and parse_fn will receive params of item of source iter and classes for parsing. What does parse_fn need, what does source iterator returns.

easycv.datasets.detection.data_sources.raw module

easycv.datasets.detection.data_sources.raw.parse_raw(source_iter, classes=None, delimeter=' ')[source]
class easycv.datasets.detection.data_sources.raw.DetSourceRaw(img_root_path, label_root_path, classes=[], cache_at_init=False, cache_on_the_fly=False, delimeter=' ', parse_fn=<function parse_raw>, num_processes=1, **kwargs)[source]

Bases: easycv.datasets.detection.data_sources.base.DetSourceBase

data dir is as follows: ``` |- data_dir

|-images

|-1.jpg |-…

|-labels

|-1.txt |-…

` Label txt file is as follows: The first column is the label id, and columns 2 to 5 are coordinates relative to the image width and height [x_center, y_center, bbox_w, bbox_h]. ` 15 0.519398 0.544087 0.476359 0.572061 2 0.501859 0.820726 0.996281 0.332178 … ``` .. rubric:: Example

data_source = DetSourceRaw(

img_root_path=’/your/data_dir/images’, label_root_path=’/your/data_dir/labels’,

)

__init__(img_root_path, label_root_path, classes=[], cache_at_init=False, cache_on_the_fly=False, delimeter=' ', parse_fn=<function parse_raw>, num_processes=1, **kwargs)[source]
Parameters

  • img_root_path – images dir path

  • label_root_path – labels dir path

  • classes (list, optional) – classes list

  • cache_at_init – if set True, will cache in memory in __init__ for faster training

  • cache_on_the_fly – if set True, will cache in memroy during training

  • delimeter – delimeter of txt file

  • parse_fn – parse function to parse item of source iterator

  • num_processes – number of processes to parse samples

get_source_iterator()[source]

Return data list iterator, source iterator will be passed to parse_fn, and parse_fn will receive params of item of source iter and classes for parsing. What does parse_fn need, what does source iterator returns.

post_process_fn(result_dict)[source]
get_ann_info(idx)[source]

Get raw annotation info, include bounding boxes, labels and so on. bboxes format is as [x1, y1, x2, y2] without normalization.

easycv.datasets.detection.data_sources.utils module

easycv.datasets.detection.data_sources.utils.exif_size(img)[source]

easycv.datasets.detection.data_sources.voc module

easycv.datasets.detection.data_sources.voc.parse_xml(source_item, classes)[source]
class easycv.datasets.detection.data_sources.voc.DetSourceVOC(path, classes=[], img_root_path=None, label_root_path=None, cache_at_init=False, cache_on_the_fly=False, img_suffix='.jpg', label_suffix='.xml', parse_fn=<function parse_xml>, num_processes=1, **kwargs)[source]

Bases: easycv.datasets.detection.data_sources.base.DetSourceBase

data dir is as follows: ``` |- voc_data

|-ImageSets
|-Main

|-train.txt |-…

|-JPEGImages

|-00001.jpg |-…

|-Annotations

|-00001.xml |-…

``` Example1:

data_source = DetSourceVOC(

path=’/your/voc_data/ImageSets/Main/train.txt’, classes=${VOC_CLASSES},

)

Example1:
data_source = DetSourceVOC(

path=’/your/voc_data/train.txt’, classes=${VOC_CLASSES}, img_root_path=’/your/voc_data/images’, img_root_path=’/your/voc_data/annotations’

)

__init__(path, classes=[], img_root_path=None, label_root_path=None, cache_at_init=False, cache_on_the_fly=False, img_suffix='.jpg', label_suffix='.xml', parse_fn=<function parse_xml>, num_processes=1, **kwargs)[source]
Parameters

  • path – path of img id list file in ImageSets/Main/

  • classes – classes list

  • img_root_path – image dir path, if None, default to detect the image dir by the relative path of the path according to the VOC data format.

  • label_root_path – label dir path, if None, default to detect the label dir by the relative path of the path according to the VOC data format.

  • cache_at_init – if set True, will cache in memory in __init__ for faster training

  • cache_on_the_fly – if set True, will cache in memroy during training

  • img_suffix – suffix of image file

  • label_suffix – suffix of label file

  • parse_fn – parse function to parse item of source iterator

  • num_processes – number of processes to parse samples

get_source_iterator()[source]

Return data list iterator, source iterator will be passed to parse_fn, and parse_fn will receive params of item of source iter and classes for parsing. What does parse_fn need, what does source iterator returns.

class easycv.datasets.detection.data_sources.voc.DetSourceVOC2012(path=None, download=True, split='train', classes=[], img_root_path=None, label_root_path=None, cache_at_init=False, cache_on_the_fly=False, img_suffix='.jpg', label_suffix='.xml', parse_fn=<function parse_xml>, num_processes=1, **kwargs)[source]

Bases: easycv.datasets.detection.data_sources.voc.DetSourceVOC

__init__(path=None, download=True, split='train', classes=[], img_root_path=None, label_root_path=None, cache_at_init=False, cache_on_the_fly=False, img_suffix='.jpg', label_suffix='.xml', parse_fn=<function parse_xml>, num_processes=1, **kwargs)[source]
Parameters

  • path – This parameter is optional. If download is True and path is not provided, a temporary directory is automatically created for downloading

  • download – If the value is True, the file is automatically downloaded to the path directory. If False, automatic download is not supported and data in the path is used

  • split – train or val

  • classes – classes list

  • img_root_path – image dir path, if None, default to detect the image dir by the relative path of the path according to the VOC data format.

  • label_root_path – label dir path, if None, default to detect the label dir by the relative path of the path according to the VOC data format.

  • cache_at_init – if set True, will cache in memory in __init__ for faster training

  • cache_on_the_fly – if set True, will cache in memroy during training

  • img_suffix – suffix of image file

  • label_suffix – suffix of label file

  • parse_fn – parse function to parse item of source iterator

  • num_processes – number of processes to parse samples

class easycv.datasets.detection.data_sources.voc.DetSourceVOC2007(path=None, download=True, split='train', classes=[], img_root_path=None, label_root_path=None, cache_at_init=False, cache_on_the_fly=False, img_suffix='.jpg', label_suffix='.xml', parse_fn=<function parse_xml>, num_processes=1, **kwargs)[source]

Bases: easycv.datasets.detection.data_sources.voc.DetSourceVOC

__init__(path=None, download=True, split='train', classes=[], img_root_path=None, label_root_path=None, cache_at_init=False, cache_on_the_fly=False, img_suffix='.jpg', label_suffix='.xml', parse_fn=<function parse_xml>, num_processes=1, **kwargs)[source]
Parameters

  • path – This parameter is optional. If download is True and path is not provided, a temporary directory is automatically created for downloading

  • download – If the value is True, the file is automatically downloaded to the path directory. If False, automatic download is not supported and data in the path is used

  • split – train or val

  • classes – classes list

  • img_root_path – image dir path, if None, default to detect the image dir by the relative path of the path according to the VOC data format.

  • label_root_path – label dir path, if None, default to detect the label dir by the relative path of the path according to the VOC data format.

  • cache_at_init – if set True, will cache in memory in __init__ for faster training

  • cache_on_the_fly – if set True, will cache in memroy during training

  • img_suffix – suffix of image file

  • label_suffix – suffix of label file

  • parse_fn – parse function to parse item of source iterator

  • num_processes – number of processes to parse samples