【数据集介绍】
数据集格式:Pascal VOC格式+YOLO格式(不包含分割路径的txt文件,仅仅包含jpg图片以及对应的VOC格式xml文件和yolo格式txt文件)
图片数量(jpg文件个数):16239
标注数量(xml文件个数):16239
标注数量(txt文件个数):16239
标注类别数:80
标注类别名称(注意yolo格式类别顺序不和这个对应,而以labels文件夹classes.txt为准):["Achnanthidium","Adlafia","Amphora","Anabaena","Aphanizomenon","Aulacoseira","Blue Green Algae","Brachysira","Caloneis","Cavinula","Chamaepinnularia","Cocconeis","Craticula","Cyclotella","Cymbopleura","Diadesmis","Diatom","Diatoma","Dinoflagellata","Encyonema","Encyonopsis","Eolimna","Euglenozoa","Eunotia","Fragilaria","Frustulia","Geissleria","Gomphonema","Green Algae","Karenia","Luticola","Meridion","Microcystis","Mycrocystis","Navicula","Neidiopsis","Neidium","Nitzschia","Noduloria","Non Toxic","Oscillatoria","Pinnularia","Planothidium","Psammothidium","Pseudostaurosira","Rossithidium","Sellaphora","Skeletonema","Stauroforma","Stauroneis","Staurosira","Staurosirella","Stephanodiscus","Surirella","Tabellaria","asteromphalus_hyalinus","calanoida","ceratium_furca","ceratium_trichorecos","chaetoceros_curvisetus","copepod","coscinodiscus_oculus_iridis","dinophysis_caudata","dinophysis_miles","eggs","eucampia_zodiacus","noctilluca_scintillans","odontella_mobiliensis","oithona","ornithocercus_thumii","parasite","planktoniella_sol","podolampas_bipes","proboscia_alata","prorocentrum_micans","protoperidinium_oceanicum","pseudo_nitzschia","skeletonema_costatum","thalassionema_nitzschioides","trichodesmium_erythraeum"]
每个类别标注的框数:
Achnanthidium 框数 = 443
Adlafia 框数 = 63
Amphora 框数 = 189
Anabaena 框数 = 64
Aphanizomenon 框数 = 480
Aulacoseira 框数 = 314
Blue Green Algae 框数 = 13743
Brachysira 框数 = 229
Caloneis 框数 = 125
Cavinula 框数 = 191
Chamaepinnularia 框数 = 79
Cocconeis 框数 = 237
Craticula 框数 = 251
Cyclotella 框数 = 139
Cymbopleura 框数 = 428
Diadesmis 框数 = 47
Diatom 框数 = 3085
Diatoma 框数 = 101
Dinoflagellata 框数 = 1783
Encyonema 框数 = 444
Encyonopsis 框数 = 388
Eolimna 框数 = 63
Euglenozoa 框数 = 795
Eunotia 框数 = 121
Fragilaria 框数 = 115
Frustulia 框数 = 111
Geissleria 框数 = 112
Gomphonema 框数 = 121
Green Algae 框数 = 16424
Karenia 框数 = 811
Luticola 框数 = 112
Meridion 框数 = 88
Microcystis 框数 = 552
Mycrocystis 框数 = 8
Navicula 框数 = 113
Neidiopsis 框数 = 63
Neidium 框数 = 158
Nitzschia 框数 = 772
Noduloria 框数 = 703
Non Toxic 框数 = 925
Oscillatoria 框数 = 678
Pinnularia 框数 = 607
Planothidium 框数 = 371
Psammothidium 框数 = 361
Pseudostaurosira 框数 = 182
Rossithidium 框数 = 101
Sellaphora 框数 = 450
Skeletonema 框数 = 514
Stauroforma 框数 = 42
Stauroneis 框数 = 593
Staurosira 框数 = 100
Staurosirella 框数 = 136
Stephanodiscus 框数 = 169
Surirella 框数 = 364
Tabellaria 框数 = 73
asteromphalus_hyalinus 框数 = 75
calanoida 框数 = 15
ceratium_furca 框数 = 31
ceratium_trichorecos 框数 = 27
chaetoceros_curvisetus 框数 = 12
copepod 框数 = 1975
coscinodiscus_oculus_iridis 框数 = 73
dinophysis_caudata 框数 = 68
dinophysis_miles 框数 = 45
eggs 框数 = 133
eucampia_zodiacus 框数 = 31
noctilluca_scintillans 框数 = 14
odontella_mobiliensis 框数 = 42
oithona 框数 = 3
ornithocercus_thumii 框数 = 77
parasite 框数 = 9
planktoniella_sol 框数 = 35
podolampas_bipes 框数 = 27
proboscia_alata 框数 = 39
prorocentrum_micans 框数 = 39
protoperidinium_oceanicum 框数 = 37
pseudo_nitzschia 框数 = 58
skeletonema_costatum 框数 = 26
thalassionema_nitzschioides 框数 = 18
trichodesmium_erythraeum 框数 = 29
总框数:52669
使用标注工具:labelImg
标注规则:对类别进行画矩形框
重要说明:暂无
特别声明:本数据集不对训练的模型或者权重文件精度作任何保证,数据集只提供准确且合理标注
图片预览:
标注例子:
【训练步骤】
首先我们获取数据集压缩包7z格式或者zip格式后解压到一个非中文或者有空格路径下面。比如解压到C:\Users\Administrator\Downloads目录,下面都是以这个目录演示训练流程。
然后我们在数据集data文件夹下面新建一个脚本split.py写入下面代码:
import os
import shutil
import random
class VOCDataSplit(object):
def __init__(self):
pass
def split(self, image_dir, label_dir, save_dir, trainval_radio=0.9, train_radio=0.9, copy_it=True,
need_test_dataset=False, shuffle_file=True):
train_images_dir = os.path.join(save_dir, 'train', 'images')
train_labels_dir = os.path.join(save_dir, 'train', 'labels')
val_images_dir = os.path.join(save_dir, 'val', 'images')
val_labels_dir = os.path.join(save_dir, 'val', 'labels')
test_images_dir = os.path.join(save_dir, 'test', 'images')
test_labels_dir = os.path.join(save_dir, 'test', 'labels')
os.makedirs(train_images_dir, exist_ok=True)
os.makedirs(train_labels_dir, exist_ok=True)
os.makedirs(val_images_dir, exist_ok=True)
os.makedirs(val_labels_dir, exist_ok=True)
if need_test_dataset:
os.makedirs(test_images_dir, exist_ok=True)
os.makedirs(test_labels_dir, exist_ok=True)
files = []
for file in os.listdir(image_dir):
if file.endswith('.jpg'):
files.append(file)
total_count = len(files)
print('find {} images'.format(total_count))
if shuffle_file:
random.shuffle(files)
if need_test_dataset:
test_count = int(total_count * (1 - trainval_radio))
train_count = int((total_count - test_count) * train_radio)
val_count = total_count - test_count - train_count
else:
train_count = int(total_count * train_radio)
val_count = total_count - train_count
test_count = 0
print('train={},val={},test={}'.format(train_count, val_count, test_count))
train_files = files[:train_count]
val_files = files[train_count:train_count + val_count]
test_files = files[train_count + val_count:]
print('start copy or move train files...')
for file in train_files:
if copy_it:
shutil.copy(os.path.join(image_dir, file), os.path.join(train_images_dir, file))
label_file = os.path.join(label_dir, file[:-3] + 'txt')
if os.path.exists(label_file):
shutil.copy(label_file, os.path.join(train_labels_dir, file[:-3] + 'txt'))
else:
shutil.move(os.path.join(image_dir, file), os.path.join(train_images_dir, file))
label_file = os.path.join(label_dir, file[:-3] + 'txt')
if os.path.exists(label_file):
shutil.move(label_file, os.path.join(train_labels_dir, file[:-3] + 'txt'))
print('start copy or move val files...')
for file in val_files:
if copy_it:
shutil.copy(os.path.join(image_dir, file), os.path.join(val_images_dir, file))
label_file = os.path.join(label_dir, file[:-3] + 'txt')
if os.path.exists(label_file):
shutil.copy(label_file, os.path.join(val_labels_dir, file[:-3] + 'txt'))
else:
shutil.move(os.path.join(image_dir, file), os.path.join(val_images_dir, file))
label_file = os.path.join(label_dir, file[:-3] + 'txt')
if os.path.exists(label_file):
shutil.move(label_file, os.path.join(val_labels_dir, file[:-3] + 'txt'))
if need_test_dataset and test_count > 0:
print('start copy or move test files...')
for file in test_files:
if copy_it:
shutil.copy(os.path.join(image_dir, file), os.path.join(test_images_dir, file))
label_file = os.path.join(label_dir, file[:-3] + 'txt')
if os.path.exists(label_file):
shutil.copy(label_file, os.path.join(test_labels_dir, file[:-3] + 'txt'))
else:
shutil.move(os.path.join(image_dir, file), os.path.join(test_images_dir, file))
label_file = os.path.join(label_dir, file[:-3] + 'txt')
if os.path.exists(label_file):
shutil.move(label_file, os.path.join(test_labels_dir, file[:-3] + 'txt'))
class_file=os.path.join(label_dir, 'classes.txt')
if os.path.exists(class_file):
class_names=[]
with open(class_file, 'r') as f:
class_names=f.read().rstrip('\n').split('\n')
nc = len(class_names)
content='train: '+train_images_dir+'\n'
content += 'val: '+val_images_dir+'\n'
if need_test_dataset:
content += 'test: '+test_images_dir+'\n'
content += 'nc: '+str(nc)+'\n'
content += 'names:\n'
content += "\n".join([f" {i}: {name}" for i, name in enumerate(class_names)])
with open(os.path.join(save_dir,'coco128.yaml'), 'w') as f:
f.write(content)
print('all done!')
if __name__ == '__main__':
images_dir=r'C:\Users\Administrator\Downloads\data\JPEGImages'
labels_dir=r'C:\Users\Administrator\Downloads\data\labels'
save_dir=r'C:\Users\Administrator\Downloads\dataset'
vs = VOCDataSplit()
vs.split(images_dir,labels_dir,save_dir,trainval_radio=0.9, train_radio=0.9, copy_it=True,shuffle_file=True,need_test_dataset=False)
注意 images_dir,labels_dir,save_dir根据自己实际情况进行调整。这个split.py脚本功能就是将JPEGImages和labels文件进行自动随机分割成标准yolo训练目录格式。脚本放在data文件夹下面即可:
然后运行脚本:
然后在C:\Users\Administrator\Downloads看到有个dataset文件夹生成
至此标准训练格式完成了。打开coco128.yaml看到下面类似格式:
train: C:\Users\Administrator\Downloads\dataset\train\images
val: C:\Users\Administrator\Downloads\dataset\val\images
nc: 3
names:
0: class_name1
1: class_name2
2: class_name3
这些都不用修改,我们只需要检查一下是不是对的就行。
之后就是开始训练了,注意训练yolov8模型需要自己提前安装好环境。
使用预训练模型开始训练
yolo task=detect mode=train model=yolov8n.pt data=coco128.yaml epochs=100 imgsz=640 batch=8 workers=2
参数说明:
model: 使用的模型类型,如 yolov8s.pt(小模型)、yolov8m.pt(中)、yolov8l.pt(大) data: 指定数据配置文件 epochs: 训练轮数 imgsz: 输入图像尺寸 batch: 批量大小(根据显存调整)
workers:指定进程数(windows最好设置0或者1或2,linux可以设置8) 训练完成后,最佳权重保存路径为:runs/detect/train/weights/best.pt,如果多次运行命令runs/detect/train2,runs/detect/train3文件夹生成只需要到数字最大文件夹查看就可以找到模型
图片预测:
from ultralytics import YOLO
# 加载训练好的模型
model = YOLO('runs/detect/train/weights/best.pt')
# 图像预测
results = model('path_to_your_image.jpg')
视频或摄像头预测
results = model('path_to_video.mp4') # 视频
#results = model(0) # 摄像头
验证集评估
yolo task=detect mode=val model=runs/detect/train/weights/best.pt data=data.yaml
输出指标图像,一般在模型训练后生成,文件位置在runs/detect/train/results.png:
上面训练结果图片常用评估参数介绍
【常用评估参数介绍】
在目标检测任务中,评估模型的性能是至关重要的。你提到的几个术语是评估模型性能的常用指标。下面是对这些术语的详细解释:
这些指标共同构成了评估目标检测模型性能的重要框架。通过比较不同模型在这些指标上的表现,可以判断哪个模型在实际应用中可能更有效。
将模型导出为ONNX、TensorRT等格式以用于部署:
yolo export model=runs/detect/train/weights/best.pt format=onnx 支持格式包括:onnx, engine, tflite, pb, torchscript 等。
经过上面训练可以使用模型做一步部署,比如使用onnx模型在嵌入式部署,使用engine模型在jetson上deepstream部署,使用torchscript模型可以在C++上部署等等。