Code×Life
0%

跑通Swin Transformer V1语义分割任务

发表于 更新于 分类于
跑通Github上，openmmlab实现的Swin-Transformer-Semantic-Segmentation代码

github仓库地址

Swin-Transformer-V2 blog

Swin-Transformer pretrain models

简介

  1. 代码是openmmlab用mmcv库和mmseg toolbox实现了训练，验证，测试过程，支持多种语义分割模型，本次实验采用Swin-Transformer为backbone的模型
  2. 实验难点：mmcv和mmseg的安装，环境配置
  3. 硬件环境：腾讯云租用V100 GPU服务器(特价)，32G显存，10核

配置环境

2023.8.8：先按照现有环境(cuda=11.8.0, torch=2.0.1)直接装mmcv和mmseg试试看

目前环境：

mmseg=1.1.1
mmcv=2.0.1
mmengine=0.8.4(mmengine为mmcv大改之后新增的库，支持很多原来放在mmcv里的接口，比方说train的代码，data_parallel的内容)

官网提供的版本依赖关系：image-20230808150904656

根据官网文档，mmcv和mmseg匹配，但是module not found，因为train的部分已经从mmcv移除

image-20230808151010143

2023.8.9：

找到的适配环境：

mmseg=0.11.0

mmcv=1.3.0

torch=1.7.0

cuda=11.0

如何找到的环境：

  1. requirements.txt没给出mmcv和mmseg的版本，但源码有mmseg文件夹，进入mmseg/version.py可以得知mmseg需要0.11.0版本，并且需要mmcv满足1.0.4-1.3.0之间

    ->mmcv=1.3.0，mmseg=0.11.0。

  2. 因为mmcv包括了一些cuda运算的内容，mmcv必须与torch、cuda版本匹配，同时也有不支持的python版本

    1. 发现一个html网页(格式为:

      downloadopenmmlab.com/mmcv/dist/cu/torch<x.x.x>/index.html，其中xxx和x.x.x代表版本号)可以查找，特定torch和cuda版本下的mmcv的.whl包。

    2. 尝试不同版本号，最终找到合适的版本为：cuda=11.0，torch=1.7.0，python=3.8。下载.whl包

      image-20230810221836793

    3. 重装系统–cuda=11.0，再安装mmcv和mmseg。

      image-20230810221728646

  3. 安装mmseg: 但不管是pip和open-mmlab提供的mim包，都找不到mmseg=0.11.0(太旧)，所以直接把github库给的源码，复制到对应虚拟环境python3.8的packages

  4. 安装mmcv: mmcv有mmcv-full, mmcv, mmcv-lite三种，直接pip或mim安装的mmcv包缺少mmseg需要的接口，需要full版本，使用下载的mmcv-full 1.3.0的.whl包，pip安装即可。（好像直接安装mmcv-full也行，不用找.whl包）

  5. PS: 因为我是单GPU训练，它的config文件(configs/_base/models/upernet_swin.py)中有一个地方接口没对好，使用了SyncBN，这个需要手动改为BN，改正就可以跑动了。

    image-20230810222549116

训练过程

2023.8.9:训练Swin-B模型

对标的结果：

image-20230810224052923

开始训练，只加载了预训练模型(github代码页提供)，改动了batch size

(configs/swin/upernet_swin_base_patch4_window7_512x512_160k_ade20k.py)中的data段改为samples_per_gpu=8(因为github是8GPUs分布式训练，我只有单GPU，并且开不到16，=8时已经占用24G显存)

image-20230810222731182

本实验的batch size影响很大，当我samples_per_gpu设为4时，对比作者的log文件，同样15950iterations下

image-20230810223522211

2023.8.10:

跑的好慢！！！应该去跑Swin-S的，参数量少，应该速度会快很多

北京时间22:45，跑到了96k iterations的checkpoint(每16k iterations一次验证), validation结果和github在96k iterations的checkpoint处结果对标：

image-20230810225538462

2023.8.11:

三天跑完160k iterations, 最终指标与github对比：(val mAcc与96k iterations对比，不升反降，openmmlab提供的log文件也有这个现象)

image-20230812004324648

附录：

validation过程中，各类别的mIoU和Acc如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
+---------------------+-------+-------+
| Class               | IoU   | Acc   |
+---------------------+-------+-------+
| wall                | 75.88 | 87.82 |
| building            | 81.5  | 91.98 |
| sky                 | 93.95 | 97.18 |
| floor               | 80.19 | 90.6  |
| tree                | 74.04 | 87.05 |
| ceiling             | 82.28 | 90.21 |
| road                | 81.98 | 89.34 |
| bed                 | 86.94 | 95.45 |
| windowpane          | 60.9  | 77.16 |
| grass               | 67.34 | 83.15 |
| cabinet             | 59.51 | 72.18 |
| sidewalk            | 64.31 | 80.8  |
| person              | 78.34 | 90.42 |
| earth               | 37.37 | 52.52 |
| door                | 45.28 | 59.92 |
| table               | 59.22 | 76.1  |
| mountain            | 57.24 | 71.79 |
| plant               | 52.32 | 63.37 |
| curtain             | 69.82 | 83.09 |
| chair               | 54.96 | 69.21 |
| car                 | 83.77 | 91.67 |
| water               | 47.62 | 61.96 |
| painting            | 72.41 | 86.03 |
| sofa                | 60.08 | 76.23 |
| shelf               | 43.32 | 63.98 |
| house               | 49.56 | 62.8  |
| sea                 | 51.04 | 70.59 |
| mirror              | 62.54 | 69.18 |
| rug                 | 60.32 | 67.46 |
| field               | 31.73 | 45.61 |
| armchair            | 37.61 | 57.33 |
| seat                | 59.81 | 79.95 |
| fence               | 42.07 | 56.82 |
| desk                | 49.49 | 66.97 |
| rock                | 47.33 | 73.48 |
| wardrobe            | 49.47 | 68.13 |
| lamp                | 60.04 | 71.42 |
| bathtub             | 75.56 | 81.97 |
| railing             | 32.42 | 47.61 |
| cushion             | 54.0  | 68.56 |
| base                | 30.54 | 42.64 |
| box                 | 24.21 | 29.27 |
| column              | 43.06 | 53.74 |
| signboard           | 36.02 | 48.73 |
| chest of drawers    | 39.78 | 55.26 |
| counter             | 24.38 | 34.5  |
| sand                | 49.0  | 61.36 |
| sink                | 72.87 | 79.28 |
| skyscraper          | 55.51 | 75.59 |
| fireplace           | 77.31 | 86.69 |
| refrigerator        | 73.98 | 85.11 |
| grandstand          | 36.52 | 66.35 |
| path                | 25.78 | 36.24 |
| stairs              | 30.06 | 36.85 |
| runway              | 64.37 | 86.92 |
| case                | 36.91 | 47.58 |
| pool table          | 91.72 | 95.42 |
| pillow              | 54.05 | 61.68 |
| screen door         | 57.43 | 76.01 |
| stairway            | 28.86 | 37.94 |
| river               | 7.04  | 18.76 |
| bridge              | 33.95 | 43.37 |
| bookcase            | 43.84 | 63.89 |
| blind               | 42.74 | 47.66 |
| coffee table        | 56.26 | 80.03 |
| toilet              | 84.39 | 91.65 |
| flower              | 41.4  | 57.24 |
| book                | 46.29 | 63.28 |
| hill                | 6.21  | 10.59 |
| bench               | 50.84 | 58.05 |
| countertop          | 47.47 | 65.09 |
| stove               | 77.27 | 82.83 |
| palm                | 49.09 | 80.35 |
| kitchen island      | 35.46 | 66.41 |
| computer            | 60.62 | 69.92 |
| swivel chair        | 43.89 | 58.2  |
| boat                | 44.83 | 51.58 |
| bar                 | 33.78 | 46.42 |
| arcade machine      | 42.46 | 45.03 |
| hovel               | 27.84 | 33.58 |
| bus                 | 85.5  | 95.7  |
| towel               | 64.81 | 77.75 |
| light               | 51.2  | 57.51 |
| truck               | 38.52 | 49.88 |
| tower               | 12.59 | 16.34 |
| chandelier          | 67.62 | 83.59 |
| awning              | 24.46 | 29.84 |
| streetlight         | 22.69 | 27.9  |
| booth               | 39.14 | 43.55 |
| television receiver | 63.52 | 76.73 |
| airplane            | 54.17 | 68.22 |
| dirt track          | 9.49  | 20.3  |
| apparel             | 41.61 | 58.49 |
| pole                | 22.26 | 31.02 |
| land                | 3.74  | 5.16  |
| bannister           | 9.37  | 11.57 |
| escalator           | 46.01 | 61.15 |
| ottoman             | 46.64 | 61.2  |
| bottle              | 39.34 | 59.83 |
| buffet              | 29.0  | 32.67 |
| poster              | 26.59 | 36.56 |
| stage               | 19.5  | 30.8  |
| van                 | 42.08 | 56.7  |
| ship                | 31.27 | 47.3  |
| fountain            | 21.48 | 22.08 |
| conveyer belt       | 56.6  | 69.16 |
| canopy              | 6.03  | 9.86  |
| washer              | 74.13 | 76.94 |
| plaything           | 22.11 | 33.75 |
| swimming pool       | 57.93 | 75.19 |
| stool               | 36.61 | 54.75 |
| barrel              | 38.85 | 65.2  |
| basket              | 29.85 | 39.16 |
| waterfall           | 61.86 | 71.23 |
| tent                | 91.31 | 98.67 |
| bag                 | 14.61 | 18.94 |
| minibike            | 70.54 | 82.9  |
| cradle              | 69.39 | 84.86 |
| oven                | 59.75 | 75.28 |
| ball                | 39.28 | 49.13 |
| food                | 44.82 | 55.14 |
| step                | 15.71 | 19.1  |
| tank                | 56.6  | 59.83 |
| trade name          | 26.23 | 30.57 |
| microwave           | 84.77 | 92.18 |
| pot                 | 43.37 | 50.01 |
| animal              | 50.4  | 52.17 |
| bicycle             | 55.69 | 76.24 |
| lake                | 42.86 | 64.07 |
| dishwasher          | 62.77 | 69.52 |
| screen              | 56.03 | 74.51 |
| blanket             | 9.13  | 10.76 |
| sculpture           | 53.36 | 74.64 |
| hood                | 58.87 | 69.21 |
| sconce              | 43.44 | 52.69 |
| vase                | 32.91 | 42.82 |
| traffic light       | 28.49 | 40.94 |
| tray                | 9.37  | 12.05 |
| ashcan              | 39.81 | 52.34 |
| fan                 | 60.07 | 72.86 |
| pier                | 54.33 | 84.51 |
| crt screen          | 5.25  | 16.89 |
| plate               | 45.0  | 57.68 |
| monitor             | 14.04 | 19.48 |
| bulletin board      | 47.86 | 55.79 |
| shower              | 0.45  | 0.7   |
| radiator            | 51.47 | 61.22 |
| glass               | 13.14 | 14.29 |
| clock               | 31.41 | 39.73 |
| flag                | 46.52 | 50.16 |
+---------------------+-------+-------+