HICO/HICO-Det 数据集介绍
1. 基本信息
-
这是HOI（Human Object Interaction）领域最常用的数据集之二。
-
PapersWithCode：HICO榜单，HICO-Det榜单
-
基本信息：包括117类行为，80类物体，600类动名词组合（查看附录）。
- HICO-DET：共47776张图片，38118属于训练集，9658属于测试集。
2. HICO 概述
- 本身就是一个多分类任务，列出了600中动名词组合（详见附录）。
- 当然，不全是正负样本，也有“模糊”样本。
2.1. 标注文件介绍
- 下载HICO数据集或HICO-DET数据集时，都包含HICO配置文件，即名为
anno.mat的标注文件。 - 标签内容（摘录自README）
list_action：600x1的ndarray，对于每一类都有name/vname/vname_ing/syn/def/synset/add_def几个属性。list_train：训练图片名称列表list_test：测试图片名称列表anno_train：训练集标签，shape是num_classes x num_train_samples- 标签包括 +1(positive), -1(negative), 0(ambiguous)
anno_test：测试集标签，shape是num_classes x num_test_samples- 标签包括 +1(positive), -1(negative), 0(ambiguous)
- README内容
a. list_action 600 x 1 struct each entry is one HOI category
- nname: noun (object)
- vname: verb
- vname_ing: verb with the -ing ending
- syn: synonyms
- def: definition
- synset: corresponding wordnet synset
- add_def: additional definition
b. list_train 38116 x 1 cell each entry is a file name of an
(training) image
c. list_test 9658 x 1 cell each entry is a file name of an
(test) image
d. anno_train 600 x 38116 entry (i,j) is the annotation of
(training) image j on action i
e. anno_test 600 x 9658 entry (i,j) is the annotation of
(test) image j on action i
Possible annotations:
+1: positive
-1: negative
0: ambiguous
NaN: no annotation
'ambiguous' is assigned due to the disagreement betweem annotators
'no annotation' incidcates the image is not annotated with the HOI. Most
of the images are annotated with the HOIs of one object category only.
3. HICO-DET
- 该任务中，每个样本其实是一个长度为三的元组，分别包括人物信息（bbox）、物体信息（bbox）以及交互行为。
3.2. 标注文件介绍
- 主要内容包括
images：包含train和test两个文件夹，其中包括若干图片。tools：包含若干 matlab 格式的工具anno.mat：anno_bbox.mat：标注文件README：介绍标注文件内容
anno_bbox.mat介绍bbox_train：- 每张图为一个样本，每个样本内容包括
filename/size/hoi，分别代表用户名、图像尺寸、人物交互信息。 - 其中人物交互信息
hoi又包含了若干样本，每个样本包括 行为编号、人物bbox（数组，多个）、物体bbox（数组，多个）、交互任务编号对（即前面两个数组中的编号）
- 每张图为一个样本，每个样本内容包括
bbox_test：与bbox_train格式相同list_actions：600类动名词组合，附录中已经罗列。
- README 介绍
1. anno_bbox.mat contains three variables:
a. bbox_train: bounding boxes annotation for the train2015 set
filename: file names
size: image width, hieght, depth
hoi: HOI annotations
id: action index of list_action
bboxhuman: human bounding boxes
bboxobject: object bounding boxes
connection: instances of HOI (human-object pairs); each row is
one instance, represented by a pair of human bbox
index and object bbox index
invis: 1: HOI invisible; bboxhuman/bboxobject/connection
will be empty
0: HOI visible; bboxhuman/bboxobject/connection
will not be empty
b. bbox_test: bounding boxes annotation for the test2015 set; same
stucture as bbox_train
c. list_action: list of HOIs
附录
80类物体列表
id object
-------------------
001 airplane
002 apple
003 backpack
004 banana
005 baseball_bat
006 baseball_glove
007 bear
008 bed
009 bench
010 bicycle
011 bird
012 boat
013 book
014 bottle
015 bowl
016 broccoli
017 bus
018 cake
019 car
020 carrot
021 cat
022 cell_phone
023 chair
024 clock
025 couch
026 cow
027 cup
028 dining_table
029 dog
030 donut
031 elephant
032 fire_hydrant
033 fork
034 frisbee
035 giraffe
036 hair_drier
037 handbag
038 horse
039 hot_dog
040 keyboard
041 kite
042 knife
043 laptop
044 microwave
045 motorcycle
046 mouse
047 orange
048 oven
049 parking_meter
050 person
051 pizza
052 potted_plant
053 refrigerator
054 remote
055 sandwich
056 scissors
057 sheep
058 sink
059 skateboard
060 skis
061 snowboard
062 spoon
063 sports_ball
064 stop_sign
065 suitcase
066 surfboard
067 teddy_bear
068 tennis_racket
069 tie
070 toaster
071 toilet
072 toothbrush
073 traffic_light
074 train
075 truck
076 tv
077 umbrella
078 vase
079 wine_glass
080 zebra
117类行为列表
id verb
-------------------
001 adjust
002 assemble
003 block
004 blow
005 board
006 break
007 brush_with
008 buy
009 carry
010 catch
011 chase
012 check
013 clean
014 control
015 cook
016 cut
017 cut_with
018 direct
019 drag
020 dribble
021 drink_with
022 drive
023 dry
024 eat
025 eat_at
026 exit
027 feed
028 fill
029 flip
030 flush
031 fly
032 greet
033 grind
034 groom
035 herd
036 hit
037 hold
038 hop_on
039 hose
040 hug
041 hunt
042 inspect
043 install
044 jump
045 kick
046 kiss
047 lasso
048 launch
049 lick
050 lie_on
051 lift
052 light
053 load
054 lose
055 make
056 milk
057 move
058 no_interaction
059 open
060 operate
061 pack
062 paint
063 park
064 pay
065 peel
066 pet
067 pick
068 pick_up
069 point
070 pour
071 pull
072 push
073 race
074 read
075 release
076 repair
077 ride
078 row
079 run
080 sail
081 scratch
082 serve
083 set
084 shear
085 sign
086 sip
087 sit_at
088 sit_on
089 slide
090 smell
091 spin
092 squeeze
093 stab
094 stand_on
095 stand_under
096 stick
097 stir
098 stop_at
099 straddle
100 swing
101 tag
102 talk_on
103 teach
104 text_on
105 throw
106 tie
107 toast
108 train
109 turn
110 type_on
111 walk
112 wash
113 watch
114 wave
115 wear
116 wield
117 zip
600类 HOI 行为（动名词组合）列表
id object verb
-----------------------------------
001 airplane board
002 airplane direct
003 airplane exit
004 airplane fly
005 airplane inspect
006 airplane load
007 airplane ride
008 airplane sit_on
009 airplane wash
010 airplane no_interaction
011 bicycle carry
012 bicycle hold
013 bicycle inspect
014 bicycle jump
015 bicycle hop_on
016 bicycle park
017 bicycle push
018 bicycle repair
019 bicycle ride
020 bicycle sit_on
021 bicycle straddle
022 bicycle walk
023 bicycle wash
024 bicycle no_interaction
025 bird chase
026 bird feed
027 bird hold
028 bird pet
029 bird release
030 bird watch
031 bird no_interaction
032 boat board
033 boat drive
034 boat exit
035 boat inspect
036 boat jump
037 boat launch
038 boat repair
039 boat ride
040 boat row
041 boat sail
042 boat sit_on
043 boat stand_on
044 boat tie
045 boat wash
046 boat no_interaction
047 bottle carry
048 bottle drink_with
049 bottle hold
050 bottle inspect
051 bottle lick
052 bottle open
053 bottle pour
054 bottle no_interaction
055 bus board
056 bus direct
057 bus drive
058 bus exit
059 bus inspect
060 bus load
061 bus ride
062 bus sit_on
063 bus wash
064 bus wave
065 bus no_interaction
066 car board
067 car direct
068 car drive
069 car hose
070 car inspect
071 car jump
072 car load
073 car park
074 car ride
075 car wash
076 car no_interaction
077 cat dry
078 cat feed
079 cat hold
080 cat hug
081 cat kiss
082 cat pet
083 cat scratch
084 cat wash
085 cat chase
086 cat no_interaction
087 chair carry
088 chair hold
089 chair lie_on
090 chair sit_on
091 chair stand_on
092 chair no_interaction
093 couch carry
094 couch lie_on
095 couch sit_on
096 couch no_interaction
097 cow feed
098 cow herd
099 cow hold
100 cow hug
101 cow kiss
102 cow lasso
103 cow milk
104 cow pet
105 cow ride
106 cow walk
107 cow no_interaction
108 dining_table clean
109 dining_table eat_at
110 dining_table sit_at
111 dining_table no_interaction
112 dog carry
113 dog dry
114 dog feed
115 dog groom
116 dog hold
117 dog hose
118 dog hug
119 dog inspect
120 dog kiss
121 dog pet
122 dog run
123 dog scratch
124 dog straddle
125 dog train
126 dog walk
127 dog wash
128 dog chase
129 dog no_interaction
130 horse feed
131 horse groom
132 horse hold
133 horse hug
134 horse jump
135 horse kiss
136 horse load
137 horse hop_on
138 horse pet
139 horse race
140 horse ride
141 horse run
142 horse straddle
143 horse train
144 horse walk
145 horse wash
146 horse no_interaction
147 motorcycle hold
148 motorcycle inspect
149 motorcycle jump
150 motorcycle hop_on
151 motorcycle park
152 motorcycle push
153 motorcycle race
154 motorcycle ride
155 motorcycle sit_on
156 motorcycle straddle
157 motorcycle turn
158 motorcycle walk
159 motorcycle wash
160 motorcycle no_interaction
161 person carry
162 person greet
163 person hold
164 person hug
165 person kiss
166 person stab
167 person tag
168 person teach
169 person lick
170 person no_interaction
171 potted_plant carry
172 potted_plant hold
173 potted_plant hose
174 potted_plant no_interaction
175 sheep carry
176 sheep feed
177 sheep herd
178 sheep hold
179 sheep hug
180 sheep kiss
181 sheep pet
182 sheep ride
183 sheep shear
184 sheep walk
185 sheep wash
186 sheep no_interaction
187 train board
188 train drive
189 train exit
190 train load
191 train ride
192 train sit_on
193 train wash
194 train no_interaction
195 tv control
196 tv repair
197 tv watch
198 tv no_interaction
199 apple buy
200 apple cut
201 apple eat
202 apple hold
203 apple inspect
204 apple peel
205 apple pick
206 apple smell
207 apple wash
208 apple no_interaction
209 backpack carry
210 backpack hold
211 backpack inspect
212 backpack open
213 backpack wear
214 backpack no_interaction
215 banana buy
216 banana carry
217 banana cut
218 banana eat
219 banana hold
220 banana inspect
221 banana peel
222 banana pick
223 banana smell
224 banana no_interaction
225 baseball_bat break
226 baseball_bat carry
227 baseball_bat hold
228 baseball_bat sign
229 baseball_bat swing
230 baseball_bat throw
231 baseball_bat wield
232 baseball_bat no_interaction
233 baseball_glove hold
234 baseball_glove wear
235 baseball_glove no_interaction
236 bear feed
237 bear hunt
238 bear watch
239 bear no_interaction
240 bed clean
241 bed lie_on
242 bed sit_on
243 bed no_interaction
244 bench inspect
245 bench lie_on
246 bench sit_on
247 bench no_interaction
248 book carry
249 book hold
250 book open
251 book read
252 book no_interaction
253 bowl hold
254 bowl stir
255 bowl wash
256 bowl lick
257 bowl no_interaction
258 broccoli cut
259 broccoli eat
260 broccoli hold
261 broccoli smell
262 broccoli stir
263 broccoli wash
264 broccoli no_interaction
265 cake blow
266 cake carry
267 cake cut
268 cake eat
269 cake hold
270 cake light
271 cake make
272 cake pick_up
273 cake no_interaction
274 carrot carry
275 carrot cook
276 carrot cut
277 carrot eat
278 carrot hold
279 carrot peel
280 carrot smell
281 carrot stir
282 carrot wash
283 carrot no_interaction
284 cell_phone carry
285 cell_phone hold
286 cell_phone read
287 cell_phone repair
288 cell_phone talk_on
289 cell_phone text_on
290 cell_phone no_interaction
291 clock check
292 clock hold
293 clock repair
294 clock set
295 clock no_interaction
296 cup carry
297 cup drink_with
298 cup hold
299 cup inspect
300 cup pour
301 cup sip
302 cup smell
303 cup fill
304 cup wash
305 cup no_interaction
306 donut buy
307 donut carry
308 donut eat
309 donut hold
310 donut make
311 donut pick_up
312 donut smell
313 donut no_interaction
314 elephant feed
315 elephant hold
316 elephant hose
317 elephant hug
318 elephant kiss
319 elephant hop_on
320 elephant pet
321 elephant ride
322 elephant walk
323 elephant wash
324 elephant watch
325 elephant no_interaction
326 fire_hydrant hug
327 fire_hydrant inspect
328 fire_hydrant open
329 fire_hydrant paint
330 fire_hydrant no_interaction
331 fork hold
332 fork lift
333 fork stick
334 fork lick
335 fork wash
336 fork no_interaction
337 frisbee block
338 frisbee catch
339 frisbee hold
340 frisbee spin
341 frisbee throw
342 frisbee no_interaction
343 giraffe feed
344 giraffe kiss
345 giraffe pet
346 giraffe ride
347 giraffe watch
348 giraffe no_interaction
349 hair_drier hold
350 hair_drier operate
351 hair_drier repair
352 hair_drier no_interaction
353 handbag carry
354 handbag hold
355 handbag inspect
356 handbag no_interaction
357 hot_dog carry
358 hot_dog cook
359 hot_dog cut
360 hot_dog eat
361 hot_dog hold
362 hot_dog make
363 hot_dog no_interaction
364 keyboard carry
365 keyboard clean
366 keyboard hold
367 keyboard type_on
368 keyboard no_interaction
369 kite assemble
370 kite carry
371 kite fly
372 kite hold
373 kite inspect
374 kite launch
375 kite pull
376 kite no_interaction
377 knife cut_with
378 knife hold
379 knife stick
380 knife wash
381 knife wield
382 knife lick
383 knife no_interaction
384 laptop hold
385 laptop open
386 laptop read
387 laptop repair
388 laptop type_on
389 laptop no_interaction
390 microwave clean
391 microwave open
392 microwave operate
393 microwave no_interaction
394 mouse control
395 mouse hold
396 mouse repair
397 mouse no_interaction
398 orange buy
399 orange cut
400 orange eat
401 orange hold
402 orange inspect
403 orange peel
404 orange pick
405 orange squeeze
406 orange wash
407 orange no_interaction
408 oven clean
409 oven hold
410 oven inspect
411 oven open
412 oven repair
413 oven operate
414 oven no_interaction
415 parking_meter check
416 parking_meter pay
417 parking_meter repair
418 parking_meter no_interaction
419 pizza buy
420 pizza carry
421 pizza cook
422 pizza cut
423 pizza eat
424 pizza hold
425 pizza make
426 pizza pick_up
427 pizza slide
428 pizza smell
429 pizza no_interaction
430 refrigerator clean
431 refrigerator hold
432 refrigerator move
433 refrigerator open
434 refrigerator no_interaction
435 remote hold
436 remote point
437 remote swing
438 remote no_interaction
439 sandwich carry
440 sandwich cook
441 sandwich cut
442 sandwich eat
443 sandwich hold
444 sandwich make
445 sandwich no_interaction
446 scissors cut_with
447 scissors hold
448 scissors open
449 scissors no_interaction
450 sink clean
451 sink repair
452 sink wash
453 sink no_interaction
454 skateboard carry
455 skateboard flip
456 skateboard grind
457 skateboard hold
458 skateboard jump
459 skateboard pick_up
460 skateboard ride
461 skateboard sit_on
462 skateboard stand_on
463 skateboard no_interaction
464 skis adjust
465 skis carry
466 skis hold
467 skis inspect
468 skis jump
469 skis pick_up
470 skis repair
471 skis ride
472 skis stand_on
473 skis wear
474 skis no_interaction
475 snowboard adjust
476 snowboard carry
477 snowboard grind
478 snowboard hold
479 snowboard jump
480 snowboard ride
481 snowboard stand_on
482 snowboard wear
483 snowboard no_interaction
484 spoon hold
485 spoon lick
486 spoon wash
487 spoon sip
488 spoon no_interaction
489 sports_ball block
490 sports_ball carry
491 sports_ball catch
492 sports_ball dribble
493 sports_ball hit
494 sports_ball hold
495 sports_ball inspect
496 sports_ball kick
497 sports_ball pick_up
498 sports_ball serve
499 sports_ball sign
500 sports_ball spin
501 sports_ball throw
502 sports_ball no_interaction
503 stop_sign hold
504 stop_sign stand_under
505 stop_sign stop_at
506 stop_sign no_interaction
507 suitcase carry
508 suitcase drag
509 suitcase hold
510 suitcase hug
511 suitcase load
512 suitcase open
513 suitcase pack
514 suitcase pick_up
515 suitcase zip
516 suitcase no_interaction
517 surfboard carry
518 surfboard drag
519 surfboard hold
520 surfboard inspect
521 surfboard jump
522 surfboard lie_on
523 surfboard load
524 surfboard ride
525 surfboard stand_on
526 surfboard sit_on
527 surfboard wash
528 surfboard no_interaction
529 teddy_bear carry
530 teddy_bear hold
531 teddy_bear hug
532 teddy_bear kiss
533 teddy_bear no_interaction
534 tennis_racket carry
535 tennis_racket hold
536 tennis_racket inspect
537 tennis_racket swing
538 tennis_racket no_interaction
539 tie adjust
540 tie cut
541 tie hold
542 tie inspect
543 tie pull
544 tie tie
545 tie wear
546 tie no_interaction
547 toaster hold
548 toaster operate
549 toaster repair
550 toaster no_interaction
551 toilet clean
552 toilet flush
553 toilet open
554 toilet repair
555 toilet sit_on
556 toilet stand_on
557 toilet wash
558 toilet no_interaction
559 toothbrush brush_with
560 toothbrush hold
561 toothbrush wash
562 toothbrush no_interaction
563 traffic_light install
564 traffic_light repair
565 traffic_light stand_under
566 traffic_light stop_at
567 traffic_light no_interaction
568 truck direct
569 truck drive
570 truck inspect
571 truck load
572 truck repair
573 truck ride
574 truck sit_on
575 truck wash
576 truck no_interaction
577 umbrella carry
578 umbrella hold
579 umbrella lose
580 umbrella open
581 umbrella repair
582 umbrella set
583 umbrella stand_under
584 umbrella no_interaction
585 vase hold
586 vase make
587 vase paint
588 vase no_interaction
589 wine_glass fill
590 wine_glass hold
591 wine_glass sip
592 wine_glass toast
593 wine_glass lick
594 wine_glass wash
595 wine_glass no_interaction
596 zebra feed
597 zebra hold
598 zebra pet
599 zebra watch
600 zebra no_interaction