Labeling Human Actions in Drone Videos

In this blog we will show you how to label actions in drone videos, as shown in the video below:

https://www.youtube.com/watch?v=y05iJ0vMnuI

We use Microsoft VoTT as our labeling tool. The summary of labeling steps are listed as follows:

'

1. Install Microsoft VoTT v2.1.0 (link) or NTUT VoTT v0.0.5 (Windows)
2. Add security token to VoTT 
   

   Name:	NTUTDrone Token
   Key:	qITjd1Rj6hFqC7uB0OGxPKYVxIFSdcMrprQAMS5rEhc=

3. Download Drone videos (DroneVideos) and labeling projects (VoTT Projects)
4. Create two folders: C:\Drone_Target\ and C:\Drone_Source\
5. Put VoTT project to C:\Drone_Target\
6. Put drone video to C:\Drone_Source\ (Make sure ONLY project videos in this folder)
7. Open VoTT project file (C:\Drone_Target\Drone_001.vott)
8. Label and check actions (action_lists) (Check Labels)

Let's see the detailed steps now.

1. Install Microsoft VoTT

Download VoTT from GitHub (https://github.com/microsoft/VoTT/releases). Install and open VoTT

2. Add security token to VoTT

Add security token: 

Name:	NTUTDrone Token
Key:	qITjd1Rj6hFqC7uB0OGxPKYVxIFSdcMrprQAMS5rEhc=

3. Download Drone videos and labeling projects

Download the source drone videos and labeling projects:

• Drone videos (link) 
• VoTT Projects (109-1 HW3) 

4. Create folders for VoTT

Create two folders: C:\Drone_Target\ and C:\Drone_Source\

5. Put VoTT project files to C:\Drone_Target\

6. Put the drone video to C:\Drone_Source\

Make sure ONLY related videos (ONE video in our case) is put in the folder.

Hint: You can configure the path at "Connection Settings"

7. Open VoTT project file

Click "Open Local Project"

VoTT will start playing the video, stop the video to see the labels:

8. Labeling Actions

We already have 13 different actions. Please check the labels and fix them if necessary.

1. stand
2. walk
3. sit
4. watchphone
5. basketball
6. baseball
7. riding
8. jump
9. push
10. skate
11. handstand
12. soccer
13. fishing

Sometimes the objects (humans) may be blocked. In this case, we need to label the occlusion rate. 

1. block25 (Object is blocked 25~50%)
2. block50 (Object is blocked 50~75%)
3. block75 (Object is blocked 75~90%)

The action tags are shown at the right side

To label the actions, select the object (human) first

Once the region is selected, click the tags at right side; To cancel it, just click the same tag again.

To see your tags, put your mouse cursor on the selected region: