Labeling Human Actions in Drone Videos

In this blog we will show you how to label actions in drone videos, as shown in the video below:

https://www.youtube.com/watch?v=y05iJ0vMnuI


We use Microsoft VoTT as our labeling tool. The summary of labeling steps are listed as follows:

'
  1. Install Microsoft VoTT v2.1.0 (link) or NTUT VoTT v0.0.5 (Windows)
  2. Add security token to VoTT 
    Name: NTUTDrone Token
    Key: qITjd1Rj6hFqC7uB0OGxPKYVxIFSdcMrprQAMS5rEhc=
  3. Download Drone videos (DroneVideos) and labeling projects (VoTT Projects)
  4. Create two folders: C:\Drone_Target\ and C:\Drone_Source\
  5. Put VoTT project to C:\Drone_Target\
  6. Put drone video to C:\Drone_Source\ (Make sure ONLY project videos in this folder)
  7. Open VoTT project file (C:\Drone_Target\Drone_001.vott)
  8. Label and check actions (action_lists) (Check Labels)

Let's see the detailed steps now.

1. Install Microsoft VoTT

Download VoTT from GitHub (https://github.com/microsoft/VoTT/releases). Install and open VoTT

2. Add security token to VoTT

Add security token: 

Name: NTUTDrone Token
Key: qITjd1Rj6hFqC7uB0OGxPKYVxIFSdcMrprQAMS5rEhc=




3. Download Drone videos and labeling projects

Download the source drone videos and labeling projects:

4. Create folders for VoTT

Create two folders: C:\Drone_Target\ and C:\Drone_Source\

5. Put VoTT project files to C:\Drone_Target\


6. Put the drone video to C:\Drone_Source\

Make sure ONLY related videos (ONE video in our case) is put in the folder.


Hint: You can configure the path at "Connection Settings"


7. Open VoTT project file

Click "Open Local Project"

VoTT will start playing the video, stop the video to see the labels:

8. Labeling Actions

We already have 13 different actions. Please check the labels and fix them if necessary.
  1. stand
  2. walk
  3. sit
  4. watchphone
  5. basketball
  6. baseball
  7. riding
  8. jump
  9. push
  10. skate
  11. handstand
  12. soccer
  13. fishing
Sometimes the objects (humans) may be blocked. In this case, we need to label the occlusion rate. 
  1. block25 (Object is blocked 25~50%)
  2. block50 (Object is blocked 50~75%)
  3. block75 (Object is blocked 75~90%)

The action tags are shown at the right side



To label the actions, select the object (human) first



Once the region is selected, click the tags at right side; To cancel it, just click the same tag again.


To see your tags, put your mouse cursor on the selected region:

Comments

Post a Comment

Popular posts from this blog

ArduPilot Software-In-The-Loop (SITL), MavLink, and AirSim

Image
In this article, I would like to share my experience of setting up ArduPilot SITL running with AirSim using Cygwin on Windows. The result is shown in the picture below. I hope this post can be complementary to the original AirSim SITL tutorial Using SITL with AirSim . First of all, let's see the architecture of AirSim SITL: The red block represents SITL, which connects to physics simulator (blue block), which is AirSim, and a Ground Control System (GCS), which is MAVProxy in our case. So we will run three programs that communicates with each other through TCP/IP.  Now we are ready to run the simulation. The major steps are listed below: Git clone ArduPilot and build it under Cygwin. You can follow ArduPilot tutorial ( link ) or see my summary at the section ( Run SITL using Cygwin ). Install Unreal and git clone AirSim and build it, OR  download pre-built AirSim precompiled binaries ( link ) Edit “Documents\AirSim\settings.json”, note the default folder is you...
Read more

Rhinoceros 3D simulation Pothole

Image
這篇主旨在於使用3D立體模型軟體來製作坑洞，從而獲得大量坑洞資料，減少人工採集資料的時間。 以下會依序介紹路面剖析、模擬坑洞等等。 路面分層 路面坑洞常見形成方式，是在 密集配瀝青混泥土層破裂 從而導致路面塌陷形成坑洞。 路面分層詳細介紹 柏油路上坑洞形成四大種類 道路寬度 快速道路者，不得小於3.2公尺 於主要道路及次要道路者，不得小於三公尺 於服務道路者，不得小於2.8公尺 坑洞設計 使用Rhinoceros 3D來模擬坑洞形成 簡介：Rhinoceros 3D是一套專業的3D立體模型製作軟體，簡稱Rhino3D 下載點： https://drive.google.com/open?id=1aYolSV0hTTM-Y1m-cMgPCc61cIBPqJ1p 解壓縮：rhinoceros67chs.rar 出現畫面，並選取紅框中安裝 安裝完後，不要啟動，回到原先壓縮檔位置，選取紅框 將紅框檔案移動到，Rhino 6 安裝檔案位置，後可啟動 啟動畫面，在 Command 輸入 Grasshopper，按下 enter Grasshopper 畫面啟動，點選 File，選取紅框 選取檔案 檔案下載點： https://drive.google.com/file/d/1tT3k-_CUdNnEdOCiXDVUaYFr9AMMHeMx/view?usp=sharing 坑洞生成，選取紅框按滑鼠中鍵 點選 Bake 回到 Rhino 畫面，坑洞形成 存檔(存檔位置自行選擇)，檔案類型選擇 Rhino 6 3D Models(*.3dm) 坑洞形成概念 先在一個方格內隨機佈點 框出點的範圍隨機分布 A 點為圓心，B 點為框裡面的隨機點分布，以 AB 畫出向量 劃出坑洞邊界 邊界向下延伸，形成柱體 ...
Read more