Scene Recorder

Scene Recorder

This chapter introduces how to use Horned Sungem to deploy GoogleNet for building a useful scene recorder tool

Introduction

Idea

HS Scene Recorder records and classifies images in real-time. It ‘descripts’ images into image feature vectors and store them into different bins for training a classifier. For instance, put HS in front of a door and record the images to bin-0 when door is closed and bin-‘1’ when door is opened. Then the scene recorder will be able to distinguish between an opened door and a closed door.

Usage example

After initialisation, target HS to the first object and press ‘1’:

the terminal shows: (assume pressed ‘1’ for 7 frames)

|        Record to bin: 1        |
|      [7]-[0]-[0]-[0]-[0]       |

then target the second object and press ‘2’:

|        Record to bin: 2        |
|      [7]-[8]-[0]-[0]-[0]       |

press ‘r’ to remove redundent image features and train a nearest neighbour classifier:

| .........Compressing.......... |
|      [2]-[2]-[0]-[0]-[0]       |
| ------Compress finished------- |

then the program enters the recognition stage automatically:

| ---------Running ANN---------- |
*           [1]: 0.74            *
*           [2]: 0.26            *
*           [3]: 0.00            *
*           [4]: 0.00            *
*           [5]: 0.00            *
| --------Probabilities--------- |
|            |||||||             |
|               ||               |
|                                |
|                                |
|                                |

Press ’s’ to save bins to /misc/record.dat. Press ‘l’ to load. You can specify save file path as below:

prob = net.record(result, key, saveFilename='/Myfolder/NewFile.dat')

Press ‘p’ to reset bins

Change number of bins

prob = net.record(result, key, numBin = 9)

Warnings

Signal-to-Noise ratio

Since the recorder takes the whole full image to produce a feature vector, the performance will be decreased if the target object occupies too less of the image.

Negative sample

In many scenario, we will need extra bin for storing ‘negative’ samples to ensure a good performance. The ‘negative sample’ can be anything that considered as nuisances, usually is background.

Path and files

  • Python：SungemSDK-Python/examples/apps/SceneRecorder/SceneRecorder.py
  • Model files：SungemSDK-Python/examples/graphs/graph_g - GoogleNet

Scene Recorder

cd SungemSDK-Python/examples/apps/SceneRecorder
python3 SceneRecorder.py

Initialisation

WEBCAM = False
net = hs.SceneRecorder(zoom = True, verbose = 2)

WEBCAM:

  • True: use USB webcam

  • False: use HS built-in camera

  • zoom (only for HS cam): if True, the resolution is set to 640*360, otherwise 1920*1080

Scene Recorder

Image and its feature vector

result = net.run(img)

result returns the image and detection result, stored in result[0] and result[1] respectively

Note that when using HS cam, no image is required (img = None)

Record and Classify

key = cv2.waitKey(5)
prob = net.record(result, key)

Key press：

  • 1-9: save face feature to bin 1-9。
  • s: save bins to a local file
  • l: load bins from a local file
  • p: reset bins

If no key pressed, the function returns SoftMax confidences