Multiple Model

This article mainly introduces how to implement a simple multi-model example using the multi-model version of HornedSungem under Android platform.

Preparation

  1. For details on the configuration environment, please refer to quick start, I will not elaborate here.
  2. Download the models graph_face_SSD and graph_object_SSD required for detection, In your Android Studio, create a new assets package under the current module, copy the downloaded model file to this directory.
  3. Because the project needs to process and display the image, the javacv library is use. Developers can download it from GitHub or copy it from the official sample project to their own project.

Implementation

1.Implementation steps:

  • Turn on device communication.
  • The two models are sent to HornedSungem in sequence using USB communication.
  • Get image data corresponding to the face model.
  • Get the result of the corresponding face model.
  • The image is preprocessed and passed into the device.
  • Get the result of the corresponding object detection and draw it on the screen.

2.Main code:

    //Executed in a thread
   int status = openDevice();
   int face_id = allocateGraphByAssets(mContext, "graph_face_SSD");
   int object_id = allocateGraphByAssets(mContext, "graph_object_SSD");
   byte[] bytes = getImage(STD, MEAN, face_id);
   float[] result_face = getResult(face_id);
   //...The preprocessed image data is converted into a float type and passed to the object detection model.
   loadTensor(float_tensor, float_tensor.length, object_id);
   float[] result_object = getResult(object_id);
  HornedSungemFrame frame = getFrameResult(image, result_face, result_object);//show

Code can refer to example project, This article only extracts important code, data processing and business logic are not listed

3.Return result analysis:

Because this article mainly uses two models of face detection and object detection, the specific analysis results can refer to the detailed introduction of Model List.

  int num_object = (int) object_result[0];
  if (num_object > 0) {
    for (int i = 0; i < num_object; i++) {
        HornedSungemFrame.ObjectInfo objectInfo = new HornedSungemFrame.ObjectInfo();
        int type = (int) (object_result[7 * (i + 1) + 1]);
        int x1 = (int) (object_result[7 * (i + 1) + 3] * 1280);
        int y1 = (int) (object_result[7 * (i + 1) + 4] * 720);
        int x2 = (int) (object_result[7 * (i + 1) + 5] * 1280);
        int y2 = (int) (object_result[7 * (i + 1) + 6] * 720);
        int wight = x2 - x1;
        int height = y2 - y1;
        int percentage = (int) (object_result[7 * (i + 1) + 2] * 100);
        if (type == 0) {
            continue;
        }
        if (percentage <= 55) {
            continue;
        }
        if (wight >= 1280 * 0.8 || height >= 720 * 0.8) {
            continue;
        }
        if (x1 < 0 || x2 < 0 || y1 < 0 || y2 < 0 || wight < 0 || height < 0) {
            continue;
        }
        objectInfo.setType(labels[type - 1]);
        objectInfo.setRect(new Rect(x1, y1, x2, y2));
        objectInfo.setScore(percentage);
        objectInfos.add(objectInfo);
    }
}

4.Showing results:

result

5.Node:

The multi-model version is in beta. If you need any help, please post us an issue on GitHub Issues or send us an email (support@hornedsungem.org). You are welcome to contact us for your suggestions, and model request.

The code can be downloaded from GitHub, the address is as follows SungemSDK-AndroidExamples