Skip to main content

detectObjects

warning

The document is a continuation of the previous document, if you have landed directly on this page then, Please read from page Get started.

What is detectObjects component ?#

detectObjects component is used to detect objects from an input image.

  • Description : detectObjects() takes an input as image in base64 string format and probability and returns the object information with each object probability greater than input given. Check Input and output parameters for details.
  • Parameters :
    • Input(Via STDIN) : detectObjects.stdin() << jsonDoc2Text(inputJson) << std::endl;
      • An inputJson String with following parameters:
        • Parameter1 : Image (.jpg/.png) in base64 format
        • Parameter2 : Float probability (value between 0 to 1)
    • Output(Via STDOUT) : detectObjects.stdout() >> outputJson;
      • An outputJson string with following contents
        • For each object
          • Bounding box co-ordinates
          • Probability being object
          • Class label
        • int requestID

List of detectObjects features in shunya stack#

  1. Customize object probability.
  2. Get object data.
  3. Currently we support 20 classes. List below
"aeroplane", "bicycle", "bird", "boat",
"bottle", "bus", "car", "cat", "chair",
"cow", "diningtable", "dog", "horse",
"motorbike", "person", "pottedplant",
"sheep", "sofa", "train", "tvmonitor"

Using detectObjects#

Requirements to use detectObjects#

  1. Shunya OS installed (supported Arm devices) or Shunya OS docker container (X86 based windows/linux devices)
  2. Shunya AI installed in Shunya OS.
  3. Shunya Video component installed in Shunya OS.

Steps to use detectObjects#

  1. Set input video location.
  2. Read Video frames
  3. Convert cv image to base64 string
  4. Customize object probability.
  5. Call API binary
  6. Print detected objects output
note

Run the steps given below inside Shunya OS installed (supported Arm devices) or Shunya OS docker container (X86 based windows/linux devices) terminals.

Lets take an example use case: Say we need to

  1. Detect all those objects which have a probability of 80% or higher in an image.
  2. Get JSON output of all the detected objects.

Steps are

Step 1: Set input video location#

  1. Start with a ready to use template for detecting objects from video/image.
    git clone https://gitlab.iotiot.in/repo-public/examples.git
  2. Open the examples in a text editor and modify as per your usecase.
    • We will use video compnent to set input video location in configuration file.
    • Please check here, how to setup the video source path in video component.

Step 2: Read video frames#

  1. Once video path is set using video component, lets read video frames one by one.

    • For CPP you will find the examples in the folder shunya-ai-examples/indiv-components/cpp-examples/object-detection/detectObjects
    • Open the file object_detect.cpp
    • Code to read first video frame. If you want to read all frames one by one, put following code in while loop.
    /* --- Capturing image from video using Video component --- */
    captureObj src = newCaptureDevice("video-source"); /* Create capture Instance */
    cv::Mat inputImage;
    int32_t outIndex = 0;
    /*################## Call Video Component functions ################*/
    inputImage = captureFrameToMem(&src); /* Capture one frame at a time in a loop*/
    if (inputImage.empty()) {
    fprintf(stderr, "End of video file!.");
    closeCapture(&src);
    return 0;
    }

Step 3: Convert cv image to base64 string#

  1. Since detectObjects API needs image in base64 string format, we will convert image from cv::Mat to base64 string.
  2. Code to do it
    /* ---- Create cv::mat image to base64 string ---- */
    std::string b64InpImg = mat2Base64(inputImage);

Step 4: Customize object probability#

  1. Modify the line to set the object probability to 80% or higher.

    /* Set value to 0.8 i.e (80/100) to set the probability to 80% or higher */
    float probability = 0.8;

Step 5: Call API binary#

  1. We will now call API binary by giving input image(parameter1) and object probability(parameter2) as an input through STDIN.

    /* Creating inputJson adding image and object probability in it */
    rapidjson::Document inputJson;
    inputJson.SetObject();
    rapidjson::Value inpImage;
    /* Call API binary */
    inpImage.SetString(b64InpImg.c_str(), strlen(b64InpImg.c_str()), inputJson.GetAllocator());
    /* Adding parameter1 in inputJson file */
    inputJson.AddMember("inputImage", inpImage, inputJson.GetAllocator());
    /* Adding parameter2 in inputJson file */
    inputJson.AddMember("probability", probability, inputJson.GetAllocator());
    /*############### Call detectObjects Component ##################*/
    subprocess::popen detectObjects("/usr/bin/detectObjects", {});
    /* Calling detectObjects by passing inputJson as an input through STDIN */
    detectObjects.stdin() << jsonDoc2Text(inputJson) << std::endl;
    detectObjects.close();
    std::string outputJson;
    /* Getting output in outputJson */
    detectObjects.stdout() >> outputJson;
  2. You will get output in outputJson string.

Step 6: Print detected objects output.#

  1. Code to print the json output, got from detectObjects API.
    /* ---- Printing detected objects information ---- */
    rapidjson::Document detectObjectsJson = readJsonString(detectObjectsOut);
    if (detectObjectsJson.HasMember("data")) {
    rapidjson::Value &results = detectObjectsJson["data"]["objects"];
    assert(results.IsArray());
    /* Checking if objects are detected or not */
    if(results.Size()>0){
    /* Reading from json file and add it in the structure */
    for (rapidjson::SizeType i = 0; i < results.Size(); i++) {
    // Printing detected objects values
    std::cout<<"\nData:";
    std::cout<<"\nObject"<<std::to_string(i)<<": ";
    std::cout<<"\nObject Label: "<<results[i]["object"].GetString();
    std::cout<<"\nObject probability: "<<results[i]["confidence"].GetFloat();
    }
    }
    else{
    std::cout<<"\nNo objects detected.";
    }
    }
    else{
    std::cout<<"\nno json member as data ";
    }

Run ready to use example.#

  1. Once you are done editing, save and run the code, by running

    git clone https://gitlab.iotiot.in/shunya/products/shunya-ai-examples.git
    cd cpp-examples/object-detection/
    mkdir build && cd build
    cmake ../
    make
    ./detectObjectsCpp
  1. Running the codes will print the JSON output on the terminal (to STDOUT).

    For Example:

    • Lets say the input image is

      Oops!, No Image to display.
    • Input JSON is

      {
      "inputImage": "4h32e898473urmcrkd947ryuemcc3x21j98k09754ycmx1k030i1d98754yn==",
      "probability": 0.8
      }
    • Then the JSON output is

      {
      "apiVersion": "1.2.0",
      "requestId": 19287918271287,
      "data": {
      "objects": [
      {
      "object": "Cat",
      "boundingBox": {
      "top": 234253,
      "left": 124.023480,
      "width": 2435,
      "height": 2345,
      },
      "confidence": 99.97504425048828
      },
      {
      "object": "Dog",
      "boundingBox": {
      "top": 234253,
      "left": 124.023480,
      "width": 2435,
      "height": 2345,
      },
      "confidence": 99.97504425048828
      },
      ],
      "image": "base64str"
      }
      }

This part is not updated yet (please do not use below example)#

Understand this component with an example (ready to use code)#

  • This is an example for object-detection and here we will be using 2 components: detectObjects and drawObjects

  • Check this ready to use example in c++ and python

  • Download the code

    git clone https://gitlab.iotiot.in/shunya/products/shunya-ai-examples.git
    cd shunya-ai-examples/cpp-examples/object-detection
    ```shell git clone https://gitlab.iotiot.in/shunya/products/shunya-ai-examples.git cd shunya-ai-examples/python-examples/object-detection ```
  • In this folder there is a file, object_detect.cpp or object_detect.py

  • detectbjects Components used

    subprocess::popen detectObjects("/usr/bin/detectObjects", {});
    ```shell detectObjects = Popen(['/usr/bin/detectObjects'], stdout=PIPE, stdin=PIPE) ```
  • drawObjects component used

    subprocess::popen drawObjects("/usr/bin/drawObjects", {});
    ```shell drawObjects = Popen(['/usr/bin/drawObjects'], stdout=PIPE, stdin=PIPE) ```
  • Run code by yourself

    mkdir build && cd build
    cmake .. && make
    ./objectDetectCpp
    ```shell python3 object_detect.py ```
    - You will get a new image stored in system.Oops!, No Image to display.

Facing errors with the component?#