I. overview

• Detectionbasedtracker provided by the official example Detectionbasedtracker under Java and JNI_ jni. CPP interaction
• When the project is migrated, the package name changes. If the content under JNI is copied to the new package, detectionbasedtracker Java cannot find detectionbasedtracker under JNI_ jni. Cpp file
• Rewrite the file under JNI to generate DetectionBasedTracker_jni.h and DetectionBasedTracker_jni.cpp

II. Relationship between FdActivity, DetectionBasedTracker and jni

2.1 calling relationship between

Calling DetectionBasedTracker. in FdActivity When using the start() method in Java

Execute detectionbasedtracker native method in Java nativeStart(long thiz)

DetectionBasedTracker. The native method in Java has been in the detectionbasedtracker in JNI_ jni. Declaration in H

DetectionBasedTracker_ jni. Detectionbasedtracker is implemented in CPP_ jni. Method declared in H

2.2 schematic diagram of calling relationship

III. key points of knowledge

• Knowledge: NDK and JNI
• Dependencies: OpenCV, javacpp and javacv

IV. project migration

4.1 create a new project (e.g. MyOpenCV)

4.2 add opencv and dependencies

4.2.1 importing opencv SDK

Click File - > New - > import module from source to import opencv SDK

4.2.2 configure NDK

After importing, the following errors may be displayed (which may be caused by not downloading NDK or configuring NDK)

Install NDK: click SDK Manager - > appearance & behavior > System Settings > Android SDK - > SDK tools to install NDK and CMake

Configure SDK: click File - > project structure - > SDK location, and select the NDK File location

settings. Configure opencv SDK in gradle (because it is in the same directory as the project, opencvsdk = ', the upper directory, opencvsdk ='... / ')

def opencvsdk=''
//def opencvsdk='/<path to OpenCV-android-sdk>'
include ':opencv'
project(':opencv').projectDir = new File(opencvsdk + '/sdk')

app/build. Add opencv and javacpp under gradle, javacv

  //opencv face detection
  implementation project(':opencv')
  //Face recognition
  implementation 'org.bytedeco:javacpp:1.5.5' //javacpp
  implementation 'org.bytedeco:javacv:1.5.5' //javac

  implementation group: 'org.bytedeco', name: 'javacv-platform', version: '1.5.5'
  implementation group: 'org.bytedeco', name: 'javacpp-platform', version: '1.5.5'

4.3 migration project code (code file + jni file + layout file)

• Code files: migrate the files (FdActivity and DetectionBasedTracker) to java / [package name] of the new project
• Layout files: layout / face_ detect_ surface_ view. Migrate XML to new project layout
• Resource file: raw/lbpcascade_frontalface.xml is migrated to the res directory of the new project
• jni: face detection / jni migrate to the main directory of the new project

4.4 generate the data under jni according to the native method h and cpp file

Under jni h and The cpp file is generated according to the package name. The native of the new project cannot recognize the old project h and Cpp file error

Detectionbasedtracker in JNI folder_ jni. H and DetectionBasedTracker_jni.cpp is deleted. At this time, only

Android.mk
Application.mk
CMakeLists.txt

Right click on main/java - > open in terminal to open the CMD terminal. At this time, the code display position in CMD is

D:\Code\Android\MyOpenCV\app\src\main\java>

Execute the javah command to generate the corresponding from the native method h header file

javah -d ../jni -jni com.example.myopencv.DetectionBasedTracker

explain:

• Javah: it is a set of javah commands, which can perform operation generation h header file
• -d: Destination file location:/ jni: indicates the jni directory at the upper level of java
• -JNI: generate JNI style header file (default) (when you enter javah, you can display the options option to view)
• com.example.myopencv.DetectionBasedTracker: the path of the file where the native method is located (package name + class name)

Delete package name prefix com_example_myopencv_， The file name is DetectionBasedTracker_jni.h. At the same time, DetectionBasedTracker_jni.h make a copy and rename it DetectionBasedTracker_jni.cpp (because Android.mk specifies the file name of cpp)

LOCAL_SRC_FILES  := DetectionBasedTracker_jni.cpp
LOCAL_C_INCLUDES += $(LOCAL_PATH)
LOCAL_LDLIBS     += -llog -ldl

LOCAL_MODULE     := detection_based_tracker

include $(BUILD_SHARED_LIBRARY)

4.5 configuring NDK

4.5.1 app/build.grale

defaultConfig

externalNativeBuild {
    cmake {
              arguments "-DOpenCV_DIR=" + project(':opencv').projectDir + "/native/jni",
                        "-DANDROID_TOOLCHAIN=clang",
                        "-DANDROID_STL=c++_shared"
              targets "detection_based_tracker"
                ///abiFilters  "armeabi-v7a" , "arm64-v8a", "x86", "x86_64"
          }
   }

android{}

sourceSets {  //Configuration address modification
        main {
            java.srcDirs = ['src/main/java']
            aidl.srcDirs = ['src/main/java']
            res.srcDirs = ['src/main/res']
            manifest.srcFile 'src/main/AndroidManifest.xml'
        }
    }
externalNativeBuild {
        cmake {
            path 'src/main/jni/CMakeLists.txt'  //Configuration address modification
        }
    }     

4.5.2 project/build.gradle(APP_ABI)

gradle.afterProject { project ->
    if (project.pluginManager.hasPlugin('com.android.application')
            || project.pluginManager.hasPlugin('com.android.library')
            || project.pluginManager.hasPlugin('com.android.test')
            || project.pluginManager.hasPlugin('com.android.feature') ) {
        if (true) {
            gradle.println("Override build ABIs for the project ${project.name}")
            project.android {
                splits {
                    abi {
                        enable true
                        universalApk false

//reset()
//include 'armeabi-v7a'
//include 'arm64-v8a'
//include 'x86'
//include 'x86_64'

                    }
                }
            }
        }

        if (true) {
            gradle.println("Override lintOptions for the project ${project.name}")
            project.android {
                lintOptions {
                    // checkReleaseBuilds false
                    abortOnError false
                }
            }
        }

        // (you still need to re-build OpenCV with debug information to debug it)
        if (true) {
            gradle.println("Override doNotStrip-debug for the project ${project.name}")
            project.android {
                buildTypes {
                    debug {
                        packagingOptions {
                            doNotStrip '**/*.so'  // controlled by OpenCV CMake scripts
                        }
                    }
                }
            }
        }
        if (false || project.hasProperty("doNotStrip")) {
            gradle.println("Override doNotStrip-release for the project ${project.name}")
            project.android {
                buildTypes {
                    release {
                        packagingOptions {
                            doNotStrip '**/*.so'  // controlled by OpenCV CMake scripts
                        }
                    }
                }
            }
        }

    }
}

4.5.3 opencv API level is android-21 (minSdkVersion of OpenCV SDK is 21)

D:\Code\Android\MyOpenCV\app\src\main\jni\CMakeLists.txt : C/C++ debug|x86 : CMake Warning at D:/Code/Android/MyOpenCV/sdk/native/jni/abi-x86/OpenCVConfig.cmake:105 (message):
  Minimum required by OpenCV API level is android-21
Call Stack (most recent call first):
  D:/Code/Android/MyOpenCV/sdk/native/jni/OpenCVConfig.cmake:44 (include)
  CMakeLists.txt:8 (find_package)

Please set minSdkVersion to 21

minSdkVersion 21

4.5.4 OS independent conflict

phenomenon

More than one file was found with OS independent path 'META-INF/native-image/ios-x86_64/jnijavacpp/reflect-config.json'.

solve

 packagingOptions {
        exclude 'META-INF/proguard/androidx-annotations.pro'
        exclude 'META-INF/native-image/**'
}        

4.5.5 modify DetectionBasedTracker_jni.cpp file

copy the header file in the sample project to DetectionBasedTracker_jni.cpp header

#include <DetectionBasedTracker_jni.h>
#include <opencv2/core.hpp>
#include <opencv2/objdetect.hpp>

#include <string>
#include <vector>

#include <android/log.h>

#define LOG_TAG "FaceDetection/DetectionBasedTracker"
#define LOGD(...) ((void)__android_log_print(ANDROID_LOG_DEBUG, LOG_TAG, __VA_ARGS__))

using namespace std;
using namespace cv;

inline void vector_Rect_to_Mat(vector<Rect>& v_rect, Mat& mat)
{
    mat = Mat(v_rect, true);
}

class CascadeDetectorAdapter: public DetectionBasedTracker::IDetector
{
public:
    CascadeDetectorAdapter(cv::Ptr<cv::CascadeClassifier> detector):
            IDetector(),
            Detector(detector)
    {
        LOGD("CascadeDetectorAdapter::Detect::Detect");
        CV_Assert(detector);
    }

    void detect(const cv::Mat &Image, std::vector<cv::Rect> &objects)
    {
        LOGD("CascadeDetectorAdapter::Detect: begin");
        LOGD("CascadeDetectorAdapter::Detect: scaleFactor=%.2f, minNeighbours=%d, minObjSize=(%dx%d), maxObjSize=(%dx%d)", scaleFactor, minNeighbours, minObjSize.width, minObjSize.height, maxObjSize.width, maxObjSize.height);
        Detector->detectMultiScale(Image, objects, scaleFactor, minNeighbours, 0, minObjSize, maxObjSize);
        LOGD("CascadeDetectorAdapter::Detect: end");
    }

    virtual ~CascadeDetectorAdapter()
    {
        LOGD("CascadeDetectorAdapter::Detect::~Detect");
    }

private:
    CascadeDetectorAdapter();
    cv::Ptr<cv::CascadeClassifier> Detector;
};

struct DetectorAgregator
{
    cv::Ptr<CascadeDetectorAdapter> mainDetector;
    cv::Ptr<CascadeDetectorAdapter> trackingDetector;

    cv::Ptr<DetectionBasedTracker> tracker;
    DetectorAgregator(cv::Ptr<CascadeDetectorAdapter>& _mainDetector, cv::Ptr<CascadeDetectorAdapter>& _trackingDetector):
            mainDetector(_mainDetector),
            trackingDetector(_trackingDetector)
    {
        CV_Assert(_mainDetector);
        CV_Assert(_trackingDetector);

        DetectionBasedTracker::Parameters DetectorParams;
        tracker = makePtr<DetectionBasedTracker>(mainDetector, trackingDetector, DetectorParams);
    }
};

The implementation copcy of each method in the sample project is to the corresponding method (nativechreateobject as an example)

Before modification

/*
 * Class:     com_example_myopencv_DetectionBasedTracker
 * Method:    nativeCreateObject
 * Signature: (Ljava/lang/String;I)J
 */
JNIEXPORT jlong JNICALL Java_com_example_myopencv_DetectionBasedTracker_nativeCreateObject
  (JNIEnv *, jclass, jstring, jint);

After modification

/*
 * Class:     com_example_myopencv_DetectionBasedTracker
 * Method:    nativeCreateObject
 * Signature: (Ljava/lang/String;I)J
 */
JNIEXPORT jlong JNICALL Java_com_example_myopencv_DetectionBasedTracker_nativeCreateObject
        (JNIEnv * jenv, jclass, jstring jFileName, jint faceSize)
{
    LOGD("Java_org_opencv_samples_facedetect_DetectionBasedTracker_nativeCreateObject enter");
    const char* jnamestr = jenv->GetStringUTFChars(jFileName, NULL);
    string stdFileName(jnamestr);
    jlong result = 0;

    LOGD("Java_org_opencv_samples_facedetect_DetectionBasedTracker_nativeCreateObject");

    try
    {
        cv::Ptr<CascadeDetectorAdapter> mainDetector = makePtr<CascadeDetectorAdapter>(
                makePtr<CascadeClassifier>(stdFileName));
        cv::Ptr<CascadeDetectorAdapter> trackingDetector = makePtr<CascadeDetectorAdapter>(
                makePtr<CascadeClassifier>(stdFileName));
        result = (jlong)new DetectorAgregator(mainDetector, trackingDetector);
        if (faceSize > 0)
        {
            mainDetector->setMinObjectSize(Size(faceSize, faceSize));
            //trackingDetector->setMinObjectSize(Size(faceSize, faceSize));
        }
    }
    catch(const cv::Exception& e)
    {
        LOGD("nativeCreateObject caught cv::Exception: %s", e.what());
        jclass je = jenv->FindClass("org/opencv/core/CvException");
        if(!je)
            je = jenv->FindClass("java/lang/Exception");
        jenv->ThrowNew(je, e.what());
    }
    catch (...)
    {
        LOGD("nativeCreateObject caught unknown exception");
        jclass je = jenv->FindClass("java/lang/Exception");
        jenv->ThrowNew(je, "Unknown exception in JNI code of DetectionBasedTracker.nativeCreateObject()");
        return 0;
    }

    LOGD("Java_org_opencv_samples_facedetect_DetectionBasedTracker_nativeCreateObject exit");
    return result;
}

4.6 adding permissions

  <uses-permission android:name="android.permission.INTERNET" />
  <uses-permission android:name="android.permission.CAMERA" />

  <uses-feature
        android:name="android.hardware.camera"
        android:required="false" />
  <uses-feature
        android:name="android.hardware.camera.autofocus"
        android:required="false" />
  <uses-feature
        android:name="android.hardware.camera.front"
        android:required="false" />
  <uses-feature
        android:name="android.hardware.camera.front.autofocus"
        android:required="false" />
  <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
  <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />

4.7 renderings