OpenCv在Android Framework层集成 | 踩坑备忘

小驰行动派

发布于 2024-11-23 08:44:55

14600

代码可运行

文章被收录于专栏：Android Camera开发Android Camera开发

运行总次数：0

代码可运行

OpenCv在计算机视觉方面大有用处，这篇文章主要记录在Android Framework集成OpenCv，对Camera数据流进行处理，遇到的问题记录备忘。

openc源码下载地址：

https://opencv.org/releases/

一、OpencV在framework层集成

（下面的三方算法，是指的三方算法采用到了opencv）

1、三方算法是源码

(如下面所示，image.cpp是三方算法源码，该源码image.cpp文件中调用到opencv)

// -----Android.bp

cc_library_shared {
    name: "libimage",
    rtti:true,
    srcs: [
       "image/image.cpp",
    ],

    include_dirs: [
        "frameworks/av/services/camera/libcameraservice/image/include",
    ],

    shared_libs: [
        "libopencv_java4",
        "liblog",
    ],

    export_include_dirs: ["."],

    cflags: [
        "-Wall",
        "-Wextra",
        "-Werror",
        "-Wno-ignored-qualifiers",
        "-fPIC",
        "-ferror-limit=0",
    ],
    cppflags: [
        "-fno-rtti",
        "-fexceptions",
    ],
}

2、三方算法是so库

1 ）如下所示，libimage是三方算法提供的so库，我们自己封装了个libProcess的so，libProcess中调用三方算法的libimage,三方的libimage算法库，会调用到opencv；

//----- Android.bp内容

cc_prebuilt_library_shared {
     name: "libopencv_java4",
     arch: {
         arm: {
             srcs: ["image/lib/armeabi-v7a/libopencv_java4.so"],
         },
         arm64: {
             srcs: ["image/lib/arm64-v8a/libopencv_java4.so"],
         }
     },

    shared_libs: ["libandroid", "libc++", "liblog", "libmediandk", "libjnigraphics","libz","libdl","libm"],

    check_elf_files: false,
}

cc_prebuilt_library_shared {
     name: "libimage",
     arch: {
         arm: {
             srcs: ["./image/lib/armeabi-v7a/libimage.so"],
         },
         arm64: {
             srcs: ["./image/lib/arm64-v8a/libimage.so"],
         }
     },

    shared_libs: ["libopencv_java4", "libandroid", "libc++", "liblog", "libmediandk", "libjnigraphics","libz","libdl","libm"],
    check_elf_files: false,
}

cc_library_shared {
    name: "libProcess",
    //编译引用到opencv相关头文件，需要设为true，否则编译报错
    rtti:true, 
    srcs: [
       "image/ImageProcess.cpp",
    ],

     //调用到的opencv,opencv相关的头文件路径
    include_dirs: [
        "frameworks/av/services/camera/libcameraservice/image/include",
    ],

    shared_libs: [
        "libimage",
        "libopencv_java4",
        "liblog",
    ],

    export_include_dirs: ["."],

    cflags: [
        "-Wall",
        "-Wextra",
        "-Werror",
        "-Wno-ignored-qualifiers",
        "-fPIC",
        "-ferror-limit=0",
    ],
    cppflags: [
        "-fno-rtti",
        "-fexceptions",
    ],
}

2 ）正常来说，我们调用算法的so库，其实是不需要再单独封装一个so库。这个封装成一个so库的原因是，三方算法调用到了opencv，opencv头文件的编译，需要 rtti 设为true，而rtti设为true,会影响到cameraservice里面其它代码的编译。

所以比较好的解决方法就是，单独封装一个调用三方算法库的so，在这个so里面，rtti 设为true，这样不影响其它代码的编译。

----- 单独封装的调用三方算法库(libimage) 的 ImageProcess.cpp文件内容：

#include <iostream>
#include <opencv2/opencv.hpp>
#include <fstream>
#include <ctime>
#include<cmath>

//三方算法库头文件
#include "process.h"

using namespace cv;
using namespace std;

void imageProcess(uint8_t* buffer, uint width, uint height)
{
    //process是三方算法库的接口
    process(buffer, width, height);
}

在framework层获取camera相关数据，可以查看下面代码。

在下面代码逻辑里面，可以拿到拍照的jpeg以及录像的yuv数据。

frameworks/av/services/camera/libcameraservice/device3/Camera3OutputStream.cpp

Camera3OutputStream::returnBufferCheckedLocked{
if (getFormat() == HAL_PIXEL_FORMAT_BLOB && getDataSpace() == HAL_DATASPACE_V0_JFIF && mImageDumpMask) {
   dumpImageToDisk(timestamp, anwBuffer, anwReleaseFence);
    }

    ALOGE("%s:getFormat():%d ,width:%d ,HAL_PIXEL_FORMAT_YCbCr_420_888:%d ", __FUNCTION__,getFormat(),buffer.stream->width,HAL_PIXEL_FORMAT_YCbCr_420_888);
     //新添加内容，判断到是yuv format,才走下面的逻辑
    if (getFormat() == 34) {
        processYuv(anwBuffer,buffer.stream->width, buffer.stream->height,anwReleaseFence);
    }
}

void Camera3OutputStream::processYuv(
         ANativeWindowBuffer* anwBuffer, int width,int height,int fence) {

    ALOGE("%s: width: %d,height: %d,usage: %d", __FUNCTION__, width, height, usage);

    // Lock the image for CPU read
    sp<GraphicBuffer> graphicBuffer = GraphicBuffer::from(anwBuffer);
    void* mapped = nullptr;
    base::unique_fd fenceFd(dup(fence));
    status_t res = graphicBuffer->lockAsync(GraphicBuffer::USAGE_SW_READ_OFTEN, &mapped,
            fenceFd.get());
    if (res != OK) {
        ALOGE("%s: Failed to lock the buffer: %s (%d)", __FUNCTION__, strerror(-res), res);
        return;
    }
    //这个地方，mapped拿到的就是yuv数据,就可以把mapped送给算法进行处理.
    processImage((uint8_t*)mapped,width,height);

   //也可以把yuv数据保存到本地进行debug.
}

二、opencv在Android源码集成遇到的问题汇总

1、not specified in shared_libs

【action】这类错误，就是我们的目标文件，需要引用到的一些so库，但是我们的bp或者mk文件没有申明。

如下图所示，错误信息里面，也有Fix suggestion，根据提示来做修改就好。

DT_NEEDED "liblog.so" is not specified in shared_libs

 Fix suggestions:

 Android.bp: shared_libs: ["libc++_shared", "libc", "libdl", "libjnigraphics", "liblog", "libm", "libmediandk", "libz"],

  Android.mk: LOCAL_SHARED_LIBRARIES := libc++_shared libc libdl libjnigraphics liblog libm libmediandk libz

  If the fix above doesn't work, bypass this check with:
  Android.bp: check_elf_files: false,

  Android.mk: LOCAL_CHECK_ELF_FILES := false

2、(native:vendor) can not link against libc++_shared (native:platform)

【action】问题原因是，android8.0之后，system和vendor库进行了区分，除了一些特殊的so库，很多so库，比vendor模块的代码，不能去访问到system下的so库。vndk ?

vendor/mediatek/proprietary/hardware/mtkcam3/***/Android.mk: error: "libopencv_java4 (native:vendor) can not link against libc++_shared (native:platform)"

类似上面这种报错，如果是vendor下去访问system下的so，我们可以找到system下该so源码的Android.bp文件，添加vendor_available: true 来进行尝试。

diff --git a/frameworks/base/native/graphics/jni/Android.bp b/frameworks/base/native/graphics/jni/Android.bp
index 1709dfd973d..22432812430 100644
--- a/frameworks/base/native/graphics/jni/Android.bp
+++ b/frameworks/base/native/graphics/jni/Android.bp
@@ -47,6 +47,9 @@ cc_library_shared {

     static_libs: ["libarect"],

+    
+    proprietary:true,
+

For Android.bp:
vendor_available: true

For Android.mk:
LOCAL_VENDOR_MODULE := true

如果是涉及到多个so的嵌套使用，就会比较麻烦，如下面的，libjnigraphics里面，会用到libhwui，而libhwui也是不允许vendor去访问的，libhwui里面，又会用到其它的vendor无法访问的so。类似这种情况？关闭vndk？

网友提供的一些方案（https://blog.csdn.net/qq_35345103/article/details/110390454），试了没能解决。

FAILED: out/soong/.intermediates/frameworks/base/native/graphics/jni/libjnigraphics/android_vendor.31_arm64_armv8-2a_cortex-a55_shared/libjnigraphics.so
echo "module libjnigraphics missing dependencies: libhwui_internal_headers{os:android,image:vendor.31,arch:arm64_armv8-2a_cortex-a55,sdk:}, libhwui{os:android,image:vendor.31,arch:arm64_armv8-2a_cortex-a55,sdk:,link:shared}, libandroid{os:android,image:vendor.31,arch:arm64_armv8-2a_cortex-a55,sdk:,link:shared}" && false
module libjnigraphics missing dependencies: libhwui_internal_headers{os:android,image:vendor.31,arch:arm64_armv8-2a_cortex-a55,sdk:}, libhwui{os:android,image:vendor.31,arch:arm64_armv8-2a_cortex-a55,sdk:,link:shared}, libandroid{os:android,image:vendor.31,arch:arm64_armv8-2a_cortex-a55,sdk:,link:shared}

vndk相关内容，可以查看官网介绍：

https://source.android.google.cn/docs/core/architecture/vndk/build-system?hl=ca

3、error: cannot use 'throw' with exceptions disabled

frameworks/av/services/camera/libcameraservice/***/include/opencv2/flann/kmeans_index.h:445:13: error: cannot use 'throw' with exceptions disabled
            FLANN_THROW(cv::Error::StsError, "Branching factor must be at least 2");
            ^
frameworks/av/services/camera/libcameraservice/image/include/opencv2/flann/general.h:55:32: note: expanded from macro 'FLANN_THROW'
#define FLANN_THROW(TYPE, STR) throw FLANNException(STR)
                               ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]

【action】

Android.bp中添加如下内容可以解决：

cppflags: [
        "-fexceptions",
    ],

Android.mk的修改

LOCAL_CPPFLAGS += -fexceptions

4、 error: use of *requires -frtti**

frameworks/av/services/camera/libcameraservice/image/include/opencv2/flann/any.h:334:34: error: use of typeid requires -frtti
        return policy->type() == typeid(T);

【action】

android.bp中添加如下内容可以解决：

rtti: true,

(如果是cppflags中添加如下内容，会继续报该错误)

cppflags: [
        "-frtti",
    ],

5、vndk能否关闭掉，怎么关闭？

本文参与腾讯云自媒体同步曝光计划，分享自微信公众号。

原始发表：2024-11-14，如有侵权请联系 cloudcommunity@tencent.com 删除

android

本文分享自小驰行动派微信公众号，前往查看

如有侵权，请联系 cloudcommunity@tencent.com 删除。

本文参与腾讯云自媒体同步曝光计划，欢迎热爱写作的你一起参与！

登录后参与评论

0 条评论

热度