英文原版链接：亚马逊《Mastering OpenCV Android Application Programming》
原书所给代码以章为单位，针对的Android版本从API 19到API 21不等，同时使用的OpenCV库版本也有2.4.9和2.4.10两种。本文给出的代码是在原书代码的基础上，针对Android 7.0（API 24）与OpenCV 3.2进行了修改，适当地修改了一些操作，以使代码整体上更合理。
原书的完整代码可以在这里获取：https://www.packtpub.com/lcode_download/22299。
更新后的代码托管在GitHub：https://github.com/johnhany/MOAAP/tree/master/MOAAP-Chp6-r3。
关于在Android Studio上配置OpenCV开发环境的方法，请参考《在Android Studio上进行OpenCV 3.1开发》。
关于在Android Studio上利用CMAKE配置OpenCV NDK开发环境的方法，请参考《Android Studio 2.3利用CMAKE进行OpenCV 3.2的NDK开发》。
本章介绍如何在Android Studio 2上利用OpenCV 3.2的Java API和Native API，通过Android NDK r15b开发一个图像拼接应用，涉及特征检测与匹配、图像匹配、光束平差法、自动校直、增益补偿、多频段融合等知识。但我们的应用只需要在C++部分调用OpenCV的Stitcher就可完成整个拼接过程。关于以上各算法的原理可以参考《深入OpenCV Android应用开发》第六章。
开发环境
Windows 10 x64专业版
Android Studio 2.3.3（Gradle 3.3，Android Plugin 2.3.3）
Android 7.0（API 24）
Android NDK r15b
JDK 8u141
OpenCV 3.2.0 Android SDK
代码及简略解释
1. 创建Android Studio项目，包命名为net.johnhany.moaap_chp6。注意我们需要建立一个NDK项目，所以需要勾选Include C++ Support。导入OpenCV-android-sdk\sdk\java到项目中，并为app模块加载模块依赖。
2. 在app\src\main\java目录中找到net.johnhany.moaap_chp6包，为MainActivity.java添加如下代码：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 |
package net.johnhany.moaap_chp6; import android.Manifest; import android.app.ProgressDialog; import android.content.Intent; import android.content.pm.PackageManager; import android.graphics.Bitmap; import android.graphics.BitmapFactory; import android.net.Uri; import android.os.AsyncTask; import android.os.Bundle; import android.os.Environment; import android.provider.MediaStore; import android.support.annotation.NonNull; import android.support.v4.app.ActivityCompat; import android.support.v4.content.ContextCompat; import android.support.v7.app.AppCompatActivity; import android.util.Log; import android.view.Menu; import android.view.MenuItem; import android.view.View; import android.widget.Button; import android.widget.ImageView; import android.widget.Toast; import org.opencv.android.BaseLoaderCallback; import org.opencv.android.LoaderCallbackInterface; import org.opencv.android.OpenCVLoader; import org.opencv.android.Utils; import org.opencv.core.CvType; import org.opencv.core.Mat; import org.opencv.core.Size; import org.opencv.imgproc.Imgproc; import java.io.File; import java.io.FileNotFoundException; import java.io.InputStream; import java.util.ArrayList; import java.util.List; public class MainActivity extends AppCompatActivity { private final int CLICK_PHOTO = 1; private final int SELECT_PHOTO = 2; private Uri fileUri; private ImageView ivImage; private Mat src; private List<Mat> listImage = new ArrayList<>(); private static final String FILE_LOCATION = Environment.getExternalStorageDirectory() + "/Download/MOAAP/Chapter6/"; static int REQUEST_READ_EXTERNAL_STORAGE = 11; static boolean read_external_storage_granted = false; private BaseLoaderCallback mOpenCVCallBack = new BaseLoaderCallback(this) { @Override public void onManagerConnected(int status) { switch (status) { case LoaderCallbackInterface.SUCCESS: System.loadLibrary("stitcher"); //DO YOUR WORK/STUFF HERE break; default: super.onManagerConnected(status); break; } } }; @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); ivImage = (ImageView)findViewById(R.id.ivImage); OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_2_0, this, mOpenCVCallBack); if (ContextCompat.checkSelfPermission(MainActivity.this, Manifest.permission.READ_EXTERNAL_STORAGE) != PackageManager.PERMISSION_GRANTED) { Log.i("permission", "request READ_EXTERNAL_STORAGE"); ActivityCompat.requestPermissions(MainActivity.this, new String[]{Manifest.permission.READ_EXTERNAL_STORAGE}, REQUEST_READ_EXTERNAL_STORAGE); }else { Log.i("permission", "READ_EXTERNAL_STORAGE already granted"); read_external_storage_granted = true; } Button bClickImage, bDone; bClickImage = (Button)findViewById(R.id.bClickImage); bDone = (Button)findViewById(R.id.bDone); bClickImage.setOnClickListener(new View.OnClickListener() { @Override public void onClick(View v) { Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE); File imagesFolder = new File(FILE_LOCATION); //imagesFolder.mkdirs(); File image = new File(imagesFolder, "panorama_"+ (listImage.size()+1) + ".jpg"); fileUri = Uri.fromFile(image); Log.d("MainActivity", "File URI = " + fileUri.toString()); intent.putExtra(MediaStore.EXTRA_OUTPUT, fileUri); // set the image file name // start the image capture Intent startActivityForResult(intent, CLICK_PHOTO); } }); bDone.setOnClickListener(new View.OnClickListener() { @Override public void onClick(View v) { if(listImage.size()==0){ Toast.makeText(getApplicationContext(), "No images clicked", Toast.LENGTH_SHORT).show(); } else if(listImage.size()==1){ Toast.makeText(getApplicationContext(), "Only one image clicked", Toast.LENGTH_SHORT).show(); Bitmap image = Bitmap.createBitmap(src.cols(), src.rows(), Bitmap.Config.ARGB_8888); Utils.matToBitmap(src, image); ivImage.setImageBitmap(image); } else { createPanorama(); } } }); } @Override public boolean onCreateOptionsMenu(Menu menu) { getMenuInflater().inflate(R.menu.menu_main, menu); return true; } @Override public boolean onOptionsItemSelected(MenuItem item) { int id = item.getItemId(); if (id == R.id.action_load_image && read_external_storage_granted) { Intent photoPickerIntent = new Intent(Intent.ACTION_PICK); photoPickerIntent.setType("image/*"); startActivityForResult(photoPickerIntent, SELECT_PHOTO); return true; } else if(!read_external_storage_granted) { Log.e("MainActivity", "pick image failed"); return true; } return super.onOptionsItemSelected(item); } @Override protected void onActivityResult(int requestCode, int resultCode, Intent imageReturnedIntent) { super.onActivityResult(requestCode, resultCode, imageReturnedIntent); Log.d("MainActivity", "request code " + requestCode + ", click photo " + CLICK_PHOTO + ", result code " + resultCode + ", result ok " + RESULT_OK); switch(requestCode) { case CLICK_PHOTO: if(resultCode == RESULT_OK){ try { Log.d("MainActivity", fileUri.toString()); final InputStream imageStream = getContentResolver().openInputStream(fileUri); final Bitmap selectedImage = BitmapFactory.decodeStream(imageStream); src = new Mat(selectedImage.getHeight(), selectedImage.getWidth(), CvType.CV_8UC4); Imgproc.resize(src, src, new Size(src.rows()/4, src.cols()/4)); Utils.bitmapToMat(selectedImage, src); Imgproc.cvtColor(src, src, Imgproc.COLOR_BGR2RGB); listImage.add(src); } catch (FileNotFoundException e) { e.printStackTrace(); } } break; case SELECT_PHOTO: if(resultCode == RESULT_OK && read_external_storage_granted){ try { final Uri imageUri = imageReturnedIntent.getData(); final InputStream imageStream = getContentResolver().openInputStream(imageUri); final Bitmap selectedImage = BitmapFactory.decodeStream(imageStream); src = new Mat(selectedImage.getHeight(), selectedImage.getWidth(), CvType.CV_8UC4); Imgproc.resize(src, src, new Size(src.rows()/4, src.cols()/4)); Utils.bitmapToMat(selectedImage, src); Imgproc.cvtColor(src, src, Imgproc.COLOR_RGBA2BGR); Log.d("MainActivity", "image height " + src.rows() + ", image width " + src.cols()); listImage.add(src); } catch (FileNotFoundException e) { e.printStackTrace(); } } break; } } @Override public void onRequestPermissionsResult(int requestCode, @NonNull String permissions[], @NonNull int[] grantResults) { if (requestCode == REQUEST_READ_EXTERNAL_STORAGE) { // If request is cancelled, the result arrays are empty. if (grantResults.length > 0 && grantResults[0] == PackageManager.PERMISSION_GRANTED) { // permission was granted Log.i("permission", "READ_EXTERNAL_STORAGE granted"); read_external_storage_granted = true; } else { // permission denied Log.i("permission", "READ_EXTERNAL_STORAGE denied"); } } } private void createPanorama(){ new AsyncTask<Void, Void, Bitmap>() { ProgressDialog dialog; @Override protected void onPreExecute() { super.onPreExecute(); dialog = ProgressDialog.show(MainActivity.this, "Building Panorama", "Please Wait"); } @Override protected Bitmap doInBackground(Void... params) { int elems = listImage.size(); long[] tempobjadr = new long[elems]; for (int i=0; i<elems; i++){ tempobjadr[i] = listImage.get(i).getNativeObjAddr(); } Mat result = new Mat(); int stitchstatus = StitchPanorama(tempobjadr, result.getNativeObjAddr()); Log.d("MainActivity", "result height " + result.rows() + ", result width " + result.cols()); if(stitchstatus != 0){ Log.e("MainActivity", "Stitching failed: " + stitchstatus); return null; } Imgproc.cvtColor(result, result, Imgproc.COLOR_BGR2RGBA); Bitmap bitmap = Bitmap.createBitmap(result.cols(), result.rows(), Bitmap.Config.ARGB_8888); Utils.matToBitmap(result, bitmap); return bitmap; } @Override protected void onPostExecute(Bitmap bitmap) { super.onPostExecute(bitmap); dialog.dismiss(); if(bitmap!=null) { ivImage.setImageBitmap(bitmap); } } }.execute(); } @Override protected void onResume() { super.onResume(); } public native int StitchPanorama(long[] imageAddressArray, long outputAddress); } |
原书给出的代码是保留一个空的MainActivity，创建一个StitchingActivity，然后所有的操作在StitchingActivity当中执行。我这里改为全部Java操作只在MainActivity中执行。另外，Java层与Native层之间图像数据的传递方式也产生了一些变化。而且，原书所给代码只提供了一个用于载入图像的菜单按钮，并没有实际提供对从内存中读取的图像进行拼接的功能。我修改为即可以通过摄像头拍照进行拼接，也可以通过读取已有图像的方式来拼接。
3. 修改app\src\main\res\layout\activity_main.xml内容如下：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
<?xml version="1.0" encoding="utf-8"?> <ScrollView xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools" android:layout_width="match_parent" android:layout_height="match_parent" tools:context="net.johnhany.moaap_chp6.MainActivity"> <LinearLayout android:orientation="vertical" android:layout_width="match_parent" android:layout_height="wrap_content"> <ImageView android:layout_width="match_parent" android:layout_height="0dp" android:layout_weight="0.5" android:id="@+id/ivImage" /> <LinearLayout android:layout_width="match_parent" android:layout_height="wrap_content" android:orientation="horizontal"> <Button android:layout_width="match_parent" android:layout_height="wrap_content" android:layout_weight="0.5" android:id="@+id/bClickImage" android:text="Click more images"/> <Button android:layout_width="match_parent" android:layout_height="wrap_content" android:layout_weight="0.5" android:id="@+id/bDone" android:text="Done"/> </LinearLayout> </LinearLayout> </ScrollView> |
4. 修改app/CMakeLists.txt内容如下：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
cmake_minimum_required(VERSION 3.4.1) set(CMAKE_VERBOSE_MAKEFILE on) set(ocvlibs "E:/dev-lib/opencv-contrib-android-sdk/sdk/native/libs") include_directories(E:/dev-lib/opencv-contrib-android-sdk/sdk/native/jni/include) add_library(libopencv_java3 SHARED IMPORTED ) set_target_properties(libopencv_java3 PROPERTIES IMPORTED_LOCATION "${ocvlibs}/${ANDROID_ABI}/libopencv_java3.so") add_library( # Sets the name of the library. stitcher # Sets the library as a shared library. SHARED # Provides a relative path to your source file(s). src/main/cpp/stitcher.cpp ) find_library( # Sets the name of the path variable. log-lib # Specifies the name of the NDK library that # you want CMake to locate. log ) target_link_libraries( # Specifies the target library. stitcher android log libopencv_java3 # Links the target library to the log library # included in the NDK. ${log-lib} ) |
关于该文件中各项的含义，请参考《Android Studio 2.3利用CMAKE进行OpenCV 3.2的NDK开发》。另外，我这里引用的是包含opencv_contrib模块的OpenCV Android SDK，其实直接引用官方预编译SDK也可。
5. 将项目默认产生的app\src\main\cpp\native-lib.cpp文件更名为stitcher.cpp，然后将文件内容修改为：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
#include <jni.h> #include <vector> #include <android/log.h> #include "opencv2/opencv.hpp" using namespace cv; using namespace std; char filepath1[100] = "/storage/emulated/0/Download/MOAAP/Chapter6/panorama_stitched.jpg"; #define LOG_TAG "MOAAP-CHP6" #define LOGD(...) __android_log_print(ANDROID_LOG_DEBUG, LOG_TAG, __VA_ARGS__) #define LOGE(...) __android_log_print(ANDROID_LOG_ERROR, LOG_TAG, __VA_ARGS__) extern "C" { JNIEXPORT jint JNICALL Java_net_johnhany_moaap_1chp6_MainActivity_StitchPanorama(JNIEnv *env, jobject, jlongArray imageAddressArray, jlong outputAddress) { jsize a_len = env->GetArrayLength(imageAddressArray); jlong *imgAddressArr = env->GetLongArrayElements(imageAddressArray,0); vector<Mat> imgVec; for(int k=0; k<a_len; k++) { Mat &curimage = *(Mat*)imgAddressArr[k]; Mat newimage; curimage.copyTo(newimage); float scale = 500.0f / curimage.rows; resize(newimage, newimage, Size((int)(scale*curimage.cols), (int)(scale*curimage.rows))); LOGD("Image height %d width %d", newimage.rows, newimage.cols); imgVec.push_back(newimage); } Mat &result = *(Mat*)outputAddress; Stitcher stitcher = Stitcher::createDefault(); Stitcher::Status status = stitcher.stitch(imgVec, result); LOGD("Result height %d width %d", result.rows, result.cols); imwrite(filepath1, result); return status; } } |
6. 在app\src\main\res目录下创建一个名为menu的Andorid resource directory，再在res\menu中创建一个menu_main.xml文件：
|
1 2 3 4 5 6 7 8 9 |
<?xml version="1.0" encoding="utf-8"?> <menu xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools" xmlns:app="http://schemas.android.com/apk/res-auto"> <item android:id="@+id/action_load_image" android:title="@string/action_load_image" android:orderInCategory="1" app:showAsAction="never" /> </menu> |
7. 修改app\src\main\res\values\strings.xml文件：
|
1 2 3 4 5 6 |
<resources> <string name="app_name">第六章 - 深入OpenCV Android应用开发</string> <string name="action_load_image">Load Image</string> <string name="title_activity_stitching">第六章 - 深入OpenCV Android应用开发</string> <string name="title_activity_home">第六章 - 深入OpenCV Android应用开发</string> </resources> |
8. 修改app\src\main\res\AndroidManifest.xml文件为如下内容：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 |
<?xml version="1.0" encoding="utf-8"?> <manifest xmlns:android="http://schemas.android.com/apk/res/android" package="net.johnhany.moaap_chp6"> <supports-screens android:anyDensity="true" android:largeScreens="true" android:normalScreens="true" android:resizeable="true" android:smallScreens="true" /> <uses-permission android:name="android.permission.CAMERA" /> <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" /> <uses-feature android:name="android.hardware.camera" android:required="false" /> <uses-feature android:name="android.hardware.camera.autofocus" android:required="false" /> <uses-feature android:name="android.hardware.camera.front" android:required="false" /> <uses-feature android:name="android.hardware.camera.front.autofocus" android:required="false" /> <application android:allowBackup="true" android:icon="@mipmap/ic_launcher" android:label="@string/app_name" android:supportsRtl="true"> <activity android:name=".MainActivity" android:label="@string/title_activity_stitching" android:theme="@style/AppTheme"> <intent-filter> <action android:name="android.intent.action.MAIN" /> <category android:name="android.intent.category.LAUNCHER" /> </intent-filter> </activity> </application> </manifest> |
9. 将app\build.gradle文件修改为：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
apply plugin: 'com.android.application' android { compileSdkVersion 24 buildToolsVersion "26.0.1" defaultConfig { applicationId "net.johnhany.moaap_chp6" minSdkVersion 16 targetSdkVersion 24 versionCode 1 versionName "1.0" testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner" externalNativeBuild { cmake { cppFlags "-std=c++11", "-frtti", "-fexceptions" abiFilters 'x86', 'x86_64', 'armeabi', 'armeabi-v7a', 'arm64-v8a', 'mips', 'mips64' } } } sourceSets { main { jniLibs.srcDirs = ['E:\\dev-lib\\opencv-contrib-android-sdk\\sdk\\native\\libs'] } } buildTypes { release { minifyEnabled false proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro' } } externalNativeBuild { cmake { path "CMakeLists.txt" } } } dependencies { compile fileTree(include: ['*.jar'], dir: 'libs') androidTestCompile('com.android.support.test.espresso:espresso-core:2.2.2', { exclude group: 'com.android.support', module: 'support-annotations' }) compile 'com.android.support:appcompat-v7:24.2.1' compile 'com.android.support.constraint:constraint-layout:1.0.2' testCompile 'junit:junit:4.12' compile project(':openCVLibrary320') } |
同样地，这里引用的是包含opencv_contrib模块的OpenCV Android SDK，读者可以直接引用官方预编译SDK。
10. openCVLibrary320\build.gradle文件修改为：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
apply plugin: 'com.android.library' android { compileSdkVersion 24 buildToolsVersion "26.0.1" defaultConfig { minSdkVersion 16 targetSdkVersion 24 } buildTypes { release { minifyEnabled false proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.txt' } } } |
11. 检查一下项目根目录的build.gradle文件是否为如下内容：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
// Top-level build file where you can add configuration options common to all sub-projects/modules. buildscript { repositories { jcenter() } dependencies { classpath 'com.android.tools.build:gradle:2.3.3' // NOTE: Do not place your application dependencies here; they belong // in the individual module build.gradle files } } allprojects { repositories { jcenter() } } task clean(type: Delete) { delete rootProject.buildDir } |
运行效果
测试图片来自https://github.com/opencv/opencv_extra/tree/master/testdata/stitching。
原始图片：
拼接后的效果：
