OpenCV的全称是Open Source Computer Vision Library,是一个跨平台的计算机视觉库。本例的代码和ffmpeg开发教程(4) : 第一个简单例子
非常类似。打开视频后,不是吧视频帧图像保存到硬盘中,而是使用OpenCV来对图像进行处理,比如显示图像的轮廓线。OpenCV本身也可以直接打开视频(OpenCV内部也使用 ffmpeg的函数)。
OpenCV 使用BGR24图像格式(AV_PIX_FMT_BGR24):
sws_getContext ( pCodecContext->width, pCodecContext->height, pCodecContext->pix_fmt, pCodecContext->width, pCodecContext->height, AV_PIX_FMT_BGR24, SWS_BILINEAR, nullptr, nullptr, nullptr );
OpenCV图像数据的基本数据机构是Mat类型,可以直接把AVFrame数据转换成Mat格式:
Mat image(height, width, CV_8UC3, pFrameRGB->data[0], pFrameRGB->linesize[0]);
然后我们就可以使用OpenCV的一些图像处理的函数来对图像进行处理,本例使用了findContours和approxPolyDP 来获取图像的外轮廓,并对外轮廓进行一些平滑处理:
void opencv_get_contours(Mat img, vector<vector<Point> > &contours, vector<Vec4i> &hierarchy) { Mat image; Mat canny_output; int thresh = 100; vector<vector<Point> > contours1; cv::cvtColor(img, image, COLOR_BGR2GRAY); Canny(image, canny_output, thresh, thresh * 2, 3); findContours(canny_output, contours1, hierarchy, RETR_TREE, CHAIN_APPROX_SIMPLE); int total_points = 0; int total_points1 = 0; contours.resize(contours1.size()); for (int i = 0; i < contours1.size(); i++) { total_points += contours1[i].size(); double area = contourArea(contours1[i]); if (area > 8) { approxPolyDP(contours1[i], contours[i], 2, true); } else { contours[i] = contours1[i]; } total_points1 += contours[i].size(); } cout << "Before approxPolyDP: " << total_points << " After approxPolyDP:" << total_points1 << " Shape:" << contours.size() << "\n"; } void opencv_process_frame(int width, int height, AVFrame *pFrameRGB, bool show, int max_size, bool color) { Mat image(height, width, CV_8UC3, pFrameRGB->data[0], pFrameRGB->linesize[0]); cv::Mat greyMat; Mat outImg; int maxSize = max(image.size().width, image.size().height); float scale = (float) max_size / (float) maxSize; cv::resize(image, outImg, cv::Size(), scale, scale); if (!color) { cv::cvtColor(outImg, greyMat, COLOR_BGR2GRAY); outImg = greyMat; } int dst_width = outImg.size().width; int dst_height = outImg.size().height; int length = outImg.total() * outImg.elemSize(); vector<vector<Point> > contours; vector<Vec4i> hierarchy; opencv_get_contours(image, contours, hierarchy); RNG rng(12345); Mat newImage = Mat::zeros(image.size(), CV_8UC3); for (int i = 0; i < contours.size(); i++) { InputArray contour = contours[i]; double area = contourArea(contour); Scalar color = Scalar(255, 255, 255); drawContours(newImage, contours, i, color, 1, 8, hierarchy, 0, Point()); } resizeWindow("opencv", dst_width, dst_height); imshow("opencv", newImage); waitKey(1); imwrite("screen_" + to_string(frame_counter++) + ".jpg", newImage); cout << frame_counter << endl; }
由于OpenCV的运算量比较大,因此在我本机是运行时,视频有些滞后,我们把处理后的图像保存到硬盘上(调用imwrite),然后使用视频处理软件(Adobe Premiere Pro)。得到如下效果: