ffmpeg开发教程(6) : 使用OpenCV处理图像

OpenCV的全称是Open Source Computer Vision Library,是一个跨平台的计算机视觉库。本例的代码和ffmpeg开发教程(4) : 第一个简单例子
非常类似。打开视频后,不是吧视频帧图像保存到硬盘中,而是使用OpenCV来对图像进行处理,比如显示图像的轮廓线。OpenCV本身也可以直接打开视频(OpenCV内部也使用 ffmpeg的函数)。

OpenCV 使用BGR24图像格式(AV_PIX_FMT_BGR24)

sws_getContext
                    (
                            pCodecContext->width,
                            pCodecContext->height,
                            pCodecContext->pix_fmt,
                            pCodecContext->width,
                            pCodecContext->height,
                            AV_PIX_FMT_BGR24,
                            SWS_BILINEAR,
                            nullptr,
                            nullptr,
                            nullptr
                    );

OpenCV图像数据的基本数据机构是Mat类型,可以直接把AVFrame数据转换成Mat格式:

Mat image(height, width, CV_8UC3, pFrameRGB->data[0], pFrameRGB->linesize[0]);

然后我们就可以使用OpenCV的一些图像处理的函数来对图像进行处理,本例使用了findContours和approxPolyDP 来获取图像的外轮廓,并对外轮廓进行一些平滑处理:

void opencv_get_contours(Mat img, vector<vector<Point> > &contours, vector<Vec4i> &hierarchy) {
    Mat image;
    Mat canny_output;
    int thresh = 100;
    vector<vector<Point> > contours1;
    cv::cvtColor(img, image, COLOR_BGR2GRAY);
    Canny(image, canny_output, thresh, thresh * 2, 3);

    findContours(canny_output, contours1, hierarchy,
                 RETR_TREE, CHAIN_APPROX_SIMPLE);

    int total_points = 0;
    int total_points1 = 0;
    contours.resize(contours1.size());
    for (int i = 0; i < contours1.size(); i++) {
        total_points += contours1[i].size();
        double area = contourArea(contours1[i]);
        if (area > 8) {
            approxPolyDP(contours1[i], contours[i], 2, true);
        } else {
            contours[i] = contours1[i];
        }
        total_points1 += contours[i].size();
    }
    cout << "Before approxPolyDP: " << total_points
         << " After approxPolyDP:" << total_points1 << " Shape:" << contours.size() << "\n";

}

void
opencv_process_frame(int width, int height, AVFrame *pFrameRGB, bool show,
                     int max_size, bool color) {
    Mat image(height, width, CV_8UC3, pFrameRGB->data[0], pFrameRGB->linesize[0]);
    cv::Mat greyMat;
    Mat outImg;

    int maxSize = max(image.size().width, image.size().height);
    float scale = (float) max_size / (float) maxSize;
    cv::resize(image, outImg, cv::Size(), scale, scale);
    if (!color) {
        cv::cvtColor(outImg, greyMat, COLOR_BGR2GRAY);
        outImg = greyMat;
    }
    int dst_width = outImg.size().width;
    int dst_height = outImg.size().height;
    int length = outImg.total() * outImg.elemSize();


    vector<vector<Point> > contours;
    vector<Vec4i> hierarchy;

    opencv_get_contours(image, contours, hierarchy);
    RNG rng(12345);
    Mat newImage = Mat::zeros(image.size(), CV_8UC3);
    for (int i = 0; i < contours.size(); i++) {
        InputArray contour = contours[i];
        double area = contourArea(contour);
        Scalar color = Scalar(255, 255, 255);
        drawContours(newImage, contours, i, color, 1, 8, hierarchy, 0, Point());
    }

    resizeWindow("opencv", dst_width, dst_height);
    imshow("opencv", newImage);
    waitKey(1);
    imwrite("screen_" + to_string(frame_counter++) + ".jpg", newImage);
    cout << frame_counter << endl;

}

由于OpenCV的运算量比较大,因此在我本机是运行时,视频有些滞后,我们把处理后的图像保存到硬盘上(调用imwrite),然后使用视频处理软件(Adobe Premiere Pro)。得到如下效果: