Which 4 distortion parameters are needed for a fisheye camera?

[Michael-Manning]

Related:
alicevision/AliceVision#45
#921
#1484
#1647
#1649

I am attempting to apply the distortion parameters created by the CameraCalibration utility/OpenCV to the CameraInit node for a fisheye camera.

The CameraCalibration utility provides:

• image size
• focal
• principal coordinates
• K1 K2 K3 radial distortion coefficients

Additionally, OpenCV will provide the p1 and p2 tangential distortion coefficients.

The common theme among all the related issues is that the parameters should be applied in this order:
0 - k1
1 - k2
2 - k3
3 - p1
4 - p2

However, as pointed out in #1649, the camera type you select might be what determines which distortion parameters are required. The camera type tooltip states that radial3 contains the 3 radial distortion parameters and that brown uses the additional 2 tangential parameters. This is all coherent with the linked issues related to this topic, but what about fisheye cameras which require 4 parameters?

How do I Identify and acquire the 4 distortion parameters for a fisheye camera?

Thank you

[Michael-Manning]

After doing some digging, it turns out OpenCV actually supports 6 radial distortion coefficients, not just 3. Although I can't get OpenCV to generate similar coefficients to what the Meshroom calibration utility generates.

The Alicevision source code for calibration will allow you ask for up to 6 coefficients, but the the error message says the max is actually 3, which is very confusing.

cvCalibFlags |= cv::CALIB_ZERO_TANGENT_DIST;
if (nbDistortionCoef < 1 || nbDistortionCoef > 6)
	throw boost::program_options::invalid_option_value(std::string("Only supports 2 or 3 radial coefs: ") + std::to_string(nbDistortionCoef));
const std::array<int, 6> fixDistortionCoefs = {cv::CALIB_FIX_K1, cv::CALIB_FIX_K2, cv::CALIB_FIX_K3, cv::CALIB_FIX_K4, cv::CALIB_FIX_K5, cv::CALIB_FIX_K6};
for (int i = nbDistortionCoef; i < 6; ++i)
	cvCalibFlags |= fixDistortionCoefs[i];

In the actual calibration module of the source, the flags for fixing coefficients 4-6 (which I believe forces them to be zero) are overwritten before calling the OpenCV calibration function

  const double rms = cv::calibrateCamera(objectPoints, imagePoints, imageSize, cameraMatrix,
                                         distCoeffs, rvecs, tvecs, cvCalibFlags | cv::CALIB_FIX_K4 | cv::CALIB_FIX_K5 | cv::CALIB_FIX_K6);

Clearly something is not lining up here. Should I submit this as a bug?

[simogasp]

In opencv k4-k6 are not radial distortion coefficients, they refer to other coefficients in the Rational model.
https://docs.opencv.org/3.4/d9/d0c/group__calib3d.html#ga3207604e4b1a1758aa66acb6ed5aa65d
That's why the message sounds ambiguous. Only K1 K2 K3 will be taken into account. If you pass cv::CALIB_FIX_K4 it will only take into account if CALIB_RATIONAL_MODEL is also passed. (ie not sure rational model is supported in AV)
So it's better to pass only 3 radial distortion coefficients and, possibly, 2 tangential coefficients.

That being said, it also depends of what kind of fisheye you are using, fisheye camera are rarely well modeled by a pinhole camera with the radial+tangential distortion unless the distortion is mild.

[Michael-Manning]

This makes sense. Now I think the model I should really be using equidistant_r3 model anyways (the initial pair isn't found when I use this setting but that's a different issue)

Thanks for answering my question

[qume]

This makes sense. Now I think the model I should really be using equidistant_r3 model anyways (the initial pair isn't found when I use this setting but that's a different issue)

I'm having the same issue with the initial pair using the equidistant_r3 model. Did you manage to find a way around it?