Camera Intrinsics Matrix: Why Two Focal Lengths?

• Introduction
• Step-by-Step Guide
• Code Example
• Additional Notes
• Summary
• Conclusion
• References

In computer vision, the camera intrinsics matrix plays a crucial role in translating between the 3D world and 2D images. A key aspect of this matrix is the representation of focal length, often denoted by two values: fx and fy. This distinction arises from the nature of camera sensors, where the pixels, the fundamental building blocks of an image, are not always perfectly square.

The camera intrinsics matrix often uses two focal length values (fx, fy) because it needs to account for the possibility of non-square pixels on the camera sensor.

Imagine a camera sensor where the physical dimensions of each pixel are not perfectly square. This means that a change of 1 unit in the x-direction on the image sensor might not correspond to the same physical distance as a change of 1 unit in the y-direction.

To accurately map 3D points in the real world to 2D points on the image plane, the camera intrinsics matrix needs to incorporate this difference in pixel dimensions. That's where fx and fy come in.

fx represents the focal length in terms of pixel width, while fy represents the focal length in terms of pixel height.

If the pixels were perfectly square, then fx and fy would be equal. However, in many real-world cameras, there's a slight difference, and using distinct fx and fy values ensures a more accurate geometric representation of the camera's projection.

This Python code demonstrates the projection of a 3D point in the real world onto a 2D image plane using a camera intrinsics matrix. The code accounts for non-square pixels by using separate focal length values (fx, fy) for the width and height. It calculates the 2D point in homogeneous coordinates and then converts it to pixel coordinates.

import numpy as np

# Example camera intrinsics matrix with non-square pixels
fx = 1200  # Focal length in pixel width units
fy = 1000  # Focal length in pixel height units
cx = 640   # Principal point x-coordinate
cy = 480   # Principal point y-coordinate

# Construct the camera intrinsics matrix
K = np.array([
    [fx, 0, cx],
    [0, fy, cy],
    [0, 0, 1]
])

# Example 3D point in the real world
point_3d = np.array([1, 2, 3])

# Project the 3D point onto the image plane
point_2d_homogeneous = K @ point_3d 
point_2d = point_2d_homogeneous[:2] / point_2d_homogeneous[2]

print("Camera Intrinsics Matrix (K):\n", K)
print("3D Point:", point_3d)
print("Projected 2D Point:", point_2d)

Explanation:

1. Camera Intrinsics Matrix (K):

   • fx and fy are the focal lengths in terms of pixel width and height, respectively. They account for the non-square pixel dimensions.
   • cx and cy represent the principal point (the optical center of the camera) in pixel coordinates.

2. 3D Point: A sample 3D point in the real world.

3. Projection:

   • We multiply the camera intrinsics matrix K with the 3D point to project it onto the image plane.
   • The result is a 2D point in homogeneous coordinates.

4. Homogeneous to Pixel Coordinates:

   • We divide the first two elements of the homogeneous coordinates by the third element to get the 2D point in pixel coordinates.

Key Point:

• Using distinct fx and fy values in the camera intrinsics matrix ensures that the projection from 3D to 2D accurately reflects the physical dimensions of the camera sensor's pixels, even if they are not perfectly square. This leads to more accurate geometric representations and measurements in computer vision applications.

• Pixel Aspect Ratio: The difference between fx and fy is directly related to the pixel aspect ratio of the camera sensor. A ratio of 1 indicates square pixels, while deviations from 1 signify non-square pixels.
• Calibration: The values of fx, fy, cx, and cy are specific to each camera and are determined through camera calibration.
• Distortion: In reality, camera lenses introduce distortions. The camera intrinsics matrix often includes additional parameters to model and correct these distortions for accurate image formation.
• Units: It's important to note that fx and fy are expressed in pixel units, not physical units like millimeters. This means they represent the focal length relative to the size and shape of the pixels on the sensor.
• Applications: Understanding the role of fx and fy is crucial for various computer vision tasks, including:
  • 3D Reconstruction: Accurately estimating the 3D structure of a scene from 2D images.
  • Augmented Reality: Superimposing virtual objects onto the real world in a geometrically consistent manner.
  • Camera Pose Estimation: Determining the position and orientation of the camera in the world.

Simplified Analogy:

Imagine a projector projecting an image onto a screen. If the screen's pixels are rectangular, the projected image will appear stretched. fx and fy act like scaling factors, adjusting the projected image to compensate for the non-square pixels, ensuring the image appears undistorted.

In conclusion, the use of two focal length values, fx and fy, in the camera intrinsics matrix is essential for accurately representing the geometry of cameras with non-square pixels. This distinction accounts for the different physical dimensions represented by a single pixel in the horizontal and vertical directions on the image sensor. By incorporating fx and fy, the camera intrinsics matrix enables precise mapping between 3D points in the real world and their corresponding 2D projections on the image plane, ultimately contributing to the accuracy of computer vision applications such as 3D reconstruction, augmented reality, and camera pose estimation.

• Dissecting the Camera Matrix, Part 3: The Intrinsic Matrix ← | Aug 13, 2013 ... Using pixel units for focal length and principal point offset allows us to represent the relative dimensions of the camera, namely, the film's ...
• opencv - How to find camera's intrinsic matrix from focal length ... | Feb 28, 2024 ... Peak detection in a 2D array · 46 · Why does the focal length in the camera intrinsics matrix have two dimensions? 2 · Camera intrinsics matrix ...
• Mono camera SLAM, Can I get camera intrinsics matrix from random ... | Posted by u/LordDaniel09 - 2 votes and 2 comments
• Getting 2 values of focal length when finding Intrinsic camera matrix ... | Nov 22, 2022 ... I believe you have an Fx and a Fy. This is so that the the matrix transforms on f can scale f in two directions x and y. IIRC this is why you get 2 f numbers.
• scripting - How can I set the focal length independent for x and y ... | Sep 26, 2018 ... A Blender camera does have a different focal length (or expressed as an angle, field of view / FOV) in the X and Y, however, ...
• camera coordinate to pixel coordinate opencv - Stack Overflow | Jul 21, 2016 ... When I use this camera matrix to multiply a point in camera ... Why does the focal length in the camera intrinsics matrix have two dimensions?
• Camera intrinsics: Axis skew -- Solution to Exercise 1.6, Computer ... | Feb 28, 2019 ... The intrinsic parameters of a camera consist of the focal length, pixel dimensions ... have 2 axes as the point lies on the 2D image plane.
• Is is possible to estimate real depth from single image? : r ... | Posted by u/pakitomasia - 33 votes and 35 comments
• Is there a definitive source for DJI drone camera specifications? | DJI ... | Is there a definitive source for DJI drone camera specifications such as sensor size (in pixels) or true focal length (not the 35mm equivalent)? I know that DJI specifies the camera's sensor size using notation such as 1/1.3 but I am looking for the actual sensor size in mm for all DJI drones...