Evolving Photogrammetry from Single-Pair Stereo to Multi-View rayCloud

Stereophotogrammetry is a technique which mimics the stereo view of human eyes by restoring the 3D positions and orientations of an overlapping image pair. With the filtering of anaglyph red-cyan glasses, 3D glasses, or 3D screens, it allows us to see only one corresponding image out of a two-image pair for each eye, which lets our brain generate the 3D from the parallax.

In traditional workflows, stereo matching based on image pairs is important for retrieving 3D information from the corresponding 2D images. It requires accurate camera interior (focal length, principal point of autocollimation – PPA, lens distortion, etc.) and exterior parameters (geolocation and orientations of the camera at the moment of image acquisition: x, y, z, roll, pitch, yaw) to restore the spatial relations of the image pair which performs a 3D view. Without the accurate parameter values, users will not be able to retrieve or visualize real 3D due to the wrong parallax computed.

Before the boost in drone usage, the majority of mapping projects were done using metric cameras on board airplanes. The interior parameters of those metric cameras are lab-calibrated and much more stable than consumer-grade ones. It was a mandatory process in the photogrammetry workflow after aerial triangulation was done, to check whether the computed camera exteriors were accurate enough to form the stereo view.

Once the accurate interior and exterior parameters were obtained, photogrammetrists would use a stereo plotter to perform the assessment or 3D mapping. However, stereo matching requires expertise and years of training, from picking the right image pairs to extracting precise elevation.

Pix4Dmapper rayCloud Provides Full-3D Vision using Multi-Ray Photogrammetry

Along with the advancement of computer technology, visualization and analysis of photogrammetry have also been improved. The 3D rayCloud view in Pix4Dmapper Pro desktop for example, is capable of presenting interactive 3D object space, thanks to a decent graphics card. Instead of extracting 3D information from stereo pairs after importing the computed camera interiors and exteriors, Pix4D algorithms perform dense matching, automatically or with custom parameters.

In the rayCloud interface, users can visualize all the images contributed for intersecting any selected 3D point. Retrieving the correspondence to all original images gives the possibility to locate a 3D point, more precisely than ever. In this full-3D interface, any imported vectors can be analyzed based on the current project. This is a much easier and more straight-forward way than checking the parallax from selected image pairs in traditional stereo matching.

Moreover, measurement and 3D extraction conducted from different image pairs can contain offsets. The selection of image pair plays an important role in final analysis. For example, picking a short-baseline* image pair from a nadir flight may result in low vertical accuracy, and selecting image pairs from the same flight line may make users underestimate the Y parallax and become over-optimistic of the relative accuracy.

Left: Different image pairs may result in obvious offset when extracting 3D information
Right: Pix4Dmapper Pro’s rayCloud displays the 3D object space from all corresponding images

In Pix4D’s rayCloud, the displayed 3D point cloud is intersected from multiple images instead of one picked image pair. Users can easily assess the calibration result by checking reprojection errors displayed in the properties panel. They can then add manual tie points in the region with larger errors by marking and adjusting the positions on corresponding images. The inserted manual tie point will be displayed in real-time in rayCloud.

This improves the complicated and expensive stereo-matching workflow, prevents human errors and gives better accuracy from multiple-image intersection. In rayCloud, you can also create a fly-through animation of your project with a few simple clicks – it will generate a path and export a video which records your views in 3D.

Using rayCloud for Applications Previously Relying on Stereo-Pair Matching

Without going into pair-to-pair assessment, rayCloud displays the entire project where users can easily perform analysis and measurements. By relying on stereo matching, some applications can now be easily accomplished in Pix4Dmapper Pro’s rayCloud: urban and cadastral mapping, environmental monitoring and forestry inventory, infrastructure measurement, watershed management, and much more.

From left to right: vector assessment and generation in rayCloud; accurate and detailed point cloud intersected from multiple images; precise measurements can be done without years of stereo-matching training; digitisation of water regions is important for terrain and watershed management

For urban or cadastral mapping cases, camera interior and exterior calibration results can be assessed by checking the reprojection error of any 3D point in the properties panel. Similar procedures can be used for analysing existing 3D vectors by importing them into the project to compare.

Mountain and forests are two tough regions for photogrammetry. Instead of spending days extracting 3D information from image pairs, computer vision technology finds the correct 3D position using multiple images. For example, it is easier to find the same leaf based on numerous images which cover various aspects, than distinguishing it on a stereo image pair, even with manual interpretations.

To precisely retrieve the elevation from stereo models requires expertise, even then risking human error. Now the rayCloud enables precise measurements by simple clicks in the reconstructed 3D space, plus the cross-check in all corresponding images. With this new method, generalisation and digitisation have never been so time efficient! Results generated in rayCloud can be preserved digitally, and the accuracy of all procedures can be assessed with displayed error values, preventing human interpretations which potentially create conflict.

3D Vision in a Future Reality World

With the quick development of 3D technologies, virtual reality (VR) and augmented reality (AR) have become popular topics, not just for the gaming industry. For industrial inspection, virtual reality from drone-mapped images enable a highly-detailed yet low-cost monitoring, without putting personnel in danger. For indoor mapping, augmented reality combined with real mapping information will also bring technology to the next level!

Pix4D has been improving its image processing ability non-stop for the past few years. The rayCloud is a big leap in the combination of photogrammetry and 3D visualisation. Paired with the advancement of related rendering technology, we aim to provide great user experiences in the future reality world. Stay with us – more development is coming soon!

—————————————————————————————–
* Baseline indicates the distance formed by two perspective centers, a key element influencing the vertical accuracy. For more information regarding the baseline and base-height ratio of image pairs, please read our article – “Are You Getting the Accuracy You Expect with Pix4Dmapper Pro?”