Modern Aerial Photogrammetry in Practice

Summary

Over the past 25 years, aerial photogrammetry has evolved from an analog/analytical to a fully digital technology. This transformation enables the provision of highly precise spatial data, which is essential for the creation of digital twins and decision-making in areas such as urbanization, climate change, and energy production. Modern aerial photogrammetry encompasses advanced technologies such as laser scanning and digital aerial cameras, which allow for efficient capture and processing of foundational geospatial data. Significant developments in the field include the introduction of hybrid sensor systems that combine both imaging and laser scanning technologies. Trends indicate increasing detail accuracy, more frequent flight repetitions, the capture of larger areas, and the integration of multisensor systems. Aerial photogrammetry plays a crucial role in providing geospatial data that is indispensable for creating spatial digital twins and deriving new products and solutions.

1. Introduction

Aerial photogrammetry is one of the most important sources for providing foundational spatial data. However, in public perception, the technology often leads a shadowy existence and is frequently equated with satellite data, as seen with Google Maps. Examining the developments in this field over the past 25 years reveals an impressive example of how a technology has completely transitioned from the analog world to the digital realm. In the face of rapid societal changes, such as urbanization, climate change, or the decarbonization of energy production, it is increasingly important to simulate processes and use digital representations or twins for decision-making. The foundation of all spatial digital twins, whether for municipalities, cities, countries, or entire continents, is current geodata, which can only be provided in the required quality by aerial (laser) surveying. The advancement of technology must address these challenges through the evolution of sensors and the optimization of process chains. The goal must be to provide decision-makers with data from modern sensors in a processed form so that the correct conclusions can be drawn.

2. Modern Aerial Photogrammetry

Modern aerial photogrammetry has developed rapidly over the past decades. In the 1990s, analytical photogrammetry was partially digitized. Analog aerial images could be digitized with special high-precision and high-resolution scanners and then digitally processed in the first digital photogrammetric workstations. Additionally, a new technology emerged: laser scanning. In the late 1990s, it became possible to directly capture elevation data over larger areas. The processing capabilities of the captured data were significantly improved by the development of powerful computer systems, and GPS enabled near-real-time positioning, allowing for the direct orientation of captured aerial images. The final step towards full digitization of aerial photogrammetry occurred in 2000 when both Leica and Zeiss-Intergraph introduced the first digital aerial cameras, the ADS40 and the DMC-1, at the ISPRS Congress in Amsterdam. In 2003, Vexcel introduced its first digital aerial system, the UltraCam-D. The three companies pursued different approaches in developing their digital aerial cameras. The Leica ADS was designed as a line scanner based on the three-line principle, the Zeiss-Intergraph DMC used four individual cameras combined into a larger aerial image, and Vexcel's UltraCam used the "syntopic" imaging concept to create a large aerial image from four individual cameras and nine sub-images. Zeiss-Intergraph and Vexcel adhered to the principle of area-based aerial imagery, as in analog photogrammetry, while Leica took a new path with the three-line pushbroom principle (Cramer, 2006; Leberl, 2012; Tempelmann et al., 2000).

A significant step towards democratizing aerial photogrammetry occurred with the advent of cost-effective and easy-to-operate drones as platforms for cameras. This led to the emergence of a new class of aerial image capture in the 2010s. Flight altitudes are generally limited to around 100 meters by regulation, and range is restricted by the required line of sight and limited flight time to a few hundred meters. This new class is ideal for capturing smaller areas of up to a few square kilometers. With increasingly better lightweight camera systems, data from drones can compete with those from "classic" aerial surveying. Driven by a significantly larger market, new approaches for processing aerial image data emerged from the drone market in conjunction with algorithms developed in computer vision (SIFF (Scale-invariant feature transform) (Lowe, D. G., 2004) or SURF (Speeded Up Robust Features)). These new methods, specifically for efficient and robust determination of tie points, allowed new image configurations to be processed together, removing the limitation to the normal case of photogrammetry.

In the field of airborne camera sensors, new configurations were developed alongside large-format area sensors, optimized for use in urban areas: oblique or oblique aerial systems. Typically, four oblique cameras (usually between 35 and 45 degrees) are combined with a nadir-looking camera in a Maltese cross arrangement. The purpose of this new sensor configuration was the combined capture of nadir images and oblique views, especially of built-up areas, for the creation of 3D city models. All relevant sensor manufacturers (Leica Geosystems, Vexcel, IGI, or PhaseOne) followed this "trend" and brought corresponding systems to market. In the area of area sensors, the development focused on making image sensors larger and simultaneously reducing the cycle time between consecutive images to capture image data more efficiently. In parallel, a new sensor class emerged for capturing very large areas, optimized to capture as many image elements as possible across the flight direction, often at the expense of a poorer base/height ratio. Examples of this sensor class, typically used for pure orthophoto production in content programs, include the Leica Geosystems ContentMapper and the Vexcel UltraCam Condor.

Another significant milestone in the evolution of modern digital aerial photogrammetry was the development of Semi-Global Matching (SGM) by Hirschmüller in 2005 (Hirschmüller, 2005). This improved the quality of elevation data derived from image data to pixel-accurate 3D point clouds, comparable to the results of laser scanning for surface areas. The SGM approach was quickly adopted and further developed, becoming the standard for DSM generation in almost all photogrammetric solutions.

Parallel to the development in image-based surveying from the air, the development of airborne laser scanners began in the 1990s. Starting with scanners with scan rates in the range of a few thousand points per second and limited multi-echo capability, the systems evolved into high-performance scanners with scan rates of over 2 million points per second and the ability to split more than 15 echoes per pulse. Parallel developments in laser technology, such as single-photon laser scanners or Geiger-mode laser scanners, enabled highly efficient capture of large areas with high point densities. Airborne laser scanners thus became a perfect complement to traditional aerial photogrammetry. Initially, there was strong competition between the two approaches, but over time, the conviction grew that the complementary technologies together represent the perfect combination. Consequently, the first hybrid aerial system, consisting of a laser scanner and camera system in one sensor, was developed and introduced by Leica Geosystems in 2016 with the CityMapper.

3. Trends

The developments in aerial photogrammetry described in the previous section have led to noticeable changes in the entire airborne surveying industry. Six areas have shown changes for some time, which will continue or even intensify (Figure 1):

1. More Details: The resolutions of aerial surveys will continue to increase. Large-scale surveys will be conducted with resolutions of 10 cm or better. For urban areas, a GSD of 5 cm will establish itself as a standard. LiDAR missions will be conducted with at least 10 pt/m² or significantly more, with initial trends indicating a need for LiDAR data with more than 40 pt/m².

2. More Frequent Repetitions and Fast Delivery: The cycles for large-scale surveys will be shortened to 1 to 2 years. For urban areas, there is a trend towards annual or even more frequent surveys. The importance of LiDAR data is increasing, and there is also a need for shorter repetition cycles in the range of 2 to 3 years. For timely use of the data, it is crucial to make the results of the survey quickly available. The time from survey to final delivery is becoming shorter, sometimes within 30 to 90 days for large projects.

3. Larger Areas: The project sizes for survey projects are increasing, with a trend towards rapid and uniform capture of entire countries. This applies to both LiDAR and aerial surveys.

4. Multisensor: Until recently, there was a strict separation between image and LiDAR surveys. This has changed with the development of new hybrid sensors. There is an increasing need for simultaneous capture of complementary image and LiDAR data. This trend will continue and likely intensify.

5. More AI: AI is already indispensable in many areas. Currently, applications in airborne surveying are still limited, primarily to the evaluation side, such as image analysis or point cloud classification. We will see further improvements and implementations in these areas, as well as many new applications. AI will play a significant role in optimizing flight planning, automatic quality control, and deriving new products.

6. Added Value: In the future, the creation of classic aerial surveying products will no longer be sufficient. Customers will increasingly demand finished solutions and direct information extraction. End-to-end solutions are becoming more prevalent, with everything from data capture to processing, analysis, and presentation of results being requested from a single source.

All these changes and new requirements for aerial surveying are only achievable if all components of the process chain are optimally coordinated and seamlessly integrated. Efficient sensors optimized for the respective application, combined with a high degree of automation, are the guarantors of high-quality data and derived products.

4. Sensors

Modern aerial sensors are characterized by a high degree of specialization. There are three main classes of aerial sensors for the following applications:

1. Urban Mapping: Capturing cities to create 3D city models.

2. Classical Aerial Photogrammetry: Capturing orthophotos, digital terrain models (DTM), and digital surface models (DSM), as well as 3D stereo evaluation.

3. Large Area Capture: Capturing very large areas for orthophoto production.

Each of these applications requires a different type of sensor. For urban mapping, a combination of oblique and nadir-looking cameras is needed. The size of the nadir camera's footprint is less important, while a narrower field of view (FoV) is advantageous. For classical aerial photogrammetry, a sensor that covers a large area in the flight direction is needed to achieve a good base/height ratio for stereo evaluation and DSM generation. Additionally, a sensor with enough image elements across the flight direction is necessary for economic capture, without causing excessive tilting at the image edges due to a large FoV. For large area capture, sensors with as many image elements as possible across the flight direction are preferred, with the base/height ratio being less relevant, as well as the tilting issue at the image edges. Details on sensor sizes and FoVs can be found in Table 1. Most sensors have models with different focal lengths to achieve comparable results from different flight altitudes.

In addition to imaging sensors, airborne laser scanners have gained increasing importance. Unlike imaging sensors, laser scanners are active systems that use a laser pulse to scan the landscape. As an active system, a laser scanner does not require sunlight and can generate a 3D point with a single measurement. Thus, data can be captured with laser scanners at night or under challenging lighting conditions. Additionally, a laser scanner can penetrate vegetation by using small gaps between leaves and branches to capture the entire 3D structure of forests, including the ground, for creating highly accurate digital terrain models (DTM). This makes laser scanners a complementary technology to image-based data capture, providing excellent synergy. The latest airborne sensor systems combine imaging sensors with laser scanners into hybrid sensors to combine the best of both worlds (Bacher, 2022). Examples of modern hybrid sensors include Leica Geosystems' CityMapper-2 and CountryMapper, or Vexcel's UltraCam Dragon 4.1. The CityMapper and Dragon are combinations of nadir and oblique cameras with a laser scanner, primarily for urban mapping, while the CountryMapper is designed for capturing large landscapes with both large-format images and laser scanning. By capturing image and laser scanning data simultaneously, the weaknesses of each subsystem can be compensated. For example, the laser scanner can capture the 3D structure of forests, while the image data provides information about the type and condition of the forest (Figure 2).

In urban mapping, each of the three subsystems contributes to creating a 3D city model. Additionally, the high horizontal accuracy of image-based data is combined with the high vertical accuracy of laser data to create a balanced 3D dataset. Details on the current sensor systems from major manufacturers can be found under Leica Geosystems, 2024; Vexcel, 2024; PhaseOne, 2024; and IGI Integrated Systems, 2024.

5. Workflow

The trends in modern aerial photogrammetry make it clear that the high expectations for the technology can only be met if all components and subprocesses work together optimally. Projects are becoming larger, resolutions are getting smaller, and the time from capture to delivery is getting shorter. This apparent contradiction can only be resolved through high automation in the processes. Considering the entire project workflow from flight planning to the delivery of finished products, there are several areas with significant potential for optimization. The data capture area has already been discussed in the sensor development section, showing how more efficient sensors can reduce flight times. The area currently showing the most significant changes is data processing, from raw data to finished products. Only a few subprocesses, such as measuring control points, require manual assistance; most processes are highly automated. There is a trend towards (partial) outsourcing of processing to cloud-based infrastructure. This allows computationally intensive processes to be scaled well, reducing the time from capture to delivery to a minimum. This shift to the cloud is becoming more common, especially for tasks that are hardware (sensor) independent, such as creating true orthophotos or 3D city models. For sensor-near processes, such as ingesting raw data into actual image or LiDAR data, local computing power is still predominantly used. More and more prepared cloud solutions are being provided for creating follow-up products.

6. Data and Products

Modern aerial photogrammetry serves more than ever as a supplier of foundational spatial data. In addition to classic products such as orthophotos (DOP), stereo models, and digital terrain models (DTM), a whole range of new products has emerged. New data products include photogrammetrically generated point clouds (pDOM), where 3D point clouds are created using "Dense Image Matching" based on Semi-Global Matching, generating a 3D coordinate for each image element. The point cloud is typically colored with the values of the corresponding pixels. The resulting point cloud largely corresponds to a digital surface model, except that vertical structures, such as facades, can also be reconstructed depending on the image data used. Regularly rasterized digital surface models (DSM) are derived from the pDOM. Due to their high level of detail and edge sharpness, pDOMs are excellent for creating true orthophotos (TDOP). Initially, TDOPs were mainly used in urban mapping as an alternative to classic orthophotos, but they are now increasingly offered for larger areas. The advantage of TDOPs over classic DOPs lies in the correct geometry of all visible objects in the image, completely eliminating tilting effects and associated blind spots. It is important to note that the flight effort for TDOPs is greater (minimum 50% side overlap) than for classic DOPs (typically 30% side overlap).

For optimal use of high-quality 3D data, the 3D mesh has established itself as a data product, initially for city models and later for larger areas. The advantage of a 3D mesh over an orthophoto or point cloud is its detailed three-dimensionality combined with the visual quality of an image product. Sometimes a 3D mesh is referred to as a 3D ortho. Artificial intelligence has also made its way into aerial photogrammetry, allowing for the derivation and analysis of new products and information. An example is the operational generation of land cover classifications. While this is not new, the corresponding neural networks are trained in such a way that very different landscape types and input data can be analyzed without needing to train new models each time. In combination with hybrid datasets, even 3D classifications are possible, where the surface can be assigned to a different class than the ground. An example would be a road partially shaded by trees, where the DSM is assigned to the tree class and the ground to the road (or sealed surface) class.

7. From Sensor to Solution

For many users of geodata, the foundational data described above is not enough, or they prefer the direct integration or interaction of foundational data with other data and corresponding analyses in dedicated solutions. Generally speaking, the results of aerial photogrammetry form the basis for spatial digital twins (Figure 3). They provide the framework in which all other information is stored. In addition to serving as a spatial reference for orientation, photogrammetric data is an important component of the digital twin itself. For example, sealing maps can be created from the data, which, in combination with terrain models, weather data, and runoff models, can simulate heavy rainfall events in detail at the local level.

Another example of how aerial photogrammetry data can help make cities future-proof in the face of climate change is the creation of a tree cadastre and the resulting modeling of shading. Conversely, it can also simulate potential areas that could become heat islands under strong sunlight. The data for this comes from aerial sensors and is highly automated, processed, and made available to potential users as dashboards. Users can focus on their specific tasks without needing special photogrammetric or GIS knowledge.

8. Outlook

Considering the rapid developments in aerial photogrammetry technology, the question arises of how the technology will continue to evolve. It is anticipated that sensor technology will continue to improve, enabling the construction of cameras with more and more image elements. This will allow for better resolutions with comparable capture effort. However, this also means that more data will need to be processed, stored, and transmitted. Higher resolutions are not always sensible and bring new problems. For example, in cities, image data is often captured with resolutions better than 5 cm. In this resolution range, considerations of privacy and data protection must always be taken into account. Not everything that is technically possible is always sensible or legally compliant. The trend towards multisensor systems in a single sensor is already in full swing, with camera systems combined with LiDAR (Leica CityMapper/CountryMapper) or optical systems combined with thermal sensors (IGI EcoMapper). More interesting combinations are likely to emerge in the future. Another point is the use of artificial intelligence in aerial photogrammetry. Its use will intensify in all areas of the process chain, providing valuable assistance in flight planning, quality assurance, and information extraction. These capabilities will increasingly be integrated into the sensors themselves. This will improve quality and significantly shorten the time from planning to delivery.

References

• Bacher, U. [2022]: HYBRID AERIAL SENSOR DATA AS BASIS FOR A GEOSPATIAL DIGITAL TWIN, Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., Vol. XLIII-B4-2022

• Cramer, M [2006]: DIGITAL AIRBORNE CAMERAS – STATUS AND FUTURE, in Proceedings ISPRS workshop "High resolution earth imaging for geospatial information", University of Hannover, Hannover/Germany, May 17-20, 2005

• Hirschmüller, H. [2005]: ACCURATE AND EFFICIENT STEREO PROCESSING BY SEMI-GLOBAL MATCHING AND MUTUAL INFORMATION. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 807-814

• Leberl, F. [2012]: THE ULTRACAM STORY, International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXIX-B1, 2012

• Lowe, D. G. [2004]: DISTINCTIVE IMAGE FEATURES FROM SCALE-INVARIANT KEYPOINTS, International Journal of Computer Vision, 60(2), 91-110

• Tempelmann, U et al [2000]: PHOTOGRAMMETRIC SOFTWARE FOR THE LH SYSTEMS ADS40 AIRBORNE DIGITAL SENSOR, International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B2. Amsterdam 2000

• Vexcel [2024]: Vexcel Imaging Product Overview, https://www.vexcel-imaging.com/products/ (last accessed September 2024)

• Leica Geosystems [2024]: Leica Geosystems Product Overview, https://leica-geosystems.com/de-de/products/airborne-systems (last accessed September 2024)

• PhaseOne [2024]: PhaseOne Product Overview, https://www.phaseone.com/solutions/geospatial-solutions/aircraft-systems/ (last accessed September 2024)

• IGI Integrated Systems [2024]: IGI Product Overview, https://www.igi-systems.com (last accessed September 2024)

Note:

The original version of the article was published in German in the Schriftenreihe des Instituts für Geodäsie an der Universität der Bundeswehr in November 2024. All rights are with the author. Copies of the article or of parts needs the approval of the Author.

Author:

Dr. Uwe Bacher

Technical Director Hexagon Innovation Hub