Methoden zur Reduktion der Messlatenz von GOBO-Projektor-basierten 3D-Sensoren. - Ilmenau : Universitätsbibliothek, 2022. - 1 Online-Ressource (95 Seiten)
Technische Universität Ilmenau, Dissertation 2022
Genaue, optische 3D-Messverfahren werden vielfältig in der Industrie, der Medizin und der Wissenschaft eingesetzt. Bei etlichen dieser Anwendungen ist eine schnelle Reaktion auf Veränderungen in der Messszene erforderlich. Dies ist z.B. der Fall, wenn eine Maschine aus Sicherheitsgründen abgeschaltet oder angehalten werden muss oder eine direkte Rückmeldung an einen Menschen gegeben werden soll. Steht das 3D-Ergebnis der Messung nach hinreichend kurzer Zeit ab Beginn der Messung zur Verfügung, kann die Reaktion basierend auf diesem Ergebnis angestoßen werden. Das Prinzip des GOBO-Projektor-basierten, aktiven Stereo-Sensors hat sich als genaues, optisches 3D-Messverfahren etabliert. Bei diesem Verfahren wird ein sich zeitlich änderndes, aperiodisches Streifenmuster auf das Messobjekt projiziert, während zwei kalibrierte Kameras jeweils eine Bildsequenz synchron aufnehmen. Innerhalb dieser Bildsequenzen werden anschließend die Abbilder von Objektpunkten, welche in beiden Kameras sichtbar sind, einander zugeordnet. Für jedes solche Paar wird dann die 3D-Koordinate des zugehörenden Objektpunktes trianguliert. Das Verfahren erlaubt auch 3D-Aufnahmen mit speziellen Anforderungen, die von anderen, genauen 3D-Sensorprinzipien nur schwer erreicht werden. Dazu gehört die Messung mit speziellen Lichtwellenlängen, wie z.B. dem Nah-Infrarotbereich, womit blendfreie Vermessung ermöglicht wird, oder die Erfassung sehr schneller Prozesse, wie die Messung von Airbag-Entfaltungen. Bisher war es jedoch nicht möglich, die 3D-Messergebnisse in so kurzer Zeit (z.B. 100 ms), d.h. mit so kurzer Messlatenz, zur Verfügung zu stellen, dass eine unmittelbare Reaktion auf eine Veränderung der Messszene erfolgen kann. Diese Verkürzung der Messlatenz ist das Ziel dieser Arbeit. Es werden Methoden beschrieben und untersucht, mit denen die Messlatenz von GOBO-Projektor-basierten, aktiven Stereo-Sensoren auf unter 100 ms verkürzt werden kann. Die Verbesserungen konzentrieren sich auf zwei Bereiche: die schnelle Rekonstruktion des 3D-Modells aus den aufgenommenen Bildsequenzen und die Reduktion der Aufnahmezeit durch Verkürzung der Bildsequenz-Länge. Letztere wird mittels einer Optimierung der Musterprojektion ermöglicht, welche bei kurzen Bildsequenz-Längen eine erhebliche Reduktion unerwünschter Messartefakte bewirkt. Abschließend werden mehrere Anwendungen gezeigt, die von diesen Verbesserungen profitieren.
https://doi.org/10.22032/dbt.53040
The duality of ray-based and pinhole-camera modeling and 3D measurement improvements using the ray-based model. - In: Sensors, ISSN 1424-8220, Bd. 22 (2022), 19, 7540, S. 1-15
Geometrical camera modeling is the precondition for 3D-reconstruction tasks using photogrammetric sensor systems. The purpose of this study is to describe an approach for possible accuracy improvements by using the ray-based-camera model. The relations between the common pinhole and the generally valid ray-based-camera model are shown. A new approach to the implementation and calibration of the ray-based-camera model is introduced. Using a simple laboratory setup consisting of two cameras and a projector, experimental measurements were performed. The experiments and results showed the possibility of easily transforming the common pinhole model into a ray-based model and of performing calibration using the ray-based model. These initial results show the model’s potential for considerable accuracy improvements, especially for sensor systems using wide-angle lenses or with deep 3D measurements. This study presents several approaches for further improvements to and the practical usage of high-precision optical 3D measurements.
https://doi.org/10.3390/s22197540
Contactless heart rate measurement in newborn infants using a multimodal 3D camera system. - In: Frontiers in Pediatrics, ISSN 2296-2360, Bd. 10 (2022), 897961, S. 01-11
Richtiger Name des 5. Verfassers: Gunther Notni
Newborns and preterm infants require accurate and continuous monitoring of their vital parameters. Contact-based methods of monitoring have several disadvantages, thus, contactless systems have increasingly attracted the neonatal communities' attention. Camera-based photoplethysmography is an emerging method of contactless heart rate monitoring. We conducted a pilot study in 42 healthy newborn and near-term preterm infants for assessing the feasibility and accuracy of a multimodal 3D camera system on heart rates (HR) in beats per min (bpm) compared to conventional pulse oximetry. Simultaneously, we compared the accuracy of 2D and 3D vision on HR measurements. The mean difference in HR between pulse oximetry and 2D-technique added up to + 3.0 bpm [CI−3.7 - 9.7; p = 0.359, limits of agreement (LOA) ± 36.6]. In contrast, 3D-technique represented a mean difference in HR of + 8.6 bpm (CI 2.0-14.9; p = 0.010, LOA ± 44.7) compared to pulse oximetry HR. Both, intra- and interindividual variance of patient characteristics could be eliminated as a source for the results and the measuring accuracy achieved. Additionally, we proved the feasibility of this emerging method. Camera-based photoplethysmography seems to be a promising approach for HR measurement of newborns with adequate precision; however, further research is warranted.
https://doi.org/10.3389/fped.2022.897961
FPGA-based multi-view stereo system with flexible measurement setup. - In: Measurement: sensors, ISSN 2665-9174, Bd. 24 (2022), 100425, S. 1-9
In recent years, stereoscopic image processing algorithms have gained importance for a variety of applications. To capture larger measurement volumes, multiple stereo systems are combined into a multi-view stereo (MVS) system. To reduce the amount of data and the data rate, calculation steps close to the sensors are outsourced to Field Programmable Gate Arrays (FPGAs) as upstream computing units. The calculation steps include lens distortion correction, rectification and stereo matching. In this paper a FPGA-based MVS system with flexible camera arrangement and partly overlapping field of view is presented. The system consists of four FPGA-based passive stereoscopic systems (Xilinx Zynq-7000 7020 SoC, EV76C570 CMOS sensor) and a downstream processing unit (Zynq Ultrascale ZU9EG SoC). This synchronizes the sensor near processing modules and receives the disparity maps with corresponding left camera image via HDMI. The subsequent computing unit calculates a coherent 3D point cloud. Our developed FPGA-based 3D measurement system captures a large measurement volume at 24 fps by combining a multiple view with eight cameras (using Semi-Global Matching for an image size of 640 px × 460 px, up to 256 px disparity range and with aggregated costs over 4 directions). The capabilities and limitation of the system are shown by an application example with optical non-cooperative surface.
https://doi.org/10.1016/j.measen.2022.100425
Investigations on the potential application of machine vision lenses for depth measurement by exploiting chromatic aberrations. - In: Measurement: sensors, ISSN 2665-9174, Bd. 23 (2022), 100410, S. 1-9
Chromatic (spectral) aberrations are image imperfections that are disadvantageous for standard image processing tasks and are typically compensated through the application of different types of glass during lens design. The longitudinal chromatic aberration causes a relative unsharpness over different spectral channels. Since this error is corrected in most multi-chromatic lenses, this paper investigates to which extent the shift of the focal planes in a standard lens can be used specifically for image processing applications. Theoretical investigations of the longitudinal chromatic aberration are carried out. Based on this, conditions and a method to generate a 3D depth reconstruction out of different spectral channels are presented.
https://doi.org/10.1016/j.measen.2022.100410
High-resolution 3D shape measurement with extended depth of field using fast chromatic focus stacking. - In: Optics express, ISSN 1094-4087, Bd. 30 (2022), 13, S. 22590-22607
Close-range 3D sensors based on the structured light principle have a constrained measuring range due to their depth of field (DOF). Focus stacking is a method to extend the DOF. The additional time to change the focus is a drawback in high-speed measurements. In our research, the method of chromatic focus stacking was applied to a high-speed 3D sensor with 180 fps frame rate. The extended DOF was evaluated by the distance-dependent 3D resolution derived from the 3D-MTF of a tilted edge. The conventional DOF of 14 mm was extended to 21 mm by stacking two foci at 455 and 520 nm wavelength. The 3D sensor allowed shape measurements with extended DOF within 44 ms.
https://doi.org/10.1364/OE.454856
Automatic detection and prediction of discontinuities in laser beam butt welding utilizing deep learning. - In: Journal of advanced joining processes, ISSN 2666-3309, Bd. 6 (2022), 100119, S. 1-11
Laser beam butt welding of thin sheets of high-alloy steel can be really challenging due to the formation of joint gaps, affecting weld seam quality. Industrial approaches rely on massive clamping systems to limit joint gap formation. However, those systems have to be adapted for each individually component geometry, making them very cost-intensive and leading to a limited flexibility. In contrast, jigless welding can be a high flexible alternative to substitute conventionally used clamping systems. Based on the collaboration of different actuators, motions systems or robots, the approach allows an almost free workpiece positioning. As a result, jigless welding gives the possibility for influencing the formation of the joint gap by realizing an active position control. However, the realization of an active position control requires an early and reliable error prediction to counteract the formation of joint gaps during laser beam welding. This paper proposes different approaches to predict the formation of joint gaps and gap induced weld discontinuities in terms of lack of fusion based on optical and tactile sensor data. Our approach achieves 97.4 % accuracy for video-based weld discontinuity detection and a mean absolute error of 0.02 mm to predict the formation of joint gaps based on tactile length measurements by using inductive probes.
https://doi.org/10.1016/j.jajp.2022.100119
A new sensor system for accurate 3D surface measurements and modeling of underwater ojects. - In: Applied Sciences, ISSN 2076-3417, Bd. 12 (2022), 9, 4139, S. 1-15
A new underwater 3D scanning device based on structured illumination and designed for continuous capture of object data in motion for deep sea inspection applications is introduced. The sensor permanently captures 3D data of the inspected surface and generates a 3D surface model in real time. Sensor velocities up to 0.7 m/s are directly compensated while capturing camera images for the 3D reconstruction pipeline. The accuracy results of static measurements of special specimens in a water basin with clear water show the high accuracy potential of the scanner in the sub-millimeter range. Measurement examples with a moving sensor show the significance of the proposed motion compensation and the ability to generate a 3D model by merging individual scans. Future application tests in offshore environments will show the practical potential of the sensor for the desired inspection tasks.
https://doi.org/10.3390/app12094139
Overview of the state of the art in the digitization of drivable forestry roads. - In: Image Sensing Technologies: Materials, Devices, Systems, and Applications IX, (2022), 120910A, S. 120910A-1-120910A-10
Mobile mapping becomes a more and more important and interesting field of sensing technologies and their application scenarios. Various applications range from airborne sensing of specific environments and characteristics to ground-based applications such as multimodal 3-dimensional registration of environments in the infrastructure sector or for assistance systems. In the specific case of infrastructure systems, known fields of application range from the detection of the surface condition of roads to the digitization of entire railroad lines, including their clearance diagrams. From the technical point of view, it also combines a wide variety of sensory approaches for sensing relevant features. For example, known systems use both LiDAR and GNSS and image processing-based subsystems. This work summarizes the state-of-the-art mobile mapping technologies in the framework of road detection and digitization concerning the application of georeferenced condition monitoring. In the first part, the relevant historical development will be briefly reviewed and compared regarding technological progress furthermore, various sensing systems will be compared regarding their applications, applicability and limitations. The aim is to clearly identify shortcomings regarding the application case of road detection in the forestry sector and thus to lay the foundation for subsequent research and development work for multimodal sensing systems. It is also the starting point for upcoming work for a multimodal sensing system that is able to digitalize and characterize the structure of forestry trails. The data obtained in this way will later be used for a planning tool that will derive measures for the maintenance and repair these forest roads.
https://doi.org/10.1117/12.2622738
Investigation on surface inspection using polarizing image sensors. - In: Image Sensing Technologies: Materials, Devices, Systems, and Applications IX, (2022), 120910F, S. 120910F-1-120910F-13
Surface inspection in industrial automated processes is very often challenging. Especially the detection of transparent liquid materials such as water represent a major challenge for standard imaging systems. One approach to overcome the limitation of these imaging systems lies in the exploitation of the polarization effect. This effect surely can only be applied if the contaminants have polarizing features but can help to use invisible characteristics of light for quality inspection tasks. In this work investigations on surfaces which are contaminated with water will be presented. Therefore, an imaging system using an RGB dome light illumination was set up in combination with a four-channel polarizing camera. The dome light, which is equipped with three different LED wavelengths, will be mixed so that the illumination which hits the sample is completely unpolarized. So, any effect on the surface which leads to a polarizing effect can be observed. The system delivers a four-channel image with different polarization angles that have to be processed. Therefore, an algorithm realizes a demosaicing which separates the four different polarized pixels into individual images. Based on this, the stokes equation which allows the calculation of the degree of polarization and the angle of polarization has to be processed for the image presentation. To achieve a better visualization of the degree of polarization an HSV-transformation based on the polarization parameters was also realized.
https://doi.org/10.1117/12.2620501