publications

2020

1. PhD Thesis

   Learning to extract features for 2D–3D multimodal registration

   Alba Pujol-Miró

   Universitat Politècnica de Catalunya, 2020

   Abs Bib HTML PDF

   The ability to capture depth information form an scene has greatly increased in the recent years. 3D sensors, traditionally high cost and low resolution sensors, are being democratized and 3D scans of indoor and outdoor scenes are becoming more and more common.

   However, there is still a great data gap between the amount of captures being per- formed with 2D and 3D sensors. Although the 3D sensors provide more information about the scene, 2D sensors are still more accessible and widely used. This trade- off between availability and information between sensors brings us to a multimodal scenario of mixed 2D and 3D data.

   This thesis explores the fundamental block of this multimodal scenario: the reg- istration between a single 2D image and a single unorganized point cloud. An unorganized 3D point cloud is the basic representation of a 3D capture. In this representation the surveyed points are represented only by their real word coordi- nates and, optionally, by their colour information. This simplistic representation brings multiple challenges to the registration, since most of the state of the art works leverage the existence of metadata about the scene or prior knowledges.

   Two different techniques are explored to perform the registration: a keypoint-based technique and an edge-based technique. The keypoint-based technique estimates the transformation by means of correspondences detected using Deep Learning, whilst the edge-based technique refines a transformation using a multimodal edge detection to establish anchor points to perform the estimation.

   An extensive evaluation of the proposed methodologies is performed. Albeit fur- ther research is needed to achieve adequate performances, the obtained results show the potential of the usage of deep learning techniques to learn 2D and 3D similari- ties. The results also show the good performance of the proposed 2D-3D iterative refinement, up to the state of the art on 3D-3D registration.

   @phdthesis{citekey,
     entry_type = {PhD Thesis},
     author = {Pujol-Miró, Alba},
     title = {Learning to extract features for 2D–3D multimodal registration},
     school = {Universitat Politècnica de Catalunya},
     year = {2020},
   }

2019

1. Journal Article

   Correspondence matching in unorganized 3D point clouds using Convolutional Neural Networks

   Alba Pujol-Miró, Josep R Casas, and Javier Ruiz-Hidalgo

   Image and Vision Computing, 2019

   Abs Bib HTML PDF

   This document presents a novel method based in Convolutional Neural Net- works (CNN) to obtain correspondence matchings between sets of keypoints of several unorganized 3D point cloud captures, independently of the sen- sor used. The proposed technique extends a state-of-the-art method for correspondence matching in standard 2D images to sets of unorganized 3D point clouds. The strategy consists in projecting the 3D neighborhood of the keypoint onto an RGBD patch, and the classification of patch pairs us- ing CNNs. The objective evaluation of the proposed 3D point matching based in CNNs outperforms existing 3D feature descriptors, especially when intensity or color data is available.

   @article{pujol2019correspondence,
     entry_type = {Journal Article},
     title = {Correspondence matching in unorganized 3D point clouds using Convolutional Neural Networks},
     author = {Pujol-Miró, Alba and Casas, Josep R and Ruiz-Hidalgo, Javier},
     journal = {Image and Vision Computing},
     volume = {83},
     pages = {51--60},
     year = {2019},
     publisher = {Elsevier},
     doi = {https://doi.org/10.1016/j.imavis.2019.02.013},
   }

2017

1. Conference Paper

   Registration of images to unorganized 3D point clouds using contour cues

   Alba Pujol-Miro, Javier Ruiz-Hidalgo, and Josep R Casas

   In 2017 25th European Signal Processing Conference (EUSIPCO), 2017

   Abs Bib HTML PDF

   Low resolution commercial 3D sensors contribute to computer vision tasks even better when the analysis is carried out in a combination with higher resolution image data. This requires registration of 2D images to unorganized 3D point clouds. In this paper we present a framework for 2D-3D data fusion to obtain directly the camera pose of a 2D color image in relation to a 3D point cloud. It includes a novel multiscale intensity feature detection algorithm and a modified ICP procedure based on point-to-line distances. The framework is generic for several data types (such as CAD designs or LiDAR data without photometric information), and results show that performance is comparable to the state of the art, while avoiding manual markers or specific patterns on the data.

   @inproceedings{pujol2017registration,
     entry_type = {Conference Paper},
     title = {Registration of images to unorganized 3D point clouds using contour cues},
     author = {Pujol-Miro, Alba and Ruiz-Hidalgo, Javier and Casas, Josep R},
     booktitle = {2017 25th European Signal Processing Conference (EUSIPCO)},
     pages = {81--85},
     year = {2017},
     organization = {IEEE},
     doi = {http://hdl.handle.net/2117/110974},
     isbn = {978-0-9928626-7-1},
   }

2016

1. Conference Paper

   Interactive registration method for 3D data fusion

   Arantxa Casanova, Alba Pujol-Miró, Javier Ruiz-Hidalgo, and 1 more author

   In 2016 International Conference on 3D Imaging (IC3D), 2016

2015

1. Master thesis

   Detection of objects above the ground in automotive images

   Alba Pujol Miró

   Universitat Autònoma de Barcelona, 2015

   Abs Bib

   This master thesis explores the 3-D reconstruction capabilities of the scene around a moving vehicle. Using as a baseline the depth reconstruction techniques available with stereo cameras, this project provides a pipeline that allows a 3-D reconstruction with an unconstrained moving monocular camera. The results obtained show that the technique presented is suitable to be implemented as an improvement to current ADAS algorithms.

   @mastersthesis{citekez,
     entry_type = {Master thesis},
     author = {Miró, Alba Pujol},
     title = {Detection of objects above the ground in automotive images},
     school = {Universitat Autònoma de Barcelona},
     year = {2015},
   }

2014

1. Grad thesis

   Real-time image stitching for automotive 360º vision systems

   Alba Pujol Miró

   Universitat Politècnica de Catalunya, 2014

   Abs Bib HTML PDF

   This document presents the development of a 360º system adapted to buses. From four cameras located around the vehicle, a bird’s eye top view is created to get the driver the vehicle’s surroundings information. The development has been focused on implementing a warping algorithm that produces the best alignment between camera images. This project analyses the state-of-the-art warping algorithms. Based on the results obtained, an adapted method to the specific 360º vision system is designed and implemented. In addition, this project also has tested different calibration patterns -both in 3D virtual and real environments- and a calibration pattern for the final application is proposed. This project has been carried out in a multidisciplinary UPC team. The developments included in this project are part of the work done in the GPI (Image Processing Group) team. This team has been working in a 1-year project commissioned by the Arcol company. This project main goal is to develop a camera-and-display based guidance system for the bus driver.

   @mastersthesis{citekfa,
     entry_type = {Grad thesis},
     author = {Miró, Alba Pujol},
     title = {Real-time image stitching for automotive 360º vision systems},
     school = {Universitat Politècnica de Catalunya},
     year = {2014},
   }