My work spans computer vision and computer graphics to computational geometry and topology and to remote sensing. With an overall goal of 3D shape analysis and understanding, my research interests include the acquisition, reconstruction and processing of 3D moving shapes. These shapes range from human bodies and organs to animals or even mechanical parts. Currently, I am especially interested in trees and plants, because of their challenging geometric and topological complexities.
3D shape analysis and understanding of plants and trees
For a computer to automatically analyze and "understand" the geometric shape of real objects such as plants (which is sometimes called digital geometry processing), several stages are necessary. Click on each picture to get an overview of my contributions to each of them.
3D data acquisition
The geometry of the input real object or plant, and possibly other information such as its colours or textures, must be transferred into the computer. This is usually done using passive systems such as cameras (a process called photogrammetry) or active ones such as laser scanners. The result is most often a 3D point cloud or a 3D mesh.
Data cleaning
The acquired data is usually noisy and incomplete because of limitations in the acquisition process (hidden parts due to occlusions, windy conditions making the plant move, etc.).
Segmentation into organs
The geometry of the plant is often complex. Subdividing it into simpler parts (ideally, its organs) helps the computer to analyze it.
Geometric model fitting
Once the 3D points of each plant organ have been isolated from the others, fitting a continuous geometric model such as a cylinder or a spline surface to them is often needed. It makes subsequent measurements easier and more robust to missing data or non-uniform sampling.
Motion/growth tracking
When several successive 3D acquisitions of a plant or tree are available, it is possible to track its motion over time. Depending on the time scale, this can be e.g., a balancing motion due to windy conditions, or the growth motion.
Once these stages have been completed, several plant traits can easily be estimated and tracked over time. They include, among others:
- the number of leaves, fruits, etc.;
- the length of the main stem [3DV'21,FPS'22] or other branches [3DV'21];
- branching angles;
- the area of individual leaf blades and the leaf area index [IJRS'16,GVC'22,FPS'22];
- the volume of the canopy.
As an example, the UrTrees project is a citizen science project that allows to compute the main features of a tree from a single smartphone video. Check the video!
Current students
- Steve de Rose (from October 2023). Constructing temporally-consistent 3D mesh models of growing plants. Other advisor: Rémi Allègre (Univ. Strasbourg). Collaboration with Julien Derr (ENS Lyon).
- Evan Zarrella (from October 2023). Estimating changes in the National Forest Inventory from heterogeneous and multimodal 3D data. Other advisors: Joris Ravaglia and Pierre-Alexis Herrault (Univ. Strasbourg).
Past work
Topics
- Understanding digital shapes from the life sciences
- Digital geometry processing for shapes in motion
- Geometric, topological and perceptual analysis of 3D meshes
Graduate students
Resources
Talk
- An introduction to digital geometry processing (in French): part 1, part 2.
Talk given in 2014 to secondary/high school students and teachers.
Software
- Plant leaf area estimation (PhD work of Mélinda Boukhana, related paper)
- 4D plant analysis (Master work of Haolin Pan, related paper)
- 3DtLaplace (related paper)
Data sets
- Chenopodium album data set (PhD work of Katia Mirande, acquisitions by Marie Tisserand, Julie Charlaix and Fabrice Besnard, related paper)
- Arabidopsis 3D+t data set (Master work of Haolin Pan, acquisitions by Julie Charlaix, related paper)
- Animation skeletons for various animals (related paper)
