Abstract

Model-based 3D reconstruction of animals from images enables the estimation of 3D shape and pose within parametric spaces, supporting a wide range of downstream tasks. Yet this problem remains challenging: it requires expressive models that can generalize across the large diversity of animal species. About ten years ago, we introduced SMAL (Skinned Multi-Animal Linear Model), a simple multi-species parametric model that has since been widely adopted to estimate 3D shape and pose directly from visual data. In this talk, I will discuss the limitations of SMAL and present AWOL (Analysis WithOut synthesis using Language), a framework that leverages natural language to steer parametric 3D models and improve controllability. Finally, I will show how synthetic data generated with AWOL can be used to train systems that predict 3D shape and pose across a broad range of animal species.

Bio of Speaker: Silvia Zuffi is a Senior Research Scientist at the IMATI (Institute of Applied Mathematics and Information Technologies) of the Italian National Research Council (CNR) in Milan. She holds a PhD in Computer Science from Brown University, where her thesis focused on “Shape models of the human body for distributed inference” under the supervision of Prof. Michael J. Black. Her academic background also includes a Master of Science in Computer Science from Brown University and a degree in Electronic Engineering from the University of Bologna. She held also a postdoctoral position at the Perceiving Systems group of the Max Planck Institute for Intelligent Systems (Tübingen, Germany). Her research lies at the intersection of computer vision, graphics, and machine learning, with a special emphasis on pose and shape estimation for humans and animals from images and video data. Over her career, she has contributed to realistic modeling of articulated bodies, and the development of generative models for animal 3D shape and pose reconstruction. Her earlier work includes research in color imaging, multispectral imaging, and visual perception. Silvia’s ongoing projects address challenges in reconstructing 3D shape and pose of animals from “in the wild” imagery, contributing tools for applications in ecology, conservation, animal wellness and visual computing.