My personal website.
There’s been lots of talk lately on how to make Europe more competitive. Grand strategies and and ambitious initiatives are often proposed. Yet these discussions often overlook the small administrative and practical hurdles entrepreneurs face daily.
Haoran Chen extended vision-language feature fields to also segment instances of objects, creating an open-vocabulary panoptic segmentation system. This means we can not only figure out which parts of the scene are relevant for a query, but also tell how many of those relevant objects there are. The work was published in IEEE Robotics and Automation Letters.
Nicolas Gorlo presented his work on few-shot instance segmentation and re-identification at WACV 2024. I had the great pleasure of advising Nicolas on this project.
Neural Implicit Vision-Language Feature Fields are an approach to dense scene understanding. It is a neural implicit scene representation with additional vision-language feature outputs, which can be correlated with text prompts, enabling zero-shot open-vocabulary scene segmentation. You can basically type in the thing you are looking for, and it will highlight the relevant parts of the scene.
Recently, the Luxonis OAK-D has come up as a contender as a go-to RGB-D sensor. It’s cheap, has high frame rate global shutter image sensors, a high quality rolling shutter RGB camera and it comes with onboard compute through the Intel Movidius chip.
Autolabel is a tool to produce dense segmentation maps for RGB-D video through volumetric segmentation. Behind the scenes, it uses a semantic NeRF model to infer a 3D segmentation of the scene from very sparse user labels. The learned representation is then used to render dense segmentation maps for given viewpoints.
The Stray CLI and Studio is a software toolkit for solving computer vision problems. It allows you to reconstruct 3D scenes from RGB-D video, annotate the scenes with high-level semantic information and fit computer vision algorithms to the annotated scenes.
Object Keypoints is a simple method to build a model that can track pre-defined points on objects in 3D. It includes a way to scan the objects using a robot arm, a tool to annotate the points, a script to fit the model and a pipeline to detect the 3D points at runtime using the trained model. It can handle multiple objects in a frame simultanously and does not require object instance segmentation.
Update 17th of March 2021: Updated links, expanded some sections.
Stray Scanner is an app for collecting RGB-D datasets with an iPhone or iPad with a LiDAR sensor.