Niladri Shekhar Dutt

I am a first year ELLIS PhD student at University College London (UCL), sponsored by Adobe Research. I am fortunate to be advised by Niloy Mitra (UCL) and Duygu Ceylan (Adobe). My research lies at the interesction of computer vision, computer graphics, and machine learning; I’m particularly interested in 2D/3D generative modelling by leveraging LLMs for reasoning.

Email Scholar Github Twitter LinkedIn

News

Oct 2024: Presented my research at The Alan Turing Institute PhD day
Oct 2024: Started a PhD in Computer Science at UCL
July 2024: Temporal Residual Jacobians has been accepted at ECCV’24
Mar 2024: ProteusNeRF has been accepted at I3D’24
Feb 2024: Diffusion 3D Features has been accepted at CVPR’24

Research

Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Features

Niladri Shekhar Dutt, Sanjeev Muralikrishnan, Niloy Mitra

CVPR 2024

A feature distiller that harnesses the expressive power of in-painting diffusion features and distills them to points on 3D surfaces to produce semantic descriptors that attain SOTA correspondence accurracy while being zero-shot.

Project Page PDF arXiv

Temporal Residual Jacobians for Rig-free Motion Transfe

Sanjeev Muralikrishnan, Niladri Shekhar Dutt, Siddhartha Chaudhuri, Noam Aigerman, Vladimir Kim, Matthew Fisher, Niloy Mitra

ECCV 2024

A representation to enable data-driven motion transfer motions without the need for any rigging or intermediate shape keyframes.

Project Page PDF arXiv

ProteusNeRF: Fast Lightweight NeRF Editing using 3D-Aware Image Context

Binglun Wang, Niladri Shekhar Dutt, Niloy Mitra

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (I3D) 2024

A fast and lightweight framework for interactive editing of NeRF assets via existing image manipulation tools or generative frameworks. This is enabled by a novel 3D-aware image context that allows linking edits across multiple views.

Project Page PDF arXiv

Advancing the Representation and Biological Interpretation of Cell Painting Readouts for Toxicity Prediction

Srijit Seal, Niladri Shekhar Dutt, Jordi Carreras-Puigvert, Maria-Anna Trapotsi, Ola Spjuth, Andreas Bender

Society of Toxicology Annual Congress 2023, 62nd Annual Meeting, Nashville, USA

Represented cell morphology in grid form layout to use CNNs to exploit spatial information from highly correlated Cell Painting features to predict biological toxicity and study biological interpretations.

PDF

Experience

Research Engineer
London, UK

Oct 2023 - Sep 2024

Ready Player Me is a Series B cross-app avatar platform for the metaverse backed by a16z.

Worked on Generative AI x 3D, avatar generation from a single image, texture generation, identitiy preservation in stylized generaive modelling.

Founding Engineer
Tallinn, Estonia

Oct 2020 - Sep 2022

Led machine learning at NFTPort from its inception through its Series A of $26M.

Developed computer vision based NFT search and duplicate detection system.

Developed and maintained the core API at an API first company.

Research Intern
Bangalore, India

Jan 2020 - May 2020

Researched how regularity affects learning in GANs by studying modelling on synthetic data.

CNeRG Lab | IIT Kharagpur
Research Intern

Jun 2019 - Jul 2019

Worked on autonomous driving research funded by HP using CARLA simulator.

Used computer vision sequential models based on Conditional Affordance Learning, keyframe extraction and subset selection to replicate SOTA results on 40x reduced data.

CITRIS Policy Lab | University of California, Berkeley
Undergraduate Researcher

Feb 2019 - May 2019

Aggregated tweets in clusters based on social network analysis.

Combined LDA with clustering on social media interactions to improve topic coherence by 20%.

Service

Reviewer
NeurIPS'24
CVPR'24,'25
ICLR'25
AAAI'25