Niladri Shekhar Dutt

I am an incoming ELLIS PhD student at University College London (UCL) advised by Prof. Niloy Mitra and sponsored by Adobe Research. My research lies at the interesction of computer vision, computer graphics, and machine learning; I’m particularly interested in 3D/4D generative modeling.

Previous research experiences: Nvidia, University of California, Berkeley, IIT Kharagpur.

Previous work experiences: Ready Player Me (Research Engineer), NFTPort (Founding Engineer; raised $26M Series A)

Email Scholar Github Twitter LinkedIn

News

Research

Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Features

Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Features

Niladri Shekhar Dutt, Sanjeev Muralikrishnan, Niloy Mitra
CVPR 2024

A feature distiller that harnesses the expressive power of in-painting diffusion features and distills them to points on 3D surfaces to produce semantic descriptors that attain SOTA correspondence accurracy while being zero-shot.

Project Page PDF arXiv
Temporal Residual Jacobians for Rig-free Motion Transfe

Temporal Residual Jacobians for Rig-free Motion Transfe

Sanjeev Muralikrishnan, Niladri Shekhar Dutt, Siddhartha Chaudhuri, Noam Aigerman, Vladimir Kim, Matthew Fisher, Niloy Mitra
ECCV 2024

A representation to enable data-driven motion transfer motions without the need for any rigging or intermediate shape keyframes.

Project Page PDF arXiv
ProteusNeRF: Fast Lightweight NeRF Editing using 3D-Aware Image Context

ProteusNeRF: Fast Lightweight NeRF Editing using 3D-Aware Image Context

Binglun Wang, Niladri Shekhar Dutt, Niloy Mitra
ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (I3D) 2024

A fast and lightweight framework for interactive editing of NeRF assets via existing image manipulation tools or generative frameworks. This is enabled by a novel 3D-aware image context that allows linking edits across multiple views.

Project Page PDF arXiv
Advancing the Representation and Biological Interpretation of Cell Painting Readouts for Toxicity Prediction

Advancing the Representation and Biological Interpretation of Cell Painting Readouts for Toxicity Prediction

Srijit Seal, Niladri Shekhar Dutt, Jordi Carreras-Puigvert, Maria-Anna Trapotsi, Ola Spjuth, Andreas Bender
Society of Toxicology Annual Congress 2023, 62nd Annual Meeting, Nashville, USA

Represented cell morphology in grid form layout to use CNNs to exploit spatial information from highly correlated Cell Painting features to predict biological toxicity and study biological interpretations.

PDF

Experience

Research Engineer
Research Engineer
London, UK
Oct 2023 - Present
  • Ready Player Me is a Series B cross-app avatar platform for the metaverse backed by a16z.
  • Working on Generative AI x 3D, avatar generation from a single image, texture generation.
    • Smart Geometry Processing Group, UCL
    Smart Geometry Processing Group, UCL
    Research Assistant
    April 2023 - Present
  • Developed 3D semantic descriptors which achieve SOTA correspondence accuracy.
  • Editing NeRFs at interactive rates (10-30x speedup) using text or image processing tools.
    • Founding Engineer
    Founding Engineer
    Tallinn, Estonia
    Oct 2020 - Sep 2022
  • Led machine learning at NFTPort from its inception through its Series A of $26M.
  • Developed computer vision based NFT search and duplicate detection system.
  • Developed and maintained the core API at an API first company.
    • Research Intern
    Research Intern
    Bangalore, India
    Jan 2020 - May 2020
  • Researched how regularity affects learning in GANs by studying modelling on synthetic data.
    • CNeRG Lab | IIT Kharagpur
    CNeRG Lab | IIT Kharagpur
    Research Intern
    Jun 2019 - Jul 2019
  • Worked on autonomous driving research funded by HP using CARLA simulator.
  • Used computer vision sequential models based on Conditional Affordance Learning, keyframe extraction and subset selection to replicate SOTA results on 40x reduced data.
    • CITRIS Policy Lab | University of California, Berkeley
    CITRIS Policy Lab | University of California, Berkeley
    Undergraduate Researcher
    Feb 2019 - May 2019
  • Aggregated tweets in clusters based on social network analysis.
  • Combined LDA with clustering on social media interactions to improve topic coherence by 20%.


    • Service

    Reviewer
    NeurIPS, 2024
    CVPR, 2024
    Elsevier Neural Networks