📍 London, UK

Niladri Shekhar Dutt

I am a Research Engineer at Ready Player Me working on Generative AI x 3D. I completed my master’s in Computer Vision at University College London (UCL) advised by Prof. Niloy Mitra. Previously, I was a Founding Engineer at NFTPort leading machine learning from its inception through its Series A round of $26M. Prior to that, I was a visiting student at the University of California, Berkeley where I worked with Dr Brandie Nonnecke on ML x social network analysis. I have also spent time as a research intern at Nvidia and IIT Kharagpur.

I am interested in generating and editing 3D content (particularly meshes that are compatible with graphics engines) using text instructions, images or other simple modalities. I believe solving this requires a fundamental understanding of shape design- what makes a sofa well-designed and functional?


Will we ever answer The Last Question? Perhaps not in our lifetime, but it's definitely worth pursuing :)
“Break the cycle, Morty. Rise above. Focus on Science.”

Research


Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Features

Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Features

Niladri Shekhar Dutt, Sanjeev Muralikrishnan, Niloy Mitra
CVPR 2024

A feature distiller that harnesses the expressive power of in-painting diffusion features and distills them to points on 3D surfaces to produce semantic descriptors that attain SOTA correspondence accurracy while being zero-shot.

ProteusNeRF: Fast Lightweight NeRF Editing using 3D-Aware Image Context

ProteusNeRF: Fast Lightweight NeRF Editing using 3D-Aware Image Context

Binglun Wang, Niladri Shekhar Dutt, Niloy Mitra
ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (I3D) 2024

A fast and lightweight framework for interactive editing of NeRF assets via existing image manipulation tools or generative frameworks. This is enabled by a novel 3D-aware image context that allows linking edits across multiple views.

Advancing the Representation and Biological Interpretation of Cell Painting Readouts for Toxicity Prediction

Advancing the Representation and Biological Interpretation of Cell Painting Readouts for Toxicity Prediction

Srijit Seal, Niladri Shekhar Dutt, Jordi Carreras-Puigvert, Maria-Anna Trapotsi, Ola Spjuth, Andreas Bender
Society of Toxicology Annual Congress 2023, 62nd Annual Meeting, Nashville, USA

Represented cell morphology in grid form layout to use CNNs to exploit spatial information from highly correlated Cell Painting features to predict biological toxicity and study biological interpretations.


Experience


Research Engineer
Research Engineer
London, UK
Oct 2023 - Present
  • Ready Player Me is a Series B cross-app avatar platform for the metaverse backed by a16z.
  • Working on Generative AI x 3D, avatar generation from a single image, texture generation.
  • Smart Geometry Processing Group, UCL
    Smart Geometry Processing Group, UCL
    Research Assistant
    April 2023 - Present
  • Developed 3D semantic descriptors which achieve SOTA correspondence accuracy.
  • Editing NeRFs at interactive rates (10-30x speedup) using text or image processing tools.
  • Founding Engineer
    Founding Engineer
    Tallinn, Estonia
    Oct 2020 - Sep 2022
  • Led machine learning at NFTPort from its inception through its Series A of $26M.
  • Developed computer vision based NFT search and duplicate detection system.
  • Developed and maintained the core API at an API first company.
  • Research Intern
    Research Intern
    Bangalore, India
    Jan 2020 - May 2020
  • Researched how regularity affects learning in GANs by studying modelling on synthetic data.
  • CNeRG Lab | IIT Kharagpur
    CNeRG Lab | IIT Kharagpur
    Research Intern
    Jun 2019 - Jul 2019
  • Worked on autonomous driving research funded by HP using CARLA simulator.
  • Used computer vision sequential models based on Conditional Affordance Learning, keyframe extraction and subset selection to replicate SOTA results on 40x reduced data.
  • CITRIS Policy Lab | University of California, Berkeley
    CITRIS Policy Lab | University of California, Berkeley
    Undergraduate Researcher
    Feb 2019 - May 2019
  • Aggregated tweets in clusters based on social network analysis.
  • Combined LDA with clustering on social media interactions to improve topic coherence by 20%.


  • Service


    Reviewer
    CVPR, 2024
    Elsevier Neural Networks