Machine learning · Computer vision
Yanwei Fu 付彦伟
Full Professor (tenured)
School of Data Science, Fudan University
Faculty, Shanghai Innovation Institute
I study how machines learn and generalize from limited data. My research connects few-shot learning and statistical methods with generative models, 3D vision, robotics, and brain decoding.
CVPR Tutorial Organizer & Speaker, 2022–2024
Three consecutive years · Topics & talks →
Research awards
-
2023
State Natural Science Award, Second Class China
多元协同的视觉计算理论与方法
Team award · Fourth contributor · 2023 award year, announced in 2024
Fudan University announcement ↗ -
2023
Shanghai Technological Invention Award, First Class
面向智能制造的跨域融合感知关键技术及应用
Team award · Third contributor
Shanghai government award list ↗ -
2023
Shanghai Natural Science Award, Second Class
面向行人重识别的高效学习理论与方法
Team award · Fourth contributor
Shanghai government award list ↗ -
2022
Ministry of Education Natural Science Award, First Class China
多元协同的视觉计算理论与方法
Team award · 2022 award year, announced in 2023
Fudan University announcement ↗
Industry recognition
- Outstanding Technical Achievement Award, 2021Huawei Shanghai Research Institute华为上海研究所 · 2021 年度优秀技术成果奖
- Tencent Rhino-Bird Visiting Scholar腾讯“犀牛鸟计划”访问学者
Research
How can models acquire new concepts, transfer knowledge, and generalize with only a few observations? This question underpins my work on learning to learn, from statistical foundations to visual and embodied intelligence.
Few-shot learning & statistical foundations
Meta-learning, sparsity, causal learning, and reliable generalization under limited supervision.
Generative models & 3D vision
Diffusion models, controllable video generation, and 3D/4D reconstruction from sparse observations.
Embodied intelligence
Vision-guided manipulation, vision-language-action models, learning from demonstrations, and egocentric & ego–exo understanding.
Multimodal learning & brain decoding
Large-scale fMRI pretraining, fMRI-to-3D decoding, and open fMRI/EEG datasets for the research community.
Selected publications
Full publication list →Research highlights
-
CVPR 2026Brain decoding
CineBrain: A Large-Scale Multi-Modal Audiovisual Brain Dataset for Brain-Conditioned Video Generation
Jianxiong Gao, Yichang Liu, Baofeng Yang, Jianfeng Feng, Yanwei Fu
First large-scale audiovisual brain dataset pairing time-aligned fMRI and EEG recordings.
-
ICML 2026Reliable generation
Conformal Reliability: A New Evaluation Metric for Conditional Generation
Yachen Gao, Xinwei Sun, Yikai Wang, Ye Shi, Jingya Wang, Jianfeng Feng, Yanwei Fu
-
CVPR 2026Video world models
VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Control
Sixiao Zheng, Minghao Yin, Wenbo Hu, Xiaoyu Li, Ying Shan, Yanwei Fu
To our knowledge, this is the first dynamic, realistic video world model with explicit 4D geometric control over camera and multi-object motion.
-
CVPR 2026Robotics
ActiveVLA: Injecting Active Perception into Vision-Language-Action Models for Precise 3D Robotic Manipulation
Zhenyang Liu, Yongchong Gu, Yikai Wang, Xiangyang Xue, Yanwei Fu
Bringing active perception into 3D robotic manipulation: virtual viewpoint selection and active 3D zoom-in overcome the constraints of fixed visual inputs.
-
ICML 2025Diffusion bridges
UniDB: A Unified Diffusion Bridge Framework via Stochastic Optimal Control
Kaizhen Zhu, Mokai Pan, Yuexin Ma, Yanwei Fu, Jingyi Yu, Jingya Wang, Ye Shi
-
ICLR 20243D reconstruction
MVSFormer++: Revealing the Devil in Transformer's Details for Multi-View Stereo
Chenjie Cao, Xinlin Ren, Yanwei Fu
-
CVPR 2021Semantic segmentation
SETR: Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers
Sixiao Zheng, Jiachen Lu, Hengshuang Zhao, Xiatian Zhu, Zekun Luo, Yabiao Wang, Yanwei Fu, Jianfeng Feng, Tao Xiang, Philip H.S. Torr, Li Zhang
To our knowledge, SETR was the first to formulate semantic segmentation as sequence-to-sequence prediction with a pure Vision Transformer encoder. It models global context at every encoder layer without progressive spatial downsampling.
Three consecutive years
CVPR tutorials · 2022–2024
Co-organizer and speaker for three consecutive years, giving opening remarks and tutorial talks at CVPR 2022, 2023, and 2024.
-
CVPR 2024
Object-centric Representations in Computer Vision
Talk: More Real-world Applications by Object-centric Representations
-
CVPR 2023
Few-shot Learning from Meta-Learning, Statistical Understanding to Applications
Talk: Few-shot Learning by Statistical Methods (with Yikai Wang)
-
CVPR 2022
Sparse Learning in Neural Networks and Robust Statistical Analysis
Talks: Introduction; Sparse Learning in Noisy Data Detection (with Yikai Wang)
Academic service
- Action EditorTransactions on Machine Learning Research
- Area Chair (multiple years)NeurIPS · ICML · ICLR
- Senior Area Chair (multiple years)AAAI
- CVPR Tutorial Organizer & Speaker2022–2024 · Three consecutive years
Fellowships & honors
- Ten-Year Honor Certificate2026 · School of Data Science, Fudan University复旦大学大数据学院十周年荣誉证书View certificate ↗
- Fellow of the British Computer Society2022 · BCS
- Young Thousand Talents Program2018 · 国家青年千人计划
- Shanghai Eastern Scholar2017 · 上海市东方学者
- ARC DECRA Fellow2016 · Australian Research CouncilDiscovery Early Career Researcher Award
- Chinese Government Award for Outstanding Self-financed Students Abroad2014 · 国家优秀自费留学生奖学金
Academic background
Full Professor, Fudan UniversitySchool of Data Science · Full Professor since December 2023
Young Associate Research Fellow, then Young Research Fellow, Fudan UniversitySchool of Data Science
Postdoctoral Researcher, Disney Research PittsburghWith Leonid Sigal
Ph.D. in Computer Vision, Queen Mary University of LondonAdvisors: Tao Xiang and Shaogang Gong
Master’s degree in Computer Science, Nanjing University
Selected paper honors
Paper awards & recognition
-
2026
Best Paper Award Candidate CVPR 2026 · Oral
Official conference program ↗ -
2025
Top Viewed Article Wiley · Journal of Advances in Modeling Earth Systems (JAMES)
Coauthored with Kerry Emanuel (MIT; Member of the U.S. National Academy of Sciences and the American Academy of Arts and Sciences), Noah S. Diffenbaugh (Stanford), and Eran Bendavid (Stanford).
Wiley recognition, 2025 · Among JAMES articles published in 2024, with views counted 12 months after publication.
-
2025
TMLR Journal-to-Conference (J2C) Certification ICML 2026 · Poster
Repositioning the Subject within Image
Published in TMLR, November 2024 · J2C certified October 2025 · Selected for the ICML 2026 Journal-to-Conference Track.
-
2025
Long Paper Outstanding Paper Award DeLTa Workshop at ICLR 2025
A Unified Diffusion Bridge Framework via Stochastic Optimal Control
Official award announcement ↗ -
2024
Highest Impact Award IEEE Computer Society Biometrics Workshop
When Person Re-Identification Meets Changing Clothes
Published at the CVPR 2020 Biometrics Workshop · Awarded in 2024
Coauthor's award record ↗ -
2023
Youth Outstanding Paper Award World Artificial Intelligence Conference (WAIC)
SETR: Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers
Coauthored work · Award recipient: Sixiao Zheng
Official award list ↗ -
2019
Best Paper Award IEEE ICME 2019
An End-to-End Architecture for Class-Incremental Object Detection with Knowledge Distillation
Official award announcement ↗ -
2019
Best Paper Finalist CVPR 2019 · Oral
Image Deformation Meta-Networks for One-Shot Learning
Official award announcement ↗
Academic recognition & media
ACM China Rising Star Awards
2017 · SIGAI China
2018 · Shanghai Chapter
World’s Top 2% Scientists · 2026
Selected for both career-long and single-year citation impact lists in the Stanford/Elsevier database.
Global Top 100 AI Talents · 2025
Also included in the Top 100 Chinese AI Talents list. The lists were jointly released by UNIDO ITPO Beijing and Dongbi Data.
World Robot Contest · 2020
Best Collaboration Award for FDU-Insight in the collaborative robotics challenge. I led the team and co-advised it with Xiangyang Xue.
White paper contribution
-
AI for Science 2025 · Embodied intelligence section
I drafted the embodied intelligence section of AI for Science 2025, a white paper jointly released by Fudan University, the Shanghai Academy of AI for Science, and Nature Research Intelligence, part of Springer Nature. The section reviews research progress, key scientific challenges, and directions for future breakthroughs.
Industry deployment
-
ZITS image inpainting in Huawei Mate 60 AI object removal
Our team’s ZITS-series image-inpainting algorithms have been deployed in Huawei Mate 60’s AI object-removal feature, bringing structure-aware image completion into everyday smartphone photo editing.
3D reconstruction & generative foundation models
-
MVSFormer series: advancing multi-view stereo with pretrained vision transformers
With results submitted in May 2022, MVSFormer ranked first among published methods on both the Intermediate and Advanced Tanks and Temples leaderboards at the time. Its mean F-scores were 66.37 and 40.87, respectively.
To our knowledge, MVSFormer is the first work to systematically study how pretrained vision transformers enhance multi-view stereo. MVSFormer++ (ICLR 2024) extends this approach with DINOv2 features, cross-view information, and refined transformer design, improving reconstruction accuracy and generalization.
The series has been introduced in the TMLR official digest and research overviews from BAAI Hub and HyperAI.
-
Pixel2Mesh series: from a single RGB image directly to a 3D mesh
Pixel2Mesh is the first end-to-end deep learning framework to directly generate a 3D triangular mesh from a single RGB image. It bypasses the extra conversion from point clouds or voxels to meshes, directly producing a mesh representation for rendering and geometric processing.
Pixel2Mesh++ (ICCV 2019) extends the series to multi-view mesh generation and refinement, improving geometric detail and consistency across viewpoints.
Pixel2Mesh was taught in Stanford University’s CS231n course for three consecutive years (2022–2024). The 3D vision lectures use Pixel2Mesh to explain learning-based reconstruction of triangle meshes from single RGB images.
PaperWeekly published dedicated features on Pixel2Mesh in and Pixel2Mesh++ in , explaining the direct-mesh breakthrough and its multi-view extension.
-
DeepSFM series: deep bundle adjustment and its research impact
The ECCV paper and TPAMI extension combine geometric constraints with learned cost volumes to iteratively refine depth and camera pose.
MaFreeI2P (2024) explicitly cites DeepSFM as a source of inspiration for its image-to-point-cloud registration method. Its introduction credits the TPAMI work when motivating a pose-based cost volume.
Independent technical commentary highlights robustness to few input views, initialization noise, and changes in image quality. In a Japanese computer-vision reading-group report, TSUKAMOTO Kenji discusses these strengths alongside the task-dependent parameters introduced by pose-hypothesis generation.
Included in Gerard Pons-Moll's 2025 University of Tübingen 3D vision course. Lecture 09 introduces DeepSFM alongside VGGSfM under “End-to-End Learnable SfM Pipelines.” The work also appears in 3D视觉工坊's ECCV 2020 paper roundup and an autonomous-driving overview hosted on EE Times China.
-
Argus-3D: a 3D-native generative foundation model at 3.6-billion-parameter scale
To our knowledge, Argus-3D was the first academic 3D-native shape generation model scaled to 3.6 billion trainable parameters, first publicly introduced on . Developed with BAAI and collaborators, it learns discrete geometric representations and autoregressive priors directly from approximately 900,000 3D objects, supporting multimodal conditional shape generation.
The original paper reported it as the largest 3D shape generation model at the time. BAAI Hub published a detailed feature on ; an expanded preprint followed on .
Controllable video generation & adoption
-
Uni3C: 3D video control with native ComfyUI camera support
Our Uni3C unifies 3D-enhanced camera and human-motion control for video generation, combining scene geometry with controllable viewpoints and character motion.
Its camera-control module is natively supported in ComfyUI for Wan models, included in the official v0.29.0 release. ComfyLab documents a hands-on camera-orbit workflow using this integration.
-
VerseCrafter: 4D geometric control of camera and multiple moving objects
To our knowledge, VerseCrafter is the first dynamic, realistic video world model with explicit 4D geometric control over camera and multi-object motion. Developed with Tencent ARC Lab and collaborators, it provides a Blender interface for interactively designing these trajectories.
Featured by Ji Qizhixin (机器之心), with the article available through the outlet’s Tencent Cloud account.
-
VidCRAFT3: camera, object, and lighting control for video generation
VidCRAFT3 is the first unified image-to-video (I2V) diffusion framework supporting independent and joint control over camera motion, object motion, and lighting direction, enabling precise control of viewpoint, scene dynamics, and illumination within one model.
Discussed in Japanese-language technical commentary by D × MirAI and a research review by The Moonlight.
Embodied intelligence
-
τ₀-WM: a unified video-action world model for robotic manipulation
I coauthored τ₀-WM, jointly released by Shanghai Innovation Institute and AGIBOT Finch. Pretrained on approximately 27,300 hours of robot and human interaction data, the model unifies video prediction, action generation, and evaluation of action consequences, allowing robots to predict, evaluate, and revise candidate actions before execution.
He You, a member of the Chinese Academy of Engineering, featured τ₀-WM in his presentation on intelligent unmanned systems at the Fourth Shanghai Computer Vision Conference (2026). His slide highlighted its training-data scale and described it at the time as the world’s largest-scale open-source pretrained embodied world model.
Featured in Wenhui Daily / Shangguan News, Xinmin Evening News, and QbitAI. The coverage explains how the model anticipates action consequences and generalizes across tasks; code and model weights are openly available.
-
Pose-VLA: reshaping VLA around 3D poses
Pose-VLA puts universal 3D poses at the core of VLA representation and pretraining. Discrete pose tokens unify spatial learning from diverse 3D datasets and robot trajectories; pretraining builds transferable geometric priors before these are aligned to a specific robot’s action space. This addresses the gap between semantic recognition and the fine 3D state distinctions that precise actions require.
Our central message to the embodied AI community: 3D spatial understanding is crucial for embodied foundation models. Published at Robotics: Science and Systems (RSS) 2026, Pose-VLA demonstrates real-world generalization across diverse objects with only 100 demonstrations per task.
I see a parallel in the recent GPT-6 Astra robot demonstrations documented in Wentao Zhu’s technical perspective: using a low-level robot SDK, the model plans bimanual coordination, active vision, and error recovery. These developments underscore, through a different approach, the importance of connecting spatial understanding with action.
-
ActiveVLA: bringing active perception into 3D robotic manipulation
ActiveVLA’s core advance is to integrate active perception into precise 3D manipulation, enabling a VLA to decide where to look and how closely to inspect. It tackles a central limitation of fixed-view VLA approaches: task-critical geometry can be obscured or too small to resolve in the original camera inputs.
Our novel coarse-to-fine framework combines task-driven virtual viewpoint selection with active 3D zoom-in. It first locates the relevant 3D region, then synthesizes informative views and magnifies local geometry before predicting an action, overcoming the viewpoint and resolution constraints of fixed visual inputs.
I am a co-corresponding author of this CVPR 2026 paper. Evaluations on RLBench, COLOSSEUM, and GemBench, together with real-robot demonstrations, show improved precision and robustness for manipulation in cluttered and occluded scenes.
-
TriVLA: unifying world knowledge and world modeling in a triple-system VLA
TriVLA was first publicly introduced on (arXiv v1). The initial version already unified world knowledge from a vision-language model, a video world model, and an action policy in a triple-system VLA architecture.
The v3 revision explicitly formalizes episodic world modeling, describing TriVLA as the first framework to do so within a unified triple-system VLA architecture, supporting complex instructions and long-horizon robotic manipulation.
Discussed in a Shenlan Academy video and research reviews by Moonlight and the CSDN developer community.
-
Visuo-tactile AI sensing: live demonstration to Premier Li Qiang
I contributed to the team’s adaptive visuo-tactile AI sensor, demonstrated live to Chinese Premier Li Qiang at WAIC 2025. The work enables delicate robotic handling of fragile objects such as tofu and jelly.
China Science Daily / ScienceNet and Wenhui Daily / Shangguan News interviewed me about the technology. Wenhui’s WAIC report discusses the sensing algorithm; China Science Daily’s covers jelly-grasping and tangyuan-scooping demonstrations and applications in precise sorting and assembly.
-
TaMMa: connecting navigation with precise manipulation across subscenes
TaMMa extends robotic manipulation across multiple tabletop subscenes, coordinating a mobile base with a robotic arm. A semantic 3D Gaussian representation first locates the target and guides the robot toward it; depth completion then refines local geometry and object poses. Gaussian inpainting represents the intended arrangement before execution, supporting real-robot pouring, stacking, and tidying tasks.
The 2025 survey Towards a Unified Understanding of Robot Manipulation discusses TaMMa in its mobile-manipulation section, highlighting sparse Gaussian localization and depth completion as a way to connect 3D scene modeling with accurate target poses.
-
Polaris: grounding open-ended dialogue in real robot actions
Polaris combines GPT-4 interaction with category-level 6D pose estimation trained on synthetic data to perform real tabletop manipulation. It connects users' open-ended requests to the geometry needed for grasping and placement, including stacking, tidying, and compositional tasks that account for object affordances. This develops our earlier dialogue-guided grasping work toward richer scene interaction.
The independent 2025 paper Text-guided RGB-P grasp generation discusses Polaris in its review of LLM-based robotic grasping, describing how language interpretation is connected to visual grounding. Polaris provides a concrete example of bringing language reasoning and precise spatial perception together for physical action.
-
WALL-E: from multi-round dialogue to visually grounded robot grasping
WALL-E connects ChatGPT dialogue with visual grounding, object pose and size estimation, and physical robot grasping. Through successive exchanges, the system turns a user's preferences into a target instruction, locates the corresponding object, and executes the grasp. Publicly released in August 2023, it documents our early exploration of language models as an interface to a real robot.
UT Dallas's Fall 2023 robot manipulation and navigation course selected WALL-E as a representative reference for language-guided manipulation, alongside SayCan and Instruct2Act in its project guide. A 2024 survey by researchers at KAIST also includes the work in its discussion of object grounding for LLM-integrated robots.
-
PourIt!: perceiving liquid for visual closed-loop robotic pouring
PourIt! learns to perceive flowing liquid using image-level supervision and uses that perception to guide a real Franka robot. It combines a single image's liquid mask with the source container's pose to recover a 3D liquid representation, without requiring direct liquid-depth measurements. The resulting liquid-to-container geometry supplies visual feedback for pouring, linking weakly supervised perception with physical control.
The project releases code and a dedicated dataset. An independent, ongoing IsaacLabPouringExtension project explicitly uses PourIt for image processing in a transparent-liquid pouring task, providing a concrete example of reuse in a subsequent robotics implementation.
-
GeoVLN: geometry-aware visual representations for language-guided navigation
GeoVLN brings explicit geometric cues into vision-and-language navigation. Depth and surface-normal information complement RGB appearance, while slot attention integrates the representations to help an agent relate language instructions to the surrounding scene. The work extends our embodied-intelligence research from object manipulation to navigation through an environment.
GeoVLN continues to appear in subsequent navigation comparisons: the independent 2026 STE-VLN preprint includes it in its R2R benchmark table. This documents its use as a comparison method in later research.
-
SAR-Net: shape alignment and recovery for grasping unseen object instances
SAR-Net estimates category-level 6D object poses and sizes through shape alignment and symmetry-aware recovery. It aligns a category template to the observed point cloud and reconstructs missing shape information to infer rotation, translation, and size, without requiring real pose-annotated training images. Experiments on a physical Baxter robot demonstrate grasping of previously unseen instances from known categories.
The independent PACE benchmark (ECCV 2024) includes SAR-Net among its evaluated category-level pose-estimation methods and publishes its baseline results. This provides direct evidence of external evaluation and establishes SAR-Net as a reference point for later research on pose estimation in cluttered environments.
-
I Know What You Draw: using a few freehand sketches to specify a grasping target
This work explores freehand object sketches as an intuitive way to communicate a grasping target to a robot. A graph representation preserves sketch structure, allowing the system to learn from around 100 hand-drawn examples, connect drawings with objects in the observed scene, and predict the corresponding grasps. Dataset evaluations and physical robot demonstrations investigate how visual intent can guide manipulation without requiring a detailed verbal description.
Toyota Frontier Research Center's subsequent Sketch-MoMa work explicitly discusses this paper as prior research on sketch-based object detection and manipulation. The connection places our ICRA 2022 study in the continuing development of drawing-based interfaces for robot control.
-
Language-guided grasping: connecting object descriptions, 6D poses, and physical action
Our ICRA 2022 work connects free-form language instructions to category-level 6D object localization and real robot grasping. Descriptions of names, attributes, and spatial relations identify the target; pose estimation supports grasping an unseen instance from a known category. Language Guided Robotic Grasping with Fine-grained Instructions (IROS 2023) further explores detailed language for specifying the intended object.
The independent MaskGrasp study (IROS 2024) cites both papers in its discussion of language-driven grasp detection. Together, these studies form a sustained research direction connecting human instructions with robot perception and action.
-
Smart Bartender: serving drinks through interactive gaze following
Our Smart Bartender lets users choose a drink by looking at it. Combining gaze following, category-level object pose estimation, and motion planning, the robot identifies the selected bottle, grasps it, and pours into a cup, generalizing to bottles and cups not seen during training.
Featured in IEEE Spectrum’s Video Friday selection of ICRA 2023 robot videos. The editorial roundup includes the demonstration and credits the Fudan University team.
-
Robotics in early 2023: vision-guided grasping and pouring
By February 2023, my group had publicly demonstrated robotic grasping and pouring using category-level generalization and 6D pose estimation, with models trained on synthetic images and applied to real scenes. Gaze tracking enabled user-directed drink serving. Fudan’s School of Data Science documented the demonstrations held on .
Egocentric vision & ego–exo understanding
-
ObjectRelator & V²-SAM: connecting first- and third-person views
ObjectRelator (ICCV 2025) and V²-SAM (CVPR 2026) were both selected as Highlight papers. The two works establish object correspondence across ego–exo views through multimodal alignment and complementary geometric and visual prompts.
The related ObjectRelator challenge method took second place in the Ego-Exo4D 2025 Correspondences track at the EgoVis Workshop held with CVPR 2025.
-
EgoSound: an open benchmark for first-person sound understanding
EgoSound extends egocentric understanding to sound perception and audiovisual reasoning. It evaluates spatial localization, causal inference, and cross-modal understanding, with publicly available data and evaluation code.
The work was selected as a CVPR 2026 Highlight, complementing our ego–exo visual correspondence research with multisensory evaluation.
-
Whose hand is this? Person Identification from Egocentric Hand Gestures
Satoshi Tsutsui, Yanwei Fu, David Crandall
Studies camera-wearer identification from hand appearance and motion, and uses identity-adversarial training to improve gesture recognition across users.
-
EgoCross Challenge: advancing cross-domain egocentric video understanding
I co-organized the first EgoCross Challenge at EgoVis, CVPR 2026, and coauthored its technical report. The challenge evaluates video question answering across surgery, industrial assembly, extreme sports, and animal perspectives.
The report records over 130 participants and over 1,500 submissions, with public challenge data, baseline code, and winning-team solutions.
Brain decoding & open research resources
-
fMRI-PTE: large-scale, multi-subject fMRI pretraining
To our knowledge, fMRI-PTE is the first fMRI foundation model pretrained at the scale of approximately 40,000 participants for cross-subject visual brain decoding. Pretrained on 39,630 UK Biobank participants, it learns shared neural representations for within-subject and cross-subject visual decoding.
Our group released the arXiv preprint on .
Science and Technology Daily highlighted the team’s pioneering fMRI pretraining research in August 2024; the report was republished by People.cn and CNR.
-
NeuroPictor: leading structural fidelity in an independent evaluation
An AAAI 2026 study by University of Cambridge researchers Weihao Xia and Cengiz Öztireli compared 14 brain visual decoding methods on NSD. NeuroPictor ranked first in structural fidelity (BASIC-L), with a score of 25.88.
ZEBRA (NeurIPS 2025), from HKUST, builds on NeuroPictor’s unified brain representation and adopts the fMRI-PTE pretrained encoder for zero-shot cross-subject visual decoding.
-
MinD-3D: establishing the fMRI-to-3D reconstruction task
Our group first introduced and defined the fMRI-to-3D reconstruction task, Recon3DMind, and demonstrated its feasibility with MinD-3D. We released fMRI-Shape, the first dataset pairing fMRI signals with 3D shapes, collected from 14 participants.
The preprint was released on , followed by publication at ECCV 2024.
Independent studies recognize fMRI-Shape’s pioneering role and use it for further brain-to-3D research. Neuro-3D (CVPR 2025) identifies it as the first paired fMRI–3D shape dataset; NeuroSculptor3D (AAAI 2026) and BrainGS (CVPR 2026) use it in their experiments. BrainGS also follows MinD-3D’s cross-subject and cross-category evaluation protocols.
-
Open brain-signal datasets: over 180,000 cumulative downloads
Our group contributes brain-signal datasets and research resources to the community through Fudan-fMRI, with 8 public repositories and over 180,000 cumulative downloads. fMRI-Shape alone has over 130,000 downloads; other resources include fMRI-Objaverse, fMRI-Video, and CineBrain.
Hugging Face cumulative repository download counts, checked on .
-
CineBrain: CVPR Daily cover feature and independent research adoption
CineBrain also presents the first systematic investigation of fMRI–EEG fusion for video reconstruction. Featured on the cover of the Sunday issue, with a five-page interview and review of our audiovisual brain decoding research (pp. 10–14).
The open dataset supports independent video and speech reconstruction research. Tongji University’s DynaMind (CVPR 2026) uses CineBrain for EEG-to-video experiments and describes Cine-Benchmark as “more comprehensive for naturalistic human-centric videos.” NeuroSonic (MICCAI 2026) uses the dataset for EEG-conditioned speech reconstruction.
Brain–Computer Interface Community published a Chinese-language overview on July 8, 2026, archived by X-Techcon.
-
EEG intention decoding: presenting a robotic grasping system
Our EEG intention-decoding robotic grasping system was presented at Fudan’s robotics and technology-transfer salon on June 5, 2025. The university’s Technology Transfer Center documented the presentation in its June 6 report.
-
fMRI dream decoding: from neural signals to visual narratives
Our exploratory study combines fMRI decoding of dream imagery with language models to construct video narratives.
Communications of the ACM featured the work in May 2026, interviewing me about dream decoding and CineBrain. Additional coverage includes a direct ScienceWriters / NASW interview and an Innovando.News feature; Science and Fiction pairs the research with an original short story.
Earlier work at Disney Research
-
Learning to recognize unseen objects from language
Coverage of my work with Leonid Sigal on semi-supervised vocabulary-informed learning, connecting language and visual recognition.
-
Understanding video events through object and scene semantics
Coverage of our Disney Research–Fudan collaboration on video understanding and recognizing events beyond the training examples.
Foundational contributions
-
CVPR 2020
TPAMI 2022Few-shot learningInstance Credibility Inference for Few-Shot Learning
Yikai Wang, Chengming Xu, Chen Liu, Li Zhang, Yanwei Fu
Journal extension
How to Trust Unlabeled Data? Instance Credibility Inference for Few-Shot Learning
Yikai Wang, Li Zhang, Yuan Yao, Yanwei Fu
Uses sparse statistical modeling to identify trustworthy pseudo-labels for few-shot learning; the journal extension develops generalized linear models and theoretical analysis.
-
TPAMI 2024Learning with noisy labels
Knockoffs-SPR: Clean Sample Selection in Learning with Noisy Labels
Yikai Wang, Yanwei Fu, Xinwei Sun
Selects clean training samples with knockoff filters, controlling the false-selection rate under the method’s theoretical assumptions.
-
ICML 2020
TPAMI 2023Statistical foundationsDessiLBI: Exploring Structural Sparsity of Deep Networks via Differential Inclusion Paths
Yanwei Fu, Chen Liu, Donghao Li, Xinwei Sun, Jinshan Zeng, Yuan Yao
Journal extension
Exploring Structural Sparsity of Deep Networks Via Inverse Scale Spaces
Yanwei Fu, Chen Liu, Donghao Li, Zuyuan Zhong, Xinwei Sun, Jinshan Zeng, Yuan Yao
Learns sparse neural structures through inverse-scale-space dynamics; the journal extension also studies adaptive network growth and transfer of sparse subnetworks.
-
2023–2025Causal learning
Causal learning: hidden confounding and reliable generalization
Develops doubly robust estimation for continuous treatments and nonparametric causal discovery using proxy variables. These methods address unmeasured confounding, clarify when single-proxy tests can miss genuine causal relationships, and introduce additional proxy restrictions to restore identification under stated assumptions.
Continuous-treatment estimation · ICLR 2024
Doubly Robust Proximal Causal Learning for Continuous Treatments
Yong Wu, Yanwei Fu, Shouyan Wang, Xinwei Sun
Proxy-based causal discovery · ICML 2025
Bivariate Causal Discovery with Proxy Variables: Integral Solving and Beyond
Yong Wu, Yanwei Fu, Shouyan Wang, Xinwei Sun
Related applications connect causal feature learning with few-shot recognition in PatchMix (TPAMI 2023), and incorporate intraoperative information into preoperative survival prediction in Causally-Aware Intraoperative Imputation (CVPR 2023).
Independent research uptake. The continuous-treatment estimator appears in Arthur Gretton’s CLeaR 2026 keynote materials and was used as a baseline, with its experimental settings reused, in an independent NeurIPS 2025 study. A 2026 follow-up preprint develops statistical inference for proximal dose-response curves.
-
TPAMI 2015Zero-shot learning
Transductive Multi-view Zero-Shot Learning
Yanwei Fu, Timothy M. Hospedales, Tao Xiang, Shaogang Gong
Identifies projection domain shift in zero-shot learning and addresses it through transductive multi-view embedding.
Prospective students & visitors
For research inquiries, doctoral applications, or summer visits, please contact yanweifu@fudan.edu.cn. International students interested in summer internships are welcome to get in touch.
2026 年招生说明：本组仅在复旦大学大数据学院和上海创智学院招生。大数据学院科学硕士及直博生由学院统一招生，请先申请并获得学院 offer，再联系我；申请普博或暑期访问可提前联系。有兴趣研究统计稀疏性的本科生，也可联系孙鑫伟老师共同合作。
本组 2027年入学的普博博士名额已满。
U.S. patents
Granted U.S. patents from my research at Fudan University and Disney Research, spanning 3D vision, image restoration and visual recognition. The 11 grants below are grouped by invention.
Fudan University Original assignee
-
Network and system for pose and size estimation
Yanwei Fu, Haitao Lin, Xiangyang Xue
-
System and method for image inpainting
Yanwei Fu, Chenjie Cao
-
Network and system for neural pose transfer
Yanwei Fu, Xiangyang Xue, Yinda Zhang, Chao Wen, Haitao Lin, Jiashun Wang, Tianyun Zou
-
Network, system and method for image processing
Yugang Jiang, Yanwei Fu, Changmao Cheng, Xiangyang Xue
-
Network, system and method for multi-view 3D mesh generation via deformation
Yanwei Fu, Chao Wen, Yinda Zhang, Zhuwen Li
-
Network, system and method for 3D shape generation
Yugang Jiang, Yanwei Fu, Nanyang Wang, Yinda Zhang, Zhuwen Li
Disney Enterprises, Inc. Original assignee
-
Robust classification by pre-conditioned lasso and transductive diffusion component analysis
Yanwei Fu, Leonid Sigal
-
System and method of vocabulary-informed categorization of items of interest included within digital information
Yanwei Fu, Leonid Sigal
-
Systems and methods for identifying activities and/or events in media contents based on object data and scene data
Zuxuan Wu, Yanwei Fu, Leonid Sigal
Published U.S. patent application
Applicant: Tencent Technology (Shenzhen) Company Limited
-
Three-dimensional image generation
Shijie Zhang, Chengming Xu, Junwei Zhu, Xuelin Qian, Ying Tai, Yanwei Fu, Chengjie Wang
US20260245287A1 ↗ · Published