Jianlong Fu

CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation

Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, Xiaofan Wang, Bei Liu, Jianlong Fu, Jianmin Bao, Dong Chen, Yuanchun Shi, Jiaolong Yang, Baining Guo

November 2024

DOI Publication Publication

PromptFix: You Prompt and We Fix the Photo

Yongsheng Yu, Ziyun Zeng, Hang Hua, Jianlong Fu, Jiebo Luo

NeurIPS 2024 | May 2024

DOI Publication Publication

Zero-Reference Low-Light Enhancement via Physical Quadruple Priors

Wenjing Wang, Huan Yang, Jianlong Fu, Jiaying Liu

CVPR 2024 | March 2024

DOI Publication Publication

Solving Diffusion ODEs with Optimal Boundary Conditions for Better Image Super-Resolution

Y. Ma, Huan Yang, Wenhan Yang, Jianlong Fu, Jiaying Liu

ArXiv | May 2023, Vol abs/2305.15357

DOI PDF

MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation

Ludan Ruan, Yiyang Ma, Huan Yang, Huiguo He, Bei Liu, Jianlong Fu, Nicholas Jing Yuan, Qin Jin, Baining Guo

arXiv | December 2022

Publication Project

Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning

Yuchong Sun, Hongwei Xue, Ruihua Song, Bei Liu, Huan Yang, Jianlong Fu

NeurIPS 2022 | November 2022

Publication

Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions

Hongwei Xue, Tiankai Hang, Yanhong Zeng, Yuchong Sun, Bei Liu, Huan Yang, Jianlong Fu, Baining Guo

CVPR 2022 | June 2022

Publication Project

Learning Trajectory-Aware Transformer for Video Super-Resolution

Chengxu Liu, Huan Yang, Jianlong Fu, Xueming Qian

CVPR 2022 | June 2022

Oral

Publication Project

MiniViT: Compressing Vision Transformers with Weight Multiplexing

Jinnian Zhang, Houwen Peng, Kan Wu, Mengchen Liu, Bin Xiao, Jianlong Fu, Lu Yuan

CVPR 2022 | May 2022

Publication

Improving Visual Quality of Image Synthesis by A Token-based Generator with Transformers

Yanhong Zeng, Huan Yang, Hongyang Chao, Jianbo Wang, Jianlong Fu

NeurIPS 2021 | December 2021

Publication Project

Probing Inter-modality: Visual Parsing with Self-Attention for Vision-and-Language Pre-training

Hongwei Xue, Yupan Huang, Bei Liu, Houwen Peng, Jianlong Fu, Houqiang Li, Jiebo Luo

NeurIPS 2021 | December 2021

Publication

Searching the Search Space of Vision Transformer

Minghao Chen, Kan Wu, Bolin Ni, Houwen Peng, Bei Liu, Jianlong Fu, Hongyang Chao, Haibin Ling

NeurIPS 2021 | December 2021

Publication

Learning Fine-Grained Motion Embedding for Landscape Animation

Hongwei Xue, Bei Liu, Huan Yang, Jianlong Fu, Houqiang Li, Jiebo Luo

ACM Multimedia 2021 | October 2021

Oral

Publication Project

A Picture is Worth a Thousand Words: A Unified System for Diverse Captions and Rich Images Generation

Yupan Huang, Bei Liu, Jianlong Fu, Yutong Lu

ACM Multimedia 2021 | October 2021

Publication Project

Domain-Aware Universal Style Transfer

Kibeom Hong, Seogkyu Jeon, Huan Yang, Jianlong Fu, Hyeran Byun

ICCV 2021 | October 2021

Publication Project

Learning Conditional Knowledge Distillation for Degraded-Reference Image Quality Assessment

Heliang Zheng, Huan Yang, Jianlong Fu, Zheng-Jun Zha, Jiebo Luo

ICCV 2021 | October 2021

Publication Project

AutoFormer: Searching Transformers for Visual Recognition

Minghao Chen, Houwen Peng, Jianlong Fu, Haibin Ling

ICCV 2021 | October 2021

Publication

Learning Spatio-Temporal Transformer for Visual Tracking

Bin Yan, Houwen Peng, Jianlong Fu, Dong Wang, Huchuan Lu

ICCV 2021 | October 2021

Publication

Rethinking and Improving Relative Position Encoding for Vision Transformer

Kan Wu, Houwen Peng, Minghao Chen, Jianlong Fu, Hongyang Chao

ICCV 2021 | October 2021

Publication

MMPT’21: International Joint Workshop on Multi-Modal Pre-Training for Multimedia Understanding

Bei Liu, Jianlong Fu, Shizhe Chen, Qin Jin, Alexander Hauptmann, Yong Rui

ICMR 2021 | August 2021

DOI Project

Reference-Based Defect Detection Network

Zhaoyang Zeng, Bei Liu, Jianlong Fu, Hongyang Chao

IEEE Transactions on Image Processing | July 2021, Vol 30: pp. 6637-6647

Publication

Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning

Zhicheng Huang*, Zhaoyang Zeng*, Yupan Huang*, Bei Liu, Dongmei Fu, Jianlong Fu

CVPR 2021 | June 2021

Oral

Publication Project

LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Search

Bin Yan, Houwen Peng, Kan Wu, Dong Wang, Jianlong Fu, Huchuan Lu

CVPR 2021 | June 2021

Publication

One-Shot Neural Ensemble Architecture Search by Diversity-Guided Search Space Shrinking

Minghao Chen, Houwen Peng, Jianlong Fu, Haibin Ling

CVPR 2021 | June 2021

Publication

Aesthetic-Aware Image Style Transfer

Bei Liu, Jianlong Fu

ACM Multimedia | October 2020

Publication Project

Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture Search

Houwen Peng, Hao Du, Hongyuan Yu, Qi Li, Jianlong Fu

NeurIPS 2020 | October 2020

Editor(s): Houwen Peng, Jianlong Fu

Publication Project

Learning Semantic-aware Normalization for Generative Adversarial Networks

Jianlong Fu

NeurIPS 2020 | October 2020

Publication Project

Learning Joint Spatial-Temporal Transformations for Video Inpainting

Yanhong Zeng, Jianlong Fu, Hongyang Chao

16th European Conference Computer Vision (ECCV 2020) | August 2020

Publication

Ocean: Object-aware Anchor-free Tracking

Zhipeng Zhang, Houwen Peng, Jianlong Fu, Bing Li, Weiming Hu

16th European Conference Computer Vision (ECCV 2020) | August 2020

Publication

Learning Joint Spatial-Temporal Transformations for Video Inpainting

Jianlong Fu

ECCV | July 2020

Publication Project

Learning Texture Transformer Network for Image Super-Resolution

Fuzhi Yang, Huan Yang, Jianlong Fu, Hongtao Lu, Baining Guo

CVPR 2020 | June 2020

Publication Project

Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers

Zhicheng Huang*, Zhaoyang Zeng*, Bei Liu, Dongmei Fu, Jianlong Fu

MSR-TR-2020-51 | April 2020

Published by Microsoft

Publication Project

Neural Storyboard Artist: Visualizing Stories with Coherent Image Sequences

Shizhe Chen, Bei Liu, Jianlong Fu, Ruihua Song, Qin Jin, Pingping Lin, Xiaoyu Qi, Chunting Wang, Jin Zhou

ACM Multimedia | October 2019

Publication Project

WSOD^2: Learning Bottom-up and Top-down Objectness Distillation for Weakly-supervised Object Detection

Zhaoyang Zeng, Bei Liu, Jianlong Fu, Hongyang Chao, Lei Zhang

ICCV | October 2019

Publication Project

Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting

Jianlong Fu, Baining Guo

CVPR | July 2019

Editor(s): Yanhong Zeng, Jianlong Fu,Hongyang Chao, Baining Guo

Publication Project

Emotion Reinforced Visual Storytelling

Nanxing Li, Bei Liu, Zhizhong Han, Yu-Shen Liu, Jianlong Fu

ICMR | June 2019

Publication Project

Learning Rich Image Region Representation for Visual Question Answering

Bei Liu, Zhicheng Huang, Zhaoyang Zeng, Zheyu Chen, Jianlong Fu

MSR-TR-2020-1 | June 2019

Published by Microsoft

CVPR 2019, VQA Workshop

Publication Project

Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captions

Qing Li, Jianlong Fu, Dongfei Yu, Tao Mei, Jiebo Luo

Conference on Empirical Methods in Natural Language Processing | October 2018

Publication

What Dress Fits Me Best? Fashion Recommendation on the Clothing Style for Personal Body Shape

Shintami Chusnul Hidayati, Jianlong Fu, Wen-Huang Cheng

ACM International Conference on Multimedia (ACM Multimedia) | October 2018

Publication

Beyond Narrative Description: Generating Poetry from Images by Multi-Adversarial Training

Bei Liu, Jianlong Fu, Makoto P. Kato, Masatoshi Yoshikawa

2018 ACM Multimedia | October 2018

Best Paper Award

DOI Publication PDF Project

Deep Attention Neural Tensor Network for Visual Question Answering

Yalong Bai, Jianlong Fu, Tiejun Zhao, Tao Mei

European Conference on Computer Vision | September 2018

Publication

DA-GAN: Instance-level Image Translation by Deep Attention Generative Adversarial Networks

Shuang Ma, Jianlong Fu, ChangWen Chen, Tao Mei

IEEE Conference on Computer Vision and Pattern Recognition (CVPR) | June 2018

Publication

Show, Reward and Tell: Automatic Generation of Narrative Paragraph from Photo Stream by Adversarial Training

Jing Wang, Jianlong Fu, Jinhui Tang, Zechao Li, Tao Mei

The AAAI Conference on Artificial Intelligence (AAAI), 2018. | February 2018

Publication Project

Self-view Grounding Given a Narrated 360 Video

Shih-Han Chou, Jianlong Fu, Min Sun

AAAI Conference on Artificial Intelligence | February 2018

Publication Project

Learning Multi-Attention Convolutional Neural Network for Fine-Grained Image Recognition (ICCV 2017 Oral)

Heliang Zheng, Jianlong Fu, Tao Mei, Jiebo Luo

IEEE International Conference on Computer Vision (ICCV) | October 2017

Publication Project

Searching Personal Photos on the Phone with Instant Visual Qery Suggestion and Joint Text-Image Hashing

Zhaoyang Zeng, Jianlong Fu, Hongyang Chao, Tao Mei

ACM International Conference on Multimedia (ACM Multimedia) | October 2017

Publication

Show, Adapt and Tell: Adversarial Training of Cross-domain Image Captioner

Tseng-Hung Chen, Jianlong Fu, Min Sun

IEEE International Conference on Computer Vision (ICCV) | October 2017

Publication Project

Advances in deep learning approaches for image tagging

Jianlong Fu, Yong Rui

APSIPA Transactions on Signal and Information Processing | October 2017

Publication

Look Closer to See Better: Recurrent Attention Convolutional Neural Network for Fine-grained Image Recognition (CVPR 2017 Oral)

Jianlong Fu (jianf), Heliang Zheng, Tao Mei

IEEE Conference on Computer Vision and Pattern Recognition (CVPR) | July 2017

Publication Project

Multi-level Attention Networks for Visual Question Answering

Dongfei Yu, Jianlong Fu, Yong Rui, Tao Mei

IEEE Conference on Computer Vision and Pattern Recognition (CVPR) | July 2017

Publication Project

Let Your Photos Talk: Generating Narrative Paragraph for Photo Stream via Bidirectional Attention Recurrent Neural Networks

Yu Liu, Jianlong Fu, Tao Mei, Chang Wen Chen

February 2017

Publication Project

Beyond Object Recognition: Visual Sentiment Analysis with Deep Coupled Adjective and Noun Neural Networks

Jingwen Wang, Jianlong Fu, Yong Xu, Tao Mei

International Joint Conference on Artificial Intelligence (IJCAI) | August 2016

Publication Project

Relaxing from Vocabulary: Robust Weakly-Supervised Deep Learning for Vocabulary-Free Image Tagging

Jianlong Fu, Yue Wu, Tao Mei, Jinqiao Wang, Hanqing Lu, Yong Rui

December 2015

Publication Project

Tagging Personal Photos with Transfer Deep Learning

Jianlong Fu, Tao Mei, Kuiyuan Yang, Hanqing Lu, Yong Rui

International World Wide Web conference (WWW) | May 2015

Publication Project

CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation

Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, Xiaofan Wang, Bei Liu, Jianlong Fu, Jianmin Bao, Dong Chen, Yuanchun Shi, Jiaolong Yang, Baining Guo

November 2024

DOI Publication Publication

Solving Diffusion ODEs with Optimal Boundary Conditions for Better Image Super-Resolution

Y. Ma, Huan Yang, Wenhan Yang, Jianlong Fu, Jiaying Liu

ArXiv | May 2023, Vol abs/2305.15357

DOI PDF

Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning

Yuchong Sun, Hongwei Xue, Ruihua Song, Bei Liu, Huan Yang, Jianlong Fu

NeurIPS 2022 | November 2022

Publication

Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning

Zhicheng Huang*, Zhaoyang Zeng*, Yupan Huang*, Bei Liu, Dongmei Fu, Jianlong Fu

CVPR 2021 | June 2021

Oral

Publication Project

Learning Semantic-aware Normalization for Generative Adversarial Networks

Jianlong Fu

NeurIPS 2020 | October 2020

Publication Project

Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture Search

Houwen Peng, Hao Du, Hongyuan Yu, Qi Li, Jianlong Fu

NeurIPS 2020 | October 2020

Editor(s): Houwen Peng, Jianlong Fu

Publication Project

Aesthetic-Aware Image Style Transfer

Bei Liu, Jianlong Fu

ACM Multimedia | October 2020

Publication Project

Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers

Zhicheng Huang*, Zhaoyang Zeng*, Bei Liu, Dongmei Fu, Jianlong Fu

MSR-TR-2020-51 | April 2020

Published by Microsoft

Publication Project

Neural Storyboard Artist: Visualizing Stories with Coherent Image Sequences

Shizhe Chen, Bei Liu, Jianlong Fu, Ruihua Song, Qin Jin, Pingping Lin, Xiaoyu Qi, Chunting Wang, Jin Zhou

ACM Multimedia | October 2019

Publication Project

Learning Rich Image Region Representation for Visual Question Answering

Bei Liu, Zhicheng Huang, Zhaoyang Zeng, Zheyu Chen, Jianlong Fu

MSR-TR-2020-1 | June 2019

Published by Microsoft

CVPR 2019, VQA Workshop

Publication Project

Emotion Reinforced Visual Storytelling

Nanxing Li, Bei Liu, Zhizhong Han, Yu-Shen Liu, Jianlong Fu

ICMR | June 2019

Publication Project

Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captions

Qing Li, Jianlong Fu, Dongfei Yu, Tao Mei, Jiebo Luo

Conference on Empirical Methods in Natural Language Processing | October 2018

Publication

What Dress Fits Me Best? Fashion Recommendation on the Clothing Style for Personal Body Shape

Shintami Chusnul Hidayati, Jianlong Fu, Wen-Huang Cheng

ACM International Conference on Multimedia (ACM Multimedia) | October 2018

Publication

Beyond Narrative Description: Generating Poetry from Images by Multi-Adversarial Training

Bei Liu, Jianlong Fu, Makoto P. Kato, Masatoshi Yoshikawa

2018 ACM Multimedia | October 2018

Best Paper Award

DOI Publication PDF Project

Deep Attention Neural Tensor Network for Visual Question Answering

Yalong Bai, Jianlong Fu, Tiejun Zhao, Tao Mei

European Conference on Computer Vision | September 2018

Publication

DA-GAN: Instance-level Image Translation by Deep Attention Generative Adversarial Networks

Shuang Ma, Jianlong Fu, ChangWen Chen, Tao Mei

IEEE Conference on Computer Vision and Pattern Recognition (CVPR) | June 2018

Publication

Show, Reward and Tell: Automatic Generation of Narrative Paragraph from Photo Stream by Adversarial Training

Jing Wang, Jianlong Fu, Jinhui Tang, Zechao Li, Tao Mei

The AAAI Conference on Artificial Intelligence (AAAI), 2018. | February 2018

Publication Project

Look Closer to See Better: Recurrent Attention Convolutional Neural Network for Fine-grained Image Recognition (CVPR 2017 Oral)

Jianlong Fu (jianf), Heliang Zheng, Tao Mei

IEEE Conference on Computer Vision and Pattern Recognition (CVPR) | July 2017

Publication Project

Multi-level Attention Networks for Visual Question Answering

Dongfei Yu, Jianlong Fu, Yong Rui, Tao Mei

IEEE Conference on Computer Vision and Pattern Recognition (CVPR) | July 2017

Publication Project

Relaxing from Vocabulary: Robust Weakly-Supervised Deep Learning for Vocabulary-Free Image Tagging

Jianlong Fu, Yue Wu, Tao Mei, Jinqiao Wang, Hanqing Lu, Yong Rui

December 2015

Publication Project

CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation

Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, Xiaofan Wang, Bei Liu, Jianlong Fu, Jianmin Bao, Dong Chen, Yuanchun Shi, Jiaolong Yang, Baining Guo

November 2024

DOI Publication Publication

PromptFix: You Prompt and We Fix the Photo

Yongsheng Yu, Ziyun Zeng, Hang Hua, Jianlong Fu, Jiebo Luo

NeurIPS 2024 | May 2024

DOI Publication Publication

Zero-Reference Low-Light Enhancement via Physical Quadruple Priors

Wenjing Wang, Huan Yang, Jianlong Fu, Jiaying Liu

CVPR 2024 | March 2024

DOI Publication Publication

MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation

Ludan Ruan, Yiyang Ma, Huan Yang, Huiguo He, Bei Liu, Jianlong Fu, Nicholas Jing Yuan, Qin Jin, Baining Guo

arXiv | December 2022

Publication Project

Learning Trajectory-Aware Transformer for Video Super-Resolution

Chengxu Liu, Huan Yang, Jianlong Fu, Xueming Qian

CVPR 2022 | June 2022

Oral

Publication Project

Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions

Hongwei Xue, Tiankai Hang, Yanhong Zeng, Yuchong Sun, Bei Liu, Huan Yang, Jianlong Fu, Baining Guo

CVPR 2022 | June 2022

Publication Project

MiniViT: Compressing Vision Transformers with Weight Multiplexing

Jinnian Zhang, Houwen Peng, Kan Wu, Mengchen Liu, Bin Xiao, Jianlong Fu, Lu Yuan

CVPR 2022 | May 2022

Publication

Improving Visual Quality of Image Synthesis by A Token-based Generator with Transformers

Yanhong Zeng, Huan Yang, Hongyang Chao, Jianbo Wang, Jianlong Fu

NeurIPS 2021 | December 2021

Publication Project

Searching the Search Space of Vision Transformer

Minghao Chen, Kan Wu, Bolin Ni, Houwen Peng, Bei Liu, Jianlong Fu, Hongyang Chao, Haibin Ling

NeurIPS 2021 | December 2021

Publication

Probing Inter-modality: Visual Parsing with Self-Attention for Vision-and-Language Pre-training

Hongwei Xue, Yupan Huang, Bei Liu, Houwen Peng, Jianlong Fu, Houqiang Li, Jiebo Luo

NeurIPS 2021 | December 2021

Publication

Learning Fine-Grained Motion Embedding for Landscape Animation

Hongwei Xue, Bei Liu, Huan Yang, Jianlong Fu, Houqiang Li, Jiebo Luo

ACM Multimedia 2021 | October 2021

Oral

Publication Project

A Picture is Worth a Thousand Words: A Unified System for Diverse Captions and Rich Images Generation

Yupan Huang, Bei Liu, Jianlong Fu, Yutong Lu

ACM Multimedia 2021 | October 2021

Publication Project

Learning Conditional Knowledge Distillation for Degraded-Reference Image Quality Assessment

Heliang Zheng, Huan Yang, Jianlong Fu, Zheng-Jun Zha, Jiebo Luo

ICCV 2021 | October 2021

Publication Project

Domain-Aware Universal Style Transfer

Kibeom Hong, Seogkyu Jeon, Huan Yang, Jianlong Fu, Hyeran Byun

ICCV 2021 | October 2021

Publication Project

AutoFormer: Searching Transformers for Visual Recognition

Minghao Chen, Houwen Peng, Jianlong Fu, Haibin Ling

ICCV 2021 | October 2021

Publication

Learning Spatio-Temporal Transformer for Visual Tracking

Bin Yan, Houwen Peng, Jianlong Fu, Dong Wang, Huchuan Lu

ICCV 2021 | October 2021

Publication

Rethinking and Improving Relative Position Encoding for Vision Transformer

Kan Wu, Houwen Peng, Minghao Chen, Jianlong Fu, Hongyang Chao

ICCV 2021 | October 2021

Publication

MMPT’21: International Joint Workshop on Multi-Modal Pre-Training for Multimedia Understanding

Bei Liu, Jianlong Fu, Shizhe Chen, Qin Jin, Alexander Hauptmann, Yong Rui

ICMR 2021 | August 2021

DOI Project

Reference-Based Defect Detection Network

Zhaoyang Zeng, Bei Liu, Jianlong Fu, Hongyang Chao

IEEE Transactions on Image Processing | July 2021, Vol 30: pp. 6637-6647

Publication

Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning

Zhicheng Huang*, Zhaoyang Zeng*, Yupan Huang*, Bei Liu, Dongmei Fu, Jianlong Fu

CVPR 2021 | June 2021

Oral

Publication Project

One-Shot Neural Ensemble Architecture Search by Diversity-Guided Search Space Shrinking

Minghao Chen, Houwen Peng, Jianlong Fu, Haibin Ling

CVPR 2021 | June 2021

Publication

LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Search

Bin Yan, Houwen Peng, Kan Wu, Dong Wang, Jianlong Fu, Huchuan Lu

CVPR 2021 | June 2021

Publication

Ocean: Object-aware Anchor-free Tracking

Zhipeng Zhang, Houwen Peng, Jianlong Fu, Bing Li, Weiming Hu

16th European Conference Computer Vision (ECCV 2020) | August 2020

Publication

Learning Joint Spatial-Temporal Transformations for Video Inpainting

Yanhong Zeng, Jianlong Fu, Hongyang Chao

16th European Conference Computer Vision (ECCV 2020) | August 2020

Publication

Learning Joint Spatial-Temporal Transformations for Video Inpainting

Jianlong Fu

ECCV | July 2020

Publication Project

Learning Texture Transformer Network for Image Super-Resolution

Fuzhi Yang, Huan Yang, Jianlong Fu, Hongtao Lu, Baining Guo

CVPR 2020 | June 2020

Publication Project

Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers

Zhicheng Huang*, Zhaoyang Zeng*, Bei Liu, Dongmei Fu, Jianlong Fu

MSR-TR-2020-51 | April 2020

Published by Microsoft

Publication Project

WSOD^2: Learning Bottom-up and Top-down Objectness Distillation for Weakly-supervised Object Detection

Zhaoyang Zeng, Bei Liu, Jianlong Fu, Hongyang Chao, Lei Zhang

ICCV | October 2019

Publication Project

Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting

Jianlong Fu, Baining Guo

CVPR | July 2019

Editor(s): Yanhong Zeng, Jianlong Fu,Hongyang Chao, Baining Guo

Publication Project

Learning Rich Image Region Representation for Visual Question Answering

Bei Liu, Zhicheng Huang, Zhaoyang Zeng, Zheyu Chen, Jianlong Fu

MSR-TR-2020-1 | June 2019

Published by Microsoft

CVPR 2019, VQA Workshop

Publication Project

Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captions

Qing Li, Jianlong Fu, Dongfei Yu, Tao Mei, Jiebo Luo

Conference on Empirical Methods in Natural Language Processing | October 2018

Publication

What Dress Fits Me Best? Fashion Recommendation on the Clothing Style for Personal Body Shape

Shintami Chusnul Hidayati, Jianlong Fu, Wen-Huang Cheng

ACM International Conference on Multimedia (ACM Multimedia) | October 2018

Publication

Deep Attention Neural Tensor Network for Visual Question Answering

Yalong Bai, Jianlong Fu, Tiejun Zhao, Tao Mei

European Conference on Computer Vision | September 2018

Publication

DA-GAN: Instance-level Image Translation by Deep Attention Generative Adversarial Networks

Shuang Ma, Jianlong Fu, ChangWen Chen, Tao Mei

IEEE Conference on Computer Vision and Pattern Recognition (CVPR) | June 2018

Publication

Show, Reward and Tell: Automatic Generation of Narrative Paragraph from Photo Stream by Adversarial Training

Jing Wang, Jianlong Fu, Jinhui Tang, Zechao Li, Tao Mei

The AAAI Conference on Artificial Intelligence (AAAI), 2018. | February 2018

Publication Project

Self-view Grounding Given a Narrated 360 Video

Shih-Han Chou, Jianlong Fu, Min Sun

AAAI Conference on Artificial Intelligence | February 2018

Publication Project

Show, Adapt and Tell: Adversarial Training of Cross-domain Image Captioner

Tseng-Hung Chen, Jianlong Fu, Min Sun

IEEE International Conference on Computer Vision (ICCV) | October 2017

Publication Project

Searching Personal Photos on the Phone with Instant Visual Qery Suggestion and Joint Text-Image Hashing

Zhaoyang Zeng, Jianlong Fu, Hongyang Chao, Tao Mei

ACM International Conference on Multimedia (ACM Multimedia) | October 2017

Publication

Learning Multi-Attention Convolutional Neural Network for Fine-Grained Image Recognition (ICCV 2017 Oral)

Heliang Zheng, Jianlong Fu, Tao Mei, Jiebo Luo

IEEE International Conference on Computer Vision (ICCV) | October 2017

Publication Project

Advances in deep learning approaches for image tagging

Jianlong Fu, Yong Rui

APSIPA Transactions on Signal and Information Processing | October 2017

Publication

Look Closer to See Better: Recurrent Attention Convolutional Neural Network for Fine-grained Image Recognition (CVPR 2017 Oral)

Jianlong Fu (jianf), Heliang Zheng, Tao Mei

IEEE Conference on Computer Vision and Pattern Recognition (CVPR) | July 2017

Publication Project

Multi-level Attention Networks for Visual Question Answering

Dongfei Yu, Jianlong Fu, Yong Rui, Tao Mei

IEEE Conference on Computer Vision and Pattern Recognition (CVPR) | July 2017

Publication Project

Let Your Photos Talk: Generating Narrative Paragraph for Photo Stream via Bidirectional Attention Recurrent Neural Networks

Yu Liu, Jianlong Fu, Tao Mei, Chang Wen Chen

February 2017

Publication Project

Beyond Object Recognition: Visual Sentiment Analysis with Deep Coupled Adjective and Noun Neural Networks

Jingwen Wang, Jianlong Fu, Yong Xu, Tao Mei

International Joint Conference on Artificial Intelligence (IJCAI) | August 2016

Publication Project

Relaxing from Vocabulary: Robust Weakly-Supervised Deep Learning for Vocabulary-Free Image Tagging

Jianlong Fu, Yue Wu, Tao Mei, Jinqiao Wang, Hanqing Lu, Yong Rui

December 2015

Publication Project

Tagging Personal Photos with Transfer Deep Learning

Jianlong Fu, Tao Mei, Kuiyuan Yang, Hanqing Lu, Yong Rui

International World Wide Web conference (WWW) | May 2015

Publication Project

CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation

Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, Xiaofan Wang, Bei Liu, Jianlong Fu, Jianmin Bao, Dong Chen, Yuanchun Shi, Jiaolong Yang, Baining Guo

November 2024

DOI Publication Publication

Learning Fine-Grained Motion Embedding for Landscape Animation

Hongwei Xue, Bei Liu, Huan Yang, Jianlong Fu, Houqiang Li, Jiebo Luo

ACM Multimedia 2021 | October 2021

Oral

Publication Project

A Picture is Worth a Thousand Words: A Unified System for Diverse Captions and Rich Images Generation

Yupan Huang, Bei Liu, Jianlong Fu, Yutong Lu

ACM Multimedia 2021 | October 2021

Publication Project

MMPT’21: International Joint Workshop on Multi-Modal Pre-Training for Multimedia Understanding

Bei Liu, Jianlong Fu, Shizhe Chen, Qin Jin, Alexander Hauptmann, Yong Rui

ICMR 2021 | August 2021

DOI Project

Neural Storyboard Artist: Visualizing Stories with Coherent Image Sequences

Shizhe Chen, Bei Liu, Jianlong Fu, Ruihua Song, Qin Jin, Pingping Lin, Xiaoyu Qi, Chunting Wang, Jin Zhou

ACM Multimedia | October 2019

Publication Project

Emotion Reinforced Visual Storytelling

Nanxing Li, Bei Liu, Zhizhong Han, Yu-Shen Liu, Jianlong Fu

ICMR | June 2019

Publication Project

Beyond Narrative Description: Generating Poetry from Images by Multi-Adversarial Training

Bei Liu, Jianlong Fu, Makoto P. Kato, Masatoshi Yoshikawa

2018 ACM Multimedia | October 2018

Best Paper Award

DOI Publication PDF Project

Show, Reward and Tell: Automatic Generation of Narrative Paragraph from Photo Stream by Adversarial Training

Jing Wang, Jianlong Fu, Jinhui Tang, Zechao Li, Tao Mei

The AAAI Conference on Artificial Intelligence (AAAI), 2018. | February 2018

Publication Project

Self-view Grounding Given a Narrated 360 Video

Shih-Han Chou, Jianlong Fu, Min Sun

AAAI Conference on Artificial Intelligence | February 2018

Publication Project

Show, Adapt and Tell: Adversarial Training of Cross-domain Image Captioner

Tseng-Hung Chen, Jianlong Fu, Min Sun

IEEE International Conference on Computer Vision (ICCV) | October 2017

Publication Project

Searching Personal Photos on the Phone with Instant Visual Qery Suggestion and Joint Text-Image Hashing

Zhaoyang Zeng, Jianlong Fu, Hongyang Chao, Tao Mei

ACM International Conference on Multimedia (ACM Multimedia) | October 2017

Publication

Learning Multi-Attention Convolutional Neural Network for Fine-Grained Image Recognition (ICCV 2017 Oral)

Heliang Zheng, Jianlong Fu, Tao Mei, Jiebo Luo

IEEE International Conference on Computer Vision (ICCV) | October 2017

Publication Project

Advances in deep learning approaches for image tagging

Jianlong Fu, Yong Rui

APSIPA Transactions on Signal and Information Processing | October 2017

Publication

Let Your Photos Talk: Generating Narrative Paragraph for Photo Stream via Bidirectional Attention Recurrent Neural Networks

Yu Liu, Jianlong Fu, Tao Mei, Chang Wen Chen

February 2017

Publication Project

Beyond Object Recognition: Visual Sentiment Analysis with Deep Coupled Adjective and Noun Neural Networks

Jingwen Wang, Jianlong Fu, Yong Xu, Tao Mei

International Joint Conference on Artificial Intelligence (IJCAI) | August 2016

Publication Project

Relaxing from Vocabulary: Robust Weakly-Supervised Deep Learning for Vocabulary-Free Image Tagging

Jianlong Fu, Yue Wu, Tao Mei, Jinqiao Wang, Hanqing Lu, Yong Rui

December 2015

Publication Project

Tagging Personal Photos with Transfer Deep Learning

Jianlong Fu, Tao Mei, Kuiyuan Yang, Hanqing Lu, Yong Rui

International World Wide Web conference (WWW) | May 2015

Publication Project

Beyond Narrative Description: Generating Poetry from Images by Multi-Adversarial Training

Bei Liu, Jianlong Fu, Makoto P. Kato, Masatoshi Yoshikawa

2018 ACM Multimedia | October 2018

Best Paper Award

DOI Publication PDF Project

CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation

Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, Xiaofan Wang, Bei Liu, Jianlong Fu, Jianmin Bao, Dong Chen, Yuanchun Shi, Jiaolong Yang, Baining Guo

November 2024

DOI Publication Publication

PromptFix: You Prompt and We Fix the Photo

Yongsheng Yu, Ziyun Zeng, Hang Hua, Jianlong Fu, Jiebo Luo

NeurIPS 2024 | May 2024

DOI Publication Publication

Zero-Reference Low-Light Enhancement via Physical Quadruple Priors

Wenjing Wang, Huan Yang, Jianlong Fu, Jiaying Liu

CVPR 2024 | March 2024

DOI Publication Publication

MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation

Ludan Ruan, Yiyang Ma, Huan Yang, Huiguo He, Bei Liu, Jianlong Fu, Nicholas Jing Yuan, Qin Jin, Baining Guo

arXiv | December 2022

Publication Project

Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning

Yuchong Sun, Hongwei Xue, Ruihua Song, Bei Liu, Huan Yang, Jianlong Fu

NeurIPS 2022 | November 2022

Publication

Learning Trajectory-Aware Transformer for Video Super-Resolution

Chengxu Liu, Huan Yang, Jianlong Fu, Xueming Qian

CVPR 2022 | June 2022

Oral

Publication Project

Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions

Hongwei Xue, Tiankai Hang, Yanhong Zeng, Yuchong Sun, Bei Liu, Huan Yang, Jianlong Fu, Baining Guo

CVPR 2022 | June 2022

Publication Project

MiniViT: Compressing Vision Transformers with Weight Multiplexing

Jinnian Zhang, Houwen Peng, Kan Wu, Mengchen Liu, Bin Xiao, Jianlong Fu, Lu Yuan

CVPR 2022 | May 2022

Publication

Improving Visual Quality of Image Synthesis by A Token-based Generator with Transformers

Yanhong Zeng, Huan Yang, Hongyang Chao, Jianbo Wang, Jianlong Fu

NeurIPS 2021 | December 2021

Publication Project

Searching the Search Space of Vision Transformer

Minghao Chen, Kan Wu, Bolin Ni, Houwen Peng, Bei Liu, Jianlong Fu, Hongyang Chao, Haibin Ling

NeurIPS 2021 | December 2021

Publication

Probing Inter-modality: Visual Parsing with Self-Attention for Vision-and-Language Pre-training

Hongwei Xue, Yupan Huang, Bei Liu, Houwen Peng, Jianlong Fu, Houqiang Li, Jiebo Luo

NeurIPS 2021 | December 2021

Publication

Learning Fine-Grained Motion Embedding for Landscape Animation

Hongwei Xue, Bei Liu, Huan Yang, Jianlong Fu, Houqiang Li, Jiebo Luo

ACM Multimedia 2021 | October 2021

Oral

Publication Project

A Picture is Worth a Thousand Words: A Unified System for Diverse Captions and Rich Images Generation

Yupan Huang, Bei Liu, Jianlong Fu, Yutong Lu

ACM Multimedia 2021 | October 2021

Publication Project

Learning Conditional Knowledge Distillation for Degraded-Reference Image Quality Assessment

Heliang Zheng, Huan Yang, Jianlong Fu, Zheng-Jun Zha, Jiebo Luo

ICCV 2021 | October 2021

Publication Project

Domain-Aware Universal Style Transfer

Kibeom Hong, Seogkyu Jeon, Huan Yang, Jianlong Fu, Hyeran Byun

ICCV 2021 | October 2021

Publication Project

AutoFormer: Searching Transformers for Visual Recognition

Minghao Chen, Houwen Peng, Jianlong Fu, Haibin Ling

ICCV 2021 | October 2021

Publication

Learning Spatio-Temporal Transformer for Visual Tracking

Bin Yan, Houwen Peng, Jianlong Fu, Dong Wang, Huchuan Lu

ICCV 2021 | October 2021

Publication

Rethinking and Improving Relative Position Encoding for Vision Transformer

Kan Wu, Houwen Peng, Minghao Chen, Jianlong Fu, Hongyang Chao

ICCV 2021 | October 2021

Publication

MMPT’21: International Joint Workshop on Multi-Modal Pre-Training for Multimedia Understanding

Bei Liu, Jianlong Fu, Shizhe Chen, Qin Jin, Alexander Hauptmann, Yong Rui

ICMR 2021 | August 2021

DOI Project

Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning

Zhicheng Huang*, Zhaoyang Zeng*, Yupan Huang*, Bei Liu, Dongmei Fu, Jianlong Fu

CVPR 2021 | June 2021

Oral

Publication Project

One-Shot Neural Ensemble Architecture Search by Diversity-Guided Search Space Shrinking

Minghao Chen, Houwen Peng, Jianlong Fu, Haibin Ling

CVPR 2021 | June 2021

Publication

LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Search

Bin Yan, Houwen Peng, Kan Wu, Dong Wang, Jianlong Fu, Huchuan Lu

CVPR 2021 | June 2021

Publication

Learning Semantic-aware Normalization for Generative Adversarial Networks

Jianlong Fu

NeurIPS 2020 | October 2020

Publication Project

Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture Search

Houwen Peng, Hao Du, Hongyuan Yu, Qi Li, Jianlong Fu

NeurIPS 2020 | October 2020

Editor(s): Houwen Peng, Jianlong Fu

Publication Project

Aesthetic-Aware Image Style Transfer

Bei Liu, Jianlong Fu

ACM Multimedia | October 2020

Publication Project

Ocean: Object-aware Anchor-free Tracking

Zhipeng Zhang, Houwen Peng, Jianlong Fu, Bing Li, Weiming Hu

16th European Conference Computer Vision (ECCV 2020) | August 2020

Publication

Learning Joint Spatial-Temporal Transformations for Video Inpainting

Yanhong Zeng, Jianlong Fu, Hongyang Chao

16th European Conference Computer Vision (ECCV 2020) | August 2020

Publication

Learning Joint Spatial-Temporal Transformations for Video Inpainting

Jianlong Fu

ECCV | July 2020

Publication Project

Learning Texture Transformer Network for Image Super-Resolution

Fuzhi Yang, Huan Yang, Jianlong Fu, Hongtao Lu, Baining Guo

CVPR 2020 | June 2020

Publication Project

Neural Storyboard Artist: Visualizing Stories with Coherent Image Sequences

Shizhe Chen, Bei Liu, Jianlong Fu, Ruihua Song, Qin Jin, Pingping Lin, Xiaoyu Qi, Chunting Wang, Jin Zhou

ACM Multimedia | October 2019

Publication Project

WSOD^2: Learning Bottom-up and Top-down Objectness Distillation for Weakly-supervised Object Detection

Zhaoyang Zeng, Bei Liu, Jianlong Fu, Hongyang Chao, Lei Zhang

ICCV | October 2019

Publication Project

Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting

Jianlong Fu, Baining Guo

CVPR | July 2019

Editor(s): Yanhong Zeng, Jianlong Fu,Hongyang Chao, Baining Guo

Publication Project

Emotion Reinforced Visual Storytelling

Nanxing Li, Bei Liu, Zhizhong Han, Yu-Shen Liu, Jianlong Fu

ICMR | June 2019

Publication Project

Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captions

Qing Li, Jianlong Fu, Dongfei Yu, Tao Mei, Jiebo Luo

Conference on Empirical Methods in Natural Language Processing | October 2018

Publication

What Dress Fits Me Best? Fashion Recommendation on the Clothing Style for Personal Body Shape

Shintami Chusnul Hidayati, Jianlong Fu, Wen-Huang Cheng

ACM International Conference on Multimedia (ACM Multimedia) | October 2018

Publication

Beyond Narrative Description: Generating Poetry from Images by Multi-Adversarial Training

Bei Liu, Jianlong Fu, Makoto P. Kato, Masatoshi Yoshikawa

2018 ACM Multimedia | October 2018

Best Paper Award

DOI Publication PDF Project

Deep Attention Neural Tensor Network for Visual Question Answering

Yalong Bai, Jianlong Fu, Tiejun Zhao, Tao Mei

European Conference on Computer Vision | September 2018

Publication

DA-GAN: Instance-level Image Translation by Deep Attention Generative Adversarial Networks

Shuang Ma, Jianlong Fu, ChangWen Chen, Tao Mei

IEEE Conference on Computer Vision and Pattern Recognition (CVPR) | June 2018

Publication

Show, Reward and Tell: Automatic Generation of Narrative Paragraph from Photo Stream by Adversarial Training

Jing Wang, Jianlong Fu, Jinhui Tang, Zechao Li, Tao Mei

The AAAI Conference on Artificial Intelligence (AAAI), 2018. | February 2018

Publication Project

Self-view Grounding Given a Narrated 360 Video

Shih-Han Chou, Jianlong Fu, Min Sun

AAAI Conference on Artificial Intelligence | February 2018

Publication Project

Show, Adapt and Tell: Adversarial Training of Cross-domain Image Captioner

Tseng-Hung Chen, Jianlong Fu, Min Sun

IEEE International Conference on Computer Vision (ICCV) | October 2017

Publication Project

Searching Personal Photos on the Phone with Instant Visual Qery Suggestion and Joint Text-Image Hashing

Zhaoyang Zeng, Jianlong Fu, Hongyang Chao, Tao Mei

ACM International Conference on Multimedia (ACM Multimedia) | October 2017

Publication

Learning Multi-Attention Convolutional Neural Network for Fine-Grained Image Recognition (ICCV 2017 Oral)

Heliang Zheng, Jianlong Fu, Tao Mei, Jiebo Luo

IEEE International Conference on Computer Vision (ICCV) | October 2017

Publication Project

Look Closer to See Better: Recurrent Attention Convolutional Neural Network for Fine-grained Image Recognition (CVPR 2017 Oral)

Jianlong Fu (jianf), Heliang Zheng, Tao Mei

IEEE Conference on Computer Vision and Pattern Recognition (CVPR) | July 2017

Publication Project

Multi-level Attention Networks for Visual Question Answering

Dongfei Yu, Jianlong Fu, Yong Rui, Tao Mei

IEEE Conference on Computer Vision and Pattern Recognition (CVPR) | July 2017

Publication Project

Let Your Photos Talk: Generating Narrative Paragraph for Photo Stream via Bidirectional Attention Recurrent Neural Networks

Yu Liu, Jianlong Fu, Tao Mei, Chang Wen Chen

February 2017

Publication Project

Beyond Object Recognition: Visual Sentiment Analysis with Deep Coupled Adjective and Noun Neural Networks

Jingwen Wang, Jianlong Fu, Yong Xu, Tao Mei

International Joint Conference on Artificial Intelligence (IJCAI) | August 2016

Publication Project

Relaxing from Vocabulary: Robust Weakly-Supervised Deep Learning for Vocabulary-Free Image Tagging

Jianlong Fu, Yue Wu, Tao Mei, Jinqiao Wang, Hanqing Lu, Yong Rui

December 2015

Publication Project

Tagging Personal Photos with Transfer Deep Learning

Jianlong Fu, Tao Mei, Kuiyuan Yang, Hanqing Lu, Yong Rui

International World Wide Web conference (WWW) | May 2015

Publication Project

Solving Diffusion ODEs with Optimal Boundary Conditions for Better Image Super-Resolution

Y. Ma, Huan Yang, Wenhan Yang, Jianlong Fu, Jiaying Liu

ArXiv | May 2023, Vol abs/2305.15357

DOI PDF

Reference-Based Defect Detection Network

Zhaoyang Zeng, Bei Liu, Jianlong Fu, Hongyang Chao

IEEE Transactions on Image Processing | July 2021, Vol 30: pp. 6637-6647

Publication

Advances in deep learning approaches for image tagging

Jianlong Fu, Yong Rui

APSIPA Transactions on Signal and Information Processing | October 2017

Publication

Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers

Zhicheng Huang*, Zhaoyang Zeng*, Bei Liu, Dongmei Fu, Jianlong Fu

MSR-TR-2020-51 | April 2020

Published by Microsoft

Publication Project

Learning Rich Image Region Representation for Visual Question Answering

Bei Liu, Zhicheng Huang, Zhaoyang Zeng, Zheyu Chen, Jianlong Fu

MSR-TR-2020-1 | June 2019

Published by Microsoft

CVPR 2019, VQA Workshop

Publication Project