Publications | Jinyi Liu (刘金毅) | PhD Candidate in RL, LLMs, and Agentic Systems

Squeeze the Soaked Sponge: Efficient Off-policy RFT for Large Language Model

Jing Liang ^* , Jinyi Liu ^* , Yi Ma ^* , Hongyao Tang , Yan Zheng , Shuyue Hu , Lei Bai , Jianye Hao

The Fourteenth International Conference on Learning Representations (ICLR 2026 Poster) · Jan 2026

LLM Post-training (RL Tuning) LLM Post-training RLVR Off-policy

CellAgent: LLM-Driven Multi-Agent Framework for Natural Language-Based Single-Cell Analysis

Yihang Xiao ^* , Jinyi Liu ^* , Yan Zheng ^* , Shaoqing Jiao ^* , Jianye Hao , Xiaohan Xie , Mingzhi Li , Ruitao Wang , Fei Ni , Yuxiao Li , Zhen Wang , Xuequn Shang , Zhijie Bao , Changxiao Yang , Jiajie Peng

The Fourteenth International Conference on Learning Representations (ICLR 2026 Poster) · Jan 2026

LLM Agent Single-cell data analysis Spatial transcriptomics ICLR 2026

Details PDF

From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulation

Yifu Yuan , Haiqin Cui , Yibin Chen , Zibin Dong , Fei Ni , Longxin Kou , Jinyi Liu , Pengyi Li , Yan Zheng , Jianye Hao

The Fourteenth International Conference on Learning Representations (ICLR 2026 Poster) · Jan 2026

Embodied AI LLM Agent Robotics VLM Manipulation Spatial Reasoning Zero-shot Learning ICLR 2026

Details PDF

Benchmarking Continual Agent Memory for Online Learning, Transfer, and Forgetting

Zihang Ma ^* , Jinyi Liu ^* , Hongyao Tang , Yi Ma , Ruitao Wang , Yifu Yuan , Yan Zheng , Jianye Hao

LLA@ICLR 2026 Poster · Jan 2026

LLM Agent Memory Continual Learning Benchmark LLA 2026

Details

Beyond Scalar Critics: A Distributional Perspective on Reinforcement Learning with Verifiable Rewards for LLMs

Jinyi Liu , Yiboyun Chen , Hongyao Tang , Yi Ma , Shuyue Hu , Yang Chen , Fei Ni , Qiaosheng Zhang , Lei Bai , Yan Zheng , Jianye Hao

SPOT@ICLR 2026 · Jan 2026

LLM Post-training (RL Tuning) RLVR Value Distribution Reasoning SPOT 2026

Details

AFE-Master: Enhancing LLM-Driven Autonomous Feature Engineering with Domain-Specific Language Parsing and Guided Local Search

Hebin Liang , Jianye Hao , Jinyi Liu , Yi Ma , Zilin Cao , Jing Liang , Kun Shao , Zhaocheng Du , Fei Ni , Yifu Yuan , Yan Zheng

ACM TheWebConf 2026 Industry · Jan 2026

LLM Agent Feature Engineering

Details

Hands-on LLM-based Agents: A Tutorial for General Audiences

Shuyue Hu , Siyue Ren , Yang Chen , Chunjiang Mu , Jinyi Liu , Zhiyao Cui , Yiqun Zhang , Hao Li , Dongzhan Zhou , Jia Xu , others

TechRxiv · Nov 2025

LLM Agent Tutorial

Details PDF

Improving Reward Models with Proximal Policy Exploration for Preference-Based Reinforcement Learning

Yiwen Zhu ^* , Jinyi Liu ^* , Pengjie Gu , Yifu Yuan , Zhenxing Ge , Wenya Wei , Zhou Fang , Yujing Hu , Bo An

NeurIPS 2025 · Sep 2025

DRL Exploration Preference Learning

Details

From Chaos to Order: The Atomic Reasoner Framework for Fine-grained Reasoning in Large Language Models

Jinyi Liu , Yan Zheng , Rong Cheng , Qiyu Wu , Wei Guo , Fei Ni , Hebin Liang , Yifu Yuan , Hangyu Mao , Fuzheng Zhang , others

SCALR@COLM 2025 · Aug 2025

LLM Post-training (TTS) LLM Agent Reasoning

Details PDF

RoboAnnotatorX: A Comprehensive and Universal Annotation Framework for Accurate Understanding of Long-horizon Robot Demonstration

Longxin Kou , Fei Ni , Jianye Hao , Peilong Han , Jinyi Liu , Haiqin Cui , Rui Liu , Yan Zheng

Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) 2025 · Jul 2025

Embodied AI Robotics Computer Vision Annotation Robot Demonstration

Details

War of Thoughts: Competition Stimulates Stronger Reasoning in Large Language Models

Yibin Chen ^* , Jinyi Liu ^* , Yan Zheng , Yifu Yuan , Jianye Hao

Findings of the Association for Computational Linguistics: ACL 2025 · May 2025

LLM Post-training (TTS) ACL 2025 ACL Findings Competition

Details

DualRAG: A Dual-Process Approach to Integrate Reasoning and Retrieval for Multi-Hop Question Answering

Rong Cheng , Jinyi Liu , Yan Zheng , Fei Ni , Jiazhen Du , Hangyu Mao , Fuzheng Zhang , Bo Wang , Jianye Hao

Proceedings of the Association for Computational Linguistics: ACL 2025 · May 2025

LLM Agent LLM Post-training (TTS) Reasoning

Details

Key Decision-Makers in Multi-Agent Debates: Who Holds the Power?

Qian Zhang , Jinyi Liu , Yan Zheng , Hebin Liang , Lanjun Wang

AAAI Conference on Artificial Intelligence, 2025 (Poster) · Feb 2025

LLM Post-training (TTS) LLM Agent

Details

SheetAgent: towards a generalist agent for spreadsheet reasoning and manipulation via large language models

Yibin Chen , Yifu Yuan , Zeyu Zhang , Yan Zheng , Jinyi Liu , Fei Ni , Jianye Hao , Hangyu Mao , Fuzheng Zhang

Proceedings of the ACM on Web Conference 2025 · Jan 2025

LLM Agent LLM WWW 2025 Oral

Details

Optimizing Reward Models with Proximal Policy Exploration in Preference-Based Reinforcement Learning

Yiwen Zhu , Jinyi Liu , Yifu Yuan , Wenya Wei , Zhenxing Ge , Zhou Fang , Yujing Hu , Bo An , others

NeurIPS 2024 Workshop on Behavioral Machine Learning · Jul 2024

DRL PbRL Exploration

Details PDF

Ovd-explorer: Optimism should not be the sole pursuit of exploration in noisy environments

Jinyi Liu , Zhi Wang , Yan Zheng , Jianye Hao , Chenjia Bai , Junjie Ye , Zhen Wang , Haiyin Piao , Yang Sun

Proceedings of the AAAI Conference on Artificial Intelligence · May 2024

DRL Exploration AAAI 2024

Details PDF

A trajectory perspective on the role of data sampling techniques in offline reinforcement learning

Jinyi Liu , Yi Ma , Jianye Hao , Yujing Hu , Yan Zheng , Tangjie Lv , Changjie Fan

Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems · May 2024

DRL Offline RL AAMAS 2024

Details PDF

Enhancing robotic manipulation with AI feedback from multimodal large language models

Jinyi Liu , Yifu Yuan , Jianye Hao , Fei Ni , Lingzhi Fu , Yibin Chen , Yan Zheng

arXiv preprint arXiv:2402.14245 · Feb 2024

Embodied AI LLM Agent DRL LLM

Details PDF

vMFER: Von Mises-Fisher experience resampling based on uncertainty of gradient directions for policy improvement

Yiwen Zhu , Jinyi Liu , Wenya Wei , Qianyi Fu , Yujing Hu , Zhou Fang , Bo An , Jianye Hao , Tangjie Lv , Changjie Fan

Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence · Jan 2024

DRL

Details

Uni-RLHF: Universal Platform and Benchmark Suite for Reinforcement Learning with Diverse Human Feedback

Yifu Yuan , HAO Jianye , Yi Ma , Zibin Dong , Hebin Liang , Jinyi Liu , Zhixin Feng , Kai Zhao , Yan Zheng

The Twelfth International Conference on Learning Representations · Jan 2024

DRL PbRL

Details

Peria: Perceive, reason, imagine, act via holistic language and vision planning for manipulation

Fei Ni , Jianye Hao , Shiguang Wu , Longxin Kou , Yifu Yuan , Zibin Dong , Jinyi Liu , Mingzhi Li , Yuzheng Zhuang , Yan Zheng

Advances in Neural Information Processing Systems · Jan 2024

Embodied AI LLM Agent LLM

Details

Kisa: A unified keyframe identifier and skill annotator for long-horizon robotics demonstrations

Longxin Kou , Fei Ni , Yan Zheng , Jinyi Liu , Yifu Yuan , Zibin Dong , Jianye Hao

Forty-first International Conference on Machine Learning · Jan 2024

Embodied AI LLM

Details

ENOTO: improving offline-to-online reinforcement learning with Q-ensembles

Kai Zhao , Jianye Hao , Yi Ma , Jinyi Liu , Yan Zheng , Zhaopeng Meng

Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence · Jan 2024

DRL Offline RL

Details

OSCAR: OOD State-Conservative Offline Reinforcement Learning for Sequential Decision Making

Yi Ma , Chao Wang , Chen Chen , Jinyi Liu , Zhaopeng Meng , Yan Zheng , Jianye Hao

CAAI Artificial Intelligence Research · Jan 2023

DRL

Details

Exploration in deep reinforcement learning: From single-agent to multiagent domain

Jianye Hao , Tianpei Yang , Hongyao Tang , Chenjia Bai , Jinyi Liu , Zhaopeng Meng , Peng Liu , Zhen Wang

IEEE Transactions on Neural Networks and Learning Systems · Jan 2023

DRL Exploration

Details

EUCLID: Towards Efficient Unsupervised Reinforcement Learning with Multi-choice Dynamics Model

Yifu Yuan , HAO Jianye , Fei Ni , Yao Mu , Yan Zheng , Yujing Hu , Jinyi Liu , Yingfeng Chen , Changjie Fan

The Eleventh International Conference on Learning Representations · Jan 2023

DRL

Details

Figcps: Effective failure-inducing input generation for cyber-physical systems with deep reinforcement learning

Shaohua Zhang , Shuang Liu , Jun Sun , Yuqi Chen , Wenzhi Huang , Jinyi Liu , Jian Liu , Jianye Hao

2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) · Jan 2021

DRL Software Engineering

Details