Hanze Dong
About
Hanze Dong is a research scientist at Salesforce Research. He earned his PhD in Mathematics from the Hong Kong University of Science and Technology (HKUST), where he was advised by Tong Zhang. He was also a visiting scholar at UC San Diego, collaborating with Yi-An Ma. Prior to his doctoral studies, he completed his Bachelor of Science in Mathematics at Fudan University under the supervision of Yanwei Fu.
He has a keen interest in comprehending the mechanisms of machine learning algorithms and aspires to develop innovative ones. His research interests encompass a broad range of topics, including generative modeling, statistical sampling, foundation models, and their theories and applications in machine learning. He also has some experience in the robustness and efficiency of machine learning algorithms.
News
[2024-07] Attending ICML 2024 at Vienna, Austria[2024-06] LMFlow has won Best Demo Award at NAACL 2024!
[2024-05] One paper has been accepted to COLT 2024, which provides the first sampling algorithm that support general non-log-Sobolev distribution with quasi-polynomial computation complexity.
[2024-05] Two papers have been accepted to ICML 2024, including Gibbs sampling from human feedback (GSHF) and stochastic proximal sampler.
[2024-03] Excited to share that LMFlow paper was accepted by NAACL 2024 Demo Track.
[2024-01] Two papers have been accepted to ICLR 2024, including rdMC and Spurious feature diversification.
[2023-12] The defense of the PhD thesis was successfully completed on November 30th, 2023.
Show More
[2023-07] Attending ICML 2023 at Honolulu, Hawaiʻi, USA.
[2023-05] Received HKUST RedBird Academic Excellence Award.
[2023-05] Attending ICLR 2023 at Kigali, Rwanda.
[2023-03] We're thrilled to release our project, LMFlow! This framework streamlines the development of LLMs, including fine-tuning, inference, and RLHF, in a more cost-effective and effortless manner. Our aspiration is that LMFlow will incite a broader range of imaginative applications of LLMs and cultivate a larger community of LLM aficionados!
Research
Generative Modeling and Sampling
-
Faster Sampling without Isoperimetry via Diffusion-based Monte Carlo
Xunpeng Huang, Difan Zou, Hanze Dong, Yian Ma, Tong Zhang;
Annual Conference on Learning Theory (COLT), 2024. [Paper] -
Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-constraint
Wei Xiong*, Hanze Dong*, Chenlu Ye*, Ziqi Wang, Han Zhong, Heng Ji, Nan Jiang, Tong Zhang;
International Conference on Machine Learning (ICML), 2024.
ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models (Oral). [Paper] -
Faster Sampling via Stochastic Gradient Proximal Sampler
Xunpeng Huang, Difan Zou, Hanze Dong, Yian Ma, Tong Zhang;
International Conference on Machine Learning (ICML), 2024. [Paper] -
Reverse Diffusion Monte Carlo.
Xunpeng Huang*, Hanze Dong*, Yifan Hao, Yian Ma, Tong Zhang;
International Conference on Learning Representations (ICLR), 2024. [Paper] -
LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models.
Shizhe Diao*, Rui Pan*, Hanze Dong*, Kashun Shum, Jipeng Zhang, Wei Xiong, Tong Zhang;
Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) -- System Demonstration Track, 2024. (Best Demo Award) [Paper] [Github] -
Raft: Reward ranked finetuning for generative foundation model alignment
Hanze Dong*, Wei Xiong*, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, Tong Zhang;
Transactions on Machine Learning Research (TMLR), 2023. [Paper] -
Particle-based Variational Inference with Preconditioned Functional Gradient Flow
Hanze Dong, Xi Wang, Yong Lin, Tong Zhang;
International Conference on Learning Representations (ICLR), 2023. [Paper] -
DetGPT: Detect What You Need via Reasoning
Renjie Pi, Jiahui Gao, Shizhe Diao, Rui Pan, Hanze Dong, Jipeng Zhang, Lewei Yao, Jianhua Han, Hang Xu, Lingpeng Kong, Tong Zhang;
Empirical Methods in Natural Language Processing (EMNLP), 2023. [Paper] -
Weakly Supervised Disentangled Generative Causal Representation Learning
Xinwei Shen, Furui Liu, Hanze Dong, Qing Lian, Zhitang Chen, Tong Zhang;
Journal of Machine Learning Research (JMLR), 2022. [Paper] -
Normalizing Flow with Variational Latent Representation
Hanze Dong*, Shizhe Diao*, Weizhong Zhang, Tong Zhang;
arXiv preprint arXiv:2211.11638, 2022. [Paper]
Robust Machine Learning
-
Spurious feature diversification improves out-of-distribution generalization
Yong Lin, Lu Tan, Yifan Hao, Honam Wong, Hanze Dong, Weizhong Zhang, Yujiu Yang, Tong Zhang;
International Conference on Learning Representations (ICLR), 2024. -
Bayesian Invariant Risk Minimization
Yong Lin*, Hanze Dong*, Hao Wang, Tong Zhang;
Conference on Computer Vision and Pattern Recognition (CVPR), 2022. (Oral) [Paper] -
Learning the Compositional Spaces for Generalized Zero-shot Learning
Hanze Dong, Yanwei Fu, Leonid Sigal, Sung Ju Hwang, Xiangyang Xue;
Computer Vision and Image Understanding (CVIU), 2022. [Paper] -
Vocabulary-informed Zero-shot and Open-set Learning
Yanwei Fu, Xiaomei Wang, Hanze Dong, Yu-Gang Jiang, Meng Wang, Xiangyang Xue, Leonid Sigal;
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019. [Paper] -
Extreme vocabulary learning
Hanze Dong, Zhenfeng Sun, Yanwei Fu, Shi Zhong, Zhengjun Zhang, Yu-Gang Jiang;
Frontiers of Computer Science, 2019. [Paper] -
Local Augmentation for Graph Neural Networks
Songtao Liu, Rex Ying, Hanze Dong, Lanqing Li, Tingyang Xu, Yu Rong, Peilin Zhao, Junzhou Huang, Dinghao Wu;
International Conference on Machine Learning (ICML), 2022. [Paper]
Optimization and Efficient Machine Learning
-
Catalyst Acceleration of Error Compensated Methods Leads to Better Communication Complexity
Xun Qian, Hanze Dong, Tong Zhang, Peter Richtárik;
International Conference on Artificial Intelligence and Statistics (AISTATS), 2023. [Paper] -
Error Compensated Loopless SVRG for Distributed Optimization
Xun Qian, Hanze Dong, Peter Richtárik, Tong Zhang;
OPT2020: 12th Annual Workshop on Optimization for Machine Learning, 2020. [Paper] -
Error Compensated Proximal SGD and RDA
Xun Qian, Hanze Dong, Peter Richtárik, Tong Zhang;
OPT2020: 12th Annual Workshop on Optimization for Machine Learning, 2020. [Paper]
Machine Learning Theory
-
Mathematical models of Overparameterized Neural Networks
Cong Fang, Hanze Dong, Tong Zhang;
Proceedings of the IEEE (PIEEE), 2021. [Paper] -
Provable Particle-based Primal-Dual Algorithm for Mixed Nash Equilibrium
Shihong Ding*, Hanze Dong*, Cong Fang, Zhouchen Lin, Tong Zhang;
arXiv preprint arXiv:2303.00970, 2023. [Paper]
Academic Services
Teaching
-
2022-2023 Fall --
MATH 2011: Introduction to Multivariable Calculus.
2022-2023 Fall -- MATH 6450J/COMP 6211E: Optimization for Machine Learning.
2021-2022 Spring -- MATH 2011: Introduction to Multivariable Calculus.
2021-2022 Fall -- MATH 1023: Honor Calculus.
2020-2021 Spring -- MATH 1014: Calculus II.
2020-2021 Fall -- MATH 1013: Calculus IB.
2019-2020 Spring -- MATH 1014: Calculus II.
2019-2020 Spring -- MATH 2411: Applied Statistics.
Awards
-
2022/2023 --
RedBird PhD Award, The Hong Kong University of Science and Technology.
2020/2021/2022 -- Best TA Award, The Hong Kong University of Science and Technology.
2019 -- Outstanding Graduates, Fudan University.
2016 -- The First Price, China Undergraduate Physics Tournament
2015 -- Outstanding Freshmen, Fudan University
2014 -- Bronze Medal and The First Price, Chinese Physics Olympiad.
Contact Me
A at B dot com
A = hendrydong
B = gmail