Hanze Dong

About

Hanze Dong is a research scientist at Salesforce Research. He earned his PhD in Mathematics from the Hong Kong University of Science and Technology (HKUST), where he was advised by Tong Zhang. He was also a visiting scholar at UC San Diego, collaborating with Yi-An Ma. Prior to his doctoral studies, he completed his Bachelor of Science in Mathematics at Fudan University under the supervision of Yanwei Fu.

He has a keen interest in comprehending the mechanisms of machine learning algorithms and aspires to develop innovative ones. His research interests encompass a broad range of topics, including generative modeling, statistical sampling, foundation models, and their theories and applications in machine learning. He also has some experience in the robustness and efficiency of machine learning algorithms.

News

[2024-05] One paper has been accepted to COLT 2024, which provides the first sampling algorithm that support general non-log-Sobolev distribution with quasi-polynomial computation complexity.
[2024-05] Two papers have been accepted to ICML 2024, including Gibbs sampling from human feedback (GSHF) and stochastic proximal sampler.
[2024-03] Excited to share that LMFlow paper was accepted by NAACL 2024 Demo Track.
[2024-01] Two papers have been accepted to ICLR 2024, including rdMC and Spurious feature diversification.
[2023-12] The defense of the PhD thesis was successfully completed on November 30th, 2023.
[2023-11] Excited to share that RAFT has been accepted to TMLR. [Link].
[2023-07] Attending ICML 2023 at Honolulu, Hawaiʻi, USA.
[2023-05] Received HKUST RedBird Academic Excellence Award.
[2023-05] Attending ICLR 2023 at Kigali, Rwanda.
[2023-03] We're thrilled to release our project, LMFlow! This framework streamlines the development of LLMs, including fine-tuning, inference, and RLHF, in a more cost-effective and effortless manner. Our aspiration is that LMFlow will incite a broader range of imaginative applications of LLMs and cultivate a larger community of LLM aficionados!

Research

* indicates equal contribution.

Generative Modeling and Sampling

  • Faster Sampling without Isoperimetry via Diffusion-based Monte Carlo
    Xunpeng Huang, Difan Zou, Hanze Dong, Yian Ma, Tong Zhang;
    Annual Conference on Learning Theory (COLT), 2024. [Paper]
  • Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-constraint
    Wei Xiong*, Hanze Dong*, Chenlu Ye*, Ziqi Wang, Han Zhong, Heng Ji, Nan Jiang, Tong Zhang;
    International Conference on Machine Learning (ICML), 2024.
    ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models (Oral). [Paper]
  • Faster Sampling via Stochastic Gradient Proximal Sampler
    Xunpeng Huang, Difan Zou, Yian Ma, Hanze Dong, Tong Zhang;
    International Conference on Machine Learning (ICML), 2024.
  • Reverse Diffusion Monte Carlo.
    Xunpeng Huang*, Hanze Dong*, Yifan Hao, Yian Ma, Tong Zhang;
    International Conference on Learning Representations (ICLR), 2024. [Paper]
  • LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models.
    Shizhe Diao*, Rui Pan*, Hanze Dong*, Kashun Shum, Jipeng Zhang, Wei Xiong, Tong Zhang;
    Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) -- System Demonstration Track, 2024. [Paper] [Github]
  • Raft: Reward ranked finetuning for generative foundation model alignment
    Hanze Dong*, Wei Xiong*, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, Tong Zhang;
    Transactions on Machine Learning Research (TMLR), 2023. [Paper]
  • Particle-based Variational Inference with Preconditioned Functional Gradient Flow
    Hanze Dong, Xi Wang, Yong Lin, Tong Zhang;
    International Conference on Learning Representations (ICLR), 2023. [Paper]
  • DetGPT: Detect What You Need via Reasoning
    Renjie Pi, Jiahui Gao, Shizhe Diao, Rui Pan, Hanze Dong, Jipeng Zhang, Lewei Yao, Jianhua Han, Hang Xu, Lingpeng Kong, Tong Zhang;
    Empirical Methods in Natural Language Processing (EMNLP), 2023. [Paper]
  • Weakly Supervised Disentangled Generative Causal Representation Learning
    Xinwei Shen, Furui Liu, Hanze Dong, Qing Lian, Zhitang Chen, Tong Zhang;
    Journal of Machine Learning Research (JMLR), 2022. [Paper]
  • Normalizing Flow with Variational Latent Representation
    Hanze Dong*, Shizhe Diao*, Weizhong Zhang, Tong Zhang;
    arXiv preprint arXiv:2211.11638, 2022. [Paper]

Robust Machine Learning

  • Spurious feature diversification improves out-of-distribution generalization
    Yong Lin, Lu Tan, Yifan Hao, Honam Wong, Hanze Dong, Weizhong Zhang, Yujiu Yang, Tong Zhang;
    International Conference on Learning Representations (ICLR), 2024.
  • Bayesian Invariant Risk Minimization
    Yong Lin*, Hanze Dong*, Hao Wang, Tong Zhang;
    Conference on Computer Vision and Pattern Recognition (CVPR), 2022. (Oral) [Paper]
  • Learning the Compositional Spaces for Generalized Zero-shot Learning
    Hanze Dong, Yanwei Fu, Leonid Sigal, Sung Ju Hwang, Xiangyang Xue;
    Computer Vision and Image Understanding (CVIU), 2022. [Paper]
  • Vocabulary-informed Zero-shot and Open-set Learning
    Yanwei Fu, Xiaomei Wang, Hanze Dong, Yu-Gang Jiang, Meng Wang, Xiangyang Xue, Leonid Sigal;
    IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019. [Paper]
  • Extreme vocabulary learning
    Hanze Dong, Zhenfeng Sun, Yanwei Fu, Shi Zhong, Zhengjun Zhang, Yu-Gang Jiang;
    Frontiers of Computer Science, 2019. [Paper]
  • Local Augmentation for Graph Neural Networks
    Songtao Liu, Rex Ying, Hanze Dong, Lanqing Li, Tingyang Xu, Yu Rong, Peilin Zhao, Junzhou Huang, Dinghao Wu;
    International Conference on Machine Learning (ICML), 2022. [Paper]

Optimization and Efficient Machine Learning

  • Catalyst Acceleration of Error Compensated Methods Leads to Better Communication Complexity
    Xun Qian, Hanze Dong, Tong Zhang, Peter Richtárik;
    International Conference on Artificial Intelligence and Statistics (AISTATS), 2023. [Paper]
  • Error Compensated Loopless SVRG for Distributed Optimization
    Xun Qian, Hanze Dong, Peter Richtárik, Tong Zhang;
    OPT2020: 12th Annual Workshop on Optimization for Machine Learning, 2020. [Paper]
  • Error Compensated Proximal SGD and RDA
    Xun Qian, Hanze Dong, Peter Richtárik, Tong Zhang;
    OPT2020: 12th Annual Workshop on Optimization for Machine Learning, 2020. [Paper]

Machine Learning Theory

  • Mathematical models of Overparameterized Neural Networks
    Cong Fang, Hanze Dong, Tong Zhang;
    Proceedings of the IEEE (PIEEE), 2021. [Paper]
  • Provable Particle-based Primal-Dual Algorithm for Mixed Nash Equilibrium
    Shihong Ding*, Hanze Dong*, Cong Fang, Zhouchen Lin, Tong Zhang;
    arXiv preprint arXiv:2303.00970, 2023. [Paper]

Academic Services

Teaching

    2022-2023 Fall -- MATH 2011: Introduction to Multivariable Calculus.
    2022-2023 Fall -- MATH 6450J/COMP 6211E: Optimization for Machine Learning.
    2021-2022 Spring -- MATH 2011: Introduction to Multivariable Calculus.
    2021-2022 Fall -- MATH 1023: Honor Calculus.
    2020-2021 Spring -- MATH 1014: Calculus II.
    2020-2021 Fall -- MATH 1013: Calculus IB.
    2019-2020 Spring -- MATH 1014: Calculus II.
    2019-2020 Spring -- MATH 2411: Applied Statistics.

Awards

    2022/2023 -- RedBird PhD Award, The Hong Kong University of Science and Technology.
    2020/2021/2022 -- Best TA Award, The Hong Kong University of Science and Technology.
    2019 -- Outstanding Graduates, Fudan University.
    2016 -- The First Price, China Undergraduate Physics Tournament
    2015 -- Outstanding Freshmen, Fudan University
    2014 -- Bronze Medal and The First Price, Chinese Physics Olympiad.

Contact Me

A at B dot com
A = hendrydong
B = gmail