Lin Xiaoqiang

I am a fourth-year PhD student from National University of Singapore, advised by Assoc. Prof. Bryan Low and Prof. See-Kiong Ng. Before that, I was a full time machine learning engineer at Ant Group. I received my Bachelor's degree from the Fudan University, where I worked with Assoc. Prof. Zhongyu Wei.

Recent Updates

[28 Nov 2024] I gave a talk at Max Planck Research School for Intelligent Systems (IMPRS-IS) and University of Stuttgart on prompt optimization, slides are available here.

[26 Sep 2024] Our 3 papers are accepted by NeurIPS 2024!

[20 Sep 2024] Our position paper is accepted by EMNLP Findings 2024!

[5 Aug 2024] I receive Research Achievement Award from NUS School of Computing.

[8 Jun 2024] Our 3 papers on prompt optimization and data attribution for LLMs are avaliable on arxiv now.

[2 May 2024] Our 3 papers are accepted by ICML 2024!

[5 Oct 2023] Our instruction optimization paper (INSTINCT) is avaliable on arxiv now.

[10 Aug 2023] I receive Research Achievement Award from NUS School of Computing.

[8 Aug 2023] Our paper "Federated Zeroth-Order Optimization using Trajectory-Informed Surrogate Gradients" is avaliable on arxiv now.

[24 Apr 2023] Our paper "Fair yet Asymptotically Equal Collaborative Learning" is accepted by ICML 2023.

Research Interests

Data-centric AI for large models (e.g., data selection/curation in different stages of training for large models and use of data at inference).

Collaborative machine learning (e.g., federated learning, incentive mechanism).

Data valuation (e.g., data pricing, data subset selection, data debugging).

Gaussian process and its applications (e.g., zeroth-order optimization, Bayesian optimization).

Publications

Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplars.
Zhaoxuan Wu*, Xiaoqiang Lin*, Zhongxiang Dai, Wenyang Hu, Yao Shu, See-Kiong Ng, Patrick Jaillet, Bryan Kian Hsiang Low.
Neural Information Processing Systems (NeurIPS), 2024.
Also presented at ICML 2024, Workshop on In-Context Learning
Paper / Code

DETAIL: Task DEmonsTration Attribution for Interpretable In-context Learning.
Zijian Zhou, Xiaoqiang Lin, Xinyi Xu, Alok Prakash, Daniela Rus, Bryan Kian Hsiang Low.
Neural Information Processing Systems (NeurIPS), 2024.
Also presented at ICML 2024, Workshop on In-Context Learning
Paper / Code

Localized Zeroth-Order Prompt Optimization.
Wenyang Hu, Yao Shu, Zongmin Yu, Zhaoxuan Wu, Xiangqiang Lin, Zhongxiang Dai, See-Kiong Ng, Bryan Kian Hsiang Low.
Neural Information Processing Systems (NeurIPS), 2024.
Also presented at ICML 2024, Workshop on In-Context Learning
Paper

Position Paper: Data-Centric AI in the Age of Large Language Models.
Xinyi Xu, Zhaoxuan Wu, Rui Qiao, Arun Verma, Yao Shu, Jingtan Wang, Xinyuan Niu, Zhenfeng He, Jiangwei Chen, Zijian Zhou, Gregory Kang Ruey Lau, Hieu Dao, Lucas Agussurja, Rachael Hwee Ling Sim, Xiaoqiang Lin, Wenyang Hu, Zhongxiang Dai, Pang Wei Koh, Bryan Kian Hsiang Low.
Empirical Methods in Natural Language Processing Findings (EMNLP Findings), 2024.
Paper

Distributionally Robust Data Valuation.
Xiaoqiang Lin, Xinyi Xu, Zhaoxuan Wu, See-Kiong Ng, Bryan Kian Hsiang Low.
International Conference on Machine Learning (ICML), 2024.
Paper

Use Your INSTINCT: INSTruction optimization for LLMs usIng Neural bandits Coupled with Transformers.
Xiaoqiang Lin*, Zhaoxuan Wu*, Zhongxiang Dai, Wenyang Hu, Yao Shu, See-Kiong Ng, Patrick Jaillet, Bryan Kian Hsiang Low.
International Conference on Machine Learning (ICML), 2024.
Also presented at NeurIPS 2023, Workshop on Instruction Tuning and Instruction Following
Project Page / Paper / Code / Bibtex

Helpful or Harmful Data? Fine-tuning-free Shapley Attribution for Explaining Language Model Predictions.
Jingtan Wang*, Xiaoqiang Lin*, Rui Qiao*, Chuan-Sheng Foo, Bryan Kian Hsiang Low.
International Conference on Machine Learning (ICML), 2024.
Paper / Code

Fair yet Asymptotically Equal Collaborative Learning.
Xiaoqiang Lin*, Xinyi Xu*, See-Kiong Ng, Chuan-Sheng Foo, Bryan Kian Hsiang Low.
International Conference on Machine Learning (ICML), 2023.
Paper / Code / Bibtex

Joint Representation Learning of Legislator and Legislation for Roll Call Prediction.
Yuqiao Yang*, Xiaoqiang Lin*, Geng Lin, Zengfeng Huang, Changjian Jiang, Zhongyu Wei.
International Joint Conferences on Artificial Intelligence (IJCAI), 2020.
Paper / Code / Bibtex

(* denotes equal contribution)

Book Chapters

Fairness in Federated Learning.
Xiaoqiang Lin, Xinyi Xu, Zhaoxuan Wu, Rachael Hwee Ling Sim, See-Kiong Ng, Chuan-Sheng Foo, Patrick Jaillet, Trong Nghia Hoang,, Bryan Kian Hsiang Low.
In L. M. Nguyen, T. N. Hoang, P.-Y. Chen, editors, Federated Learning: Theory and Practice, chapter 8, pages 143-160, Academic Press, 2024.

Data Valuation in Federated Learning.
Zhaoxuan Wu, Xinyi Xu, Rachael Hwee Ling Sim, Yao Shu, Xiaoqiang Lin, Lucas Agussurja, Zhongxiang Dai, See-Kiong Ng, Chuan-Sheng Foo, Patrick Jaillet, Trong Nghia Hoang, Bryan Kian Hsiang Low.
In L. M. Nguyen, T. N. Hoang, P.-Y. Chen, editors, Federated Learning: Theory and Practice, chapter 8, pages 143-160, Academic Press, 2024.

Incentives in Federated Learning.
Rachael Hwee Ling Sim, Sebastian Shenghong Tay, Xinyi Xu, Yehong Zhang, Zhaoxuan Wu, Xiaoqiang Lin, See-Kiong Ng, Chuan-Sheng Foo, Patrick Jaillet, Trong Nghia Hoang, Bryan Kian Hsiang Low.
In L. M. Nguyen, T. N. Hoang, P.-Y. Chen, editors, Federated Learning: Theory and Practice, chapter 8, pages 143-160, Academic Press, 2024.

Workshop Papers & Pre-Prints

Neural Dueling Bandits.
Arun Verma, Zhongxiang Dai, Xiaoqiang Lin, Patrick Jaillet, Bryan Kian Hsiang Low.
ICML 2024, Workshop on Foundations of RL and Control.
Paper

Prompt Optimization with Human Feedback.
Xiaoqiang Lin, Zhongxiang Dai, Arun Verma, See-Kiong Ng, Patrick Jaillet, Bryan Kian Hsiang Low.
ICML 2024, Workshop on Models of Human Feedback for AI Alignment.
Selected as Oral
Oral presentation / Paper / Code

Federated Zeroth-Order Optimization using Trajectory-Informed Surrogate Gradients.
Yao Shu, Xiaoqiang Lin, Zhongxiang Dai, Bryan Kian Hsiang Low.
Pre-print, 2023. [arXiv]
Paper / Bibtex

Invited Talks

Use Your INSTINCT: INSTruction optimization usIng Neural bandits Coupled with Transformers.
@ Deep Learning and Optimization Seminar (Jointly organized by Westlake University, City University of Hong Kong, Peking University). Oct 24, 2023. Video

Prompt Optimization in the Wild - Challenges and Opportunities.
@ Max Planck Research School for Intelligent Systems (IMPRS-IS) and University of Stuttgart. Nov 28, 2024. Slides

Awards and Honors

Award of Teaching Fellowship, National University of Singapore, 2023 (1 of 3 selected CS Ph.D. students).
Research Achievement Award × 2, NUS, School of Computing, 2023 & 2024.
Outstanding Graduates, Fudan University, 2020.
Excellent Student Scholarship, Fudan University, 2017, 2018, 2019.

Professional Services

Conference Reviewer for ICLR’2024.
Conference Reviewer for NeurIPS’2024.
Conference Reviewer for AISTATS’2024.
Conference Reviewer for AAAI’2023, 2024.
Conference Reviewer for ACML’2023.
Conference Reviewer for ICML’2021, 2024.

Contact

You are very welcome to contact me regarding my research. I am open to any talk invitations on my research. I typically respond within a few days.
I can be contacted directly at xiaoqiang.lin [at] u.nus.edu