Haiyue Song

宋海越 · LLM Engineer at Preferred Networks

prof_pic.jpg

haiyuesong at preferred.jp

I am an LLM engineer at Preferred Networks, working on post-training. Before that, I was a technical researcher at NICT, working on machine translation. I received my Ph.D. from Kyoto University in 2024, advised by Prof. Sadao Kurohashi and Prof. Chenhui Chu.

My research interests center on large language models, including agent harness, long-context, reinforcement learning, and continual pre-training. Previously, I worked on machine translation, especially in low-resource, multilingual, and multimodal scenarios.

[Publications]

News

Sep 19, 2025 Four papers accepted to IJCNLP-AACL 2025 (three Main, one Demo)!
Aug 19, 2025 One paper accepted to EMNLP 2025 Findings!
May 31, 2025 Organizing the English-Indic Language Document Translation Task at WAT 2025.
Mar 31, 2025 Obtained a Grant-in-Aid for Early-Career Scientists (若手研究)!
Jan 31, 2025 Serving as Website Chair for EMNLP 2025.
Jan 19, 2025 Presented a tutorial at COLING 2025.

Education

Kyoto University

Supervised by Prof. Sadao Kurohashi and Prof. Chenhui Chu.

  • Ph.D. in Intelligence Science and Technology, October 2020 - March 2024
  • Master in Intelligence Science and Technology, October 2018 - September 2020

Shanghai Jiao Tong University

Supervised by Prof. Li Jiang.

  • Bachelor of Computer Science and Technology, September 2014 - July 2018
  • Minor in Japanese, School of Foreign Languages, February 2015 - July 2018

Nagoya University

  • Exchange student, October 2017 - February 2018

Work Experience

Preferred Networks

  • Engineer, April 2026 - present

National Institute of Information and Communications Technology (NICT)

  • Technical researcher, July 2023 - March 2026
  • Research internship, October 2019 - June 2023

JSPS

Kyoto University

  • Research assistant, November 2020 - March 2021

LINE

  • Internship, machine learning team, February 2019 - March 2019 (summary report)

Selected Publications

  1. arXiv
    optimer.png
    OptiMer: Optimal Distribution Vector Merging Is Better than Data Mixing for Continual Pre-Training
    Haiyue Song and Masao Utiyama
    arXiv preprint, 2026
  2. IJCNLP-AACL
    formatrl.png
    Structured Document Translation via Format Reinforcement Learning
    In Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics, acceptance rate 19% , Dec 2025
  3. IWSLT
    iwslt.png
    NICT’s Cascaded and End-To-End Speech Translation Systems using Whisper and IndicTrans2 for the Indic Task
    Raj Dabre and Haiyue Song
    In Proceedings of the 21st International Conference on Spoken Language Translation (IWSLT 2024), Aug 2024
  4. EAMT
    submerge.png
    SubMerge: Merging Equivalent Subword Tokenizations for Subword Regularized Models in Neural Machine Translation
    In Proceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 1), Jun 2024
  5. LREC-COLING
    nglueni.png
    NGLUEni: Benchmarking and Adapting Pretrained Language Models for Nguni Languages
    Francois Meyer, Haiyue Song, Abhisek Chakrabarty, Jan Buys, Raj Dabre, and Hideki Tanaka
    In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), May 2024
  6. JNLP
    diverseg.png
    DiverSeg: Leveraging Diverse Segmentations with Cross-granularity Alignment for Neural Machine Translation
    Journal of Natural Language Processing, 2024
  7. TALLIP
    selfseg.png
    SelfSeg: A Self-Supervised Sub-Word Segmentation Method for Neural Machine Translation
    ACM Transactions on Asian and Low-Resource Language Information Processing, Aug 2023
  8. AACL
    bertseg.png
    BERTSeg: BERT Based Unsupervised Subword Segmentation for Neural Machine Translation
    In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), Nov 2022
  9. LREC
    coursera.png
    Coursera Corpus Mining and Multistage Fine-Tuning for Improving Lectures Translation
    In Proceedings of the 12th Language Resources and Evaluation Conference (LREC 2020), May 2020
  10. ICCAD
    iccad.png
    Invocation-driven Neural Approximate Computing with a Multiclass-Classifier and Multiple Approximators
    Haiyue Song, Chengwen Xu, Qiang Xu, Zhuoran Song, Naifeng Jing, Xiaoyao Liang, and Li Jiang
    In Proceedings of the International Conference on Computer-Aided Design (ICCAD 2018), Nov 2018

Full publication list

Academic Service

  • Area Chair: ARR 2025, ARR 2026 March.
  • Website Chair: EMNLP 2025.
  • Candidate for KAKENHI Review Committee Member 2026.
  • Co-organizer: WAT 2024, and the English-Indic Language Document Translation Task at WAT 2025.
  • Mentor: AACL 2020 Student Research Workshop.
  • Award Selection Committee: ANLP (in Japan) 2026.
  • Reviewer: JNLP 2026, EAMT 2026, TALLIP 2026, TASLP 2026, IEICE 2026, ARR 2026 May, ARR 2026 January, LREC 2026, IJCNLP-AACL 2025, WAT 2025, EMNLP 2025, ACL 2025, COLING 2025, NLPCC 2024, TASLP 2024, AMTA 2024, LREC-COLING 2024, four rounds of ARR 2024, TALLIP 2024, TASLP 2023, TALLIP 2023, ARR 2023, APSIPA ASC 2023, EMNLP 2023, ACL 2023, EMNLP 2022, EMNLP 2021, EMNLP 2020, IJCNLP 2020, WAT 2020, etc.
  • One patent application in progress.

Hobbies

Competitive Programming

  • Silver medal at the National Olympiad in Informatics (NOI) 2013, Chengdu, China.
  • Bronze medal at the Asia-Pacific Informatics Olympiad (APIO) 2013.
  • AtCoder profile.

Japanese

  • Passed the Japanese Language Proficiency Test N1 (the highest level) in 2017.

Sports