Haiyue Song 宋海越

Sòng Hǎiyuè /sʊŋ xaɪ ɥɛ/

My family name 宋 (Sòng) comes from the State of Song (1100-286 BC).

My given name 海越 (Hǎiyuè) means "crossing the sea", where 海 means "sea" and 越 means "to cross".

prof_pic.jpg

Greetings! I am an LLM engineer at Preferred Networks. Previously, I worked at the National Institute of Information and Communications Technology with Masao Utiyama and Raj Dabre.
I received my Ph.D. in Informatics from Kyoto University in 2024, advised by Prof. Sadao Kurohashi and Prof. Chenhui Chu. I completed my Bachelor’s degree in Computer Science and Technology at Shanghai Jiao Tong University in 2018, with a minor in Japanese.

My research interests are in large language models and natural language processing. I primarily work on pre-, mid-, post-training methods for LLMs, including data crawling for low-resource languages, model merging, and reinforcement learning. I persistently work on multilingual, multicultural, and multimodal aspects of natural language processing. I periodically work on machine translation, tokenization, and language resources.

News

Oct 27, 2026 Our paper won the Outstanding Paper Award at EMNLP 2026! 🏆
Aug 20, 2026 Three papers accepted to EMNLP 2026!
Oct 24, 2025 Four papers accepted to IJCNLP-AACL 2025!

Selected Publications

  1. EMNLP
    2026
    optimer.png
    OptiMer: Optimal Distribution Vector Merging Is Better than Data Mixing for Continual Pre-Training
    Haiyue Song, Huy Hien Vu, and Masao Utiyama
    EMNLP 2026 Main
  2. IJCNLP-AACL
    2025
    formatrl.png
    Structured Document Translation via Format Reinforcement Learning
    IJCNLP-AACL 2025 Main
  3. IWSLT
    2024
    iwslt.png
    NICT’s Cascaded and End-To-End Speech Translation Systems using Whisper and IndicTrans2 for the Indic Task
    Raj Dabre and Haiyue Song
    IWSLT 2024
  4. EAMT
    2024
    submerge.png
    SubMerge: Merging Equivalent Subword Tokenizations for Subword Regularized Models in Neural Machine Translation
    EAMT 2024
  5. LREC-COLING
    2024
    nglueni.png
    NGLUEni: Benchmarking and Adapting Pretrained Language Models for Nguni Languages
    Francois Meyer, Haiyue Song, Abhisek Chakrabarty, Jan Buys, Raj Dabre, and Hideki Tanaka
    LREC-COLING 2024
  6. JNLP
    2024
    diverseg.png
    DiverSeg: Leveraging Diverse Segmentations with Cross-granularity Alignment for Neural Machine Translation
    Journal of Natural Language Processing, 2024
  7. TALLIP
    2023
    selfseg.png
    SelfSeg: A Self-supervised Sub-word Segmentation Method for Neural Machine Translation
    TALLIP, Aug 2023
  8. AACL
    2022
    bertseg.png
    BERTSeg: BERT Based Unsupervised Subword Segmentation for Neural Machine Translation
    AACL-IJCNLP 2022 Main
  9. LREC
    2020
    coursera.png
    Coursera Corpus Mining and Multistage Fine-Tuning for Improving Lectures Translation
    LREC 2020
  10. ICCAD
    2018
    iccad.png
    Invocation-driven neural approximate computing with a multiclass-classifier and multiple approximators
    Haiyue Song, Chengwen Xu, Qiang Xu, Zhuoran Song, Naifeng Jing, Xiaoyao Liang, and Li Jiang
    ICCAD 2018

Tutorials

  1. COLING
    2025
    Connecting Ideas in ’Lower-Resource’ Scenarios: NLP for National Varieties, Creoles and Other Low-resource Scenarios
    Aditya Joshi, Diptesh Kanojia, Heather Lent, Hour Kaing, and Haiyue Song
    COLING 2025 Tutorial
  2. EAMT
    2024
    Linguistically Motivated Neural Machine Translation
    Haiyue Song, Hour Kaing, and Raj Dabre
    EAMT 2024 Tutorial

Academic Service

  • Website Chair of EMNLP 2025. The design of the Website.
  • Area Chair of ARR 2025, ARR 2026 March, ARR 2026 October.
  • Workshop Co-organizer of WAT 2024, 2025, 2026. The English-Indic Language Document Translation Task at WAT 2025.
  • Mentor in AACL 2020 Student Research Workshop.
  • Award Selection Committee of ANLP, Japan, 2026.
  • Candidate for KAKENHI Review Committee Member 2026.
  • Reviewer of JNLP 2026, EAMT 2026, TALLIP 2026, TASLP 2026, IEICE 2026, ARR 2026 August, ARR 2026 May, ARR 2026 January, LREC 2026, IJCNLP-AACL 2025, WAT 2025, EMNLP 2025, ACL 2025, COLING 2025, NLPCC 2024, TASLP 2024, AMTA 2024, LREC-COLING 2024, four rounds of ARR 2024, TALLIP 2024, TASLP 2023, TALLIP 2023, ARR 2023, APSIPA ASC 2023, EMNLP 2023, ACL 2023, EMNLP 2022, EMNLP 2021, EMNLP 2020, IJCNLP 2020, WAT 2020, etc.

Grants

  1. KAKENHI
    2025
    Improving Large Language Models for Low-Resource Languages with Diverse Writing Systems
    Haiyue Song
    2025
    Grant-in-Aid for Early-Career Scientists(若手研究), 2025 to 2026, KAKENHI 25K21290.
  2. KAKENHI
    2021
    Multilingual Corpus Construction and Domain Adaptation for Low-Resource Machine Translation
    Haiyue Song
    2021
    JSPS Research Fellowship for Young Scientists (DC1), 2021 to 2023, KAKENHI 22KJ1724.

Accepted application forms with notes in Japanese: 科研費(若手研究・学振)申請書の公開.

My Hobbies

Sports

Japanese

  • The Japanese Language Proficiency Test N1, the highest level, 2017.

Competitive Programming

  • Silver medal at NOI 2013, the National Olympiad in Informatics, Chengdu, China.
  • Bronze medal at APIO 2013, the Asia-Pacific Informatics Olympiad, Online.
  • AtCoder profile.