Zhisong Zhang

Hi, I am a researcher working in the field of NLP and AI. Currently, I am working at Tencent AI Lab.

I received my PhD from Language Technologies Institute at Carnegie Mellon University, advised by Prof. Eduard Hovy and Prof. Emma Strubell. Before that, I graduated with BS and MS in Computer Science from Shanghai Jiao Tong University, advised by Prof. Hai Zhao.

I am currently interested in the following directions:

• Building efficient and reliable agent systems.
• Designing suitable representations to connect natural languages to machine actions.
• Understanding the underlying mechanisms of language models.

Email: zerozones17 at gmail.com

Selected Papers:

[Google Scholar] [Semantic Scholar] [dblp]

• Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models. [arxiv]
  Z. Zhang, Y. Wang, X. Huang, T. Fang, H. Zhang, C. Deng, S. Li, and D. Yu. Preprint, 2024.

• A Silver Bullet or a Compromise for Full Attention? A Comprehensive Study of Gist Token-based Context Compression. [arxiv]
  C. Deng, Z. Zhang^†, K. Mao, S. Li, X. Huang, D. Yu, and Z. Dou^†. Preprint, 2024.

• LoGU: Long-form Generation with Uncertainty Expressions. [arxiv]
  R. Yang*, C. Zhang*, Z. Zhang^†, X. Huang, S. Yang, N. Collier, D. Yu, and Deqing Yang^†. Preprint, 2024.

• Atomic Calibration of LLMs in Long-Form Generations. [arxiv]
  C. Zhang, R. Yang, Z. Zhang^†, X. Huang, S. Yang, D. Yu, and N. Collier^†. Preprint, 2024.

• Data-efficient Active Learning for Structured Prediction with Partial Annotation and Self-Training. [paper]
  Z. Zhang, E. Strubell, and E. Hovy. In Proceedings of EMNLP-Findings, 2023.

• On the Interactions of Structural Constraints and Data Resources for Structured Prediction. [paper]
  Z. Zhang, E. Strubell, and E. Hovy. In Proceedings of SustaiNLP-2023, 2023.

• Towards More Efficient Insertion Transformer with Fractional Positional Encoding. [paper] [code]
  Z. Zhang, Y. Zhang, and B. Dolan. In Proceedings of EACL-2023, 2023.

• A Survey of Active Learning for Natural Language Processing. [paper] [table]
  Z. Zhang, E. Strubell, and E. Hovy. In Proceedings of EMNLP-2022, 2022.

• Transfer Learning from Semantic Role Labeling to Event Argument Extraction with Template-based Slot Querying. [paper] [code]
  Z. Zhang, E. Strubell, and E. Hovy. In Proceedings of EMNLP-2022, 2022.

• On the Benefit of Syntactic Supervision for Cross-lingual Transfer in Semantic Role Labeling. [paper] [code]
  Z. Zhang, E. Strubell, and E. Hovy. In Proceedings of EMNLP-2021, 2021.

• Comparing Span Extraction Methods for Semantic Role Labeling. [paper] [code]
  Z. Zhang, E. Strubell, and E. Hovy. In Proceedings of SPNLP-2021, 2021.

• An Empirical Exploration of Local Ordering Pre-training for Structured Prediction. [paper] [code]
  Z. Zhang, X. Kong, L. Levin, and E. Hovy. In EMNLP-Findings, 2020.

• Incorporating a Local Translation Mechanism into Non-autoregressive Translation. [paper]
  X. Kong*, Z. Zhang*, and E. Hovy. In Proceedings of EMNLP-2020, 2020.

• A Two-Step Approach for Implicit Event Argument Detection. [paper] [code]
  Z. Zhang, X. Kong, Z. Liu, X. Ma and E. Hovy. In Proceedings of ACL-2020, 2020.

• An Empirical Investigation of Structured Output Modeling for Graph-based Neural Dependency Parsing. [paper] [code]
  Z. Zhang, X. Ma and E. Hovy. In Proceedings of ACL-2019, 2019.

• On Difficulties of Cross-Lingual Transfer with Order Differences: A Case Study on Dependency Parsing. [paper] [code1] [code2]
  W. Ahmad*, Z. Zhang*, X. Ma, E. Hovy, K. Chang and N. Peng. In Proceedings of NAACL-2019, 2019.

• Exploring Recombination for Efficient Decoding of Neural Machine Translation. [paper] [code]
  Z. Zhang, R.Wang, M. Utiyama, E. Sumita and H. Zhao. In Proceedings of EMNLP-2018, 2018.

• Probabilistic Graph-based Dependency Parsing with Convolutional Neural Network. [paper] [code]
  Z. Zhang, H. Zhao and L. Qin. In Proceedings of ACL-2016, 2016.

• High-order Graph-based Neural Dependency Parsing. [paper] [code]
  Z. Zhang and H. Zhao. In Proceedings of PACLIC-29, 2015.

Projects:

• ZMSP: The Mingled Strucutured Predictor. [code]
  This repo currently contains the implementation of several neural dependency parsers (both graph-based and transition-based ones), semantic role labelers, as well as a simple event extraction system.

• NNPGDParser: Neural Network Based Probablistic Graph Dependency Parser. [code]
  This is the work of my ACL-2016 paper. This parser adopts tree-CRF probabilistic training criterion and Convolutional Neural Network model for the task of dependency parsing.

• NNGDParser: Neural Network Based Probablistic Graph Dependency Parser. [code]
  This is the predecessor of NNPGDParser and is a simple Feed-Forward Neural Network parser. It is the work of my PACLIC-29 paper.

• TIGER-Compiler: Compiler for the programming language of TIGER. [code]
  This is the course project of the Compiler course, and I finished a complete compiler for a mini-language TIGER. It accepts TIGER source code and generates MIPS code.