IPEval: A Bilingual Intellectual Property Agency Consultation Evaluation Benchmark for Large Language Models

Qiyao Wang1, Jianguo Huang1, Shule Lu1, Yuan Lin1*, Kan Xu1, Liang Yang1, Hongfei Lin1,
1Dalian University of Technology
Paper arXiv Code Data
Evaluation System

Abstract

IPEval is a pioneering bilingual Intellectual Property (IP) agency consultation evaluation benchmark, meticulously crafted to assess the competencies of Large Language Models (LLMs) in the intricate domain of intellectual property. This benchmark is the first of its kind, encompassing a diverse spectrum of 2,657 multiple-choice questions that are intricately divided across four major capability dimensions: creation, application, protection, and management.

BibTeX

@article{wang2024ipeval,
  title={IPEval: A Bilingual Intellectual Property Agency Consultation Evaluation Benchmark for Large Language Models},
  author={Wang, Qiyao and Huang, Jianguo and Lu, Shule and Lin, Yuan and Xu, Kan and Yang, Liang and Lin, Hongfei},
  journal={arXiv preprint arXiv:2406.12386},
  year={2024}
}