Command Palette
Search for a command to run...
Rui Ribeiro; Joao P. Carvalho; Luísa Coheur
Abstract
Recent approaches have attempted to personalize dialogue systems by leveraging profile information into models. However, this knowledge is scarce and difficult to obtain, which makes the extraction/generation of profile information from dialogues a fundamental asset. To surpass this limitation, we introduce the Profile Generation Task (PGTask). We contribute with a new dataset for this problem, comprising profile sentences aligned with related utterances, extracted from a corpus of dialogues. Furthermore, using state-of-the-art methods, we provide a benchmark for profile generation on this novel dataset. Our experiments disclose the challenges of profile generation, and we hope that this introduces a new research direction.
Code Repositories
Benchmarks
| Benchmark | Methodology | Metrics |
|---|---|---|
| pgtask-on-pgdataset | gpt2-small | BLEU-1: 61.30 BLEU-2: 32.3 BLEU-3: 20.62 BLEU-4: 9.44 BertScore: 94.39 ROUGE-1: 50.07 ROUGE-2: 28.31 ROUGE-L: 50.00 |
| pgtask-on-pgdataset | gpt2-medium | BLEU-1: 59.31 BLEU-2: 25.94 BLEU-3: 15.3 BLEU-4: 9.17 BertScore: 94.76 ROUGE-1: 46.32 ROUGE-2: 24.14 ROUGE-L: 45.88 |
Build AI with AI
From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.