Оценка неопределенности в задачах обработки естественного языкаUncertainty Quantification in Natural Language Processing

Uncertainty quantification (UQ) has emerged as a promising approach for addressing several key challenges in natural language processing (NLP), particularly in mitigating the risks of errors in text classification and detecting hallucinations or low-quality outputs in text generation. Although UQ is a rapidly growing field within classification tasks, state-of-the-art methods often show poor performance or underperform trivial methods for ambiguous tasks such as toxicity detection. Furthermore, despite significant progress in UQ techniques for text classification tasks, applying UQ to large language models (LLMs) introduces additional complexity due to the conditional dependency between generation steps and the varying influence of tokens on the predictions in autoregressive models. As a result, many UQ techniques that are effective for classification models are either ineffective or not directly applicable to LLMs. This thesis addresses these challenges by developing novel methods for robust uncertainty quantification for both text classification and text generation tasks. For classification tasks, we propose a hybrid approach that combines both epistemic and aleatoric uncertainty, outperforming existing methods and providing a more reliable selective classification. For LLMs, we introduce several innovative techniques that leverage attention-based features or token embeddings to quantify uncertainty effectively. These methods are designed to handle the sequential and conditional nature of LLM outputs, enabling improved selective generation and fact-checking.