Enhancing RAGs for Rheumatology Triage: Strategies for Optimized Knowledge Retrieval

Managing the triage of chronic patients with rheumatic diseases is a significant challenge for general practitioners. Limited consultation time and a high volume of patients make it difficult to accurately identify those requiring specialist care, often leading to unnecessary referrals to specialists. Large language models can be integrated with retrieval-augmented generation to develop a generative AI system that assists general practitioners in identifying criteria for referring patients to rheumatology specialists, delivering fast and clinically reliable responses grounded in validated medical sources. To improve diagnostic efficiency while reducing the risk of errors, designing a curated knowledge base representation and a highly accurate retriever is indispensable. This study explored various retrieval strategies, including hybrid similarity computation methods, query augmentation, and noise-filtering techniques leveraging keywords and document structure. For initial tests, the knowledge base was populated with articles selected and validated by specialists from ICS Maugeri Hospital in Pavia, Italy. Preliminary results indicate that a hybrid lexical-semantic retrieval approach is the most effective for accessing the rheumatology knowledge base. Additionally, straightforward filtering strategies based on keywords and document structure can significantly reduce query times while minimizing noise.