# Retrieval Verification: Retrieval_Augmented_Generation_for_Knowledge_Intensive_NLP_Tasks_parsed

## Pilar: Background
**Query:** _Penelitian ini dilatarbelakangi oleh urgensi masalah, konteks situasi saat ini, serta motivasi mendasar yang mendorong perlunya dilakukan kajian._

### Rank 1 (Score: 0.8191)
**Header:** **3.2 Abstractive Question Answering** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
RAG models can go beyond simple extractive QA and answer questions with free-form, abstractive text generation. To test RAG’s natural language generation (NLG) in a knowledge-intensive setting, we use the MSMARCO NLG task v2.1 [43]. The task consists of questions, ten gold passages retrieved from a search engine for each question, and a full sentence answer annotated from the retrieved passages. We do not use the supplied passages, only the questions and answers, to treat  
MSMARCO as an open-domain abstractive QA task. MSMARCO has some questions that cannot be answered in a way that matches the reference answer without access to the gold passages, such as “What is the weather in Volcano, CA?” so performance will be lower without using gold passages. We also note that some MSMARCO questions cannot be answered using Wikipedia alone. Here, RAG can rely on parametric knowledge to generate reasonable responses.
```

### Rank 2 (Score: 0.8156)
**Header:** **1 Introduction** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
Margin-
Fact Verification:
alize
Label Generation
The Divine
Comedy
(x)
MIPS
Pe
This 14th century work
is divided into 3
Jeopardy Question
sectlons: "Inferno"
Generation:
"purgatorio" &
Answer Query
"paradiso"
(y)
Question Generation  
**----- Start of picture text -----**<br>
Define�"middle�ea�" (x) The�middle�ea��incl�de�<br>Q�e��i�� A���e�i�g: End-to-End Backprop through  �  and� pθ �he���mpanic�ca�i���and�he��h�ee�o��icle�.�� (y)<br>Q�e��i�� Q�e�� E�c�de�Q�e�� Re��ie�e� �η D�c�me��I�de� Gene�a�o���� Q�e��i�� A���e�i�g:A���e� Ge�e�a�i��<br>Ba�ack�Obama��a� (N��-Pa�a�e���c) (Pa�a�e���c)<br>bo�n�in�Ha�aii. (x) �(�) d(�) z4 ��ppo���� (y)<br>Fac� Ve�i�ca�i��: Fac� Q�e�� z2 z3 Ma�gin-ali�e Labe� Ge�e�a�i��Fac� Ve�i�ca�i��:<br>The�Di�ineComed�� (x) � MIPS z1 pθ Thi��14�h�cen������o�ki��di�ided�in�o�3<br>Je��a�d� Q�e��i�� �ec�ion�:�"Infe�no",<br>Ge�e�a�i��: "P��ga�o�io"�&<br>A���e� Q�e�� "Pa�adi�o"��������� (y)<br>Q�e��i�� Ge�e�a�i��<br>**----- End of picture text -----**<br>  
Figure 1: Overview of our approach. We combine a pre-trained retriever ( _Query Encoder_ + _Document Index_ ) with a pre-trained seq2seq model ( _Generator_ ) and fine-tune end-to-end. For query _x_ , we use Maximum Inner Product Search (MIPS) to find the top-K documents _zi_ . For final prediction _y_ , we treat _z_ as a latent variable and marginalize over seq2seq predictions given different documents.
```

### Rank 3 (Score: 0.8122)
**Header:** **4.1 Open-domain Question Answering** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
Table 2: Generation and classification Test Scores. MS-MARCO SotA is [4], FEVER-3 is [68] and FEVER-2 is [57] *Uses gold context/evidence. Best model without gold access underlined.
```

## Pilar: Methodology
**Query:** _Tahapan penelitian, arsitektur sistem, desain eksperimen, dan prosedur pengujian dieksekusi menggunakan metodologi teknis berikut._

### Rank 1 (Score: 0.8275)
**Header:** **2.5 Decoding** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
At test time, RAG-Sequence and RAG-Token require different ways to approximate arg max _y p_ ( _y|x_ ).
```

### Rank 2 (Score: 0.8261)
**Header:** **1 Introduction** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
Margin-
Fact Verification:
alize
Label Generation
The Divine
Comedy
(x)
MIPS
Pe
This 14th century work
is divided into 3
Jeopardy Question
sectlons: "Inferno"
Generation:
"purgatorio" &
Answer Query
"paradiso"
(y)
Question Generation  
**----- Start of picture text -----**<br>
Define�"middle�ea�" (x) The�middle�ea��incl�de�<br>Q�e��i�� A���e�i�g: End-to-End Backprop through  �  and� pθ �he���mpanic�ca�i���and�he��h�ee�o��icle�.�� (y)<br>Q�e��i�� Q�e�� E�c�de�Q�e�� Re��ie�e� �η D�c�me��I�de� Gene�a�o���� Q�e��i�� A���e�i�g:A���e� Ge�e�a�i��<br>Ba�ack�Obama��a� (N��-Pa�a�e���c) (Pa�a�e���c)<br>bo�n�in�Ha�aii. (x) �(�) d(�) z4 ��ppo���� (y)<br>Fac� Ve�i�ca�i��: Fac� Q�e�� z2 z3 Ma�gin-ali�e Labe� Ge�e�a�i��Fac� Ve�i�ca�i��:<br>The�Di�ineComed�� (x) � MIPS z1 pθ Thi��14�h�cen������o�ki��di�ided�in�o�3<br>Je��a�d� Q�e��i�� �ec�ion�:�"Infe�no",<br>Ge�e�a�i��: "P��ga�o�io"�&<br>A���e� Q�e�� "Pa�adi�o"��������� (y)<br>Q�e��i�� Ge�e�a�i��<br>**----- End of picture text -----**<br>  
Figure 1: Overview of our approach. We combine a pre-trained retriever ( _Query Encoder_ + _Document Index_ ) with a pre-trained seq2seq model ( _Generator_ ) and fine-tune end-to-end. For query _x_ , we use Maximum Inner Product Search (MIPS) to find the top-K documents _zi_ . For final prediction _y_ , we treat _z_ as a latent variable and marginalize over seq2seq predictions given different documents.
```

### Rank 3 (Score: 0.8233)
**Header:** **4.5 Additional Results** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
**Effect of Retrieving more documents** Models are trained with either 5 or 10 retrieved latent documents, and we do not observe significant differences in performance between them. We have the flexibility to adjust the number of retrieved documents at test time, which can affect performance and runtime. Figure 3 (left) shows that retrieving more documents at test time monotonically improves Open-domain QA results for RAG-Sequence, but performance peaks for RAG-Token at 10 retrieved documents. Figure 3 (right) shows that retrieving more documents leads to higher Rouge-L for RAG-Token at the expense of Bleu-1, but the effect is less pronounced for RAG-Sequence.
```

## Pilar: Gap
**Query:** _Apa kelemahan, keterbatasan ruang lingkup, atau masalah spesifik pada metode penelitian terdahulu yang belum terselesaikan?_

### Rank 1 (Score: 0.8362)
**Header:** **3.2 Abstractive Question Answering** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
RAG models can go beyond simple extractive QA and answer questions with free-form, abstractive text generation. To test RAG’s natural language generation (NLG) in a knowledge-intensive setting, we use the MSMARCO NLG task v2.1 [43]. The task consists of questions, ten gold passages retrieved from a search engine for each question, and a full sentence answer annotated from the retrieved passages. We do not use the supplied passages, only the questions and answers, to treat  
MSMARCO as an open-domain abstractive QA task. MSMARCO has some questions that cannot be answered in a way that matches the reference answer without access to the gold passages, such as “What is the weather in Volcano, CA?” so performance will be lower without using gold passages. We also note that some MSMARCO questions cannot be answered using Wikipedia alone. Here, RAG can rely on parametric knowledge to generate reasonable responses.
```

### Rank 2 (Score: 0.8342)
**Header:** **1 Introduction** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
Margin-
Fact Verification:
alize
Label Generation
The Divine
Comedy
(x)
MIPS
Pe
This 14th century work
is divided into 3
Jeopardy Question
sectlons: "Inferno"
Generation:
"purgatorio" &
Answer Query
"paradiso"
(y)
Question Generation  
**----- Start of picture text -----**<br>
Define�"middle�ea�" (x) The�middle�ea��incl�de�<br>Q�e��i�� A���e�i�g: End-to-End Backprop through  �  and� pθ �he���mpanic�ca�i���and�he��h�ee�o��icle�.�� (y)<br>Q�e��i�� Q�e�� E�c�de�Q�e�� Re��ie�e� �η D�c�me��I�de� Gene�a�o���� Q�e��i�� A���e�i�g:A���e� Ge�e�a�i��<br>Ba�ack�Obama��a� (N��-Pa�a�e���c) (Pa�a�e���c)<br>bo�n�in�Ha�aii. (x) �(�) d(�) z4 ��ppo���� (y)<br>Fac� Ve�i�ca�i��: Fac� Q�e�� z2 z3 Ma�gin-ali�e Labe� Ge�e�a�i��Fac� Ve�i�ca�i��:<br>The�Di�ineComed�� (x) � MIPS z1 pθ Thi��14�h�cen������o�ki��di�ided�in�o�3<br>Je��a�d� Q�e��i�� �ec�ion�:�"Infe�no",<br>Ge�e�a�i��: "P��ga�o�io"�&<br>A���e� Q�e�� "Pa�adi�o"��������� (y)<br>Q�e��i�� Ge�e�a�i��<br>**----- End of picture text -----**<br>  
Figure 1: Overview of our approach. We combine a pre-trained retriever ( _Query Encoder_ + _Document Index_ ) with a pre-trained seq2seq model ( _Generator_ ) and fine-tune end-to-end. For query _x_ , we use Maximum Inner Product Search (MIPS) to find the top-K documents _zi_ . For final prediction _y_ , we treat _z_ as a latent variable and marginalize over seq2seq predictions given different documents.
```

### Rank 3 (Score: 0.8280)
**Header:** **Abstract** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
Large pre-trained language models have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on downstream NLP tasks. However, their ability to access and precisely manipulate knowledge is still limited, and hence on knowledge-intensive tasks, their performance lags behind task-specific architectures. Additionally, providing provenance for their decisions and updating their world knowledge remain open research problems. Pre-trained models with a differentiable access mechanism to explicit nonparametric memory can overcome this issue, but have so far been only investigated for extractive downstream tasks. We explore a general-purpose fine-tuning recipe for retrieval-augmented generation (RAG) — models which combine pre-trained parametric and non-parametric memory for language generation. We introduce RAG models where the parametric memory is a pre-trained seq2seq model and the non-parametric memory is a dense vector index of Wikipedia, accessed with a pre-trained neural retriever. We compare two RAG formulations, one which conditions on the same retrieved passages across the whole generated sequence, and another which can use different passages per token. We fine-tune and evaluate our models on a wide range of knowledge-intensive NLP tasks and set the state of the art on three open domain QA tasks, outperforming parametric seq2seq models and task-specific retrieve-and-extract architectures. For language generation
```

## Pilar: Objective
**Query:** _Apa tujuan utama, sasaran akhir, atau kontribusi spesifik yang dihasilkan oleh penelitian ini untuk menyelesaikan masalah tersebut?_

### Rank 1 (Score: 0.8270)
**Header:** **1 Introduction** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
Margin-
Fact Verification:
alize
Label Generation
The Divine
Comedy
(x)
MIPS
Pe
This 14th century work
is divided into 3
Jeopardy Question
sectlons: "Inferno"
Generation:
"purgatorio" &
Answer Query
"paradiso"
(y)
Question Generation  
**----- Start of picture text -----**<br>
Define�"middle�ea�" (x) The�middle�ea��incl�de�<br>Q�e��i�� A���e�i�g: End-to-End Backprop through  �  and� pθ �he���mpanic�ca�i���and�he��h�ee�o��icle�.�� (y)<br>Q�e��i�� Q�e�� E�c�de�Q�e�� Re��ie�e� �η D�c�me��I�de� Gene�a�o���� Q�e��i�� A���e�i�g:A���e� Ge�e�a�i��<br>Ba�ack�Obama��a� (N��-Pa�a�e���c) (Pa�a�e���c)<br>bo�n�in�Ha�aii. (x) �(�) d(�) z4 ��ppo���� (y)<br>Fac� Ve�i�ca�i��: Fac� Q�e�� z2 z3 Ma�gin-ali�e Labe� Ge�e�a�i��Fac� Ve�i�ca�i��:<br>The�Di�ineComed�� (x) � MIPS z1 pθ Thi��14�h�cen������o�ki��di�ided�in�o�3<br>Je��a�d� Q�e��i�� �ec�ion�:�"Infe�no",<br>Ge�e�a�i��: "P��ga�o�io"�&<br>A���e� Q�e�� "Pa�adi�o"��������� (y)<br>Q�e��i�� Ge�e�a�i��<br>**----- End of picture text -----**<br>  
Figure 1: Overview of our approach. We combine a pre-trained retriever ( _Query Encoder_ + _Document Index_ ) with a pre-trained seq2seq model ( _Generator_ ) and fine-tune end-to-end. For query _x_ , we use Maximum Inner Product Search (MIPS) to find the top-K documents _zi_ . For final prediction _y_ , we treat _z_ as a latent variable and marginalize over seq2seq predictions given different documents.
```

### Rank 2 (Score: 0.8216)
**Header:** **3.2 Abstractive Question Answering** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
RAG models can go beyond simple extractive QA and answer questions with free-form, abstractive text generation. To test RAG’s natural language generation (NLG) in a knowledge-intensive setting, we use the MSMARCO NLG task v2.1 [43]. The task consists of questions, ten gold passages retrieved from a search engine for each question, and a full sentence answer annotated from the retrieved passages. We do not use the supplied passages, only the questions and answers, to treat  
MSMARCO as an open-domain abstractive QA task. MSMARCO has some questions that cannot be answered in a way that matches the reference answer without access to the gold passages, such as “What is the weather in Volcano, CA?” so performance will be lower without using gold passages. We also note that some MSMARCO questions cannot be answered using Wikipedia alone. Here, RAG can rely on parametric knowledge to generate reasonable responses.
```

### Rank 3 (Score: 0.8177)
**Header:** **5 Related Work** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
**Retrieve-and-Edit approaches** Our method shares some similarities with retrieve-and-edit style approaches, where a similar training input-output pair is retrieved for a given input, and then edited to provide a final output. These approaches have proved successful in a number of domains including Machine Translation [18, 22] and Semantic Parsing [21]. Our approach does have several differences, including less of emphasis on lightly editing a retrieved item, but on aggregating content from several pieces of retrieved content, as well as learning latent retrieval, and retrieving evidence documents rather than related training pairs. This said, RAG techniques may work well in these settings, and could represent promising future work.
```

## Pilar: Futurework
**Query:** _Apa saran, rekomendasi penelitian lanjutan, atau agenda di masa depan yang diusulkan oleh penulis untuk mengatasi keterbatasan studi saat ini?_

### Rank 1 (Score: 0.8296)
**Header:** **1 Introduction** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
Margin-
Fact Verification:
alize
Label Generation
The Divine
Comedy
(x)
MIPS
Pe
This 14th century work
is divided into 3
Jeopardy Question
sectlons: "Inferno"
Generation:
"purgatorio" &
Answer Query
"paradiso"
(y)
Question Generation  
**----- Start of picture text -----**<br>
Define�"middle�ea�" (x) The�middle�ea��incl�de�<br>Q�e��i�� A���e�i�g: End-to-End Backprop through  �  and� pθ �he���mpanic�ca�i���and�he��h�ee�o��icle�.�� (y)<br>Q�e��i�� Q�e�� E�c�de�Q�e�� Re��ie�e� �η D�c�me��I�de� Gene�a�o���� Q�e��i�� A���e�i�g:A���e� Ge�e�a�i��<br>Ba�ack�Obama��a� (N��-Pa�a�e���c) (Pa�a�e���c)<br>bo�n�in�Ha�aii. (x) �(�) d(�) z4 ��ppo���� (y)<br>Fac� Ve�i�ca�i��: Fac� Q�e�� z2 z3 Ma�gin-ali�e Labe� Ge�e�a�i��Fac� Ve�i�ca�i��:<br>The�Di�ineComed�� (x) � MIPS z1 pθ Thi��14�h�cen������o�ki��di�ided�in�o�3<br>Je��a�d� Q�e��i�� �ec�ion�:�"Infe�no",<br>Ge�e�a�i��: "P��ga�o�io"�&<br>A���e� Q�e�� "Pa�adi�o"��������� (y)<br>Q�e��i�� Ge�e�a�i��<br>**----- End of picture text -----**<br>  
Figure 1: Overview of our approach. We combine a pre-trained retriever ( _Query Encoder_ + _Document Index_ ) with a pre-trained seq2seq model ( _Generator_ ) and fine-tune end-to-end. For query _x_ , we use Maximum Inner Product Search (MIPS) to find the top-K documents _zi_ . For final prediction _y_ , we treat _z_ as a latent variable and marginalize over seq2seq predictions given different documents.
```

### Rank 2 (Score: 0.8290)
**Header:** **3.2 Abstractive Question Answering** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
RAG models can go beyond simple extractive QA and answer questions with free-form, abstractive text generation. To test RAG’s natural language generation (NLG) in a knowledge-intensive setting, we use the MSMARCO NLG task v2.1 [43]. The task consists of questions, ten gold passages retrieved from a search engine for each question, and a full sentence answer annotated from the retrieved passages. We do not use the supplied passages, only the questions and answers, to treat  
MSMARCO as an open-domain abstractive QA task. MSMARCO has some questions that cannot be answered in a way that matches the reference answer without access to the gold passages, such as “What is the weather in Volcano, CA?” so performance will be lower without using gold passages. We also note that some MSMARCO questions cannot be answered using Wikipedia alone. Here, RAG can rely on parametric knowledge to generate reasonable responses.
```

### Rank 3 (Score: 0.8258)
**Header:** **4.5 Additional Results** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
**Effect of Retrieving more documents** Models are trained with either 5 or 10 retrieved latent documents, and we do not observe significant differences in performance between them. We have the flexibility to adjust the number of retrieved documents at test time, which can affect performance and runtime. Figure 3 (left) shows that retrieving more documents at test time monotonically improves Open-domain QA results for RAG-Sequence, but performance peaks for RAG-Token at 10 retrieved documents. Figure 3 (right) shows that retrieving more documents leads to higher Rouge-L for RAG-Token at the expense of Bleu-1, but the effect is less pronounced for RAG-Sequence.
```
