# Retrieval Verification: Retrieval_Augmented_Generation_for_Knowledge_Intensive_NLP_Tasks_parsed

## Pilar: Background
**Query:** _latar belakang masalah, urgensi penelitian, motivasi mendasar, dilatar belakangi oleh, fenomena krusial, konteks situasi saat ini, dan alasan utama studi ini dilakukan._

### Rank 1 (Score: 0.8397)
**Header:** **4.5 Additional Results** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
**Effect of Retrieving more documents** Models are trained with either 5 or 10 retrieved latent documents, and we do not observe significant differences in performance between them. We have the flexibility to adjust the number of retrieved documents at test time, which can affect performance and runtime. Figure 3 (left) shows that retrieving more documents at test time monotonically improves Open-domain QA results for RAG-Sequence, but performance peaks for RAG-Token at 10 retrieved documents. Figure 3 (right) shows that retrieving more documents leads to higher Rouge-L for RAG-Token at the expense of Bleu-1, but the effect is less pronounced for RAG-Sequence.
```

### Rank 2 (Score: 0.8330)
**Header:** **4.1 Open-domain Question Answering** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
Table 2: Generation and classification Test Scores. MS-MARCO SotA is [4], FEVER-3 is [68] and FEVER-2 is [57] *Uses gold context/evidence. Best model without gold access underlined.
```

### Rank 3 (Score: 0.8318)
**Header:** **1 Introduction** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
Margin-
Fact Verification:
alize
Label Generation
The Divine
Comedy
(x)
MIPS
Pe
This 14th century work
is divided into 3
Jeopardy Question
sectlons: "Inferno"
Generation:
"purgatorio" &
Answer Query
"paradiso"
(y)
Question Generation  
**----- Start of picture text -----**<br>
Define�"middle�ea�" (x) The�middle�ea��incl�de�<br>Q�e��i�� A���e�i�g: End-to-End Backprop through  �  and� pθ �he���mpanic�ca�i���and�he��h�ee�o��icle�.�� (y)<br>Q�e��i�� Q�e�� E�c�de�Q�e�� Re��ie�e� �η D�c�me��I�de� Gene�a�o���� Q�e��i�� A���e�i�g:A���e� Ge�e�a�i��<br>Ba�ack�Obama��a� (N��-Pa�a�e���c) (Pa�a�e���c)<br>bo�n�in�Ha�aii. (x) �(�) d(�) z4 ��ppo���� (y)<br>Fac� Ve�i�ca�i��: Fac� Q�e�� z2 z3 Ma�gin-ali�e Labe� Ge�e�a�i��Fac� Ve�i�ca�i��:<br>The�Di�ineComed�� (x) � MIPS z1 pθ Thi��14�h�cen������o�ki��di�ided�in�o�3<br>Je��a�d� Q�e��i�� �ec�ion�:�"Infe�no",<br>Ge�e�a�i��: "P��ga�o�io"�&<br>A���e� Q�e�� "Pa�adi�o"��������� (y)<br>Q�e��i�� Ge�e�a�i��<br>**----- End of picture text -----**<br>  
Figure 1: Overview of our approach. We combine a pre-trained retriever ( _Query Encoder_ + _Document Index_ ) with a pre-trained seq2seq model ( _Generator_ ) and fine-tune end-to-end. For query _x_ , we use Maximum Inner Product Search (MIPS) to find the top-K documents _zi_ . For final prediction _y_ , we treat _z_ as a latent variable and marginalize over seq2seq predictions given different documents.
```

## Pilar: Methodology
**Query:** _rincian teknis metodologi, arsitektur sistem, desain eksperimen, teknik pengumpulan dataset, algoritma yang diusulkan, sampel, dan prosedur pengujian._

### Rank 1 (Score: 0.8232)
**Header:** **3 Experiments** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
We experiment with RAG in a wide range of knowledge-intensive tasks. For all experiments, we use a single Wikipedia dump for our non-parametric knowledge source. Following Lee et al. [31] and Karpukhin et al. [26], we use the December 2018 dump. Each Wikipedia article is split into disjoint 100-word chunks, to make a total of 21M documents. We use the document encoder to compute an embedding for each document, and build a single MIPS index using FAISS [23] with a Hierarchical Navigable Small World approximation for fast retrieval [37]. During training, we retrieve the top _k_ documents for each query. We consider _k 2 {_ 5 _,_ 10 _}_ for training and set _k_ for test time using dev data. We now discuss experimental details for each task.
```

### Rank 2 (Score: 0.8194)
**Header:** **4.5 Additional Results** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
**Effect of Retrieving more documents** Models are trained with either 5 or 10 retrieved latent documents, and we do not observe significant differences in performance between them. We have the flexibility to adjust the number of retrieved documents at test time, which can affect performance and runtime. Figure 3 (left) shows that retrieving more documents at test time monotonically improves Open-domain QA results for RAG-Sequence, but performance peaks for RAG-Token at 10 retrieved documents. Figure 3 (right) shows that retrieving more documents leads to higher Rouge-L for RAG-Token at the expense of Bleu-1, but the effect is less pronounced for RAG-Sequence.
```

### Rank 3 (Score: 0.8170)
**Header:** **2.2 Retriever: DPR** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
The retrieval component _p⌘_ ( _z|x_ ) is based on DPR [26]. DPR follows a bi-encoder architecture:  
_p⌘_ ( _z|x_ ) _/_ exp � **d** ( _z_ ) _[>]_ **q** ( _x_ )� **d** ( _z_ ) = BERT _d_ ( _z_ ) _,_ **q** ( _x_ ) = BERT _q_ ( _x_ )
```

## Pilar: Gap
**Query:** _keterbatasan penelitian terdahulu, kelemahan metode sebelumnya, kekosongan literatur, masalah yang belum terselesaikan, namun demikian, akan tetapi, dan hasil yang belum jelas._

### Rank 1 (Score: 0.8399)
**Header:** **4.5 Additional Results** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
**Effect of Retrieving more documents** Models are trained with either 5 or 10 retrieved latent documents, and we do not observe significant differences in performance between them. We have the flexibility to adjust the number of retrieved documents at test time, which can affect performance and runtime. Figure 3 (left) shows that retrieving more documents at test time monotonically improves Open-domain QA results for RAG-Sequence, but performance peaks for RAG-Token at 10 retrieved documents. Figure 3 (right) shows that retrieving more documents leads to higher Rouge-L for RAG-Token at the expense of Bleu-1, but the effect is less pronounced for RAG-Sequence.
```

### Rank 2 (Score: 0.8365)
**Header:** **1 Introduction** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
Margin-
Fact Verification:
alize
Label Generation
The Divine
Comedy
(x)
MIPS
Pe
This 14th century work
is divided into 3
Jeopardy Question
sectlons: "Inferno"
Generation:
"purgatorio" &
Answer Query
"paradiso"
(y)
Question Generation  
**----- Start of picture text -----**<br>
Define�"middle�ea�" (x) The�middle�ea��incl�de�<br>Q�e��i�� A���e�i�g: End-to-End Backprop through  �  and� pθ �he���mpanic�ca�i���and�he��h�ee�o��icle�.�� (y)<br>Q�e��i�� Q�e�� E�c�de�Q�e�� Re��ie�e� �η D�c�me��I�de� Gene�a�o���� Q�e��i�� A���e�i�g:A���e� Ge�e�a�i��<br>Ba�ack�Obama��a� (N��-Pa�a�e���c) (Pa�a�e���c)<br>bo�n�in�Ha�aii. (x) �(�) d(�) z4 ��ppo���� (y)<br>Fac� Ve�i�ca�i��: Fac� Q�e�� z2 z3 Ma�gin-ali�e Labe� Ge�e�a�i��Fac� Ve�i�ca�i��:<br>The�Di�ineComed�� (x) � MIPS z1 pθ Thi��14�h�cen������o�ki��di�ided�in�o�3<br>Je��a�d� Q�e��i�� �ec�ion�:�"Infe�no",<br>Ge�e�a�i��: "P��ga�o�io"�&<br>A���e� Q�e�� "Pa�adi�o"��������� (y)<br>Q�e��i�� Ge�e�a�i��<br>**----- End of picture text -----**<br>  
Figure 1: Overview of our approach. We combine a pre-trained retriever ( _Query Encoder_ + _Document Index_ ) with a pre-trained seq2seq model ( _Generator_ ) and fine-tune end-to-end. For query _x_ , we use Maximum Inner Product Search (MIPS) to find the top-K documents _zi_ . For final prediction _y_ , we treat _z_ as a latent variable and marginalize over seq2seq predictions given different documents.
```

### Rank 3 (Score: 0.8323)
**Header:** **Abstract** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
Large pre-trained language models have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on downstream NLP tasks. However, their ability to access and precisely manipulate knowledge is still limited, and hence on knowledge-intensive tasks, their performance lags behind task-specific architectures. Additionally, providing provenance for their decisions and updating their world knowledge remain open research problems. Pre-trained models with a differentiable access mechanism to explicit nonparametric memory can overcome this issue, but have so far been only investigated for extractive downstream tasks. We explore a general-purpose fine-tuning recipe for retrieval-augmented generation (RAG) — models which combine pre-trained parametric and non-parametric memory for language generation. We introduce RAG models where the parametric memory is a pre-trained seq2seq model and the non-parametric memory is a dense vector index of Wikipedia, accessed with a pre-trained neural retriever. We compare two RAG formulations, one which conditions on the same retrieved passages across the whole generated sequence, and another which can use different passages per token. We fine-tune and evaluate our models on a wide range of knowledge-intensive NLP tasks and set the state of the art on three open domain QA tasks, outperforming parametric seq2seq models and task-specific retrieve-and-extract architectures. For language generation
```

## Pilar: Objective
**Query:** _tujuan utama penelitian, sasaran spesifik, kontribusi inti makalah, hasil yang ingin dicapai, mengeksplorasi, dan solusi yang diusulkan untuk menyelesaikan masalah._

### Rank 1 (Score: 0.8257)
**Header:** **Abstract** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
Large pre-trained language models have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on downstream NLP tasks. However, their ability to access and precisely manipulate knowledge is still limited, and hence on knowledge-intensive tasks, their performance lags behind task-specific architectures. Additionally, providing provenance for their decisions and updating their world knowledge remain open research problems. Pre-trained models with a differentiable access mechanism to explicit nonparametric memory can overcome this issue, but have so far been only investigated for extractive downstream tasks. We explore a general-purpose fine-tuning recipe for retrieval-augmented generation (RAG) — models which combine pre-trained parametric and non-parametric memory for language generation. We introduce RAG models where the parametric memory is a pre-trained seq2seq model and the non-parametric memory is a dense vector index of Wikipedia, accessed with a pre-trained neural retriever. We compare two RAG formulations, one which conditions on the same retrieved passages across the whole generated sequence, and another which can use different passages per token. We fine-tune and evaluate our models on a wide range of knowledge-intensive NLP tasks and set the state of the art on three open domain QA tasks, outperforming parametric seq2seq models and task-specific retrieve-and-extract architectures. For language generation
```

### Rank 2 (Score: 0.8246)
**Header:** **1 Introduction** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
Margin-
Fact Verification:
alize
Label Generation
The Divine
Comedy
(x)
MIPS
Pe
This 14th century work
is divided into 3
Jeopardy Question
sectlons: "Inferno"
Generation:
"purgatorio" &
Answer Query
"paradiso"
(y)
Question Generation  
**----- Start of picture text -----**<br>
Define�"middle�ea�" (x) The�middle�ea��incl�de�<br>Q�e��i�� A���e�i�g: End-to-End Backprop through  �  and� pθ �he���mpanic�ca�i���and�he��h�ee�o��icle�.�� (y)<br>Q�e��i�� Q�e�� E�c�de�Q�e�� Re��ie�e� �η D�c�me��I�de� Gene�a�o���� Q�e��i�� A���e�i�g:A���e� Ge�e�a�i��<br>Ba�ack�Obama��a� (N��-Pa�a�e���c) (Pa�a�e���c)<br>bo�n�in�Ha�aii. (x) �(�) d(�) z4 ��ppo���� (y)<br>Fac� Ve�i�ca�i��: Fac� Q�e�� z2 z3 Ma�gin-ali�e Labe� Ge�e�a�i��Fac� Ve�i�ca�i��:<br>The�Di�ineComed�� (x) � MIPS z1 pθ Thi��14�h�cen������o�ki��di�ided�in�o�3<br>Je��a�d� Q�e��i�� �ec�ion�:�"Infe�no",<br>Ge�e�a�i��: "P��ga�o�io"�&<br>A���e� Q�e�� "Pa�adi�o"��������� (y)<br>Q�e��i�� Ge�e�a�i��<br>**----- End of picture text -----**<br>  
Figure 1: Overview of our approach. We combine a pre-trained retriever ( _Query Encoder_ + _Document Index_ ) with a pre-trained seq2seq model ( _Generator_ ) and fine-tune end-to-end. For query _x_ , we use Maximum Inner Product Search (MIPS) to find the top-K documents _zi_ . For final prediction _y_ , we treat _z_ as a latent variable and marginalize over seq2seq predictions given different documents.
```

### Rank 3 (Score: 0.8232)
**Header:** **4.5 Additional Results** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
**Effect of Retrieving more documents** Models are trained with either 5 or 10 retrieved latent documents, and we do not observe significant differences in performance between them. We have the flexibility to adjust the number of retrieved documents at test time, which can affect performance and runtime. Figure 3 (left) shows that retrieving more documents at test time monotonically improves Open-domain QA results for RAG-Sequence, but performance peaks for RAG-Token at 10 retrieved documents. Figure 3 (right) shows that retrieving more documents leads to higher Rouge-L for RAG-Token at the expense of Bleu-1, but the effect is less pronounced for RAG-Sequence.
```

## Pilar: Futurework
**Query:** _saran untuk penelitian selanjutnya, rekomendasi pekerjaan di masa depan, arah pengembangan masa depan, dan peluang studi berikutnya untuk mengatasi batasan saat ini._

### Rank 1 (Score: 0.8395)
**Header:** **4.5 Additional Results** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
**Effect of Retrieving more documents** Models are trained with either 5 or 10 retrieved latent documents, and we do not observe significant differences in performance between them. We have the flexibility to adjust the number of retrieved documents at test time, which can affect performance and runtime. Figure 3 (left) shows that retrieving more documents at test time monotonically improves Open-domain QA results for RAG-Sequence, but performance peaks for RAG-Token at 10 retrieved documents. Figure 3 (right) shows that retrieving more documents leads to higher Rouge-L for RAG-Token at the expense of Bleu-1, but the effect is less pronounced for RAG-Sequence.
```

### Rank 2 (Score: 0.8309)
**Header:** **1 Introduction** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
Margin-
Fact Verification:
alize
Label Generation
The Divine
Comedy
(x)
MIPS
Pe
This 14th century work
is divided into 3
Jeopardy Question
sectlons: "Inferno"
Generation:
"purgatorio" &
Answer Query
"paradiso"
(y)
Question Generation  
**----- Start of picture text -----**<br>
Define�"middle�ea�" (x) The�middle�ea��incl�de�<br>Q�e��i�� A���e�i�g: End-to-End Backprop through  �  and� pθ �he���mpanic�ca�i���and�he��h�ee�o��icle�.�� (y)<br>Q�e��i�� Q�e�� E�c�de�Q�e�� Re��ie�e� �η D�c�me��I�de� Gene�a�o���� Q�e��i�� A���e�i�g:A���e� Ge�e�a�i��<br>Ba�ack�Obama��a� (N��-Pa�a�e���c) (Pa�a�e���c)<br>bo�n�in�Ha�aii. (x) �(�) d(�) z4 ��ppo���� (y)<br>Fac� Ve�i�ca�i��: Fac� Q�e�� z2 z3 Ma�gin-ali�e Labe� Ge�e�a�i��Fac� Ve�i�ca�i��:<br>The�Di�ineComed�� (x) � MIPS z1 pθ Thi��14�h�cen������o�ki��di�ided�in�o�3<br>Je��a�d� Q�e��i�� �ec�ion�:�"Infe�no",<br>Ge�e�a�i��: "P��ga�o�io"�&<br>A���e� Q�e�� "Pa�adi�o"��������� (y)<br>Q�e��i�� Ge�e�a�i��<br>**----- End of picture text -----**<br>  
Figure 1: Overview of our approach. We combine a pre-trained retriever ( _Query Encoder_ + _Document Index_ ) with a pre-trained seq2seq model ( _Generator_ ) and fine-tune end-to-end. For query _x_ , we use Maximum Inner Product Search (MIPS) to find the top-K documents _zi_ . For final prediction _y_ , we treat _z_ as a latent variable and marginalize over seq2seq predictions given different documents.
```

### Rank 3 (Score: 0.8298)
**Header:** **3.2 Abstractive Question Answering** / **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**
**Content:**
```text
RAG models can go beyond simple extractive QA and answer questions with free-form, abstractive text generation. To test RAG’s natural language generation (NLG) in a knowledge-intensive setting, we use the MSMARCO NLG task v2.1 [43]. The task consists of questions, ten gold passages retrieved from a search engine for each question, and a full sentence answer annotated from the retrieved passages. We do not use the supplied passages, only the questions and answers, to treat  
MSMARCO as an open-domain abstractive QA task. MSMARCO has some questions that cannot be answered in a way that matches the reference answer without access to the gold passages, such as “What is the weather in Volcano, CA?” so performance will be lower without using gold passages. We also note that some MSMARCO questions cannot be answered using Wikipedia alone. Here, RAG can rely on parametric knowledge to generate reasonable responses.
```
