<jupyter_start><jupyter_text>Open-Domain QA on Tables[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/deepset-ai/haystack/blob/master/tutorials/Tutorial15_TableQA.ipynb)This tutorial shows you how to perform question-answering on tables using the `TableTextRetriever` or `ElasticsearchRetriever` as retriever node and the `TableReader` as reader node.  Prepare environment Colab: Enable the GPU runtimeMake sure you enable the GPU runtime to experience decent speed in this tutorial.**Runtime -> Change Runtime type -> Hardware accelerator -> GPU**<jupyter_code># Make sure you have a GPU running
!nvidia-smi
# Install the latest release of Haystack in your own environment 
#! pip install farm-haystack

# Install the latest master of Haystack
!pip install grpcio-tools==1.34.1
!pip install git+https://github.com/deepset-ai/haystack.git

# The TaPAs-based TableReader requires the torch-scatter library
!pip install torch-scatter -f https://data.pyg.org/whl/torch-1.9.0+cu111.html

# If you run this notebook on Google Colab, you might need to
# restart the runtime after installing haystack.<jupyter_output><empty_output><jupyter_text>Start an Elasticsearch serverYou can start Elasticsearch on your local machine instance using Docker. If Docker is not readily available in your environment (e.g. in Colab notebooks), then you can manually download and execute Elasticsearch from source.<jupyter_code># Recommended: Start Elasticsearch using Docker via the Haystack utility function
from haystack.utils import launch_es

launch_es()
# In Colab / No Docker environments: Start Elasticsearch from source
! wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.9.2-linux-x86_64.tar.gz -q
! tar -xzf elasticsearch-7.9.2-linux-x86_64.tar.gz
! chown -R daemon:daemon elasticsearch-7.9.2

import os
from subprocess import Popen, PIPE, STDOUT
es_server = Popen(['elasticsearch-7.9.2/bin/elasticsearch'],
                   stdout=PIPE, stderr=STDOUT,
                   preexec_fn=lambda: os.setuid(1)  # as daemon
                  )
# wait until ES has started
! sleep 30
# Connect to Elasticsearch
from haystack.document_stores import ElasticsearchDocumentStore

# We want to use a small model producing 512-dimensional embeddings, so we need to set embedding_dim to 512
document_store = ElasticsearchDocumentStore(host="localhost",
                                            username="",
                                            password="",
                                            index="document",
                                            embedding_dim=512)<jupyter_output><empty_output><jupyter_text>Add Tables to DocumentStoreTo quickly demonstrate the capabilities of the `TableTextRetriever` and the `TableReader` we use a subset of 1000 tables of the [Open Table-and-Text Question Answering (OTT-QA) dataset](https://github.com/wenhuchen/OTT-QA).Just as text passages, tables are represented as `Document` objects in Haystack. The content field, though, is a pandas DataFrame instead of a string.<jupyter_code># Let's first fetch some tables that we want to query
# Here: 1000 tables from OTT-QA
from haystack.utils import fetch_archive_from_http

doc_dir = "data"
s3_url = "https://s3.eu-central-1.amazonaws.com/deepset.ai-farm-qa/datasets/documents/ottqa_tables_sample.json.zip"
fetch_archive_from_http(url=s3_url, output_dir=doc_dir)
# Add the tables to the DocumentStore

import json
from haystack import Document
import pandas as pd

def read_ottqa_tables(filename):
    processed_tables = []
    with open(filename) as tables:
        tables = json.load(tables)
        for key, table in tables.items():
            current_columns = table["header"]
            current_rows = table["data"]
            current_df = pd.DataFrame(columns=current_columns, data=current_rows)
            current_doc_title = table["title"]
            current_section_title = table["section_title"]
            document = Document(
                content=current_df,
                content_type="table",
                meta={"title": current_doc_title, "section_title": current_section_title},
                id=key
            )
            processed_tables.append(document)

    return processed_tables


tables = read_ottqa_tables("data/ottqa_tables_sample.json")
document_store.write_documents(tables, index="document")

# Showing content field and meta field of one of the Documents of content_type 'table'
print(tables[0].content)
print(tables[0].meta)<jupyter_output>Result  ...            Score
0     Winner  ...        6-1 , 6-1
1     Winner  ...  6-2 , 4-6 , 6-3
2     Winner  ...        6-2 , 6-2
3  Runner-up  ...        3-6 , 2-6
4     Winner  ...  6-7 , 6-3 , 6-0
5     Winner  ...        6-1 , 6-0
6     Winner  ...  6-2 , 2-6 , 6-2
7     Winner  ...        6-0 , 6-4

[8 rows x 8 columns]
{'title': '<NAME>', 'section_title': 'ITF finals ( 7–3 ) -- Doubles ( 7–1 )'}<jupyter_text>Initalize Retriever, Reader, & Pipeline RetrieverRetrievers help narrowing down the scope for the Reader to a subset of tables where a given question could be answered.They use some simple but fast algorithm.**Here:** We use the `TableTextRetriever` capable of retrieving relevant content among a databaseof texts and tables using dense embeddings. It is an extension of the `DensePassageRetriever` and consists of three encoders (one query encoder, one text passage encoder and one table encoder) that create embeddings in the same vector space. More details on the `TableTextRetriever` and how it is trained can be found in [this paper](https://arxiv.org/abs/2108.04049).**Alternatives:**- `ElasticsearchRetriever` that uses BM25 algorithm<jupyter_code>from haystack.nodes.retriever import TableTextRetriever

retriever = TableTextRetriever(
    document_store=document_store,
    query_embedding_model="deepset/bert-small-mm_retrieval-question_encoder",
    passage_embedding_model="deepset/bert-small-mm_retrieval-passage_encoder",
    table_embedding_model="deepset/bert-small-mm_retrieval-table_encoder",
    embed_meta_fields=["title", "section_title"]
)
# Add table embeddings to the tables in DocumentStore
document_store.update_embeddings(retriever=retriever)
## Alternative: ElasticsearchRetriever
#from haystack.nodes.retriever import ElasticsearchRetriever
#retriever = ElasticsearchRetriever(document_store=document_store)
# Try the Retriever
from haystack.utils import print_documents

retrieved_tables = retriever.retrieve("How many twin buildings are under construction?", top_k=5)
# Get highest scored table
print(retrieved_tables[0].content)<jupyter_output>Name  ...              Status
0                  Twin Towers II  ...         Never built
1              World Trade Center  ...           Destroyed
2                Three Sixty West  ...  Under construction
3                  Gateway Towers  ...  Under construction
4                 Rustomjee Crown  ...  Under construction
5                  Orchid Heights  ...             On-hold
6                Hermitage Towers  ...            Proposed
7             Lokhandwala Minerva  ...  Under construction
8                    Lamar Towers  ...  Under construction
9            Indonesia One Towers  ...  Under construction
10                       Sky link  ...            Approved
11                  Vida Za'abeel  ...            Proposed
12  Broadway Corridor Twin Towers  ...         Never built
13   India Bulls Sky Forest Tower  ...  Under construction
14                 Capital Towers  ...  Under construction
15               One Avighna Park  ...  Under constructi[...]<jupyter_text>ReaderThe `TableReader` is based on TaPas, a transformer-based language model capable of grasping the two-dimensional structure of a table. It scans the tables returned by the retriever and extracts the anser. The available TableReader models can be found [here](https://huggingface.co/models?pipeline_tag=table-question-answering&sort=downloads).**Notice**: The `TableReader` will return an answer for each table, even if the query cannot be answered by the table. Furthermore, the confidence scores are not useful as of now, given that they will *always* be very high (i.e. 1 or close to 1).<jupyter_code>from haystack.nodes import TableReader

reader = TableReader(model_name_or_path="google/tapas-base-finetuned-wtq", max_seq_len=512)
# Try the TableReader on one Table (highest-scored retrieved table from previous section)

table_doc = document_store.get_document_by_id("List_of_tallest_twin_buildings_and_structures_in_the_world_1")
print(table_doc.content)
from haystack.utils import print_answers

prediction = reader.predict(query="How many twin buildings are under construction?", documents=[table_doc])
print_answers(prediction, details="all")<jupyter_output>{   'answers': [   Answer(answer='12', type='extractive', score=1.0, context=                             Name  ...              Status
0                  Twin Towers II  ...         Never built
1              World Trade Center  ...           Destroyed
2                Three Sixty West  ...  Under construction
3                  Gateway Towers  ...  Under construction
4                 Rustomjee Crown  ...  Under construction
5                  Orchid Heights  ...             On-hold
6                Hermitage Towers  ...            Proposed
7             Lokhandwala Minerva  ...  Under construction
8                    Lamar Towers  ...  Under construction
9            Indonesia One Towers  ...  Under construction
10                       Sky link  ...            Approved
11                  Vida Za'abeel  ...            Proposed
12  Broadway Corridor Twin Towers  ...         Never built
13   India Bulls Sky Forest Tower  ...  Under construction
14                 Capital Towers  ...[...]<jupyter_text>The offsets in the `offsets_in_document` and `offsets_in_context` field indicate the table cells that the model predicts to be part of the answer. They need to be interpreted on the linearized table, i.e., a flat list containing all of the table cells.In the `Answer`'s meta field, you can find the aggreagtion operator used to construct the answer (in this case `COUNT`) and the answer cells as strings.<jupyter_code>print(f"Predicted answer: {prediction['answers'][0].answer}")
print(f"Meta field: {prediction['answers'][0].meta}")<jupyter_output>Predicted answer: 12
Meta field: {'aggregation_operator': 'COUNT', 'answer_cells': ['Three Sixty West', 'Gateway Towers', 'Rustomjee Crown', 'Lokhandwala Minerva', 'Lamar Towers', 'Indonesia One Towers', 'India Bulls Sky Forest Tower', 'Capital Towers', 'One Avighna Park', 'The Destiny ( Tower )', 'Oberoi Esquire Towers', 'Bhoomi Celestia']}<jupyter_text>PipelineThe Retriever and the Reader can be sticked together to a pipeline in order to first retrieve relevant tables and then extract the answer.**Notice**: Given that the `TableReader` does not provide useful confidence scores and returns an answer for each of the tables, the sorting of the answers might be not helpful.<jupyter_code># Initialize pipeline
from haystack import Pipeline

table_qa_pipeline = Pipeline()
table_qa_pipeline.add_node(component=retriever, name="TableTextRetriever", inputs=["Query"])
table_qa_pipeline.add_node(component=reader, name="TableReader", inputs=["TableTextRetriever"])
prediction = table_qa_pipeline.run("How many twin buildings are under construction?")
print_answers(prediction, details="minimum")<jupyter_output>[   {   'answer': '12',
        'context':                              Name  ...              Status
0                  Twin Towers II  ...         Never built
1              World Trade Center  ...           Destroyed
2                Three Sixty West  ...  Under construction
3                  Gateway Towers  ...  Under construction
4                 Rustomjee Crown  ...  Under construction
5                  Orchid Heights  ...             On-hold
6                Hermitage Towers  ...            Proposed
7             Lokhandwala Minerva  ...  Under construction
8                    Lamar Towers  ...  Under construction
9            Indonesia One Towers  ...  Under construction
10                       Sky link  ...            Approved
11                  <NAME>  ...            Proposed
12  Broadway Corridor Twin Towers  ...         Never built
13   India Bulls Sky Forest Tower  ...  Under construction
14                 Capital Towers  ...  Under construction
15           [...]
{'fraction_non_alphanumeric': 0.08322008154129028, 'fraction_numerical': 0.016228315612758813, 'mean_word_length': 2.7989675068326756, 'pattern_counts': {'":': 2, '<': 26, '<?xml version=': 0, '>': 29, 'https://': 9, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 16, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '128767', 'license': 'Apache-2.0', 'n_tokens_mistral': 3570, 'n_tokens_neox': 3389, 'n_words': 1147, 'path': 'tutorials/Tutorial15_TableQA.ipynb', 'repo_name': 'shabib0786/haystack', 'size': '12521'}