Large dataset in Squad2-format
I'm pretty sure you can get some inspiration from the squad_v2 script. It looks like the dataset is quite big so it will take some time for the users to generate it, but it should be reasonable.

Also you are saying that you are still making the dataset grow in size right ?
It's probably good practice to let the users do their training/evaluations with the exact same version of the dataset.
We allow for each dataset to specify a version (ex: 1.0.0) and increment this number every time there are new samples in the dataset for example. Does it look like a good solution for you ? Or would you rather have one final version with the full dataset ?
At the moment we are building an large question answering dataset and think about sharing it with the huggingface community.
Caused the computing power we splitted it into multiple tiles, but they are all in the same format.
Right now the most important facts about are this:
- Contexts: 1.047.671
- questions: 1.677.732
- Answers: 6.742.406
- unanswerable: 377.398

It is already cleaned

<pre><code>
train_data = [
    {
        'context': "this is the context",
        'qas': [
            {
                'id': "00002",
                'is_impossible': False,
                'question': "whats is this",
                'answers': [
                    {
                        'text': "answer",
                        'answer_start': 0
                    }
                ]
            },
            {
                'id': "00003",
                'is_impossible': False,
                'question': "question2",
                'answers': [
                    {
                        'text': "answer2",
                        'answer_start': 1
                    }
                ]
            }
        ]
    }
]
</code></pre>

Cause it is growing every day we are thinking about an structure like this:
We host an Json file, containing all the download links and the script can load it dynamically.
At the moment it is around ~20GB

Any advice how to handle this, or an ready to use template ?
Large dataset in Squad2-format 
 At the moment we are building an large question answering dataset and think about sharing it with the huggingface community.
Caused the computing power we splitted it into multiple tiles, but they are all in the same format.
Right now the most important facts about are this:
- Contexts: 1.047.671
- questions: 1.677.732
- Answers: 6.742.406
- unanswerable: 377.398

It is already cleaned

<pre><code>
train_data = [
    {
        'context': "this is the context",
        'qas': [
            {
                'id': "00002",
                'is_impossible': False,
                'question': "whats is this",
                'answers': [
                    {
                        'text': "answer",
                        'answer_start': 0
                    }
                ]
            },
            {
                'id': "00003",
                'is_impossible': False,
                'question': "question2",
                'answers': [
                    {
                        'text': "answer2",
                        'answer_start': 1
                    }
                ]
            }
        ]
    }
]
</code></pre>

Cause it is growing every day we are thinking about an structure like this:
We host an Json file, containing all the download links and the script can load it dynamically.
At the moment it is around ~20GB

Any advice how to handle this, or an ready to use template ? 
 I'm pretty sure you can get some inspiration from the squad_v2 script. It looks like the dataset is quite big so it will take some time for the users to generate it, but it should be reasonable.

Also you are saying that you are still making the dataset grow in size right ?
It's probably good practice to let the users do their training/evaluations with the exact same version of the dataset.
We allow for each dataset to specify a version (ex: 1.0.0) and increment this number every time there are new samples in the dataset for example. Does it look like a good solution for you ? Or would you rather have one final version with the full dataset ?