<commit_before>..
    Public facing docs for non-developers go here. Please do not add any Python
    dependencies for code introspection here (we may temporarily host it some
    place where those dependencies are cumbersome to build).

edX Data Documentation
======================
The following documents are targeted at those who are working with various data formats consumed and produced by the edX platform -- primarily course authors and those who are conducting research on data in our system. Developer oriented discussion of architecture and strictly internal APIs should be documented elsewhere.

Course Data Formats
-------------------
These are data formats written by people to specify course structure and content. Some of this is abstracted away if you are using the Studio authoring user interface.

.. toctree::
   :maxdepth: 2

   course_data_formats/course_xml.rst
   course_data_formats/grading.rst

Specific Problem Types
^^^^^^^^^^^^^^^^^^^^^^
.. toctree::
   :maxdepth: 1

   course_data_formats/drag_and_drop/drag_and_drop_input.rst
   course_data_formats/graphical_slider_tool/graphical_slider_tool.rst
   course_data_formats/poll_module/poll_module.rst
   course_data_formats/lti_module/lti.rst
   course_data_formats/conditional_module/conditional_module.rst
   course_data_formats/word_cloud/word_cloud.rst
   course_data_formats/custom_response.rst
   course_data_formats/symbolic_response.rst
   course_data_formats/jsinput.rst
   course_data_formats/formula_equation_input.rst


Internal Data Formats
---------------------
These documents describe how we store course structure, student state/progress, and events internally. Useful for developers or researchers who interact with our raw data exports.

.. toctree::
   :maxdepth: 2

   internal_data_formats/change_log.rst
   internal_data_formats/sql_schema.rst
   internal_data_formats/discussion_data.rst
   internal_data_formats/wiki_data.rst
   internal_data_formats/tracking_logs.rst

Indices and tables
==================

* :ref:`search`
<commit_msg>Added internal refs to help researchers find the data format files<commit_after>..
    Public facing docs for non-developers go here. Please do not add any Python
    dependencies for code introspection here (we may temporarily host it some
    place where those dependencies are cumbersome to build).

edX Data Documentation
======================
The following documents are targeted at those who are working with various data formats consumed and produced by the edX platform -- primarily course authors and those who are conducting research on data in our system. 

* If you are a course author working with course XML, see :ref:`Course Data Formats`.

* If you are a researcher working with data, see :ref:`Internal Data Formats`.

Developer-oriented discussion of architecture and strictly internal APIs will be documented elsewhere.

.. _Course Data Formats:

Course Data Formats
-------------------
These are data formats written by people to specify course structure and content. Some of this is abstracted away if you are using the Studio authoring user interface.

.. toctree::
   :maxdepth: 2

   course_data_formats/course_xml.rst
   course_data_formats/grading.rst

Specific Problem Types
^^^^^^^^^^^^^^^^^^^^^^
.. toctree::
   :maxdepth: 1

   course_data_formats/drag_and_drop/drag_and_drop_input.rst
   course_data_formats/graphical_slider_tool/graphical_slider_tool.rst
   course_data_formats/poll_module/poll_module.rst
   course_data_formats/lti_module/lti.rst
   course_data_formats/conditional_module/conditional_module.rst
   course_data_formats/word_cloud/word_cloud.rst
   course_data_formats/custom_response.rst
   course_data_formats/symbolic_response.rst
   course_data_formats/jsinput.rst
   course_data_formats/formula_equation_input.rst

.. _Internal Data Formats:

Internal Data Formats
---------------------
These documents describe how we store course structure, student state/progress, and events internally. Useful for developers or researchers who interact with our raw data exports.

.. toctree::
   :maxdepth: 2

   internal_data_formats/change_log.rst
   internal_data_formats/sql_schema.rst
   internal_data_formats/discussion_data.rst
   internal_data_formats/wiki_data.rst
   internal_data_formats/tracking_logs.rst

Indices and tables
==================

* :ref:`search`

{'fraction_non_alphanumeric': 0.09019967867798945, 'fraction_numerical': 0.001377094330961671, 'mean_word_length': 4.461152882205514, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7297988', 'n_tokens_mistral': 1248, 'n_tokens_neox': 1166, 'n_words': 393}