<commit_before>- Backport support for ocaml 4.00
- catch "different assumptions" exception
- there is a bug in cmi refreshing
- write syntax error message generating heuristic
- have a way to get the types of modules
  partially done by 6f7e89e6f6d0915f5b2f76d892c99434399e9a76
  now we need a clean/generic way
- Add support for ppx
- Add support for existing p4 extensions on 4.02
- Replace Browse by a propper wrapping of Typedtree (~60%)
- Find proper API for incremental parser
  -> goto table should not be manipulated explicitly
  -> exceptions from semantic action should be caught and treated differently
  -> relying on special handling of stack bottom is not good either
     Bottom handling code improved, but we should make sure it s stable.
- Test:
  -> completion
  -> locate
  -> type enclosing / type expr
  -> occurrences
- Reimplement verbosity, expand module aliases

IDEA (later)
- perf: send batches of token rather than entering/exiting the whole
  parser barrier for each token

DONE
- recovery heuristic is really not sufficient, a few test cases easily trigger
  bad recursion:
  let f =
    let bad
    let x = () in
  fixed in b5cf1991705ea7f2138602875fe774e20b4a0a49

- with this input:
    let xxy =
      let xx = with
    let xxz : int = "hello"<commit_msg>Update todo<commit_after>- Add support for existing p4 extensions on 4.02
- Backport support for ocaml 4.00
- catch "different assumptions" exception
- there is a bug in cmi refreshing
- write syntax error message generating heuristic
- have a way to get the types of modules
  partially done by 6f7e89e6f6d0915f5b2f76d892c99434399e9a76
  now we need a clean/generic way
- Add support for ppx
- Replace Browse by a propper wrapping of Typedtree (~60%)
- Find proper API for incremental parser
  -> goto table should not be manipulated explicitly
  -> exceptions from semantic action should be caught and treated differently
  -> relying on special handling of stack bottom is not good either
     Bottom handling code improved, but we should make sure it s stable.
- Test:
  -> completion
  -> locate
  -> type enclosing / type expr
  -> occurrences
- Reimplement verbosity, expand module aliases
- When resuming after a (, lexer can get confused if it receives *:
  it will be lexed as STAR token and not beginning of comment

IDEA (later)
- perf: send batches of token rather than entering/exiting the whole
  parser barrier for each token

DONE
- recovery heuristic is really not sufficient, a few test cases easily trigger
  bad recursion:
  let f =
    let bad
    let x = () in
  fixed in b5cf1991705ea7f2138602875fe774e20b4a0a49

- with this input:
    let xxy =
      let xx = with
    let xxz : int = "hello"
{'fraction_non_alphanumeric': 0.04767184035476718, 'fraction_numerical': 0.04804138950480414, 'mean_word_length': 3.4892205638474296, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5113329', 'n_tokens_mistral': 867, 'n_tokens_neox': 761, 'n_words': 375}