<commit_before>Introduction
=================================================

Owl is an emerging numerical platform for engineering and scientific computing. The library is (mostly) developed in the OCaml language and inherits all its powerful features such as static type checking, powerful module system, and superior runtime efficiency. Owl allows you to write succinct type-safe numerical applications in a functional language without sacrificing performance, significantly reduces the cost from prototype to production use.



Motivation
-------------------------------------------------

If you are from Python world and familiar with its ecosystem for scientific computing. You may see Owl as a mixture of Numpy, Scipy, Pandas, and other libraries. You may be curious why I have packed so many stuff together. As you learn more and more about OCaml, I am almost certain your curiosity will grow even bigger because this seems against the *minimalist*, a popular design principle adopted by many OCaml libraries.

However, I argue that these functionality should be co-designed and implemented together. This is exactly what we have learnt in the past two decades struggling to build a modern numerical system. The current co-design choice avoids a lot of redundant code and duplicated efforts, and makes optimisation a lot easier. Numpy, Scipy, Pandas, and other software took so many years to well define the boundary of their functionality. In the end, Numpy becomes data representation, Scipy builds high-level analytical functions atop of such representation, Pandas is a combination of table manipulation and analytical functions, and Pytorch bridges these functions between heterogenous devices. However, there is still a significant amount of overlap if you look deep into the code.

Back to the OCaml world, the co-design becomes even more important because of the language's strict static typing. Especially if every small numerical library wraps its data representation into abstract types, then they will not play together nicely whenever you try to build a large and complex application. This further indicates that by having a huge number of small libraries in the ecosystem will not really improve a programmers' productivity. Owl is supposed to address this issue with a consistent whole design, with a strong focus on scientific computing.<commit_msg>edit ...<commit_after>Introduction
=================================================

Owl is an emerging numerical platform for engineering and scientific computing. The library is (mostly) developed in the OCaml language and inherits all its powerful features such as static type checking, powerful module system, and superior runtime efficiency. Owl allows you to write succinct type-safe numerical applications in a functional language without sacrificing performance, significantly reduces the cost from prototype to production use.



Motivation
-------------------------------------------------

If you are from Python world and familiar with its ecosystem for scientific computing. You may see Owl as a mixture of Numpy, Scipy, Pandas, and other libraries. You may be curious why I have packed so many stuff together. As you learn more and more about OCaml, I am almost certain your curiosity will grow even bigger because this seems against the *minimalist*, a popular design principle adopted by many OCaml libraries.

However, I would like to argue that these functionality should be co-designed and implemented together. This is exactly what we have learnt in the past two decades struggling to build a modern numerical system. The current co-design choice avoids a lot of redundant code and duplicated efforts, and makes optimisation a lot easier. Numpy, Scipy, Pandas, and other software took so many years to well define the boundary of their functionality. In the end, Numpy becomes data representation, Scipy builds high-level analytical functions atop of such representation, Pandas is a combination of table manipulation and analytical functions, and Pytorch bridges these functions between heterogenous devices. However, there is still a significant amount of overlap if you look deep into the code.

Back to the OCaml world, the co-design becomes even more important because of the language's strict static typing. Especially if every small numerical library wraps its data representation into abstract types, then they will not play together nicely whenever you try to build a large and complex application. This further indicates that by having a huge number of small libraries in the ecosystem will not really improve a programmers' productivity. Owl is supposed to address this issue with a consistent *wholistic* design, with a strong focus on scientific computing.

Different choice of design principle also reveals the difference between system programming and numerical programming. System programming is built atop of a wide range of complicated and heterogenous hardware, it abstracts out the realworld complexity by providing a (relatively) small set of APIs (recall how many system calls in Unix operating system). On the other hand, numerical computing is built atop of a small amount of abstract number types (e.g. real and complex), then derives a fat set of advanced numerical operations for various fields (recall how many APIs in a numerical library). As a result, reductionism is preferred in system programming whereas wholism is preferred in numerical one.
{'fraction_non_alphanumeric': 0.059027777777777776, 'fraction_numerical': 0.0, 'mean_word_length': 4.6306584362139915, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2060650', 'n_tokens_mistral': 1108, 'n_tokens_neox': 1053, 'n_words': 802}