<commit_before>JakWorkers
==========

My very own thread pool implementation.

Use cases
=========

TODO

* continuation
* delayed execution

The checkbox thing means it has one (or more) test case(s).

* [ ] Data parallelization (e.g. create an array of 1M floats, compute the squares on 4 workers) -> EXIT_WHEN_ALL_JOBS_COMPLETE = 0
* [ ] Performing separate tasks which don't inter-block themselves -> EXIT_WHEN_ALL_JOBS_COMPLETE = 1
* [ ] Continuation (TBD) (also, TODO)
* [ ] Delayed execution (TBD) (also, TODO)

Continuation
============

Yeah... the code you put in a worker should not, in turn, block, or you might bring the entire framework to a grinding halt. You should instead try to block, and if you can't, add a new job that tries to block and continues where you left off.

TODO In the future, I might add a flag on a job that might be blocking which will mean that the framework will try to keep at least one worker which undergoes only jobs which promise to never block. This way, there will be at least one worker that will never block, which means stuff will get done, which may mean that the other blocked threads will be able to continue where they left off assuming no dead-locks.

If you create jobs which inter-lock themselves on mutexes and expect the dispatcher to somehow magically guess what order they need to be run in in order to avoid dead-locks, that's your problem. This framework is not necessarily designed to perform tasks which are SUPER interdependent. Also, if you manage to have an application which requires inter-locking jobs, it means either a) you're doing it wrong, or b) you're doing it right, but what you need is something more complex than JakWorkers.

Caller
======

```C
int sigReceived = 0;
void SIGINT_handler(int signum) {
    sigReceived++;
}
someFunc {
    // ...
    waitpit blabla...
    while(sigReceived) {
        sigReceived--;
        struct job_t job = { &my_code, &my_data };
        jw_add_job(job);
    }
    // ...
}
...
jw_main(3);
```

Callee
======

```C
void my_code(void* data) {
    int* datadata = (int*)data;
    do_stuff(data[0], &data[1]);
}
```

Framework
=========

```C
// the main job queue
queue_t jw_jobQueue;
pthread_mutex_t jw_jobQueue_lock;
pthread_cond_t jw_jobAdded; // set to 0 when queue is empty,
                            // set to 1 when new job is added
pthread_mutext_t jw_jobAddedMutext;
sem_t jw_queueSem;  // starts off at numthreads, fw starts decrementing it
                    // threads return it when their job is done
                    // framework blocks if empty (i.e. worker pool is full)

// add a new job
// blocking if someone else is using the jobQueue
// wakes up the framework if it was idle
short jw_add_job(struct job_t job); // thread safe

// the workers
struct {
    job_t job;
    pthread_cond_t job_cond; // thread waits for 0->1 transition to continue
                             // used to give a new job to a worker
    pthread_mutex_t job_condMutex;
} jw_workers[];

jw_exit(int retCode); // clean up resources, call jw_exit_ret_code
                      // which can be a stub;

jw_main(int const numWorkers)
{
    ////BOOL no_jobs = FALSE;

    // init threads
    for(int i = 0; i < numWorkers; ++i) {
        // ...
        pthread_create(&fw_worker, &jw_workers[i]);
    }

    for(;;) {
        ////// check if there are jobs in the queue
        ////lock jw_jobQueue;
        ////if(jw_jobQueue is empty) pthread_cond_wait(jw_jobAdded);
        ////release jw_jobQueue;

        ////if(no_jobs) {
        ////    no_jobs = FALSE;
        ////    pthread_cond_timedwait(jw_jobAdded, 500ms);
        ////    continue;
        ////}

        sem_wait(jw_queueSem);

        lock jw_jobQueue;
        if(!EXIT_WHEN_ALL_JOBS_COMPLETE)
            while(jw_jobQueue is empty) pthread_cond_wait(jw_jobAdded);
        else {
            jw_exit(0);
            return;
        }
        take msg from jw_jobQueue;
        release jw_jobQueue;

        find free thread => t;
        pthread_mutex_lock(t->job_condMutex);<commit_msg>remove commented code<commit_after>JakWorkers
==========

My very own thread pool implementation.

Use cases
=========

TODO

* continuation
* delayed execution

The checkbox thing means it has one (or more) test case(s).

* [ ] Data parallelization (e.g. create an array of 1M floats, compute the squares on 4 workers) -> EXIT_WHEN_ALL_JOBS_COMPLETE = 0
* [ ] Performing separate tasks which don't inter-block themselves -> EXIT_WHEN_ALL_JOBS_COMPLETE = 1
* [ ] Continuation (TBD) (also, TODO)
* [ ] Delayed execution (TBD) (also, TODO)

Continuation
============

Yeah... the code you put in a worker should not, in turn, block, or you might bring the entire framework to a grinding halt. You should instead try to block, and if you can't, add a new job that tries to block and continues where you left off.

TODO In the future, I might add a flag on a job that might be blocking which will mean that the framework will try to keep at least one worker which undergoes only jobs which promise to never block. This way, there will be at least one worker that will never block, which means stuff will get done, which may mean that the other blocked threads will be able to continue where they left off assuming no dead-locks.

If you create jobs which inter-lock themselves on mutexes and expect the dispatcher to somehow magically guess what order they need to be run in in order to avoid dead-locks, that's your problem. This framework is not necessarily designed to perform tasks which are SUPER interdependent. Also, if you manage to have an application which requires inter-locking jobs, it means either a) you're doing it wrong, or b) you're doing it right, but what you need is something more complex than JakWorkers.

Caller
======

```C
int sigReceived = 0;
void SIGINT_handler(int signum) {
    sigReceived++;
}
someFunc {
    // ...
    waitpit blabla...
    while(sigReceived) {
        sigReceived--;
        struct job_t job = { &my_code, &my_data };
        jw_add_job(job);
    }
    // ...
}
...
jw_main(3);
```

Callee
======

```C
void my_code(void* data) {
    int* datadata = (int*)data;
    do_stuff(data[0], &data[1]);
}
```

Framework
=========

```C
// the main job queue
queue_t jw_jobQueue;
pthread_mutex_t jw_jobQueue_lock;
pthread_cond_t jw_jobAdded; // set to 0 when queue is empty,
                            // set to 1 when new job is added
pthread_mutext_t jw_jobAddedMutext;
sem_t jw_queueSem;  // starts off at numthreads, fw starts decrementing it
                    // threads return it when their job is done
                    // framework blocks if empty (i.e. worker pool is full)

// add a new job
// blocking if someone else is using the jobQueue
// wakes up the framework if it was idle
short jw_add_job(struct job_t job); // thread safe

// the workers
struct {
    job_t job;
    pthread_cond_t job_cond; // thread waits for 0->1 transition to continue
                             // used to give a new job to a worker
    pthread_mutex_t job_condMutex;
} jw_workers[];

jw_exit(int retCode); // clean up resources, call jw_exit_ret_code
                      // which can be a stub;

jw_main(int const numWorkers)
{
    init jw_queueSem to numWorkers;

    // init threads
    for(int i = 0; i < numWorkers; ++i) {
        // ...
        pthread_create(&fw_worker, &jw_workers[i]);
    }

    for(;;) {
        sem_wait(jw_queueSem);

        lock jw_jobQueue;
        if(!EXIT_WHEN_ALL_JOBS_COMPLETE)
            while(jw_jobQueue is empty) pthread_cond_wait(jw_jobAdded);
        else {
            jw_exit(0);
            return;
        }
        take msg from jw_jobQueue;
        release jw_jobQueue;

        find free thread => t;
        pthread_mutex_lock(t->job_condMutex);
{'fraction_non_alphanumeric': 0.09370593513652097, 'fraction_numerical': 0.003973849506473529, 'mean_word_length': 2.7312290769966525, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4832906', 'n_tokens_mistral': 2431, 'n_tokens_neox': 2333, 'n_words': 940}