<commit_before>camel-file-loadbalancer
===================
Sample code to have multiple camel file uri routes (either in same context or different), watching the same folder, i.e.: 'load balancer' for files

Current implementation works by the following logic:

1. Each file endpoint has a reference to a PriorityFileFilterFactory
2. The PriorityFileFilterFactory is assigned X number of endpoints which are watching the folder
3. The PriorityFileFilterFactory creates X PriorityFileFilters
4. Each PriorityFileFilter is given a priority starting from 0
5. Each endpoint 'Filter', is set to a PriorityFileFilter
6. Each endpoint 'MaxMessagesPerPoll' is set to the number of PriorityFileFilters created
7. Each endpoint 'Move' is set to .camelX - X being the priority of the PriorityFileFilter

This means that you end up with the following:

- 3 endpoints watching 1 folder
- Endpoint 0 has a priority of 0, thus gets the first file
- Endpoint 1 has a priority of 1, thus gets the second file
- Endpoint 2 has a priority of 2, thus gets the third file
- and so on...

If you are going to deploy the same endpoint, multiple times within the same context (as per lb-example), you will also need to set a value for 'uniqueKey'.
This is because Camel checks for endpoint uniqueness, we get around this with a slight hack

future
===================
Am also going to look into creating a 'fabric' (i.e. zookeeper backed) endpoint, which automatically scales.
Thus you don't have to set how many endpoints are watching the folder, it knows because of the number which has registered<commit_msg>Added to read me as project name isn't best currently<commit_after>camel-file-loadbalancer
===================
Sample code to have multiple camel file uri routes (either in same context or different), watching the same folder, i.e.: 'load balancer' for files

Current implementation works by the following logic:

1. Each file endpoint has a reference to a PriorityFileFilterFactory
2. The PriorityFileFilterFactory is assigned X number of endpoints which are watching the folder
3. The PriorityFileFilterFactory creates X PriorityFileFilters
4. Each PriorityFileFilter is given a priority starting from 0
5. Each endpoint 'Filter', is set to a PriorityFileFilter
6. Each endpoint 'MaxMessagesPerPoll' is set to the number of PriorityFileFilters created
7. Each endpoint 'Move' is set to .camelX - X being the priority of the PriorityFileFilter

This means that you end up with the following:

- 3 endpoints watching 1 folder
- Endpoint 0 has a priority of 0, thus gets the first file
- Endpoint 1 has a priority of 1, thus gets the second file
- Endpoint 2 has a priority of 2, thus gets the third file
- and so on...

If you are going to deploy the same endpoint, multiple times within the same context (as per lb-example), you will also need to set a value for 'uniqueKey'.
This is because Camel checks for endpoint uniqueness, we get around this with a slight hack

future
===================
Am also going to look into creating a 'fabric' (i.e. zookeeper backed) endpoint, which automatically scales.
Thus you don't have to set how many endpoints are watching the folder, it knows because of the number which has registered

rename project/work to (few ideas)
- camel-file-multiple-read-priority
- camel-file-priority
-camel-file-multiple-folder
- ....???
{'fraction_non_alphanumeric': 0.06690561529271206, 'fraction_numerical': 0.009557945041816009, 'mean_word_length': 4.517298187808896, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3240216', 'n_tokens_mistral': 866, 'n_tokens_neox': 808, 'n_words': 504}