<commit_before># Google-Trends-with-Solr-PHP-UI
A basic UI to search a keyword &amp; visualize its frequencei in a chart


## How to do it

---> All the capital letters have to be changed according to your requiriments <---



### Backend
---------
1. Extract or download Solr; //I used solr-5.0.0 
* add the CLASSPATH (it is needed if you want to use Update.java) with the following shell command
```export CLASSPATH=$HOME/SOLR-DIRECTORY/dist/*:.:$HOME/SOLR-DIRECTORY/dist/solrj-lib/*```
* lunch this long command to create the solr core, new fields and load your data
``` $HOME/SOLR-DIRECTORY/bin/solr start; $HOME/SOLR-DIRECTORY/bin/solr create -c SOLR-CORE-NAME; curl -X POST -H 'Content-type:application/json' --data-binary '{ "add-field" : { "name":"DATE", "type":"tdate", "indexed":true, "stored":true, "default":"1992-07-10T17:33:18Z"}, "add-field" : { "name":"CONTENT", "type":"string", "indexed":true, "stored":true}, "add-copy-field" : { "source":"_text", "dest": [ "CONTENT"]}}' http://localhost:8983/solr/SOLR-CORE-NAME/schema; $HOME/SOLR-DIRECTORY/bin/solr stop; $HOME/SOLR-DIRECTORY/bin/solr start; $HOME/SOLR-DIRECTORY/bin/post -c SOLR-CORE-NAME -Dauto -Drecursive $HOME/DATA-DIRECTORY; $HOME/SOLR-DIRECTORY/bin/solr stop; $HOME/SOLR-DIRECTORY/bin/solr start ```
* Finally, go the directory of ```Update.java```, check the SOLR-CORE-NAME in the 28th line and lunch it (it fixes all contents with the field DATE set to  1992-07-10T17:33:18Z).


### Frontend
---------
1. Copy the Solr-PHP-UI modified version on your web servers working folder;
* check SOLR-CORE-NAME (30th line) in ```SOLR-PHP-UI-DIRECTORY/config/config.php```;
* That's it! Check ```SOLR-PHP-UI-DIRECTORY/templates/view.list.php``` to modify the chart and ```view.index.topbar.php```, in the same directory, to modify the WEBSITE TITLE.


### Suggestions, useful pointers
---------
It could be useful to add these lines to $HOME/.bashrc to have some shortcuts:
> * alias start='$HOME/SOLR-DIRECTORY/bin/solr start'
> * alias stop='$HOME/SOLR-DIRECTORY/bin/solr stop -all'
> * alias update='$HOME/SOLR-DIRECTORY/bin/post -c SOLR-CORE-NAME -Dauto -Drecursive $HOME/DATA-DIRECTORY; cd; java Update' //that's working only if Update.java is inside $HOME<commit_msg>markdown updated<commit_after># Google-Trends-with-Solr-PHP-UI
A basic UI to search a keyword &amp; visualize its frequencei in a chart


## How to do it

---> All the capital letters have to be changed according to your requiriments <---



### Backend
---------
1. Extract or download Solr; //I used solr-5.0.0 
1. add the CLASSPATH (it is needed if you want to use Update.java) with the following shell command
```export CLASSPATH=$HOME/SOLR-DIRECTORY/dist/*:.:$HOME/SOLR-DIRECTORY/dist/solrj-lib/*```
1. lunch this long command to create the solr core, new fields and load your data
``` $HOME/SOLR-DIRECTORY/bin/solr start; $HOME/SOLR-DIRECTORY/bin/solr create -c SOLR-CORE-NAME; curl -X POST -H 'Content-type:application/json' --data-binary '{ "add-field" : { "name":"DATE", "type":"tdate", "indexed":true, "stored":true, "default":"1992-07-10T17:33:18Z"}, "add-field" : { "name":"CONTENT", "type":"string", "indexed":true, "stored":true}, "add-copy-field" : { "source":"_text", "dest": [ "CONTENT"]}}' http://localhost:8983/solr/SOLR-CORE-NAME/schema; $HOME/SOLR-DIRECTORY/bin/solr stop; $HOME/SOLR-DIRECTORY/bin/solr start; $HOME/SOLR-DIRECTORY/bin/post -c SOLR-CORE-NAME -Dauto -Drecursive $HOME/DATA-DIRECTORY; $HOME/SOLR-DIRECTORY/bin/solr stop; $HOME/SOLR-DIRECTORY/bin/solr start ```
1. Finally, go the directory of ```Update.java```, check the SOLR-CORE-NAME in the 28th line and lunch it (it fixes all contents with the field DATE set to  1992-07-10T17:33:18Z).


### Frontend
---------
1. Copy the Solr-PHP-UI modified version on your web servers working folder;
1. check SOLR-CORE-NAME (30th line) in ```SOLR-PHP-UI-DIRECTORY/config/config.php```;
1. That's it! Check ```SOLR-PHP-UI-DIRECTORY/templates/view.list.php``` to modify the chart and ```view.index.topbar.php```, in the same directory, to modify the WEBSITE TITLE.


### Suggestions, useful pointers
---------
It could be useful to add these lines to $HOME/.bashrc to have some shortcuts:
> * alias start='$HOME/SOLR-DIRECTORY/bin/solr start'
> * alias stop='$HOME/SOLR-DIRECTORY/bin/solr stop -all'
> * alias update='$HOME/SOLR-DIRECTORY/bin/post -c SOLR-CORE-NAME -Dauto -Drecursive $HOME/DATA-DIRECTORY; cd; java Update' //that's working only if Update.java is inside $HOME
{'fraction_non_alphanumeric': 0.16814946619217083, 'fraction_numerical': 0.01935053380782918, 'mean_word_length': 4.1630309988518945, 'pattern_counts': {'":': 22, '<': 5, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '33695', 'n_tokens_mistral': 1656, 'n_tokens_neox': 1534, 'n_words': 453}