<commit_before># Master List

Developed initially for the DES SL group to handle list of candidates for follow up, but can handle any list of objects

## Cloning the project

```
  $ git clone https://github.com/linea-it/masterlist.git
```

## Create a virtualenv and install software dependencies
```
  $ cd masterlist
  $ virtualenv env
  $ source env/bin/activate
  $ pip install -r requirements.txt
```

## Run the app in development mode
```
  $ cd masterlist
  $ ./run.sh
```

the first time you will be asked to set a database password, use ```<PASSWORD>```.


The app will run in your browser at http://localhost:8000

## Changing the page layout
The layout of the page is defined at ```masterlist/candidates/templates```

## Customizing the table content
See ```masterlist/candidates/models.py``` to customize the columns of your table, once you changed the model, just
remove the old ```db.sqlite3``` databse and execute the ```run.sh``` script again to create a new db.

You can add/edit entries in the database using the Administration interface at http://localhost:8000/admin.

Use the username: ```nobody``` and password: ```<PASSWORD>``` to access the admin interface.

The table allows an image column, if you place an image under ```static/img``` and register the name of
the image file in the ```thumb``` column the corresponding image will appear in the table.
<commit_msg>general update of the Readme with instructions<commit_after># Master List

Developed initially for the DES SL group to handle list of candidates for follow up, but can handle any list of objects

## Getting started 
### Cloning the project

```
  $ git clone https://github.com/linea-it/masterlist.git
```

### Create a virtualenv and install software dependencies
```
  $ cd masterlist
  $ virtualenv env
  $ source env/bin/activate
  $ pip install -r requirements.txt
```

## Run a sample of the code on your local host (only for example purposes)
### Run the app in development mode
```
  $ cd masterlist
  $ ./run.sh
```
1. The first time you will be asked to set a database password, use ```<PASSWORD>```.
2. The app will run in your browser at http://localhost:8000

## Modifying the Web Page Back end
remove the old ```db.sqlite3``` databse and execute the ```run.sh``` script again to create a new db.
2. You can add/edit entries in the database using the Administration interface at http://localhost:8000/admin.
3. Use the username: ```nobody``` and password: ```<PASSWORD>``` to access the admin interface.
4. The table allows an image column, if you place an image under ```static/img``` and register the name of
the image file in the ```thumb``` column the corresponding image will appear in the table.

### Changing the page layout
1. The layout of the page is defined at ```masterlist/candidates/templates```

### Customizing the table content
1. See ```masterlist/candidates/models.py``` to customize the columns of your table, once you changed the model

### Migrating changes
1. Perform the following if you have updated the page layout or the column definitions (from path ~/masterlist/masterlist/)):

``` 
$ python manage.py makemigrations
$ python manage.py migrate
```


## Updating the master catalog with a list of new objects

### Running the updater
1. Run the updater (update_masterlist.py) from the command line

```
$ python update_masterlist.py <input file> 
```

2. Run the updater with help to get the available options

```
$ python update_master.py -h

usage: update_masterlist.py [-h] [-t] [-b] [-i I]

description: update the masterlist by adding a list of objects from an input
file.

optional arguments:
  -h, --help       show this help message and exit
  -t, --test       test appending a list
  -b, --backup     create a backup of the masterlist only
  -i I, --input I  input file name
```


### File format for the list of new objects
1. The file format must be ".csv"
2. The required headings are:
``` ra, dec, data_season, rank ```

{'fraction_non_alphanumeric': 0.10209226115452483, 'fraction_numerical': 0.007562389715149987, 'mean_word_length': 3.9538077403245944, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 8, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6826664', 'n_tokens_mistral': 1134, 'n_tokens_neox': 1090, 'n_words': 549}