<commit_before>Arches 6.0.0 release notes
------------------------


### Major enhancements
- Moves geometry management from a materialized view to a table #7121
- Adds support for sass stylesheets and preprocessing #7396
- Adds support for geojson & kml file drag and drop in map widget/card #7030
- Allows user to sort search results #7821
- Adds an iiif image service manager (aka manifest manager) #6992
- Shows annotations as overlays in iiif viewer/editor components #6989
- Adds coordinate editor feature to map card and widget #7124
- Advanced search cards collapse by resource type #7414
- Allows users to create new geojson features from a geom buffer #7225
- Introduces component to create related instance with top card data, #7397, #7386
- Adds URL datatype and widget #7226
- Orders graph data before export for more readable diffs #7821
- Adds command to index resources by model type #7490
- Adds transaction id to edit log to group associated tile/resource edit #7559, #7563
- Allow transaction rollback #7595
- Adds an api endpoint to validate tiles and instances #7211, #7220 
- Introduces v2 (beta) version of the resources json endpoint #7589
- Allows summary reports to leverage label based graph #7441
- Adds option to include a resource report link in search export results #7586
- Allows admin to restrict background search export for guest users #7866
- Allows users to export search results by tile to .xlsx format in search export #7507
- Allows users to export search results as a html report #7825
- Adds geojson format to search export #6912
- Column order in search export can be configured to match card order #7867

### Additional highlights
- Fixes null search bug e7e66b35a
- Allow float values for cluster distance, re #7120 f7d0b3b2b
- Fix whatisthis command by casting vw_annotations featureid to uuid, 85ea624ff
- Adds put method to the resource api endpoint e5d5cb8e8
- Removes double-scroll behavior from reports #7232 d640c71cb
- In the photo gallery card show only widgets configured as visible, re #7247 f48c11ae4
- Move astimezone bug fix to stable/5.2.x 10f36f4c7
- Update FileList displayname as url not name, #5617 c8530db8f
- When deleting resources make sure documents in custom indexes are del… 03460b4d0
- Fix related resource report section f5310032c
- Fixes bug that prevented cloning graphs with map styles 629842634
- Fixes bug preventing node-value datatype data from loading 7b34c2241
- Fixes bug preventing annotation datatype data from loading 1c7102b70
- Fixes intermittent failure to save/index annotation nodes  9dac0fdd9
- Put cardinality n tiles in an array, even if there is only one tile 25b91e909
- Removes unused graph manager files 78352a110
- Improve get_display_value method of resource instance datatype, re #7364 482abea72
- Apply term filter even if term is an empty string. 874d110a4
- Fixes loading state of search view #7330 bb8dc93c4
- Remove update of '_tileData' causing widget to lose focus 13c68c0ae
- Fix erroneous prompt to run `manage.py makemigrations` e56986bd0
- Update street layer in dml 59194952e
- Remove unused arches.css file d094518e3
- Modify .eslintrc file to permit ES 6 syntax d16d0b0cb
- Changes installation of yarn via apt instead of npm, re #7024 e904bced5
- Adds get_display_value to BooleanDataType (overriding BaseDataType wh… f9c086a56
- Fixes edit log issue on resource creation through API 225f4383a
- Updated community forum link 626c2b86a
- Adds missing nodes to to tiles created through the API fac0dd7be
- Pass permissions info to search components 9961571b6
- Update models to return promises e0a0087cc
- Change 'Manifest' to 'Image Service' 99927c170
- Fix grouping card in default report template. 9490bf557
- Fix hide null values position b7036b638
- Fixes mapping integer outofbounds bug cfed41a30
- Updates django and pyprind, re #7877 b7bedd7f9
- Fix number datatype to save 0 rather than None in search export #7894


### Dependency changes:
```
Python:
    Upgraded:
        django 2.2.13 -> 2.2.24
        pyprind 2.11.2 -> 2.11.3

    Added:
        openpyxl 3.0.7
        django_compressor 2.2
        django-libsass 0.7

Javascript:
    Upgraded:
        chosen-js 1.8.2 -> 1.8.7

    Added:
        @tmcw/togeojson 4.3.0
        cytoscape 3.18.2
        cytoscape-cola 2.4.0
```


### Upgrading Arches
1. You must be upgraded to at least version 5.1.1 before proceeding.

2. Be sure to import any 5.1 graphs and resource instance data that you need in your system as you won't be able to do it after the upgrade.

3. Be sure to backup your database before proceeding

4. Upgrade to Arches 6.0.0

        pip install --upgrade arches

5. Within your project with your Python 3 virtual environment activated:

        python manage.py migrate
        python manage.py es reindex_database

6. **Important:** If you are maintaining an Arches package, be sure to export your graphs and resource instance data before reimporting or reloading your package. 


### Upgrading an Arches project
1. Update your JavaScript dependencies

    Your project's package.json file looks something like this (There may be differences, but what's important is that arches is listed in the dependencies):

    ```    
    {
        "name": "myproject",
        "dependencies": {
            "arches": "archesproject/arches#stable/5.1.4",
        }
    }
    ```
    Change the arches version number from `#stable/5.1.4` (or some other version of 5.1) to `#stable/6.0.0`\
    Then cd to your project's package.json file directory and run: `yarn install`

    **If your package.json does not have arches listed as a dependency**, follow the instructions of [step 1 here](https://github.com/archesproject/arches/blob/master/releases/5.1.0.md#upgrading-an-arches-project) to update your package.json file. Be sure to use #stable/6.0.0 as the version number for arches when you update your package.json file.
    
    Once your package.json file is updated, run:

    ```yarn install```

2. Add ```compressor``` to the end of your INSTALLED_APPS setting

    ```
    INSTALLED_APPS = (
        'django.contrib.admin',
        'django.contrib.auth',
        'django.contrib.contenttypes',
        'django.contrib.sessions',
        'django.contrib.messages',
        'django.contrib.staticfiles',
        'django.contrib.gis',
        'arches',
        'compressor',
    )
    ```

3. Update STATIC_ROOT in your settings
   
    If you are running Arches **locally** and the STATIC_ROOT setting is \
    ```STATIC_ROOT = "/var/www/media"```, \

      then change it to \
      ```STATIC_ROOT = ""```

    If you are running Arches on **Apache**, you'll need to navigate to your STATIC_ROOT directory. Then run:

    ```
    mkdir CACHE
    sudo chgrp www-data CACHE
    ```

4. Finally, if you are running Arches on Apache, be sure to run:

    ```
    python manage.py collectstatic
    ```
    and restart your server.
    ```
    sudo service apache2 reload
    ```<commit_msg>Remind admin to restart celery, re #7883<commit_after>Arches 6.0.0 release notes
------------------------


### Major enhancements
- Moves geometry management from a materialized view to a table #7121
- Adds support for sass stylesheets and preprocessing #7396
- Adds support for geojson & kml file drag and drop in map widget/card #7030
- Allows user to sort search results #7821
- Adds an iiif image service manager (aka manifest manager) #6992
- Shows annotations as overlays in iiif viewer/editor components #6989
- Adds coordinate editor feature to map card and widget #7124
- Advanced search cards collapse by resource type #7414
- Allows users to create new geojson features from a geom buffer #7225
- Introduces component to create related instance with top card data, #7397, #7386
- Adds URL datatype and widget #7226
- Orders graph data before export for more readable diffs #7821
- Adds command to index resources by model type #7490
- Adds transaction id to edit log to group associated tile/resource edit #7559, #7563
- Allow transaction rollback #7595
- Adds an api endpoint to validate tiles and instances #7211, #7220 
- Introduces v2 (beta) version of the resources json endpoint #7589
- Allows summary reports to leverage label based graph #7441
- Adds option to include a resource report link in search export results #7586
- Allows admin to restrict background search export for guest users #7866
- Allows users to export search results by tile to .xlsx format in search export #7507
- Allows users to export search results as a html report #7825
- Adds geojson format to search export #6912
- Column order in search export can be configured to match card order #7867

### Additional highlights
- Fixes null search bug e7e66b35a
- Allow float values for cluster distance, re #7120 f7d0b3b2b
- Fix whatisthis command by casting vw_annotations featureid to uuid, 85ea624ff
- Adds put method to the resource api endpoint e5d5cb8e8
- Removes double-scroll behavior from reports #7232 d640c71cb
- In the photo gallery card show only widgets configured as visible, re #7247 f48c11ae4
- Move astimezone bug fix to stable/5.2.x 10f36f4c7
- Update FileList displayname as url not name, #5617 c8530db8f
- When deleting resources make sure documents in custom indexes are del… 03460b4d0
- Fix related resource report section f5310032c
- Fixes bug that prevented cloning graphs with map styles 629842634
- Fixes bug preventing node-value datatype data from loading 7b34c2241
- Fixes bug preventing annotation datatype data from loading 1c7102b70
- Fixes intermittent failure to save/index annotation nodes  9dac0fdd9
- Put cardinality n tiles in an array, even if there is only one tile 25b91e909
- Removes unused graph manager files 78352a110
- Improve get_display_value method of resource instance datatype, re #7364 482abea72
- Apply term filter even if term is an empty string. 874d110a4
- Fixes loading state of search view #7330 bb8dc93c4
- Remove update of '_tileData' causing widget to lose focus 13c68c0ae
- Fix erroneous prompt to run `manage.py makemigrations` e56986bd0
- Update street layer in dml 59194952e
- Remove unused arches.css file d094518e3
- Modify .eslintrc file to permit ES 6 syntax d16d0b0cb
- Changes installation of yarn via apt instead of npm, re #7024 e904bced5
- Adds get_display_value to BooleanDataType (overriding BaseDataType wh… f9c086a56
- Fixes edit log issue on resource creation through API 225f4383a
- Updated community forum link 626c2b86a
- Adds missing nodes to to tiles created through the API fac0dd7be
- Pass permissions info to search components 9961571b6
- Update models to return promises e0a0087cc
- Change 'Manifest' to 'Image Service' 99927c170
- Fix grouping card in default report template. 9490bf557
- Fix hide null values position b7036b638
- Fixes mapping integer outofbounds bug cfed41a30
- Updates django and pyprind, re #7877 b7bedd7f9
- Fix number datatype to save 0 rather than None in search export #7894


### Dependency changes:
```
Python:
    Upgraded:
        django 2.2.13 -> 2.2.24
        pyprind 2.11.2 -> 2.11.3

    Added:
        openpyxl 3.0.7
        django_compressor 2.2
        django-libsass 0.7

Javascript:
    Upgraded:
        chosen-js 1.8.2 -> 1.8.7

    Added:
        @tmcw/togeojson 4.3.0
        cytoscape 3.18.2
        cytoscape-cola 2.4.0
```


### Upgrading Arches
1. You must be upgraded to at least version 5.1.1 before proceeding.

2. Be sure to import any 5.1 graphs and resource instance data that you need in your system as you won't be able to do it after the upgrade.

3. Be sure to backup your database before proceeding

4. Upgrade to Arches 6.0.0

        pip install --upgrade arches

5. Within your project with your Python 3 virtual environment activated:

        python manage.py migrate
        python manage.py es reindex_database

6. **Important:** If you are maintaining an Arches package, be sure to export your graphs and resource instance data before reimporting or reloading your package. 


### Upgrading an Arches project
1. Update your JavaScript dependencies

    Your project's package.json file looks something like this (There may be differences, but what's important is that arches is listed in the dependencies):

    ```    
    {
        "name": "myproject",
        "dependencies": {
            "arches": "archesproject/arches#stable/5.1.4",
        }
    }
    ```
    Change the arches version number from `#stable/5.1.4` (or some other version of 5.1) to `#stable/6.0.0`\
    Then cd to your project's package.json file directory and run: `yarn install`

    **If your package.json does not have arches listed as a dependency**, follow the instructions of [step 1 here](https://github.com/archesproject/arches/blob/master/releases/5.1.0.md#upgrading-an-arches-project) to update your package.json file. Be sure to use #stable/6.0.0 as the version number for arches when you update your package.json file.
    
    Once your package.json file is updated, run:

    ```yarn install```

2. Add ```compressor``` to the end of your INSTALLED_APPS setting

3. Update STATIC_ROOT in your settings
   
    If you are running Arches **locally** and the STATIC_ROOT setting is \
    ```STATIC_ROOT = "/var/www/media"```, \

      then change it to \
      ```STATIC_ROOT = ""```

    If you are running Arches on **Apache**, you'll need to navigate to your STATIC_ROOT directory. Then run:

    ```
    mkdir CACHE
    sudo chgrp www-data CACHE
    ```

4. If you are running Arches on Apache, be sure to run:

    ```
    python manage.py collectstatic
    ```
    and restart your server.
    ```
    sudo service apache2 reload
    ```
    
5. Finally, If you are running Celery, you should also restart your Celery worker(s). The process for doing this depends on how Celery is being run.
{'fraction_non_alphanumeric': 0.06633835457705678, 'fraction_numerical': 0.06351390498261877, 'mean_word_length': 3.4090038314176243, 'pattern_counts': {'":': 6, '<': 3, '<?xml version=': 0, '>': 9, 'https://': 2, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1022088', 'n_tokens_mistral': 4519, 'n_tokens_neox': 3946, 'n_words': 1829}