<commit_before># Known Issues

- ### MySQL versions below 5.7

  When the policy server is backed by MySQL versions < 5.7, a user may see this error when trying to create a policy:

  ```
  FAILED
  adding policies: failed to make request to policy server
  ```

  An operator inspecting the logs of the `policy-server` BOSH job may see this error:

  > creating destination: Error 1064: You have an error in your SQL syntax;
  check the manual that corresponds to your MySQL server version for the right
  syntax to use near 'WHERE\n\t\tNOT EXISTS (\n\t\t\tSELECT *\n\t\t\tFROM destinations\n\t\t\tWHERE group_id = ? AND '

  This issue can be resolved by upgrading your MySQL server to version 5.7+


- ### Missing Feature Parity For Application Security Groups
  Current support for application security groups in netman is incomplete:
  - The only supported protocols are `tcp` and `udp`, this means `icmp` protocol,
    code and type are not supported
  - Only single ports are supported, not ranges
  - We currently do not support logging in ASGs

- ###  Behavior Changes From Existing Application Security Groups
  Current implementations of ASGs allow opening security groups to other containers
  via the NATed port on the diego cell. With container networking we only support
  direct addressing of other containers through the overlay network and app-to-app
  policy system. Direct addressing of other containers (without going through the gorouter)
  on the underlay is not supported and may result in undefined behavior.

- ### CIDR blocks other than /16
  It is possible to configure the CIDR block for containers to be something
  other than the default of /16. This hasn't been tested.
  We don't know what happens. Good luck.

- ### Blue/Green deploys of apps must reconfigure policies
  Following the instructions
  [here](https://docs.cloudfoundry.org/devguide/deploy-apps/blue-green.html),
  when the green app is deployed it will have a different app guid than blue,
  meaning any container to container policies that blue has configured will need
  to be configured for green as well.


- ### Stale policies not cleaned up
  If you push an app and configure a policy for that app, when you delete the app
  without deleting the policy, then the policy will stay in the policies database and be seen when you run (as cf admin):
  ```bash
  cf curl /networking/v0/external/policies
  ```
  Under normal circumstances, these stale policies should not cause any major issues.
  If you want to delete every policy from the server run (as cf admin):
  ```bash
  cf curl /networking/v0/external/policies -X DELETE -d `cf curl /networking/v0/external/policies`
  ```

- ### Upgrading/Downgrading between garden-runc and garden-runc + netman requires a recreate
garden-runc and netman may leave around iptables rules or networking devices when switching networking stacks.
The safest way to upgrade from one to the other is to run:
  ```bash
  bosh deploy --recreate
  ```

- ### Flannel watchdog fails on bosh-lite
  Flannel on bosh-lite often gets into a state where the overlay network is not functioning.
  A process called `flannel-watchdog` runs on the cells and checks for this error and will cause BOSH to consider the VM unhealthy.
  If you run `bosh vms` and see output similar to this:
  ```
  Deployment 'cf-warden-diego'

  Director task 939

  Task 939 done<commit_msg>docs/known-issues: describe how to clean up stale policies<commit_after># Known Issues

- ### MySQL versions below 5.7

  When the policy server is backed by MySQL versions < 5.7, a user may see this error when trying to create a policy:

  ```
  FAILED
  adding policies: failed to make request to policy server
  ```

  An operator inspecting the logs of the `policy-server` BOSH job may see this error:

  > creating destination: Error 1064: You have an error in your SQL syntax;
  check the manual that corresponds to your MySQL server version for the right
  syntax to use near 'WHERE\n\t\tNOT EXISTS (\n\t\t\tSELECT *\n\t\t\tFROM destinations\n\t\t\tWHERE group_id = ? AND '

  This issue can be resolved by upgrading your MySQL server to version 5.7+


- ### Missing Feature Parity For Application Security Groups
  Current support for application security groups in netman is incomplete:
  - The only supported protocols are `tcp` and `udp`, this means `icmp` protocol,
    code and type are not supported
  - Only single ports are supported, not ranges
  - We currently do not support logging in ASGs

- ###  Behavior Changes From Existing Application Security Groups
  Current implementations of ASGs allow opening security groups to other containers
  via the NATed port on the diego cell. With container networking we only support
  direct addressing of other containers through the overlay network and app-to-app
  policy system. Direct addressing of other containers (without going through the gorouter)
  on the underlay is not supported and may result in undefined behavior.

- ### CIDR blocks other than /16
  It is possible to configure the CIDR block for containers to be something
  other than the default of /16. This hasn't been tested.
  We don't know what happens. Good luck.

- ### Blue/Green deploys of apps must reconfigure policies
  Following the instructions
  [here](https://docs.cloudfoundry.org/devguide/deploy-apps/blue-green.html),
  when the green app is deployed it will have a different app guid than blue,
  meaning any container to container policies that blue has configured will need
  to be configured for green as well.


- ### Stale policies not cleaned up automatically
  If you push an app and configure a policy for that app, when you delete the app
  without deleting the policy, then the policy will stay in the policies database.

  Such "stale policies" are not visible from the CF CLI, but remain in the database.

  Under normal circumstances, stale policies should not cause any major issues.

  The policy server exposes a manual "policy cleanup" endpoint that will
  remove all stale policies.  To perform a "cleanup":

  ```bash
  cf curl -X POST /networking/v0/external/policies/cleanup
  ```


- ### Upgrading/Downgrading between garden-runc and garden-runc + netman requires a recreate
garden-runc and netman may leave around iptables rules or networking devices when switching networking stacks.
The safest way to upgrade from one to the other is to run:
  ```bash
  bosh deploy --recreate
  ```

- ### Flannel watchdog fails on bosh-lite
  Flannel on bosh-lite often gets into a state where the overlay network is not functioning.
  A process called `flannel-watchdog` runs on the cells and checks for this error and will cause BOSH to consider the VM unhealthy.
  If you run `bosh vms` and see output similar to this:
  ```
  Deployment 'cf-warden-diego'

  Director task 939

  Task 939 done
{'fraction_non_alphanumeric': 0.05604203152364273, 'fraction_numerical': 0.00642148277875073, 'mean_word_length': 3.768963117606124, 'pattern_counts': {'":': 1, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5245796', 'n_tokens_mistral': 1887, 'n_tokens_neox': 1775, 'n_words': 979}