Q:

Hybrid environments, 404 with on-premise mailboxes, 200 for Exchange online

I've written an application which is able to talk to Exchange Online accounts, I'm currently trying to test to see the steps required to get it to work with on-premise accounts running in Hybrid.
I have:

Windows Server 2012 R2
Exchange 2016 CU8 (with /PrepareAD run)
Internet accessible paths to /api/v2.0 and /autodiscover/autodiscover.json
Run the HCW
AD Connect is running
Active Directory is synced to Azure Active Directory
Permissions look good on the application that I have registered (read calendar events / read-write calendar events)

Unfortunately, I'm getting a 404 when trying to access calendar events:
curl -v -H 'Content-Type: application/json' -H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJub25jZSI6IkFRQUJBQUFBQUFCSGg0a21TX2FLVDVYcmp6eFJBdEh6RE1mbEZNYTYwaktGRHRhUXp0ZGVkM2V6Z0ZfUzlLMjdDRmQxSHlfZGdRcnR6WlJBczRDV095R3E1Vl9OZW9MSFNKTGpzblNCSDNCQU9oQnBzU18wVmlBQSIsImFsZyI6IlJTMjU2IiwieDV0IjoiU1NRZGhJMWNLdmhRRURTSnhFMmdHWXM0MFEwIiwia2lkIjoiU1NRZGhJMWNLdmhRRURTSnhFMmdHWXM0MFEwIn0.eyJhdWQiOiJodHRwczovL2dyYXBoLm1pY3Jvc29mdC5jb20iLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC8wODY1YjBhYi02ZjYwLTQzNjMtYTEwZi05NWU5ZTc5ZjlmODEvIiwiaWF0IjoxNTE4OTc5NzQ0LCJuYmYiOjE1MTg5Nzk3NDQsImV4cCI6MTUxODk4MzY0NCwiYWlvIjoiWTJOZ1lKRE0reDVXbGhWNUlrajUwbDB4THNacEFBPT0iLCJhcHBfZGlzcGxheW5hbWUiOiJIQy1XZWIiLCJhcHBpZCI6ImVkZjlkY2M0LThjNjAtNDg3ZS1hYmUyLTI4MjcyYTRlZGJlMCIsImFwcGlkYWNyIjoiMiIsImlkcCI6Imh0dHBzOi8vc3RzLndpbmRvd3MubmV0LzA4NjViMGFiLTZmNjAtNDM2My1hMTBmLTk1ZTllNzlmOWY4MS8iLCJvaWQiOiJkNTUzMTE2My01MTcxLTRmZjctYTNjMS04NWFlNzUzMjkzNTIiLCJyb2xlcyI6WyJNYWlsLlJlYWRXcml0ZSIsIkNvbnRhY3RzLlJlYWRXcml0ZSIsIkRpcmVjdG9yeS5SZWFkLkFsbCIsIk1haWwuUmVhZCIsIkNhbGVuZGFycy5SZWFkV3JpdGUiXSwic3ViIjoiZDU1MzExNjMtNTE3MS00ZmY3LWEzYzEtODVhZTc1MzI5MzUyIiwidGlkIjoiMDg2NWIwYWItNmY2MC00MzYzLWExMGYtOTVlOWU3OWY5ZjgxIiwidXRpIjoiNkxJT3g1bWQ4ay05ajhBUUtiY2hBQSIsInZlciI6IjEuMCJ9.AWt_ANsH8sk15WeH1AgD6SD0Ki8VILMvzkbSMju_YFGKc5cVkrGp7Skzt64uDM8rI6Py5Y-1c3srXwON2oSihkRskfz5vG4nIlbFnuYd3Ij2Vz1ktpNnCeMAnAK2T8ifk2visRSvchRbuBNZZyamwRjActdDF9BS8NygUgmmygK4mPjOIab17PJPz5PisvRbCA2jBLWLvbu9RYrLH-xGuoLd2PLTbsn2WSVi3er4XztZCcK7XfVWe-0wjrV6qBufd5z0hH_KpQLdzPtLOzSUGUAcXGa0mBPceTWULQvQ-LPcAJO57F0ir5k22fWzlkOfUxQb9eGWREUm1cAPWk3CPw" "https://graph.microsoft.com/v1.0/users/oq@healthcentrified.co.uk/calendar/events"
*   Trying 137.116.241.64...
* Connected to graph.microsoft.com (137.116.241.64) port 443 (#0)
* found 148 certificates in /etc/ssl/certs/ca-certificates.crt
* found 592 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_256_CBC_SHA384
*        server certificate verification OK
*        server certificate status verification SKIPPED
*        common name: graph.microsoft.com (matched)
*        server certificate expiration date OK
*        server certificate activation date OK
*        certificate public key: RSA
*        certificate version: #3
*        subject: C=US,ST=WA,L=Redmond,O=Microsoft Corporation,OU=Microsoft Corporation,CN=graph.microsoft.com
*        start date: Wed, 03 Jan 2018 17:32:18 GMT
*        expire date: Fri, 03 Jan 2020 17:32:18 GMT
*        issuer: C=US,ST=Washington,L=Redmond,O=Microsoft Corporation,OU=Microsoft IT,CN=Microsoft IT TLS CA 4
*        compression: NULL
* ALPN, server did not agree to a protocol
> GET /v1.0/users/oq@healthcentrified.co.uk/calendar/events HTTP/1.1
> Host: graph.microsoft.com
> User-Agent: curl/7.47.0
> Accept: */*
> Content-Type: application/json
> Authorization: Bearer eyJ0eXAiOiJKV1QiLCJub25jZSI6IkFRQUJBQUFBQUFCSGg0a21TX2FLVDVYcmp6eFJBdEh6RE1mbEZNYTYwaktGRHRhUXp0ZGVkM2V6Z0ZfUzlLMjdDRmQxSHlfZGdRcnR6WlJBczRDV095R3E1Vl9OZW9MSFNKTGpzblNCSDNCQU9oQnBzU18wVmlBQSIsImFsZyI6IlJTMjU2IiwieDV0IjoiU1NRZGhJMWNLdmhRRURTSnhFMmdHWXM0MFEwIiwia2lkIjoiU1NRZGhJMWNLdmhRRURTSnhFMmdHWXM0MFEwIn0.eyJhdWQiOiJodHRwczovL2dyYXBoLm1pY3Jvc29mdC5jb20iLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC8wODY1YjBhYi02ZjYwLTQzNjMtYTEwZi05NWU5ZTc5ZjlmODEvIiwiaWF0IjoxNTE4OTc5NzQ0LCJuYmYiOjE1MTg5Nzk3NDQsImV4cCI6MTUxODk4MzY0NCwiYWlvIjoiWTJOZ1lKRE0reDVXbGhWNUlrajUwbDB4THNacEFBPT0iLCJhcHBfZGlzcGxheW5hbWUiOiJIQy1XZWIiLCJhcHBpZCI6ImVkZjlkY2M0LThjNjAtNDg3ZS1hYmUyLTI4MjcyYTRlZGJlMCIsImFwcGlkYWNyIjoiMiIsImlkcCI6Imh0dHBzOi8vc3RzLndpbmRvd3MubmV0LzA4NjViMGFiLTZmNjAtNDM2My1hMTBmLTk1ZTllNzlmOWY4MS8iLCJvaWQiOiJkNTUzMTE2My01MTcxLTRmZjctYTNjMS04NWFlNzUzMjkzNTIiLCJyb2xlcyI6WyJNYWlsLlJlYWRXcml0ZSIsIkNvbnRhY3RzLlJlYWRXcml0ZSIsIkRpcmVjdG9yeS5SZWFkLkFsbCIsIk1haWwuUmVhZCIsIkNhbGVuZGFycy5SZWFkV3JpdGUiXSwic3ViIjoiZDU1MzExNjMtNTE3MS00ZmY3LWEzYzEtODVhZTc1MzI5MzUyIiwidGlkIjoiMDg2NWIwYWItNmY2MC00MzYzLWExMGYtOTVlOWU3OWY5ZjgxIiwidXRpIjoiNkxJT3g1bWQ4ay05ajhBUUtiY2hBQSIsInZlciI6IjEuMCJ9.AWt_ANsH8sk15WeH1AgD6SD0Ki8VILMvzkbSMju_YFGKc5cVkrGp7Skzt64uDM8rI6Py5Y-1c3srXwON2oSihkRskfz5vG4nIlbFnuYd3Ij2Vz1ktpNnCeMAnAK2T8ifk2visRSvchRbuBNZZyamwRjActdDF9BS8NygUgmmygK4mPjOIab17PJPz5PisvRbCA2jBLWLvbu9RYrLH-xGuoLd2PLTbsn2WSVi3er4XztZCcK7XfVWe-0wjrV6qBufd5z0hH_KpQLdzPtLOzSUGUAcXGa0mBPceTWULQvQ-LPcAJO57F0ir5k22fWzlkOfUxQb9eGWREUm1cAPWk3CPw
>
< HTTP/1.1 404 Not Found
< Cache-Control: private
< Transfer-Encoding: chunked
< Content-Type: text/plain
< request-id: f499015e-325b-45e8-9716-0a8a7160b82d
< client-request-id: f499015e-325b-45e8-9716-0a8a7160b82d
< x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"North Europe","Slice":"SliceA","Ring":"3","ScaleUnit":"003","Host":"AGSFE_IN_0","ADSiteName":"DUB"}}
< Duration: 1764.3754
< Date: Sun, 18 Feb 2018 19:18:28 GMT
<
* Connection #0 to host graph.microsoft.com left intact

If I do a similar request on a mailbox that has been migrated to Exchange Online, this works (I get an HTTP 200 and a list of events back in JSON)
Office 365 support is unsure of what to do here (it is probably beyond their scope).
The documentation says that it is in preview, but, should still work

Microsoft Graph has always provided access to customer mailboxes in the cloud on Exchange Online as part of Office 365. Exchange 2016 Cumulative Update 3 (CU3), released in September 2016 for Exchange on-premises servers, adds support for REST API integration with Office 365. If your app uses v1.0 of the Mail, Calendar, or Contacts API, you will now also find a seamless authentication and application experience in hybrid deployments, regardless of whether the mailbox is on-premises or in the cloud, provided that the deployment meets specific requirements.
Behind the scenes, when Microsoft Graph identifies that a REST API
  call is attempting to access an on-premises mailbox in a hybrid
  deployment, it proxies the REST request to an on-premises REST
  endpoint which then processes the request. This discovery makes
  accessing the REST API possible.

I will state that I have very limited Windows experience and this is my first foray into using Windows servers for anything, however, the lack of documentation on this scenario, since this is probably something that a lot of large corporate organizations would want to do.
Is there anything obviously wrong?
Update
Rasmus asked if I had any requests going to my web server and whilst I can see a lot of traffic on /rpc I get nothing hitting autodiscover.json and the only requests to /API are
2018-02-25 18:58:24 ::1 
GET /api/v1.0/users/HealthMailboxda9cb9ff7af047cf9878a9b7be391e14@healthcentrified.co.uk/Messages 
$top=1
&request_id=4f17c7a2-f753-46f7-853d-36f7a5281932 444 
- ::1 Odata_AM_Probe/Local - 401 0 0 0

And others to this mailbox from this user-agent

A:

I've checked your token on https://jwt.ms and found out it's an Application token from the Client Credentials Flow. This isn't supported out of the box but today I've found out that you can enable these tokens on your on-premise exchange environment.
See my answer in this post: https://stackoverflow.com/a/56131954/639153
Original answer here

