Q:

jersey + grizzly + hk2: Dependency injection, but not into resource

Following up on Jersey + HK2 + Grizzly: Proper way to inject EntityManager?, I would like to understand how it is possible use dependency injection in classes which are not jersey resources. 
As an example, I might have background tasks running in an ExecutorService, and they might need an EntityManager. If I attempt to @Inject the EntityManager into the class, nothing happens. Injecting it into a @Path-annotated jersey resource class, injecting works fine.
The application is running as a standalone JVM, not on a Java EE application server.
Update: I have created a test scenario to demonstrate what I mean. The code is running a standalone Grizzly server with a Jersey resource, as well as an ExecutorService. A Callable is submitted to the ExecutorService.
Injection of the EntityManager into the resource works, but not into the Callable. There the EntityManager remains null.
Please advise if the code is better kept here than on github.

A:

So to really understand how HK2 works, you should become familiar with its ServiceLocator. It is analogous to Spring ApplicationContext, which is the main container for the DI framework.
In a standalone app, you could bootstrap the DI container simply by doing
ServiceLocatorFactory locatorFactory = ServiceLocatorFactory.getInstance();
ServiceLocator serviceLocator = locatorFactory.create("TestLocator");
ServiceLocatorUtilities.bind(serviceLocator, new EntityManagerProvider());

Now your EntityManagerProvider is registered into the container. You can lookup the EntityManager simply by doing
EntityManager em = serviceLocator.getService(EntityManager.class);

Now in order to be able to take advantage of injection by the container, the service needs to be managed by the container. For example say you have this
public class BackgroundTask implements Callable<String> {

    @Inject
    EntityManager em;

    @Override
    public String call() throws Exception {
        ...
}

which you actually do. The problem is, the BackgroundTask is not managed by the container. So even in a standalone bootstrap (like the three lines of code above), instantiating the task
BackgroundTask task = new BackgroundTask();

does nothing, as far as injection, as the task class is not managed by the container, and you are creating it yourself. If you wanted it managed, there a few ways to register it to the container. You've discovered one already (use an AbstractBinder) and register the binder to the ServiceLocator. Then instead of instantiating the class yourself, you just request it, like the EntityManager example above.
Or you can simply explicitly inject the task, i.e
BackgroundTask task = new BackgroundTask(); 
serviceLocator.inject(task);

What that did was cause the locator to lookup the EntityManager and inject it into your task. 
So how does this all fit in with Jersey? Jersey (partly) handles lookup of services and injection into resources during it's runtime. That's why it work's in your Jersey application. When the EntityManager is needed, it looks up the service an injects it into the resource instance.
So the next question is, if the tasks are being run outside the scope the Jersey application, how can you inject the task? For the most part, all the above is pretty much the gist of it. Jersey has it's own ServiceLocator, and it's not easy to try a obtain a reference to it. We could give Jersey our ServiceLocator, but Jersey ultimately still creates it's own locator and will populate it with our locator. So ultimately there would still be two locators. You can see an example of what I mean in the refactored code below, where it check the references in the ServiceLocatorFeature. 
But if you do want to provide the ServiceLocator to Jersey, you can pass it to the Grizzly server factory method
server = GrizzlyHttpServerFactory.createHttpServer(
        URI.create(BASE_URI),
        config, 
        serviceLocator
);

Now you can still use your locator outside of Jersey. Honestly though, in this case, you could not involve Jersey at all and just keep your own locator, and just register the EntityManagerProvider with both Jersey and your ServiceLocator. I don't see it really making much difference, except for the extra line of code. Functionally, I don't see any change.
To learn more about HK2, I highly recommend thoroughly going through the user guide. You'll learn a lot about what goes on under the hood with Jersey, and also learn about features that you can incorporate into a Jersey application.
Below is the complete refactor of your test. I didn't really change much. Any changes I made are pretty much discussed above.
public class DependencyInjectionTest {

    private final ServiceLocatorFactory locatorFactory
            = ServiceLocatorFactory.getInstance();
    private ServiceLocator serviceLocator;

    private final static String BASE_URI = "http://localhost:8888/";
    private final static String OK = "OK";
    private HttpServer server;
    private ExecutorService backgroundService;

    public class EntityManagerProvider extends AbstractBinder
            implements Factory<EntityManager> {

        private final EntityManagerFactory emf;

        public EntityManagerProvider() {
            emf = Persistence.createEntityManagerFactory("derbypu");
        }

        @Override
        protected void configure() {
            bindFactory(this).to(EntityManager.class);
            System.out.println("EntityManager binding done");
        }

        @Override
        public EntityManager provide() {
            EntityManager em = emf.createEntityManager();
            System.out.println("New EntityManager created");
            return em;
        }

        @Override
        public void dispose(EntityManager em) {
            em.close();
        }
    }

    public class BackgroundTask implements Callable<String> {

        @Inject
        EntityManager em;

        @Override
        public String call() throws Exception {
            System.out.println("Background task started");
            Assert.assertNotNull(em);   // will throw exception

            System.out.println("EntityManager is not null");
            return OK;
        }
    }

    public class ServiceLocatorFeature implements Feature {

        @Override
        public boolean configure(FeatureContext context) {
            ServiceLocator jerseyLocator
                    = org.glassfish.jersey.ServiceLocatorProvider
                            .getServiceLocator(context);

            System.out.println("ServiceLocators are the same: "
                    + (jerseyLocator == serviceLocator));

            return true;
        }
    }

    @Path("/test")
    public static class JerseyResource {

        @Inject
        EntityManager em;

        @GET
        @Produces(MediaType.TEXT_PLAIN)
        public Response doGet() {
            System.out.println("GET request received");
            Assert.assertNotNull(em);

            System.out.println("EntityManager is not null");
            return Response.ok()
                    .entity(OK)
                    .build();
        }
    }

    @Before
    public void setUp() {
        serviceLocator = locatorFactory.create("TestLocator");
        ServiceLocatorUtilities.bind(serviceLocator, new EntityManagerProvider());

        System.out.println("Setting up");
        ResourceConfig config = new ResourceConfig();
        config.register(new ServiceLocatorFeature());
        //config.register(new EntityManagerProvider());
        config.register(JerseyResource.class);
        // can't find a better way to register the resource
        //config.registerInstances(JerseyResource.class);   

        server = GrizzlyHttpServerFactory.createHttpServer(
                URI.create(BASE_URI),
                config, serviceLocator
        );

        backgroundService = Executors.newSingleThreadScheduledExecutor();
    }

    @After
    public void tearDown() {
        System.out.println("Shutting down");
        server.shutdownNow();
        backgroundService.shutdownNow();
    }

    @Test
    public void testScheduledBackgroundTask() throws Exception {
        Assert.assertTrue(server.isStarted());

        BackgroundTask task = new BackgroundTask();
        serviceLocator.inject(task);
        Future<String> f = backgroundService.submit(task);
        System.out.println("Background task submitted");

        try {
            Assert.assertEquals(OK, f.get());   // forces Exception
        } catch (ExecutionException | InterruptedException ex) {
            System.out.println("Caught exception " + ex.getMessage());
            ex.printStackTrace();

            Assert.fail();
        }
    }

    @Test
    public void testBackgroundTask() throws Exception {
        Assert.assertTrue(server.isStarted());

        BackgroundTask task = new BackgroundTask();
        serviceLocator.inject(task);
        System.out.println("Background task instantiated");

        Assert.assertEquals(OK, task.call());
    }

    @Test
    public void testResource() {
        Assert.assertTrue(server.isStarted());

        Client client = ClientBuilder.newClient();
        WebTarget target = client.target(BASE_URI);

        Response r = target.path("test")
                .request()
                .get();
        Assert.assertEquals(200, r.getStatus());
        Assert.assertEquals(OK, r.readEntity(String.class));
    }
}

Another thing I might mention is that you should need only one EntityManagerFactory for the application. It's expensive to create, and creating one every time the EntityManager is needed is not a good idea. See one solution here.

