def test(test_config_path):
    'Runs an object detection test configuration\n    \n    This runs an object detection test configuration.  This involves\n    \n    1. Download and build a model architecture (or use cached).\n    2. Optimize the model architecrue\n    3. Benchmark the optimized model against a dataset\n    4. (optional) Run assertions to check the benchmark output\n\n    The input to this function is a JSON file which specifies the test\n    configuration.\n\n    example_test_config.json:\n\n        {\n            "model_config": { ... },\n            "optimization_config": { ... },\n            "benchmark_config": { ... },\n            "assertions": [ ... ]\n        }\n\n    model_config: A dictionary of arguments passed to build_model, which\n        specify the pre-optimized model architure.  The model will be passed\n        to optimize_model.\n    optimization_config: A dictionary of arguments passed to optimize_model.\n        Please see help(optimize_model) for more details.\n    benchmark_config: A dictionary of arguments passed to benchmark_model.\n        Please see help(benchmark_model) for more details.\n    assertions: A list of strings containing python code that will be \n        evaluated.  If the code returns false, an error will be thrown.  These\n        assertions can reference any variables local to this \'test\' function.\n        Some useful values are\n\n            statistics[\'map\']\n            statistics[\'avg_latency\']\n            statistics[\'avg_throughput\']\n\n    Args\n    ----\n        test_config_path: A string corresponding to the test configuration\n            JSON file.\n    '
    with open(args.test_config_path, 'r') as f:
        test_config = json.load(f)
        print(json.dumps(test_config, sort_keys=True, indent=4))
    frozen_graph = build_model(**test_config['model_config'])
    frozen_graph = optimize_model(frozen_graph, **test_config['optimization_config'])
    statistics = benchmark_model(frozen_graph=frozen_graph, **test_config['benchmark_config'])
    print_statistics = statistics
    if ('runtimes_ms' in print_statistics):
        print_statistics.pop('runtimes_ms')
    print(json.dumps(print_statistics, sort_keys=True, indent=4))
    if ('assertions' in test_config):
        for a in test_config['assertions']:
            if (not eval(a)):
                raise AssertionError(('ASSERTION FAILED: %s' % a))
            else:
                print(('ASSERTION PASSED: %s' % a))
Runs an object detection test configuration

This runs an object detection test configuration.  This involves

1. Download and build a model architecture (or use cached).
2. Optimize the model architecrue
3. Benchmark the optimized model against a dataset
4. (optional) Run assertions to check the benchmark output

The input to this function is a JSON file which specifies the test
configuration.

example_test_config.json:

    {
        "model_config": { ... },
        "optimization_config": { ... },
        "benchmark_config": { ... },
        "assertions": [ ... ]
    }

model_config: A dictionary of arguments passed to build_model, which
    specify the pre-optimized model architure.  The model will be passed
    to optimize_model.
optimization_config: A dictionary of arguments passed to optimize_model.
    Please see help(optimize_model) for more details.
benchmark_config: A dictionary of arguments passed to benchmark_model.
    Please see help(benchmark_model) for more details.
assertions: A list of strings containing python code that will be 
    evaluated.  If the code returns false, an error will be thrown.  These
    assertions can reference any variables local to this 'test' function.
    Some useful values are

        statistics['map']
        statistics['avg_latency']
        statistics['avg_throughput']

Args
----
    test_config_path: A string corresponding to the test configuration
        JSON file.
def test(test_config_path):
    'Runs an object detection test configuration\n    \n    This runs an object detection test configuration.  This involves\n    \n    1. Download and build a model architecture (or use cached).\n    2. Optimize the model architecrue\n    3. Benchmark the optimized model against a dataset\n    4. (optional) Run assertions to check the benchmark output\n\n    The input to this function is a JSON file which specifies the test\n    configuration.\n\n    example_test_config.json:\n\n        {\n            "model_config": { ... },\n            "optimization_config": { ... },\n            "benchmark_config": { ... },\n            "assertions": [ ... ]\n        }\n\n    model_config: A dictionary of arguments passed to build_model, which\n        specify the pre-optimized model architure.  The model will be passed\n        to optimize_model.\n    optimization_config: A dictionary of arguments passed to optimize_model.\n        Please see help(optimize_model) for more details.\n    benchmark_config: A dictionary of arguments passed to benchmark_model.\n        Please see help(benchmark_model) for more details.\n    assertions: A list of strings containing python code that will be \n        evaluated.  If the code returns false, an error will be thrown.  These\n        assertions can reference any variables local to this \'test\' function.\n        Some useful values are\n\n            statistics[\'map\']\n            statistics[\'avg_latency\']\n            statistics[\'avg_throughput\']\n\n    Args\n    ----\n        test_config_path: A string corresponding to the test configuration\n            JSON file.\n    '
    with open(args.test_config_path, 'r') as f:
        test_config = json.load(f)
        print(json.dumps(test_config, sort_keys=True, indent=4))
    frozen_graph = build_model(**test_config['model_config'])
    frozen_graph = optimize_model(frozen_graph, **test_config['optimization_config'])
    statistics = benchmark_model(frozen_graph=frozen_graph, **test_config['benchmark_config'])
    print_statistics = statistics
    if ('runtimes_ms' in print_statistics):
        print_statistics.pop('runtimes_ms')
    print(json.dumps(print_statistics, sort_keys=True, indent=4))
    if ('assertions' in test_config):
        for a in test_config['assertions']:
            if (not eval(a)):
                raise AssertionError(('ASSERTION FAILED: %s' % a))
            else:
                print(('ASSERTION PASSED: %s' % a))