In my test environment we are running the same set of integration (and sometimes unit) tests against different systems that have different capabilities.Some of the tests cannot be executed against some of the systems because a feature that enables this scenario is missing or we need completely different behavior.Currently we are using MSTest and have tests with this pattern:[TestMethod]public void TestMethod1(){    if(targetSystem.SupportsFeatureX == false)        Assert.Inconclusive(Not supported by underlying system);    // the code for the actual test follows}Should we mark the test as Inconclusive or just let it Pass if the feature check is not satisfied?Please note that targetSystem is a third party system that we have no control over so we just live with features that are present.
How to mark unit/integration tests when they do not apply for the current test configuration - Inconclusive or Passed?
programming practices;testing
If your test framework supports it, you should mark tests that can't be executed due to missing (external) dependencies as skipped. That gives the clearest picture in the overview.If the test framework does not support a 'skipped' status, then 'inconclusive' is a next-best option. At least, 'inconclusive' won't give as much of a misleading result.Using 'pass' for tests that were not actually executed can give a misleading impression regarding the stability of a feature. Just imagine the questions you would get if the statistics show that the tests for Feature X consistently pass on systems A, B and C, but regularly fail on systems Y and Z. Is there a problem specific to systems Y and Z or do systems A, B and C not have feature X and is there a generic problem in the feature?