Running and Filtering Tests with the .NET Core Command Line Test Runner

01 September 2017 (0)

In a previous article we saw how to use Visual Studio’s Test Explorer to filter and run subsets of automated tests.

If we’re working with .NET Core, in addition to using Test Explorer (or other 3rd party runners such as ReSharper) we can also execute tests at the command line.

Assuming we have two test projects (as follows), the tests contained within these test projects can be executed with the dotnet test command.

ClassLibrary1.sln
│
├───XUnitTestProject1
│   │   TestClass1.cs
│   │   TestClass2.cs
│   └───XUnitTestProject1.csproj
│
└───XUnitTestProject2
    │   TestClass3.cs
    │   TestClass4.cs
    │   XUnitTestProject2.csproj
    │
    └───SomeNameSpace
            TestClass5.cs

Assuming were at the  command/PowerShell prompt, and we’re at the root directory (where the .sln is), we can run the following command to list (but not run) all the tests that are discovered:

dotnet test --list-tests

This produces the following:

Build started, please wait...
Build started, please wait...
Build completed.

Test run for C:\root\XUnitTestProject1\bin\Debug\netcoreapp2.0\XUnitTestProject1.dll(.NETCoreApp,Version=v2.0)
Microsoft (R) Test Execution Command Line Tool Version 15.3.0-preview-20170628-02
Copyright (c) Microsoft Corporation.  All rights reserved.

The following Tests are available:
Build completed.

Test run for C:\root\XUnitTestProject2\bin\Debug\netcoreapp2.0\XUnitTestProject2.dll(.NETCoreApp,Version=v2.0)
Microsoft (R) Test Execution Command Line Tool Version 15.3.0-preview-20170628-02
Copyright (c) Microsoft Corporation.  All rights reserved.

The following Tests are available:
[xUnit.net 00:00:00.3003911]   Discovering: XUnitTestProject1
[xUnit.net 00:00:00.3646372]   Discovered:  XUnitTestProject1
    XUnitTestProject1.TestClass1.Test1
    XUnitTestProject1.TestClass1.Test2
    XUnitTestProject1.TestClass1.Test3
    XUnitTestProject1.TestClass1.Test4
    XUnitTestProject1.TestClass2.Test1
    XUnitTestProject1.TestClass2.Test2
    XUnitTestProject1.TestClass2.Test3
    XUnitTestProject1.TestClass2.Test4
[xUnit.net 00:00:00.2826272]   Discovering: XUnitTestProject2
    XUnitTestProject2.TestClass3.Test1
    XUnitTestProject2.TestClass3.Test2
    XUnitTestProject2.TestClass3.Test3
    XUnitTestProject2.TestClass3.Test4
    XUnitTestProject2.TestClass4.Test1
    XUnitTestProject2.TestClass4.Test2
    XUnitTestProject2.TestClass4.Test3
    XUnitTestProject2.TestClass4.Test4
    XUnitTestProject2.SomeNameSpace.TestClass5.Test1
    XUnitTestProject2.SomeNameSpace.TestClass5.Test2
[xUnit.net 00:00:00.3863338]   Discovered:  XUnitTestProject2
    XUnitTestProject2.SomeNameSpace.TestClass5.Test3
    XUnitTestProject2.SomeNameSpace.TestClass5.Test4

In the preceding output we can see that dotnet test has built the two test projects and then discovered the test classes and test methods within.

To actually run the test we can simply call dotnet test with no additional arguments; this will rebuild the projects and then execute all the tests.

Filtering Tests

If we don’t want all the tests executed we can limit them, for example just running the tests from one of the test projects:

dotnet test XUnitTestProject1

The ––filter option can be used to fine tune which tests are executed.

For example to run the single test Test1 in TestClass1:

dotnet test --filter DisplayName=XUnitTestProject1.TestClass1.Test1

To run all the tests in a single test class, the ~ “contains” operator can be used, for example:

dotnet test --filter DisplayName~XUnitTestProject1.TestClass1

To run all tests with a specific category (trait), for example all the “Smoke Tests” (in xUnit.net this would be the attribute [Trait("Category", "Smoke Test")]):

dotnet test --filter Category="Smoke Test"

The ! “not” operator can be used as part of the filter expression, for example to run all tests except for Test1:

dotnet test --filter FullyQualifiedName!=XUnitTestProject1.TestClass1.Test1

For more info and examples, check out the docs or my xUnit.net course from Pluralsight - You can start watching with a Pluralsight free trial.

SHARE:

Add comment

Loading