Please document the IPC handshake mechanisms and open up the classes under `Microsoft.Testing.Platform.IPC`

[bradwilson]

The protocol documentation is a good start, but it leaves a lot of unanswered questions about how runners are intended to actually communicate with tests. Right now that information appears to be locked up inside of Microsoft.Testing.Extensions.VSTestBridge (for which I've been unable to find the official source). Reverse engineering the dotnet test interactions seems to indicate that it uses a command line switch (--internal-msbuild-node) that's expected to let the test know that it should open a named pipe for communication. I have not yet tried to determine how Test Explorer (in VS or in VS Code) works (or doesn't yet work) here.

This leaves me with two problems.

Understanding the expectations on unit test projects

I do not intend to use the VSTest bridge in xUnit.net v3, so I'm left trying to decode the interactions and support them myself. What are the command line switches that I'm expected to recognize and handle? How can I directly create and use an instance of NamedPipeClient? I'm currently hacking in using TestApplication but I'd like to understand how I can do what that does myself, as the API there isn't necessarily a good fit for me.

Understanding the expectations on unit test runners

In the process of supporting meta-runners like xunit.v3.console, which can run multiple test assemblies, I would like to become the server side of the IPC protocol. Given that there is no abstraction equivalent to TestApplication, how can I directly create and use an instance of NamedPipeServer to be the test host?

[MarcoRossignoli]

I do not intend to use the VSTest bridge in xUnit.net v3, so I'm left trying to decode the interactions and support them myself. What are the command line switches that I'm expected to recognize and handle? How can I directly create and use an instance of NamedPipeClient? I'm currently hacking in using TestApplication but I'd like to understand how I can do what that does myself, as the API there isn't necessarily a good fit for me.

You should not use the named pipe directly, at the moment the IPC namespace is used internally for quick communication between processes to support out-of-process extension for instance codecoverage or hang dump, where the platform will restart itself under "observation" by the out out of process extensions.

i.e.

mytest.exe pid 1("test controller" contains codecoverage, hang dump etc...)
         -> mytest.exe pid 2 ("test host" where adapter run tests)

The current internal pipeline infra is also used to have a "structured" communication between the msbuild node and the test application when we run under "dotnet test" using the current msbuild infrastructure and shipped in https://www.nuget.org/packages/Microsoft.Testing.Platform.MSBuild

The Microsoft.Testing.Platform.MSBuild is the testing platform MSBuild support and allows:

• integration with the current dotnet test infrastructure
• autogenerate the entry point using the MSBuild static extensions registration.
• move to the bin and interact with the new configuration file (a json).
• in future it could generate the launcher file used by IDEs(TBD)

This is the reason why the option starts with --internal-, options that start with it are hidden internal.

At the moment we have a "draft" for the json/tcp protocol and the implementation is built-in inside the platform and it's enabled using the --server and @drognanar can help with it.

[bradwilson]

How will the replacement dotnet test know that it's interacting with a test project that wants --server vs. --internal-msbuild-node?

And please don't forget that I am interested in being the test runner as well. I would like xunit.v3.console (and any third party runners written against xunit.v3.runner.utility) to be able to run v3 tests via the same IPC that is used for the new testing platform if possible, since there is negative value (on my personal time 😂) in maintaining two nearly identical IPC mechanisms.

[MarcoRossignoli]

How will the replacement dotnet test know that it's interacting with a test project that wants --server vs. --internal-msbuild-node?

Nuget package https://nuget.info/packages/Microsoft.Testing.Platform.MSBuild ships a MSBuild task/props/targets that integrates with the "current" dotnet test infra and start the testing application passing this "internal" argument for "structured" communication to have "better" UX and avoid overlapping.

I would not take any dependency at the moment on IPC for dotnet test. The protocol at the moment is internal and also we're planning a big change to dotnet test to have a UX that's similar the dotnet vstest but "lighter" that will use most probably pipes and custom protocol.
To be supported there you will have to take dependency on the new core platform because we don't have plan to share an "open protocol" for it for the moment, it's too early; or you have to have custom implementation of the MSBuild infra like we did with the package above.

The only current "draft" for a public protocol we have at the moment is the TCP/json-rpc one used by VS/VS-Code, but it's not complete enough to support all the features we need to have a "perfect" dotnet test experience. We will eventually replace this internal protocol with the public documented one when it(the tcp one) will be "complete".

cc: @drognanar @Evangelink

[bradwilson]

Okay, sounds like I should just wait until things are documented and public classes are available for both sides of the IPC mechanism.

[bradwilson]

Do you have a timeline for when this will be usable?

[MarcoRossignoli]

Do you have a timeline for when this will be usable?

I'll let you know soon cc: @pavelhorak @Evangelink

[MarcoRossignoli]

Do you have a timeline for when this will be usable?

We don't plan for the moment to expose an IPC public protocol if you want you can experiment with the current tcp/json "draft" we have published that is supported in VS/VS-Code in preview for which we don't have yet an ETA to release as stable.

cc: @pavelhorak @Evangelink @drognanar