V11.1.0/support for integration testing#63
Merged
Conversation
Core abstractions for Program.cs-based host integration testing:
- ApplicationHostFactory for creating started IHost instances from entry points
- ApplicationTest{TEntryPoint,T} base class for host testing patterns
- IApplicationFixture{TEntryPoint} for fixture-based lifecycle management
- ManagedApplicationFixture{TEntryPoint} implementing host factory integration
- ApplicationFixtureExtensions for convenient fixture setup methods
- ProgramHostFactoryResolver for resolving host factories from entry points
- DeferredHostBuilder for deferred host configuration
Includes functional test coverage validating hosting abstractions.
ASP.NET Core specific abstractions for Program.cs-based TestServer testing:
- WebApplicationTest{TEntryPoint,T} base class for web application testing
- IWebApplicationFixture{TEntryPoint} for web application lifecycle fixtures
- ManagedWebApplicationFixture{TEntryPoint} implementing web fixture integration
- WebApplicationFixtureExtensions for convenient web fixture setup methods
- WebApplicationHostFactory for creating started web application instances
Includes functional test coverage for ASP.NET Core Minimal Hosting and
classic Startup.cs patterns with TestServer integration.
Eight example applications demonstrating host patterns for testing: Generic Host patterns: - BootstrapperConsole.App with classic Startup.cs pattern - BootstrapperMinimalConsole.App with minimal hosting API - BootstrapperWorker.App with BackgroundService and Startup.cs - BootstrapperMinimalWorker.App minimal worker service pattern ASP.NET Core patterns: - BootstrapperWeb.App with ASP.NET Core and Startup.cs - BootstrapperMinimalWeb.App minimal ASP.NET Core pattern - BootstrapperClassicProgram.App top-level statement example - BootstrapperProgram.App advanced Program.cs customization Reference implementations for integration test validation.
Added new bootstrapper packages for integration test references: - Codebelt.Bootstrapper.Console (5.1.0) - Codebelt.Bootstrapper.Web (5.1.0) - Codebelt.Bootstrapper.Worker (5.1.0) Updated existing packages: - Codebelt.Extensions.BenchmarkDotNet.Console: 1.2.6 → 1.2.7 - Microsoft.NET.Test.Sdk: 18.5.1 → 18.6.0 All target framework versions remain compatible.
Solution structure: - Added /app/ folder containing 8 bootstrapper reference applications - Added Codebelt.Extensions.Xunit.Hosting.FunctionalTests to /test/ folder - Added Codebelt.Extensions.Xunit.Hosting.AspNetCore.FunctionalTests to /test/ folder Project updates: - Updated Codebelt.Extensions.Xunit.Hosting.AspNetCore.Tests.csproj with new test references
Codebelt.Extensions.Xunit.Hosting 11.1.0:
- Added ApplicationHostFactory for IHost creation from entry points
- Added ApplicationTest{TEntryPoint,T} and related fixture abstractions
- Updated version and availability across supported frameworks
- Preserved existing version history with formatting cleanup
Codebelt.Extensions.Xunit.Hosting.AspNetCore 11.1.0:
- Added WebApplicationTest{TEntryPoint,T} and fixture abstractions
- Added web application lifecycle and TestServer integration support
- Updated version for .NET 10 and .NET 9 availability
- Preserved existing version history with formatting cleanup
Release documentation for version 11.1.0 including changelog updates and per-package release notes for Codebelt.Extensions.Xunit and Codebelt.Extensions.Xunit.App packages.
Replaced by BlockingManagedApplicationFixture and BlockingManagedWebApplicationFixture as part of the factory-based testing pattern overhaul. The new approach centralizes test lifecycle management through factory methods (ApplicationTestFactory and WebApplicationTestFactory) rather than inheritance-based fixtures.
Add ApplicationTestFactory static factory method for creating IHostTest instances from Program.cs entry points. Implement BlockingManagedApplicationFixture for non-blocking host fixture lifecycle management. Add internal ApplicationTest base class supporting host integration testing patterns with optional IApplicationFixture implementation override. Modify ApplicationHostFactory to remove automatic host.Start() call, allowing tests to control startup timing and resource initialization through the new factory pattern.
Add WebApplicationTestFactory static factory method for creating IWebApplicationTest instances from Program.cs entry points. Implement BlockingManagedWebApplicationFixture for non-blocking web fixture lifecycle management. Add internal WebApplicationTest base class supporting ASP.NET Core TestServer integration testing patterns with optional IWebApplicationFixture implementation override. These additions provide WebApplicationFactory-like testing capabilities aligned with the Generic Host factory pattern introduced for consistency across the entire .NET application stack.
Remove BootstrapperEntryPointTest.cs from both FunctionalTests projects and ManagedWebApplicationFixtureTest.cs from AspNetCore.FunctionalTests. These tests were specific to now-removed fixture implementations and entry point resolver patterns. New factory-based test coverage through ApplicationTestFactoryTest, WebApplicationTestFactoryTest, and updated bootstrapper-specific test classes provides comprehensive replacement coverage.
Refactor all bootstrapper and integration test classes across FunctionalTests projects to adopt the new ApplicationTestFactory and WebApplicationTestFactory static factory methods. Update test initialization to remove dependency on inheritance-based fixture management and instead use the BlockingManagedApplicationFixture and BlockingManagedWebApplicationFixture implementations provided through factory methods. Modify TestTest.cs in unit tests to reflect updated Test base class signatures. These changes align all integration tests with the new standardized factory-based testing pattern.
Add ApplicationTestFactoryTest to FunctionalTests providing full coverage of ApplicationTestFactory method signatures, fixture overrides, and host lifecycle management. Add WebApplicationTestFactoryTest to AspNetCore.FunctionalTests validating WebApplicationTestFactory behavior and TestServer integration. Add BlockingManagedWebApplicationFixtureTest covering the new blocking web fixture implementation. These tests ensure the new factory pattern works reliably across Generic Host and ASP.NET Core scenarios with all bootstrapper configuration variations.
Regenerate and update the official DocFX namespace pages for Codebelt.Extensions.Xunit.Hosting and Codebelt.Extensions.Xunit.Hosting.AspNetCore namespaces to document the new factory classes (ApplicationTestFactory, WebApplicationTestFactory), base classes (ApplicationTest, WebApplicationTest), and fixture implementations (BlockingManagedApplicationFixture, BlockingManagedWebApplicationFixture). Documentation pages serve as the authoritative source for public API conventions and are updated to reflect the factory-based testing pattern.
Add new 'Official Documentation' section to AGENTS.md establishing that public API conventions belong in .docfx/api/namespaces/ and should be treated as the authoritative documentation source for library behavior and naming vocabulary. Clarify that public APIs should have corresponding updates to the relevant namespace page when introducing or clarifying a convention. Emphasize that internal reasoning, exploratory notes, and agent discussion should be excluded from DocFX pages, which should contain only stable public guidance.
Update CHANGELOG.md [11.1.0] entry to emphasize the release brings WebApplicationFactory-like integration testing patterns to the entire .NET application stack through ApplicationHostFactory and ApplicationTest abstractions for Generic Host, and WebApplicationTest for ASP.NET Core. Reorganize Added section to highlight ApplicationTestFactory and WebApplicationTestFactory along with BlockingManagedApplicationFixture and BlockingManagedWebApplicationFixture. Update per-assembly package release notes for both Hosting and AspNetCore namespaces with version 11.1.0 feature summaries and availability information.
Update .gitignore to exclude build outputs, generated artifacts, and temporary files that are created during the build and test process but should not be committed to version control.
Greptile SummaryThis PR introduces v11.1.0, expanding integration testing support to generic-host and console/worker applications (not just ASP.NET Core) by adding new factory and fixture abstractions, eight reference bootstrapper apps, and functional test suites for both the hosting and ASP.NET Core packages. CI is also extended with an optional macOS test matrix and a
Confidence Score: 4/5Safe to merge after fixing the undisposed host in one functional test; the core abstractions and CI changes are solid. One functional test ( test/Codebelt.Extensions.Xunit.Hosting.AspNetCore.FunctionalTests/WebApplicationTestTest.cs — the Important Files Changed
Sequence DiagramsequenceDiagram
participant Test as ApplicationTest / WebApplicationTest
participant Fixture as BlockingManagedApplicationFixture
participant Factory as ApplicationHostFactory
participant Resolver as ProgramHostFactoryResolver
participant DHB as DeferredHostBuilder
participant EP as EntryPoint thread
Test->>Fixture: ConfigureHost(this)
Fixture->>Factory: "Create<TEntryPoint>(configureHost)"
Factory->>Resolver: ResolveHostBuilderFactory(assembly)
alt CreateHostBuilder method found
Resolver-->>Factory: "Func<string[], IHostBuilder>"
Factory->>Factory: BuildHost(hostBuilder, configureHost)
else Top-level / minimal hosting
Factory->>Resolver: ResolveHostFactory(assembly, stopApp, ...)
Resolver-->>Factory: HostingListener factory
Factory->>DHB: new DeferredHostBuilder()
Factory->>DHB: SetHostFactory(hostFactory)
DHB->>DHB: Build() calls _hostFactory(args)
DHB->>EP: Start background thread (InvokeEntryPoint)
EP-->>DHB: "HostBuilding diagnostic -> ConfigureHostBuilder"
EP-->>DHB: "HostBuilt diagnostic -> _host.TrySetResult"
DHB-->>Factory: IHost (DeferredHost)
Factory->>Factory: BuildHost(deferredHostBuilder, configureHost)
end
Factory-->>Fixture: IHost
Fixture->>Fixture: host.Start()
Fixture-->>Test: Host, Configuration, Environment set
Prompt To Fix All With AIFix the following 2 code review issues. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 2
test/Codebelt.Extensions.Xunit.Hosting.AspNetCore.FunctionalTests/WebApplicationTestTest.cs:80-92
**Undisposed host leaks TestServer and background entry-point thread**
The test creates a `BlockingManagedWebApplicationFixture<ModernProgram>`, calls `ConfigureHost`, and thereby starts a fully running host (TestServer + background thread executing `ModernProgram`'s entry point). Neither `fixture` nor the host it owns is disposed before the method returns. In a parallel test suite the leaked background threads accumulate for the lifetime of the test runner process. The fix is to wrap `fixture` in a `using` block — since `BlockingManagedWebApplicationFixture<TEntryPoint>` inherits from `HostFixture`, which implements `IDisposable`, a `using` statement is sufficient.
### Issue 2 of 2
test/Codebelt.Extensions.Xunit.Hosting.AspNetCore.FunctionalTests/WebApplicationTestTest.cs:81
**`Test_VerifyAbstractions` does not follow the repository's test-method naming convention**
AGENTS.md requires all test method names to use the `Should{ExpectedResult}_When{Condition}` pattern. This name starts with `Test_` and has no `Should` prefix or `When` clause, which is inconsistent with every other test in this file and the broader project.
```suggestion
public void ShouldImplementExpectedAbstractions_WhenClassIsInstantiated()
```
Reviews (3): Last reviewed commit: "✅ Add async disposal test for Applicatio..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+35
to
+39
| public static async Task<HttpResponseMessage> RunAsync<TEntryPoint>(Action<IWebHostBuilder> webHostSetup = null, Func<HttpClient, Task<HttpResponseMessage>> responseFactory = null, IWebApplicationFixture<TEntryPoint> hostFixture = null) where TEntryPoint : class | ||
| { | ||
| using var client = Create<TEntryPoint>(webHostSetup, hostFixture).Host.GetTestClient(); | ||
| return await client.ToHttpResponseMessageAsync(responseFactory).ConfigureAwait(false); | ||
| } |
There was a problem hiding this comment.
IHostTest returned by Create<TEntryPoint>(...) is immediately dereferenced — only .Host.GetTestClient() is captured. The WebApplicationTest, its BlockingManagedWebApplicationFixture, and the underlying TestServer/IHost are never disposed. Every call to RunAsync leaks a running ASP.NET Core host and its background entry-point thread until the process terminates.
Suggested change
| public static async Task<HttpResponseMessage> RunAsync<TEntryPoint>(Action<IWebHostBuilder> webHostSetup = null, Func<HttpClient, Task<HttpResponseMessage>> responseFactory = null, IWebApplicationFixture<TEntryPoint> hostFixture = null) where TEntryPoint : class | |
| { | |
| using var client = Create<TEntryPoint>(webHostSetup, hostFixture).Host.GetTestClient(); | |
| return await client.ToHttpResponseMessageAsync(responseFactory).ConfigureAwait(false); | |
| } | |
| public static async Task<HttpResponseMessage> RunAsync<TEntryPoint>(Action<IWebHostBuilder> webHostSetup = null, Func<HttpClient, Task<HttpResponseMessage>> responseFactory = null, IWebApplicationFixture<TEntryPoint> hostFixture = null) where TEntryPoint : class | |
| { | |
| using var hostTest = Create<TEntryPoint>(webHostSetup, hostFixture); | |
| using var client = hostTest.Host.GetTestClient(); | |
| return await client.ToHttpResponseMessageAsync(responseFactory).ConfigureAwait(false); | |
| } |
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/Codebelt.Extensions.Xunit.Hosting.AspNetCore/WebApplicationTestFactory.cs
Line: 35-39
Comment:
The `IHostTest` returned by `Create<TEntryPoint>(...)` is immediately dereferenced — only `.Host.GetTestClient()` is captured. The `WebApplicationTest`, its `BlockingManagedWebApplicationFixture`, and the underlying `TestServer`/`IHost` are never disposed. Every call to `RunAsync` leaks a running ASP.NET Core host and its background entry-point thread until the process terminates.
```suggestion
public static async Task<HttpResponseMessage> RunAsync<TEntryPoint>(Action<IWebHostBuilder> webHostSetup = null, Func<HttpClient, Task<HttpResponseMessage>> responseFactory = null, IWebApplicationFixture<TEntryPoint> hostFixture = null) where TEntryPoint : class
{
using var hostTest = Create<TEntryPoint>(webHostSetup, hostFixture);
using var client = hostTest.Host.GetTestClient();
return await client.ToHttpResponseMessageAsync(responseFactory).ConfigureAwait(false);
}
```
How can I resolve this? If you propose a fix, please make it concise.Add optional stopApplication parameter to ApplicationHostFactory.Create overload, allowing callers to control whether the entry point is stopped after the host is built. Updates ProgramHostFactoryResolver exception handling and WebApplicationHostFactory parameter passing to support this new configuration option.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #63 +/- ##
==========================================
- Coverage 95.68% 92.45% -3.23%
==========================================
Files 46 82 +36
Lines 950 1498 +548
Branches 115 169 +54
==========================================
+ Hits 909 1385 +476
- Misses 32 99 +67
- Partials 9 14 +5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Simplifies marker classes by removing the unused static state property that was only used for testing purposes.
Verifies that ApplicationTestFactory properly disposes the fixture when the application is disposed asynchronously.
|
Want your agent to iterate on Greptile's feedback? Try greploops. |
Comment on lines
+80
to
+92
| [Fact] | ||
| public void Test_VerifyAbstractions() | ||
| { | ||
| Assert.IsAssignableFrom<IHostTest>(this); | ||
| Assert.IsAssignableFrom<IConfigurationTest>(this); | ||
| Assert.IsAssignableFrom<IEnvironmentTest>(this); | ||
| Assert.IsAssignableFrom<ITest>(this); | ||
| Assert.IsAssignableFrom<IDisposable>(this); | ||
| Assert.IsAssignableFrom<IAsyncDisposable>(this); | ||
| } | ||
|
|
||
| protected override void ConfigureWebHost(IWebHostBuilder builder) | ||
| { |
There was a problem hiding this comment.
The test creates a BlockingManagedWebApplicationFixture<ModernProgram>, calls ConfigureHost, and thereby starts a fully running host (TestServer + background thread executing ModernProgram's entry point). Neither fixture nor the host it owns is disposed before the method returns. In a parallel test suite the leaked background threads accumulate for the lifetime of the test runner process. The fix is to wrap fixture in a using block — since BlockingManagedWebApplicationFixture<TEntryPoint> inherits from HostFixture, which implements IDisposable, a using statement is sufficient.
Prompt To Fix With AI
This is a comment left during a code review.
Path: test/Codebelt.Extensions.Xunit.Hosting.AspNetCore.FunctionalTests/WebApplicationTestTest.cs
Line: 80-92
Comment:
**Undisposed host leaks TestServer and background entry-point thread**
The test creates a `BlockingManagedWebApplicationFixture<ModernProgram>`, calls `ConfigureHost`, and thereby starts a fully running host (TestServer + background thread executing `ModernProgram`'s entry point). Neither `fixture` nor the host it owns is disposed before the method returns. In a parallel test suite the leaked background threads accumulate for the lifetime of the test runner process. The fix is to wrap `fixture` in a `using` block — since `BlockingManagedWebApplicationFixture<TEntryPoint>` inherits from `HostFixture`, which implements `IDisposable`, a `using` statement is sufficient.
How can I resolve this? If you propose a fix, please make it concise.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces version 11.1.0, a significant minor release that expands integration testing support for .NET applications beyond ASP.NET Core, providing new abstractions and patterns for Program.cs-based testing in both generic host and web scenarios. It adds new factory and fixture classes, comprehensive documentation, and a suite of reference applications and functional tests. The solution structure and dependencies are also updated for consistency and modernization.
Key changes:
New Features and Abstractions
ApplicationHostFactory,ApplicationTestFactory,ApplicationTest<TEntryPoint,T>,IApplicationFixture<TEntryPoint>,BlockingManagedApplicationFixture<TEntryPoint>, andApplicationFixtureExtensionsin theCodebelt.Extensions.Xunit.Hostingnamespace, enabling Program.cs-based integration testing for console, worker, and generic host applications. [1] [2] [3]WebApplicationTestFactory,WebApplicationTest<TEntryPoint,T>,IWebApplicationFixture<TEntryPoint>,BlockingManagedWebApplicationFixture<TEntryPoint>, andWebApplicationFixtureExtensionsin theCodebelt.Extensions.Xunit.Hosting.AspNetCorenamespace, supporting TestServer-based integration tests for ASP.NET Core applications. [1] [2] [3]Reference Applications and Functional Tests
/app/solution folder. [1] [2] [3] [4] [5]Documentation and Conventions
.docfx/api/namespaces/to clarify fixture naming conventions and public API patterns for both hosting and ASP.NET Core namespaces. [1] [2] [3]Dependency and Solution Modernization
Microsoft.NET.Test.Sdk. [1] [2] [3] [4] [5] [6] [7] [8]Please review the new abstractions, reference applications, and updated documentation to familiarize yourself with the expanded integration testing capabilities and conventions.