The Problem
Your .NET application is running in production but you are flying blind. When response times spike, you find out from angry users, not your tooling. Exceptions vanish into log files nobody reads. Database queries silently degrade and you have no idea which endpoint triggered the slowdown.
You have looked at Datadog, New Relic, and Application Insights, but they come with eye-watering per-host pricing, data residency concerns, and vendor lock-in. You need full observability — runtime metrics, exception tracking, HTTP monitoring, database query analysis, and structured log aggregation — without handing your production telemetry to a third-party cloud.
Noundry Sentinel is a self-hosted APM and log aggregation platform built specifically for .NET applications. You own the infrastructure, you own the data, and it integrates natively with the Noundry stack you are already using.
Ingredients
Noundry.Sentinel.Client
vlatest
APM client SDK — collects runtime metrics, HTTP telemetry, exceptions, and database query traces
Noundry.Sentinel.Serilog
vlatest
Serilog sink that ships structured logs to the Sentinel Ingest API
Serilog.AspNetCore
vlatest
Serilog integration for ASP.NET Core request logging
.NET 10 SDK
v10.0
Target framework for the monitored application
Noundry.Sentinel.Ingest.Api
vlatest
Self-hosted data ingestion API that receives telemetry from your apps
Noundry.Sentinel.Dashboard
vlatest
Razor Pages web UI for viewing metrics, traces, logs, and configuring alerts
PostgreSQL
v17
Database backend for Sentinel tenant data storage
Docker Desktop
vlatest
Container runtime for hosting the Sentinel infrastructure
Gather these packages and tools before we begin. The Client SDK and Serilog sink go into your application. The Ingest API and Dashboard run as separate services in your infrastructure.
Sentinel uses a multi-tenant architecture — each application gets its own isolated database, so metrics from your billing service never mix with your public API.
Preparation
dotnet add package Noundry.Sentinel.Client
dotnet add package Noundry.Sentinel.Serilog
dotnet add package Serilog.AspNetCore
2. Deploy the Sentinel infrastructure
The Ingest API and Dashboard are shipped as Docker images. A minimal docker-compose.yml gets them running alongside PostgreSQL:
version: '3.8'
services:
sentinel-db:
image: postgres:17-alpine
environment:
POSTGRES_USER: sentinel
POSTGRES_PASSWORD: s3nt1n3l_pr0d!
POSTGRES_DB: sentinel_master
ports:
- "5433:5432"
volumes:
- sentinel_data:/var/lib/postgresql/data
sentinel-ingest:
image: noundry/sentinel-ingest:latest
environment:
ConnectionStrings__MasterDb: "Host=sentinel-db;Database=sentinel_master;Username=sentinel;Password=s3nt1n3l_pr0d!"
Sentinel__IngestApiKey: "your-ingest-api-key-here"
ports:
- "5100:8080"
depends_on:
- sentinel-db
sentinel-dashboard:
image: noundry/sentinel-dashboard:latest
environment:
ConnectionStrings__MasterDb: "Host=sentinel-db;Database=sentinel_master;Username=sentinel;Password=s3nt1n3l_pr0d!"
ports:
- "5200:8080"
depends_on:
- sentinel-db
volumes:
sentinel_data:
docker compose up -d
3. Create a tenant in the Dashboard
Browse to http://localhost:5200, create an account, and register a new tenant for your application. The Dashboard will generate:
- Tenant Host — the Ingest API base URL your app sends data to
- API Key — a per-tenant secret that authenticates your telemetry
Copy both values. You will need them in the next step.
4. Configure your application
Add the Sentinel configuration block to your appsettings.json. This is where the magic starts.
The Code — Configuration
Start by adding the Sentinel configuration to your application settings. Every option has a sensible default, but you will want to fine-tune what gets collected based on your environment.
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning"
}
},
"ConnectionStrings": {
"DefaultConnection": "Host=localhost;Database=myapp;Username=postgres;Password=secret"
},
"Sentinel": {
"Enabled": true,
"TenantHost": "http://localhost:5100",
"ApiKey": "your-tenant-api-key-here",
"AppName": "MyApp.Api",
"Environment": "Production",
"Metrics": {
"Enabled": true,
"IntervalSeconds": 15,
"CollectGcMetrics": true,
"CollectThreadMetrics": true,
"CollectProcessMetrics": true
},
"Http": {
"Enabled": true,
"CaptureRequestHeaders": [ "Content-Type", "Authorization", "X-Request-Id" ],
"CaptureResponseHeaders": [ "X-Request-Id", "X-Correlation-Id" ],
"IgnorePaths": [ "/health", "/favicon.ico", "/_framework" ],
"SlowRequestThresholdMs": 500
},
"Exceptions": {
"Enabled": true,
"CaptureHttpContext": true,
"CaptureUser": true,
"MaxStackTraceLength": 8000
},
"Database": {
"Enabled": true,
"SlowQueryThresholdMs": 200,
"CaptureQueryParameters": false
},
"Logging": {
"Enabled": true,
"MinimumLevel": "Information",
"BatchSize": 50,
"FlushIntervalSeconds": 5
}
}
}The Code — Program.cs Integration
Wire everything together in Program.cs. The Sentinel client registers background services that periodically flush metrics to the Ingest API. The middleware captures HTTP request/response pairs and exceptions automatically. Serilog ties structured logging into the same pipeline.
using Serilog;
using Noundry.Sentinel.Client;
using Noundry.Sentinel.Serilog;
var builder = WebApplication.CreateBuilder(args);
// -------------------------------------------------------
// 1. Configure Serilog with the Sentinel sink
// -------------------------------------------------------
var sentinelConfig = builder.Configuration.GetSection("Sentinel");
Log.Logger = new LoggerConfiguration()
.ReadFrom.Configuration(builder.Configuration)
.Enrich.FromLogContext()
.Enrich.WithMachineName()
.Enrich.WithEnvironmentName()
.WriteTo.Console(
outputTemplate: "[{Timestamp:HH:mm:ss} {Level:u3}] {Message:lj}{NewLine}{Exception}")
.WriteTo.Sentinel(
tenantHost: sentinelConfig["TenantHost"]!,
apiKey: sentinelConfig["ApiKey"]!,
appName: sentinelConfig["AppName"] ?? "MyApp",
environment: sentinelConfig["Environment"] ?? "Production",
batchSize: int.Parse(sentinelConfig["Logging:BatchSize"] ?? "50"),
flushInterval: TimeSpan.FromSeconds(
int.Parse(sentinelConfig["Logging:FlushIntervalSeconds"] ?? "5")),
restrictedToMinimumLevel: Serilog.Events.LogEventLevel.Information)
.CreateLogger();
builder.Host.UseSerilog();
// -------------------------------------------------------
// 2. Register Sentinel services
// -------------------------------------------------------
builder.Services.AddNoundrySentinel(options =>
{
options.TenantHost = sentinelConfig["TenantHost"]!;
options.ApiKey = sentinelConfig["ApiKey"]!;
options.AppName = sentinelConfig["AppName"] ?? "MyApp";
options.Environment = sentinelConfig["Environment"] ?? "Production";
// Runtime metrics: CPU, memory, GC, threads, handles
options.Metrics.Enabled = true;
options.Metrics.IntervalSeconds = 15;
options.Metrics.CollectGcMetrics = true;
options.Metrics.CollectThreadMetrics = true;
options.Metrics.CollectProcessMetrics = true;
// HTTP monitoring: response times, status codes
options.Http.Enabled = true;
options.Http.SlowRequestThresholdMs = 500;
options.Http.IgnorePaths.AddRange(new[] { "/health", "/favicon.ico" });
// Exception tracking: full stack traces with HTTP context
options.Exceptions.Enabled = true;
options.Exceptions.CaptureHttpContext = true;
options.Exceptions.CaptureUser = true;
// Database query tracking via Tuxedo ORM
options.Database.Enabled = true;
options.Database.SlowQueryThresholdMs = 200;
});
// Register your application services
builder.Services.AddRazorPages();
builder.Services.AddEndpointsApiExplorer();
var app = builder.Build();
// -------------------------------------------------------
// 3. Add Sentinel middleware (order matters!)
// -------------------------------------------------------
app.UseSerilogRequestLogging();
app.UseSentinelRequestTracking(); // Captures HTTP metrics
app.UseSentinelExceptionTracking(); // Captures unhandled exceptions
app.UseStaticFiles();
app.UseRouting();
app.UseAuthentication();
app.UseAuthorization();
app.MapRazorPages();
// -------------------------------------------------------
// 4. Graceful shutdown — flush remaining telemetry
// -------------------------------------------------------
app.Lifetime.ApplicationStopping.Register(() =>
{
Log.Information("Application shutting down — flushing Sentinel telemetry");
Log.CloseAndFlush();
});
app.Run();The Code — Exception Tracking in Action
Sentinel captures full exception context automatically via the middleware, but you can also enrich exceptions with custom metadata. Here is a service that demonstrates both unhandled exception capture and explicit error reporting with business context.
When an exception flows through UseSentinelExceptionTracking(), it captures the full stack trace, the HTTP request that triggered it (method, path, headers, query string, authenticated user), and any Serilog log context properties that were active at the time.
using System.Data;
using Noundry.Tuxedo;
using Noundry.Tuxedo.Contrib;
using Noundry.Sentinel.Client;
using Serilog;
using Serilog.Context;
namespace MyApp.Services;
public class OrderService
{
private readonly IDbConnection _db;
private readonly ISentinelClient _sentinel;
private readonly ILogger _logger = Log.ForContext<OrderService>();
public OrderService(IDbConnection db, ISentinelClient sentinel)
{
_db = db;
_sentinel = sentinel;
}
public async Task<OrderResult> ProcessOrderAsync(Guid orderId, Guid customerId)
{
// Push properties onto the Serilog context — these travel
// with every log entry AND with any captured exceptions
using (LogContext.PushProperty("OrderId", orderId))
using (LogContext.PushProperty("CustomerId", customerId))
{
_logger.Information("Processing order {OrderId} for customer {CustomerId}",
orderId, customerId);
var order = await _db.QueryFirstOrDefaultAsync<Order>(
"SELECT * FROM orders WHERE id = @Id AND customer_id = @CustomerId",
new { Id = orderId, CustomerId = customerId });
if (order is null)
{
_logger.Warning("Order {OrderId} not found for customer {CustomerId}",
orderId, customerId);
return new OrderResult { Success = false, Error = "Order not found" };
}
try
{
// Business logic that might fail
await ValidateInventoryAsync(order);
await ChargePaymentAsync(order);
await _db.ExecuteAsync(
"UPDATE orders SET status = @Status, processed_at = @ProcessedAt WHERE id = @Id",
new { Status = "completed", ProcessedAt = DateTime.UtcNow, Id = orderId });
_logger.Information("Order {OrderId} processed successfully — total: {Total:C}",
orderId, order.Total);
return new OrderResult { Success = true, OrderId = orderId };
}
catch (PaymentDeclinedException ex)
{
// Report to Sentinel with custom business metadata.
// This creates an exception record with all the context:
// stack trace, HTTP request, log properties, plus your tags.
_sentinel.CaptureException(ex, tags: new Dictionary<string, string>
{
["order_id"] = orderId.ToString(),
["customer_id"] = customerId.ToString(),
["payment_provider"] = ex.Provider,
["decline_reason"] = ex.Reason
});
_logger.Error(ex, "Payment declined for order {OrderId}: {Reason}",
orderId, ex.Reason);
return new OrderResult
{
Success = false,
Error = $"Payment declined: {ex.Reason}"
};
}
}
}
private async Task ValidateInventoryAsync(Order order)
{
var outOfStock = await _db.QueryAsync<OrderItem>(
@"SELECT oi.* FROM order_items oi
JOIN products p ON p.id = oi.product_id
WHERE oi.order_id = @OrderId AND p.stock_count < oi.quantity",
new { OrderId = order.Id });
if (outOfStock.Any())
throw new InvalidOperationException(
$"{outOfStock.Count()} item(s) are out of stock");
}
private Task ChargePaymentAsync(Order order)
{
// Payment gateway integration
return Task.CompletedTask;
}
}
public class Order
{
public Guid Id { get; set; }
public Guid CustomerId { get; set; }
public decimal Total { get; set; }
public string Status { get; set; } = string.Empty;
}
public class OrderItem
{
public Guid Id { get; set; }
public Guid OrderId { get; set; }
public Guid ProductId { get; set; }
public int Quantity { get; set; }
}
public class OrderResult
{
public bool Success { get; set; }
public Guid OrderId { get; set; }
public string? Error { get; set; }
}
public class PaymentDeclinedException : Exception
{
public string Provider { get; }
public string Reason { get; }
public PaymentDeclinedException(string provider, string reason)
: base($"Payment declined by {provider}: {reason}")
{
Provider = provider;
Reason = reason;
}
}The Code — Health Check and Diagnostics Endpoint
Expose a diagnostics endpoint that queries the Sentinel client for real-time application health. This is useful for load balancers, Kubernetes readiness probes, or your own status pages. The ISentinelClient provides snapshot access to the latest metrics the background collector has gathered.
using Microsoft.AspNetCore.Mvc;
using Noundry.Sentinel.Client;
namespace MyApp.Endpoints;
public static class DiagnosticsEndpoints
{
public static void MapDiagnostics(this WebApplication app)
{
app.MapGet("/api/diagnostics", async ([FromServices] ISentinelClient sentinel) =>
{
var snapshot = await sentinel.GetCurrentMetricsAsync();
return Results.Ok(new
{
status = "healthy",
timestamp = DateTime.UtcNow,
app = new
{
name = snapshot.AppName,
environment = snapshot.Environment,
uptime = snapshot.Uptime.ToString(@"dd\.hh\:mm\:ss")
},
runtime = new
{
dotnetVersion = snapshot.DotNetVersion,
cpuPercent = Math.Round(snapshot.CpuUsagePercent, 2),
memoryMb = Math.Round(snapshot.WorkingSetMb, 1),
gcGen0 = snapshot.GcGen0Collections,
gcGen1 = snapshot.GcGen1Collections,
gcGen2 = snapshot.GcGen2Collections,
gcHeapMb = Math.Round(snapshot.GcHeapSizeMb, 1),
threadPoolWorkers = snapshot.ThreadPoolWorkerThreads,
threadPoolIo = snapshot.ThreadPoolIoThreads,
activeHandles = snapshot.ActiveHandleCount
},
http = new
{
totalRequests = snapshot.TotalHttpRequests,
avgResponseMs = Math.Round(snapshot.AvgResponseTimeMs, 1),
p95ResponseMs = Math.Round(snapshot.P95ResponseTimeMs, 1),
p99ResponseMs = Math.Round(snapshot.P99ResponseTimeMs, 1),
errorRate = Math.Round(snapshot.HttpErrorRatePercent, 2),
slowRequests = snapshot.SlowRequestCount
},
database = new
{
totalQueries = snapshot.TotalDatabaseQueries,
avgQueryMs = Math.Round(snapshot.AvgQueryTimeMs, 1),
slowQueries = snapshot.SlowQueryCount
},
exceptions = new
{
totalCaptured = snapshot.TotalExceptions,
last24Hours = snapshot.ExceptionsLast24Hours,
topException = snapshot.MostFrequentExceptionType
}
});
})
.WithName("GetDiagnostics")
.WithTags("Diagnostics")
.AllowAnonymous();
app.MapGet("/api/diagnostics/ready", async ([FromServices] ISentinelClient sentinel) =>
{
var isConnected = await sentinel.PingIngestApiAsync();
return isConnected
? Results.Ok(new { status = "ready", sentinel = "connected" })
: Results.Json(
new { status = "degraded", sentinel = "disconnected" },
statusCode: 503);
})
.WithName("ReadinessCheck")
.WithTags("Diagnostics")
.AllowAnonymous();
}
}The Code — Configuring Alert Rules
Sentinel supports alerting via email (powered by Noundry Sanquhar) and webhooks. You can define alert rules either through the Dashboard UI or programmatically at startup. Here is how to configure alerts in code so they are version-controlled alongside your application.
Alert rules evaluate conditions against the metrics stream. When a condition is met, Sentinel fires the configured notification channels and applies a cooldown period to prevent alert storms.
using Noundry.Sentinel.Client;
using Noundry.Sentinel.Client.Alerting;
namespace MyApp.Configuration;
public static class SentinelAlerts
{
public static void ConfigureAlertRules(this ISentinelClient sentinel)
{
// --------------------------------------------------
// Alert 1: High error rate
// Fires when more than 5% of HTTP responses are 5xx
// over a 5-minute sliding window
// --------------------------------------------------
sentinel.AddAlertRule(new AlertRule
{
Name = "High Error Rate",
Description = "HTTP 5xx error rate exceeded 5% over 5 minutes",
Metric = SentinelMetric.HttpErrorRatePercent,
Condition = AlertCondition.GreaterThan,
Threshold = 5.0,
WindowMinutes = 5,
CooldownMinutes = 15,
Severity = AlertSeverity.Critical,
Channels =
{
new EmailChannel
{
Recipients = { "[email protected]", "[email protected]" },
Subject = "[CRITICAL] {AppName} - High Error Rate: {Value}%"
},
new WebhookChannel
{
Url = "https://hooks.slack.com/services/T00/B00/xxxxx",
Method = HttpMethod.Post,
PayloadTemplate = "{\"text\": \"*{AppName}* error rate is {Value}% (threshold: {Threshold}%)\", \"channel\": \"#alerts\"}"
}
}
});
// --------------------------------------------------
// Alert 2: Slow response times
// Fires when P95 response time exceeds 2 seconds
// --------------------------------------------------
sentinel.AddAlertRule(new AlertRule
{
Name = "Slow P95 Response Time",
Description = "P95 response time exceeded 2000ms over 10 minutes",
Metric = SentinelMetric.P95ResponseTimeMs,
Condition = AlertCondition.GreaterThan,
Threshold = 2000,
WindowMinutes = 10,
CooldownMinutes = 30,
Severity = AlertSeverity.Warning,
Channels =
{
new EmailChannel
{
Recipients = { "[email protected]" },
Subject = "[WARNING] {AppName} - P95 Response Time: {Value}ms"
}
}
});
// --------------------------------------------------
// Alert 3: Memory pressure
// Fires when working set exceeds 1.5 GB
// --------------------------------------------------
sentinel.AddAlertRule(new AlertRule
{
Name = "High Memory Usage",
Description = "Working set memory exceeded 1.5 GB",
Metric = SentinelMetric.WorkingSetMb,
Condition = AlertCondition.GreaterThan,
Threshold = 1536,
WindowMinutes = 5,
CooldownMinutes = 60,
Severity = AlertSeverity.Warning,
Channels =
{
new WebhookChannel
{
Url = "https://hooks.slack.com/services/T00/B00/xxxxx",
Method = HttpMethod.Post,
PayloadTemplate = "{\"text\": \"*{AppName}* memory usage is {Value}MB (threshold: {Threshold}MB)\", \"channel\": \"#alerts\"}"
}
}
});
// --------------------------------------------------
// Alert 4: Slow database queries
// Fires when average query time exceeds 500ms
// --------------------------------------------------
sentinel.AddAlertRule(new AlertRule
{
Name = "Slow Database Queries",
Description = "Average database query time exceeded 500ms over 5 minutes",
Metric = SentinelMetric.AvgQueryTimeMs,
Condition = AlertCondition.GreaterThan,
Threshold = 500,
WindowMinutes = 5,
CooldownMinutes = 20,
Severity = AlertSeverity.Warning,
Channels =
{
new EmailChannel
{
Recipients = { "[email protected]", "[email protected]" },
Subject = "[WARNING] {AppName} - Slow DB Queries: avg {Value}ms"
}
}
});
// --------------------------------------------------
// Alert 5: Exception spike
// Fires when more than 20 exceptions in 10 minutes
// --------------------------------------------------
sentinel.AddAlertRule(new AlertRule
{
Name = "Exception Spike",
Description = "More than 20 exceptions captured in 10 minutes",
Metric = SentinelMetric.ExceptionCount,
Condition = AlertCondition.GreaterThan,
Threshold = 20,
WindowMinutes = 10,
CooldownMinutes = 30,
Severity = AlertSeverity.Critical,
Channels =
{
new EmailChannel
{
Recipients = { "[email protected]" },
Subject = "[CRITICAL] {AppName} - Exception Spike: {Value} in {Window}min"
},
new WebhookChannel
{
Url = "https://hooks.slack.com/services/T00/B00/xxxxx",
Method = HttpMethod.Post,
PayloadTemplate = "{\"text\": \"*{AppName}* exception spike: {Value} exceptions in {Window} minutes\", \"channel\": \"#alerts\"}"
}
}
});
}
}// After app.Build(), register alert rules:
var sentinel = app.Services.GetRequiredService<ISentinelClient>();
sentinel.ConfigureAlertRules();
// Full startup sequence with alerts:
// var app = builder.Build();
// app.UseSerilogRequestLogging();
// app.UseSentinelRequestTracking();
// app.UseSentinelExceptionTracking();
// app.MapDiagnostics(); // from DiagnosticsEndpoints.cs
//
// var sentinel = app.Services.GetRequiredService<ISentinelClient>();
// sentinel.ConfigureAlertRules();
//
// app.Run();Plating
Your .NET application now has full-stack observability — entirely self-hosted, with zero cloud dependencies:
- Runtime Metrics — CPU, memory, GC generations, thread pool, and handle counts streamed every 15 seconds to the Sentinel Dashboard
- HTTP Monitoring — Every request tracked with response time, status code, and slow-request flagging at your configured threshold
- Exception Tracking — Full stack traces with HTTP context, authenticated user info, and Serilog log context properties captured automatically
- Database Query Analysis — SQL execution times monitored through Tuxedo ORM integration, with slow query alerts
- Structured Log Aggregation — Serilog logs shipped to Sentinel in batches, searchable by level, message template, and custom properties
- Alerting — Email via Sanquhar and webhooks to Slack, Teams, PagerDuty, or any HTTP endpoint, with sliding window evaluation and cooldown periods
Key takeaways:
- Sentinel adds less than 0.1% CPU overhead to your application — the client batches telemetry and flushes on a background thread
- The multi-tenant architecture means each application gets an isolated database, making Sentinel ideal for SaaS platforms monitoring many services
- Alert rules defined in code are version-controlled and deploy with your application — no manual Dashboard configuration to drift out of sync
- The diagnostics endpoint gives you a machine-readable health snapshot perfect for Kubernetes probes or load balancer health checks
Next steps:
- Explore the Sentinel Dashboard to view real-time metrics charts, exception drilldowns, and log search
- Set up Grafana integration by pointing it at the Sentinel PostgreSQL tenant database for custom dashboards
- Configure log retention policies in the Dashboard to automatically purge logs older than 30/60/90 days
- Add Sentinel to your CI/CD pipeline to track deployment markers and correlate releases with metric changes
Was this recipe helpful?
Chef George
.NET Architect & Head Chef
25+ years crafting enterprise C# solutions. Creator of the Noundry platform. Passionate about clean architecture, cloud-agnostic deployments, and code that cooks.