Secrets and hosting

As a responsible developer, you are using gitleaks and you're using SonarCube on your commits and PRs to make sure you don't leak secrets into github.

Both of them are objecting to your connection strings and other secrets in appsettings.config. How do you fix this?

User Secrets (or maybe not...)

You'll first discover UserSecrets, so let's look at it, and find out why it's not what you're looking for.

user-secrets init

In folder with the .csproj file (not .slnx or anywhere else), open a command line and run:


dotnet user-secrets init

If it errors, you just ignored the comment about the command line being in the folder with the csproj file.

user-secrets init does two things:

adds a <UserSecretsId>guid...</UserSecretsId> to the .csproj file
creates a folder and file %APPDATA%\Microsoft\UserSecrets\guid...\secrets.json

user-secrets set

Add secrets in the command line (note colon delimiter for nested values):


dotnet user-secrets set "ConnectionStrings:DefaultConnection" "Server=.;Database=MI6;Trusted_Connection=True;"
dotnet user-secrets set "MySettings:Name" "James Bond"

Tne file %APPDATA%\Microsoft\UserSecrets\guid...\secrets.json will contain the following:


{
  "ConnectionStrings:DefaultConnection": "Server=.;Database=MI6;Trusted_Connection=True;",
  "MySettings:Name": "James Bond"
}

Using UserSecrets

Our scenario is consoles and integration tests, not asp hosts.

Add the Microsoft.Extensions.Configuration.UserSecrets NuGet package
Add the Microsoft.Extensions.Configuration.Json NuGet package

Then use ConfigurationBuilder to load the json and override it with UserSecrets.


using Microsoft.Extensions.Configuration;

var config = new ConfigurationBuilder()
    .SetBasePath(Directory.GetCurrentDirectory())
    .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
    .AddUserSecrets<Program>()   // enables user-secrets
    .Build();
var conn = config.GetConnectionString("DefaultConnection");
//or via Microsoft.Extensions.Configuration.Binder
var mySettings = config.GetSection("MySettings").Get<MySettings>();

Share the secrets

As a developer, you normally work in a team, and your colleagues need to run and debug the code. So you need to share the secrets (but not in the repository, where others can see your secrets).

UserSecrets is designed for single developers. If they run user-secrets init, they get a different guid in the csproj. You can copy the %APPDATA%\Microsoft\UserSecrets\guid folder, and every team member copies it to their local %APPDATA%.

If you use Host.CreateApplicationBuilder(), it will load UserSecrets automatically ... if there is a UserSecretsId in the csproj AND it is in development mode (app.Environment.IsDevelopment()).

There is a slightly simpler way...

Environmental Variables

Environmental variables are also specific to your machine, and will not get into the repository.
There is no UserSecretsId in the csproj
Instead of AddUserSecrets() you AddEnvironmentVariables().
Again, Host.CreateApplicationBuilder() does it automatically, but not just in debug.

You have to share the command line among the team, but it's just the command line, not a secrets.json that has to be coied to a specific location.


setx ConnectionStrings__DefaultConnection "Server=.;Database=MI6;Trusted_Connection=True;"
setx MySettings__Name "James Bond"

setx (set persistent environment variable) uses __ (two underscores) for nested values, not : (colon) like user-secrets.

Console with hosting

Here's a full example for a console (which could also be used for an integration test).
In addition to configuration, it includes logging (NLog) and Dependency Injection (Autofac)

The Runner here is IAsyncDisposable, hence the "await using".


using Autofac;
using Autofac.Extensions.DependencyInjection;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using NLog.Extensions.Logging;

// configure NLog from nlog.config
NLog.LogManager.LoadConfiguration("nlog.config");

var builder = Host.CreateApplicationBuilder();
//get the configuration from the builder
var configSettings = builder.Configuration.GetRequiredSection("MySettings")
    .Get<MySettings>()!;
//connection string also available, we'll put it manually into the MySettings
configSettings.ConnectionString = builder.Configuration.GetConnectionString("DefaultConnection");

// logging using NLog
builder.Logging.ClearProviders();
builder.Logging.SetMinimumLevel(LogLevel.Information);
builder.Logging.AddNLog();

//Dependency injection with autofac
builder.ConfigureContainer(
    new AutofacServiceProviderFactory(),
    container =>
    {
        // register the configured MySettings instance so it is available to Runner via DI
        container.RegisterInstance(configSettings).AsSelf().SingleInstance();
        container.RegisterModule(new RegistryModule());
    });

var app = builder.Build();

//just resolve the service, with configuration injected
await using var runner = app.Services.GetRequiredService<IRunner>();
runner.Run();

In theory the Host.CreateApplicationBuilder() should also do UserSecrets in development mode, but you might have to be explicit. Another reason just to do Environmental Variables, which work reliably.


var builder = Host.CreateApplicationBuilder();
// ensure user secrets are included (optional: true so missing secrets will not error)
builder.Configuration.AddUserSecrets(optional: true);

In asp, you use WebApplication.CreateBuilder(args), configure it, var app = builder.Build() and then app.Run()

You can do this in consoles too, but you need to implement an IHostedService is a singleton with StartAsync and StopAsync methods.
I prefer to just get the service fully loaded with DI and logging, as shown above.


builder.Services.AddHostedService<Runner>();
var app = builder.Build();
await app.RunAsync();

Deployment to servers

Azure Devops Release Pipelines have pipeline variables where you can add your secrets, including connection strings. Link them with scopes per target (eg for tst connection string, prod connection string),

I prefer to use Pipelines/Library/Variable Groups with a group per scope (eg TST variables, PROD variables), which can be linked to multiple release pipelines. You can also like Variable Groups to Azure Key Vault (there's a toggle for this).

static void