This post shows how to use Elasticsearch in .NET Aspire. Elasticsearch is setup to use HTTPS with the dotnet developer certificates and and simple client can be implemented to query the data.
Code: https://github.com/damienbod/keycloak-backchannel
Setup
Two services are setup to run in .NET Aspire. The first service is the official Elasticsearch docker container and deployed using dotnet developer certificates. The second service is an ASP.NET Core application using the Elastic.Clients.Elasticsearch Nuget package. The App.Host project is used to set this up and to link the services together.
Elasticsearch development server
The Elasticsearch container is configured in the program class of the App.Host project. The container is run using HTTPS and takes the Aspire parameters for configuration of the default account.
var elasticsearch = builder.AddElasticsearch("elasticsearch",
password: passwordElastic)
.WithDataVolume()
.RunElasticWithHttpsDevCertificate(port: 9200);
The developer certificates needs to be created and copied to the specific folder inside the Elasticsearch docker container. This is implemented using a shared folder and the Elasticsearch xpack.security.http.ssl properties are set to match. The following three properties are used:
- xpack.security.http.ssl.enabled
- xpack.security.http.ssl.certificate
- xpack.security.http.ssl.key
using System.Diagnostics;
using System.IO.Hashing;
using System.Text;
namespace Aspire.Hosting;
// original src: https://github.com/dotnet/aspire-samples/tree/damianedwards/keycloak-sample/samples/Keycloak
public static class HostingElasticExtensions
{
public static IResourceBuilder<ElasticsearchResource> RunElasticWithHttpsDevCertificate(this IResourceBuilder<ElasticsearchResource> builder, int port = 9200, int targetPort = 9200)
{
if (builder.ApplicationBuilder.ExecutionContext.IsRunMode)
{
builder
.RunElasticWithHttpsDevCertificate()
.WithHttpsEndpoint(port: port, targetPort: targetPort)
.WithEnvironment("QUARKUS_HTTP_HTTP2", "false");
}
return builder;
}
public static IResourceBuilder<TResource> RunElasticWithHttpsDevCertificate<TResource>(this IResourceBuilder<TResource> builder)
where TResource : IResourceWithEnvironment
{
const string DEV_CERT_DIR = "/usr/share/elasticsearch/config/certificates";
if (builder.ApplicationBuilder.ExecutionContext.IsRunMode)
{
// Export the ASP.NET Core HTTPS development certificate & private key to PEM files, bind mount them into the container
// and configure it to use them via the specified environment variables.
var (certPath, _) = ExportElasticDevCertificate(builder.ApplicationBuilder);
var bindSource = Path.GetDirectoryName(certPath) ?? throw new UnreachableException();
if (builder.Resource is ContainerResource containerResource)
{
builder.ApplicationBuilder.CreateResourceBuilder(containerResource)
.WithBindMount(bindSource, DEV_CERT_DIR, isReadOnly: false);
}
builder
.WithEnvironment("xpack.security.http.ssl.enabled", "true")
.WithEnvironment("xpack.security.http.ssl.certificate", $"{DEV_CERT_DIR}/dev-cert.pem")
.WithEnvironment("xpack.security.http.ssl.key", $"{DEV_CERT_DIR}/dev-cert.key");
}
return builder;
}
private static (string, string) ExportElasticDevCertificate(IDistributedApplicationBuilder builder)
{
var appNameHashBytes = XxHash64.Hash(Encoding.Unicode.GetBytes(builder.Environment.ApplicationName).AsSpan());
var appNameHash = BitConverter.ToString(appNameHashBytes).Replace("-", "").ToLowerInvariant();
var tempDir = Path.Combine(Path.GetTempPath(), $"aspire.{appNameHash}");
var certExportPath = Path.Combine(tempDir, "dev-cert.pem");
var certKeyExportPath = Path.Combine(tempDir, "dev-cert.key");
if (File.Exists(certExportPath) && File.Exists(certKeyExportPath))
{
// Certificate already exported, return the path.
return (certExportPath, certKeyExportPath);
}
else if (Directory.Exists(tempDir))
{
Directory.Delete(tempDir, recursive: true);
}
Directory.CreateDirectory(tempDir);
var exportProcess = Process.Start("dotnet", $"dev-certs https --export-path \"{certExportPath}\" --format Pem --no-password");
var exited = exportProcess.WaitForExit(TimeSpan.FromSeconds(5));
if (exited && File.Exists(certExportPath) && File.Exists(certKeyExportPath))
{
return (certExportPath, certKeyExportPath);
}
else if (exportProcess.HasExited && exportProcess.ExitCode != 0)
{
throw new InvalidOperationException($"HTTPS dev certificate export failed with exit code {exportProcess.ExitCode}");
}
else if (!exportProcess.HasExited)
{
exportProcess.Kill(true);
throw new InvalidOperationException("HTTPS dev certificate export timed out");
}
throw new InvalidOperationException("HTTPS dev certificate export failed for an unknown reason");
}
}
When the App.Host project is started, the Elasticsearch containers boot up and the server can be tested using the “_cat” HTTP Get requests or the default base URL will give a server information about Elasticsearch.
Elasticsearch client
The Elasticsearch client was implemented using the Elastic.Clients.Elasticsearch Nuget package. The client project in .NET Aspire needs to reference the Elasticsearch server using the WithReference method.
builder.AddProject<Projects.ElasticsearchAuditTrail>(
"elasticsearchaudittrail")
.WithExternalHttpEndpoints()
.WithReference(elasticsearch);
Elasticsearch can be queried used a simple query search.
public async Task<IEnumerable<T>> QueryAuditLogs(string filter = "*",
AuditTrailPaging auditTrailPaging = null)
{
var from = 0;
var size = 10;
EnsureElasticClient(_indexName, _options.Value);
await EnsureAlias();
if (auditTrailPaging != null)
{
from = auditTrailPaging.Skip;
size = auditTrailPaging.Size;
if (size > 1000)
{
// max limit 1000 items
size = 1000;
}
}
var searchRequest = new SearchRequest<T>(Indices.Parse(_alias))
{
Size = size,
From = from,
Query = new SimpleQueryStringQuery
{
Query = filter
},
Sort = BuildSort()
};
var searchResponse = await _elasticsearchClient
.SearchAsync<T>(searchRequest);
return searchResponse.Documents;
}
See the source code: https://github.com/damienbod/keycloak-backchannel/blob/main/AuditTrail/AuditTrailProvider.cs
Notes
With this setup, it is easy to develop using Elasticsearch as a container and no service needs to be implemented on the developer host PC. Setting up HTTPS is a little bit complicated and it would be nice to see this supported better. The development environment should be as close as possible to the deployed versions. HTTPS should be used in development.
Links
https://learn.microsoft.com/en-us/dotnet/aspire/search/elasticsearch-integration
https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html
https://www.elastic.co/products/elasticsearch
Software Engineering 
[…] Using Elasticsearch with .NET Aspire (Damien Bowden) […]
very well explained thank you for the detailed answer,
[…] Using Elasticsearch with .NET Aspire […]