ASP.NET Core logging with NLog and Elasticsearch

This article shows how to Log to Elasticsearch using NLog in an ASP.NET Core application. NLog is a free open-source logging for .NET.

Code: https://github.com/damienbod/AspNetCoreNlog

2018-04-04 Updated to NLog.Web.AspNetCore 4.5.1, ASP.NET Core 2.0
2017.02.08 Updated to NLog.Web.AspNetCore 4.3.0 and VS2017 RC3
17.12.2016 Updated to ASP.NET Core 1.1

NLog posts in this series:

  1. ASP.NET Core logging with NLog and Microsoft SQL Server
  2. ASP.NET Core logging with NLog and Elasticsearch
  3. Settings the NLog database connection string in the ASP.NET Core appsettings.json
  4. .NET Core logging to MySQL using NLog
  5. .NET Core logging with NLog and PostgreSQL

NLog.Extensions.Logging is required to use NLog in an ASP.NET Core application. This is added to the dependencies of the project. NLog.Targets.ElasticSearch is also added to the dependencies. This project is at present NOT the NuGet package from ReactiveMarkets, but the source code from ReactiveMarkets and updated to dotnetcore. Thanks to ReactiveMarkets for this library, hopefully the NuGet package will be updated and the NuGet package can be used directly.

The NLog configuration file also needs to be added to the publishOptions in the csproj file.

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>netcoreapp2.0</TargetFramework>
    <PreserveCompilationContext>true</PreserveCompilationContext>
    <AssemblyName>AspNetCoreNlog</AssemblyName>
    <OutputType>Exe</OutputType>
    <PackageId>AspNetCoreNlog</PackageId>
    <PackageTargetFallback>$(PackageTargetFallback);dotnet5.6;portable-net45+win8</PackageTargetFallback>
  </PropertyGroup>
  <ItemGroup>
    <Content Update="wwwroot\**\*;Views;Areas\**\Views;appsettings.json;nlog.config;web.config">
      <CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory>
    </Content>
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.All" Version="2.0.6" />
    <PackageReference Include="NLog.Targets.ElasticSearch" Version="5.0.0" />
    <PackageReference Include="NLog.Web.AspNetCore" Version="4.5.1" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Tools" Version="2.0.2" PrivateAssets="All" />
  </ItemGroup>
  <ItemGroup>
    <DotNetCliToolReference Include="Microsoft.EntityFrameworkCore.Tools.DotNet" Version="2.0.0" />
    <DotNetCliToolReference Include="Microsoft.Extensions.SecretManager.Tools" Version="2.0.0" />
    <DotNetCliToolReference Include="Microsoft.VisualStudio.Web.CodeGeneration.Tools" Version="2.0.0" />
  </ItemGroup>
  <ItemGroup>
    <Content Update="nlog.config">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </Content>
  </ItemGroup>
</Project>

The NLog configuration is added to the Startup.cs class in the Configure method.

public void ConfigureServices(IServiceCollection services)
{
	services.AddSingleton<IHttpContextAccessor, HttpContextAccessor>();
	// Add framework services.
	services.AddMvc();

	services.AddScoped<LogFilter>();
}

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
	GlobalDiagnosticsContext.Set("configDir", "C:\\git\\damienbod\\AspNetCoreNlog\\Logs");
	GlobalDiagnosticsContext.Set("connectionString", "Data Source=N051\\MSSQLSERVER2014;Initial Catalog=Nlogs;Integrated Security=True;");

	loggerFactory.AddNLog();

	app.UseMvc();
}

The nlog.config target and rules can be configured to log to Elasticsearch. NLog.Targets.ElasticSearch is an extension and needs to be added using the extensions tag.

<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      autoReload="true"
      internalLogLevel="Warn"
      internalLogFile="C:\git\damienbod\AspNetCoreNlog\Logs\internal-nlog.txt">
    
    <extensions>
        <add assembly="NLog.Targets.ElasticSearch"/>
    </extensions>
            
  <targets>

    <target name="ElasticSearch" xsi:type="BufferingWrapper" flushTimeout="5000">
      <target xsi:type="ElasticSearch"/>
    </target>
   
  </targets>

  <rules>
    <logger name="*" minlevel="Trace" writeTo="ElasticSearch" />
      
  </rules>
</nlog>

The NLog.Targets.ElasticSearch package Elasticsearch URL can be configured using the ElasticsearchUrl property. This can be defined in the appsettings configuration file.

{
    "Logging": {
        "IncludeScopes": false,
        "LogLevel": {
            "Default": "Debug",
            "System": "Information",
            "Microsoft": "Information"
        }
    },
    "ElasticsearchUrl": "http://localhost:9200"
}

NLog.Targets.ElasticSearch ( ReactiveMarkets )

When the application is started the logs are written to Elasticsearch. These logs can be viewed in Elasticsearch

http://localhost:9200/logstash-‘date’/_search

{
	"took": 2,
	"timed_out": false,
	"_shards": {
		"total": 5,
		"successful": 5,
		"failed": 0
	},
	"hits": {
		"total": 18,
		"max_score": 1.0,
		"hits": [{
			"_index": "logstash-2016.08.19",
			"_type": "logevent",
			"_id": "AVaiJHPycDWw4BKmTWqP",
			"_score": 1.0,
			"_source": {
				"@timestamp": "2016-08-19T09:31:44.5790894Z",
				"level": "Debug",
				"message": "2016-08-19 11:31:44.5790|DEBUG|Microsoft.AspNetCore.Hosting.Internal.WebHost|Hosting starting"
			}
		},
		{
			"_index": "logstash-2016.08.19",
			"_type": "logevent",
			"_id": "AVaiJHPycDWw4BKmTWqU",
			"_score": 1.0,
			"_source": {
				"@timestamp": "2016-08-19T09:31:45.4788003Z",
				"level": "Info",
				"message": "2016-08-19 11:31:45.4788|INFO|Microsoft.AspNetCore.Hosting.Internal.WebHost|Request starting HTTP/1.1 DEBUG http://localhost:55423/  0"
			}
		},
		{
			"_index": "logstash-2016.08.19",
			"_type": "logevent",
			"_id": "AVaiJHPycDWw4BKmTWqW",
			"_score": 1.0,
			"_source": {
				"@timestamp": "2016-08-19T09:31:45.6248512Z",
				"level": "Debug",
				"message": "2016-08-19 11:31:45.6248|DEBUG|Microsoft.AspNetCore.Server.Kestrel|Connection id \"0HKU82EHFC0S9\" completed keep alive response."
			}
		},

Links

https://github.com/NLog/NLog.Extensions.Logging

https://github.com/ReactiveMarkets/NLog.Targets.ElasticSearch

https://github.com/NLog

https://docs.asp.net/en/latest/fundamentals/logging.html

https://msdn.microsoft.com/en-us/magazine/mt694089.aspx

https://github.com/nlog/NLog/wiki/Database-target

https://www.elastic.co/products/elasticsearch

https://github.com/elastic/logstash

https://github.com/elastic/elasticsearch-net

https://www.nuget.org/packages/Elasticsearch.Net/

https://github.com/nlog/NLog/wiki/File-target#size-based-file-archival

http://www.danesparza.net/2014/06/things-your-dad-never-told-you-about-nlog/

Advertisements

10 comments

  1. […] Asp.net Core Logging With Nlog And Elasticsearch | Software Engineering […]

  2. […] Produire les logs avec .NET Core et Elasticsearch. […]

  3. ASP.Net core 1.0 support Log4Net ?

  4. […] ASP.NET Core logging with NLog and Elasticsearch (Damien Bod) […]

  5. Sending logs over http seems poor to me. Is there a plan to log via UDP.

  6. Please, please, please log to file system and use filebeat (or any alternative) to send the logs to elasticsearch. More information about ELK stack:
    https://github.com/dzharii/awesome-elasticsearch

    1. Hi Dmytro, thanks for the comment. I never used filebeat, I’ll have a look a this. Thanks for the link

      Greetings Damien

  7. […] Asp.Net Core Logging with Nlog and Elasticsearch by Damien Bowden […]

  8. Thank you for the post, that’s cool, Can you also share the source code as an attachment?

  9. noumanbhatti · · Reply

    Reblogged this on Help Drive.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s

%d bloggers like this: