MVC 6 ASP.NET 5 full text search with Elasticsearch (dotnet .NETCoreApp 1.0 )

This tutorial shows how to create an MVC 6 ASP.NET Core application (dotnet .NETCoreApp 1.0) which does a full text search using Elasticsearch. This application uses jQuery-UI to send the search requests and ElasticsearchCRUD (pre-release version) to send the requests to Elasticsearch. ElasticsearchCRUD supports dnxcore50 and netstandard 1.4. The source code can be found in the ASP.NET Core folder:

Code: https://github.com/damienbod/WebAppAspNet50ElasticsearchCrud
Library: https://github.com/damienbod/ElasticsearchCRUD/tree/master/ElasticsearchCrudAspNet5

Update 2016.07.01: Updated to ASP.NET RTM and Elasticsearch 2.3.3.1
Update 2016.05.20: Updated to ASP.NET RC2 dotnet and Elasticsearch 2.3.1
Update 2015.11.24: Application updated to ASP.NET rc1 and Elasticsearch 2.0.0

Step 1: Create a new ASP.NET Web Application

In Visual Studio 2015, create a new project and select ASP.NET Web .NET Core Application:

Click ASP.NET Web Application. This template is used, so that the NPM and the bower tools don’t have to be setup manually. These are used to add jQuery-UI.

Step 2: Remove the AccountController class, Account views, config.json, clean up the startup class and the other views

It might be quicker to setup everything from scratch, only NPM and bower are required for this tutorial. I find it quicker to use the template and remove what’s not needed. Remove the AccountController, and its views. Rename the HomeController to SearchController and remove the About and Contact razor views. The layout can then be cleaned up. The config.json is also removed as this is not required. Then the Startup.cs can be programmed for our needs:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json.Serialization;
using WebAppAspNet50ElasticsearchCrud.Providers;

namespace WebAppAspNet50ElasticsearchCrud
{
    public class Startup
    {
        public Startup(IHostingEnvironment env)
        {
            var builder = new ConfigurationBuilder()
                .SetBasePath(env.ContentRootPath)
                .AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
                .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true)
                .AddEnvironmentVariables();
            Configuration = builder.Build();
        }

        public IConfigurationRoot Configuration { get; }

        // This method gets called by the runtime. Use this method to add services to the container.
        public void ConfigureServices(IServiceCollection services)
        {
            services.AddMvc().AddJsonOptions(options =>
            {
                options.SerializerSettings.ContractResolver = new DefaultContractResolver();
            });
            services.AddScoped<ISearchProvider, SearchProvider>();
        }

        // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
        public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
        {
            loggerFactory.AddConsole(Configuration.GetSection("Logging"));
            loggerFactory.AddDebug();

            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
                app.UseBrowserLink();
            }
            else
            {
                app.UseExceptionHandler("/Home/Error");
            }

            app.UseStaticFiles();

            app.UseMvc(routes =>
            {
                routes.MapRoute(
                    name: "default",
                    template: "{controller=Search}/{action=Index}/{id?}");
            });
        }
    }
}

Rename the Home folder in the views to Search (to match the Controller). Also remove the Migrations and the Model folders.

Step 3: Add ElasticsearchCRUD to the application

Add “ElasticsearchCRUD”: “2.3.3.1” to the dependencies. You need to activate nuget.org in the package configurations to add this.

WebAppAspNet50ElasticsearchCrud_nuget_03

{
    "dependencies": {
        "Microsoft.NETCore.App": {
            "version": "1.0.0",
            "type": "platform"
        },
        "Microsoft.AspNetCore.Diagnostics": "1.0.0",
        "Microsoft.AspNetCore.Mvc": "1.0.0",
        "Microsoft.AspNetCore.Razor.Tools": {
            "version": "1.0.0-preview2-final",
            "type": "build"
        },
        "Microsoft.AspNetCore.Server.IISIntegration": "1.0.0",
        "Microsoft.AspNetCore.Server.Kestrel": "1.0.0",
        "Microsoft.AspNetCore.StaticFiles": "1.0.0",
        "Microsoft.Extensions.Configuration.EnvironmentVariables": "1.0.0",
        "Microsoft.Extensions.Configuration.Json": "1.0.0",
        "Microsoft.Extensions.Logging": "1.0.0",
        "Microsoft.Extensions.Logging.Console": "1.0.0",
        "Microsoft.Extensions.Logging.Debug": "1.0.0",
        "Microsoft.VisualStudio.Web.BrowserLink.Loader": "14.0.0",
        "ElasticsearchCrud": "2.3.3.1"
    },

  "tools": {
    "Microsoft.AspNetCore.Razor.Tools": {
      "version": "1.0.0-preview2-final",
      "imports": "portable-net45+win8+dnxcore50"
    },
    "Microsoft.AspNetCore.Server.IISIntegration.Tools": {
      "version": "1.0.0-preview2-final",
      "imports": "portable-net45+win8+dnxcore50"
    }
  },

  "frameworks": {
    "netcoreapp1.0": {
      "imports": [
        "dotnet5.6",
        "dnxcore50",
        "portable-net45+win8"
      ]
    }
  },

  "buildOptions": {
    "emitEntryPoint": true,
    "preserveCompilationContext": true
  },

  "runtimeOptions": {
    "gcServer": true
  },

  "publishOptions": {
    "include": [
      "wwwroot",
      "Views",
      "appsettings.json",
      "web.config"
    ]
  },

  "scripts": {
    "prepublish": [ "npm install", "bower install", "gulp clean", "gulp min" ],
    "postpublish": [ "dotnet publish-iis --publish-folder %publish:OutputPath% --framework %publish:FullTargetFramework%" ]
  }
}

Step 4: Add an Elasticsearch Provider and a Skill class for the document index and type

The SearchProvider class is used to access Elasticsearch. This creates, updates, deletes or searches for documents in Elasticsearch. A query_string Query is used for the search. This works well, but if you have a large amount of documents, a different Query can be used.

using System;
using System.Collections.Generic;
using System.Linq;
using ElasticsearchCRUD;
using ElasticsearchCRUD.Model.SearchModel;
using ElasticsearchCRUD.Model.SearchModel.Queries;

namespace WebAppAspNet50ElasticsearchCrud.Providers
{
	public class SearchProvider : ISearchProvider, IDisposable
	{
		public ElasticSearchProvider()
		{
			_context = new ElasticsearchContext(ConnectionString, _elasticSearchMappingResolver);
		}

		private const string ConnectionString = "http://localhost:9200/";
		private readonly IElasticsearchMappingResolver _elasticSearchMappingResolver = new ElasticsearchMappingResolver();
		private readonly ElasticsearchContext _context;

		public IEnumerable<Skill> QueryString(string term)
		{
			var results = _context.Search<Skill>(BuildQueryStringSearch(term));
			 return results.PayloadResult.Hits.HitsResult.Select(t => t.Source);
		}

		/*			
		{
		  "query": {
					"query_string": {
					   "query": "*"

					}
				}
		}
		 */
		private Search BuildQueryStringSearch(string term)
		{
			var names = "";
			if (term != null)
			{
				names = term.Replace("+", " OR *");
			}

			var search = new Search
			{
				Query = new Query(new QueryStringQuery(names + "*"))
			};

			return search;
		}

		public void AddUpdateEntity(Skill skill)
		{
			_context.AddUpdateDocument(skill, skill.Id);
			_context.SaveChanges();
		}

		public void UpdateSkill(long updateId, string updateName, string updateDescription)
		{
			var skill = _context.GetDocument<Skill>(updateId);
			skill.Updated = DateTime.UtcNow;
			skill.Name = updateName;
			skill.Description = updateDescription;
			_context.AddUpdateDocument(skill, skill.Id);
			_context.SaveChanges();
		}

		public void DeleteSkill(long deleteId)
		{
			_context.DeleteDocument<Skill>(deleteId);
			_context.SaveChanges();
		}

		private bool isDisposed;
		public void Dispose()
		{
			if (isDisposed)
			{
				isDisposed = true;
				_context.Dispose();
			}
		}
	}
}

Update the SearchController

using Microsoft.AspNetCore.Mvc;
using System;
using WebAppAspNet50ElasticsearchCrud.Providers;

namespace WebAppAspNet50ElasticsearchCrud.Controllers
{
    public class SearchController : Controller
    {
		readonly ISearchProvider _searchProvider;

		public SearchController(ISearchProvider searchProvider)
		{
			_searchProvider = searchProvider;
		}

		[HttpGet]
		public ActionResult Index()
		{
			return View();
		}

		[HttpPost]
		public ActionResult Index(Skill model)
		{
			if (ModelState.IsValid)
			{
				model.Created = DateTime.UtcNow;
				model.Updated = DateTime.UtcNow;
				_searchProvider.AddUpdateEntity(model);

				return Redirect("Search/Index");
			}

			return View("Index", model);
		}

		[HttpPost]
		public ActionResult Update(long updateId, string updateName, string updateDescription)
		{
			_searchProvider.UpdateSkill(updateId, updateName, updateDescription);
			return Redirect("Index");
		}

		[HttpPost]
		public ActionResult Delete(long deleteId)
		{
			_searchProvider.DeleteSkill(deleteId);
			return Redirect("Index");
		}

		public JsonResult Search(string term)
		{
			return Json(_searchProvider.QueryString(term));
		}

		public IActionResult Error()
        {
            return View("~/Views/Shared/Error.cshtml");
        }
    }
}

Step 5: Add jQuery-UI

In the bower.json file, add jquery-ui to the dependencies. jquery-ui is configured in this example to use the base theme.

{
  "name": "WebAppAspNet50ElasticsearchCrud",
  "private": true,
  "dependencies": {
    "bootstrap": "~3.0.0",
    "jquery-validation": "~1.11.1",
    "jquery-validation-unobtrusive": "~3.2.2",
    "jquery-ui": "~1.11.3",
    "jquery": "~2.1.3"
  },
  "exportsOverride": {
    "bootstrap": {
      "js": "dist/js/*.*",
      "css": "dist/css/*.*",
      "fonts": "dist/fonts/*.*"
    },
    "jquery": {
      "js": "jquery.{js,min.js,min.map}"
    },
    "jquery-validation": {
      "": "jquery.validate.js"
    },
    "jquery-validation-unobtrusive": {
      "": "jquery.validate.unobtrusive.{js,min.js}"
    },
    "jquery-ui": {
      "js": "*.js",
      "css": "themes/base/*.*"
    }
  }
}

Right click the Bower folder and click Restore Packages.

Now jquery-ui is download, but is not included in the wwwroot.

Right click the gruntfile.js and Click Task Runner Explorer.

Double click bower in the Tasks and this will install the bower packages to the wwwroot.

WebAppAspNet50ElasticsearchCrud_bowerInstall_04

jQuery-UI can be added to the _layout razor view

<!DOCTYPE html>
<html>
<head>
	<meta charset="utf-8" />
	<meta name="viewport" content="width=device-width, initial-scale=1.0">
	<title>@ViewBag.Title - ElasticsearchCRUD example</title>

	<link rel="stylesheet" href="~/lib/bootstrap/css/bootstrap.css" />
	<link rel="stylesheet" href="~/css/site.css" />
	<link rel="stylesheet" href="~/lib/jquery-ui/css/autocomplete.css" />
	<link rel="stylesheet" href="~/lib/jquery-ui/css/base.css" />
	<link rel="stylesheet" href="~/lib/jquery-ui/css/all.css" />
	<link rel="stylesheet" href="~/lib/jquery-ui/css/theme.css" />
	<link rel="stylesheet" href="~/lib/jquery-ui/css/core.css" />
	<link rel="stylesheet" href="~/lib/jquery-ui/css/jquery-ui.css" />
	<link rel="stylesheet" href="~/lib/jquery-ui/css/selectable.css" />
	<link rel="stylesheet" href="~/lib/jquery-ui/css/selectmenu.css" />
</head>
<body>
	<div class="container body-content">
		@RenderBody()
	</div>
	<script src="~/lib/jquery/js/jquery.js"></script>
	<script src="~/lib/jquery-ui/js/jquery-ui.js"></script>
	<script src="~/lib/bootstrap/js/bootstrap.js"></script>
	@RenderSection("scripts", required: false)
</body>
</html>

The index view can be updated with the Search controller and scripts.

@model WebAppAspNet50ElasticsearchCrud.Providers.Skill

<fieldset class="form">
    <legend>SEARCH for a document in the search engine</legend>
    <table width="500">
        <tr>
            <th></th>
        </tr>
        <tr>
            <td>
                <label for="autocomplete">Search: </label>
            </td>
        </tr>
        <tr>
            <td>
                <input id="autocomplete" type="text" style="width:500px" />
            </td>
        </tr>
    </table>
</fieldset>

@section scripts
{
    <script type="text/javascript">
        var items;
        $(document).ready(function() {
            $("input#autocomplete").autocomplete({
                source: function(request, response) {
                    $.ajax({
                        url: "search",
                        dataType: "json",
                        data: {
                            term: request.term,
                        },
                        success: function(data) {
                            var itemArray = new Array();
                            for (i = 0; i < data.length; i++) {
                                itemArray[i] = { label: data[i].Name, value: data[i].Name, data: data[i] }
                            }
 
                            console.log(itemArray);
                            response(itemArray);
                        },
                        error: function(data, type) {
                            console.log(type);
                        }
                    });
                },
                select: function (event, ui) {
                    $("#spanupdateId").text(ui.item.data.Id);
                    $("#spanupdateCreated").text(new Date(parseInt(ui.item.data.Created.substr(6))));
                    $("#spanupdateUpdated").text(new Date(parseInt(ui.item.data.Updated.substr(6))));

                    $("#updateName").text(ui.item.data.Name);
                    $("#updateDescription").text(ui.item.data.Description);
                    $("#updateName").val(ui.item.data.Name);
                    $("#updateDescription").val(ui.item.data.Description);

                    $("#updateId").val(ui.item.data.Id);
                    $("#updateCreated").val(ui.item.data.Created);
                    $("#updateUpdated").val(ui.item.data.Updated);

                    $("#spandeleteId").text(ui.item.data.Id);
                    $("#deleteId").val(ui.item.data.Id);
                    $("#deleteName").text(ui.item.data.Name);

                    console.log(ui.item);
                }
            });

        });

    </script>
}


<form name="input" action="update" method="post">
    <fieldset class="form">
        <legend>UPDATE an existing document in the search engine</legend>
        <table width="500">
            <tr>
                <th></th>
                <th></th>
            </tr>
            <tr>
                <td>
                    <span>Id:</span>
                </td>
                <td>
                    <span id="spanupdateId">-</span>
                    <input id="updateId" name="updateId" type="hidden" />
                </td>
            </tr>
            <tr>
                <td>
                    <span>Name:</span>
                </td>
                <td>
                    <input id="updateName" name="updateName" type="text" />
                </td>
            </tr>
            <tr>
                <td>
                    <span>Description:</span>
                </td>
                <td>
                    <input id="updateDescription" name="updateDescription" type="text" />
                </td>
            </tr>
            <tr>
                <td>
                    <span>Created:</span>
                </td>
                <td>
                    <span id="spanupdateCreated">-</span>
                    <input id="updateCreated" name="updateCreated" type="hidden" />
                </td>
            </tr>
            <tr>
                <td>
                    <span>Updated:</span>
                </td>
                <td>
                    <span id="spanupdateUpdated">-</span>
                    <input id="updateUpdated" name="updateUpdated" type="hidden" />
                </td>
            </tr>
            <tr>
                <td>
                    <br />
                    <input type="submit" value="Update Skill" style="width: 200px" />
                </td>
                <td></td>
            </tr>
        </table>
    </fieldset>
</form>

<form name="input" action="delete" method="post">
    <fieldset class="form">
        <legend>DELETE an existing document in the search engine</legend>
        <table width="500">
            <tr>
                <th></th>
                <th></th>
            </tr>
            <tr>
                <td>
                    <span id="deleteName">-</span>
                </td>
                <td>
                    <span id="spandeleteId">-</span>
                    <input id="deleteId" name="deleteId" type="hidden" />
                </td>
            </tr>

            <tr>
                <td>
                    <br />
                    <input type="submit" value="Delete Skill" style="width: 200px" />
                </td>
                <td></td>
            </tr>
        </table>
    </fieldset>
</form>

@using (Html.BeginForm("Index", "Search"))
{
    @Html.ValidationSummary(true)

    <fieldset class="form">
        <legend>CREATE a new document in the search engine</legend>
        <table width="500">
            <tr>
                <th></th>
                <th></th>
            </tr>
            <tr>
                <td>
                    @Html.Label("Id:")
                </td>
                <td>
                    @Html.EditorFor(model => model.Id)
                    @Html.ValidationMessageFor(model => model.Id)
                </td>
            </tr>
            <tr>
                <td>
                    @Html.Label("Name:")
                </td>
                <td>
                    @Html.EditorFor(model => model.Name)
                    @Html.ValidationMessageFor(model => model.Name)
                </td>
            </tr>
            <tr>
                <td>
                    @Html.Label("Description:")
                </td>
                <td>
                    @Html.EditorFor(model => model.Description)
                    @Html.ValidationMessageFor(model => model.Description)
                </td>
            </tr>
            <tr>
                <td>
                    <br />
                    <input type="submit" value="Add Skill" style="width:200px" />
                </td>
                <td></td>
            </tr>
        </table>
    </fieldset>
}

The application is complete and ElasticsearchCRUD can be used in a MVC 6 just like a MVC 5 application.

WebAppAspNet50ElasticsearchCrud_index_04

MVC 6 is still in development. Things like using NPM and bower or grunt will probably be made easier. It is also very difficult to find out which packages are required for the different .NET classes. Thanks to Eilon Lipton who provided a good link to find out which classes are in which package.
http://packagesearch.azurewebsites.net/

Package management has also got more complicated. Now you have NPM as well as NuGet for packages in the default template. MVC 6 is in the early stages, but is looking very promising.

Links:

https://github.com/aspnet

http://packagesearch.azurewebsites.net/

https://damienbod.wordpress.com/2014/10/01/full-text-search-with-asp-net-mvc-jquery-autocomplete-and-elasticsearch/

http://www.ndcvideos.com/#/app/videos?q=ASP.NET

6 comments

  1. […] MVC 6 aspnet50 full text search with Elasticsearch – Damienbod […]

  2. if you can inject data service into controller, your solution looks better.

    1. Hi David

      Sounds like a good idea, I’ll update the app and add it using the default DI

      greetings Damien

    2. DI is now added, thanks for the tip

  3. […] Faire de la recherche avec Elasticsearch et ASP.NET 5. […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: