Full Text Search with ASP.NET MVC, jQuery autocomplete and Elasticsearch

This article demonstrates how to do a full text search using jQuery Autocomplete with an ASP.NET MVC application and an Elasticsearch search engine. CRUD operations are also implemented for Elasticsearch ( ElasticsearchCRUD ).

To download and install Elasticsearch, use the instructions here.

Code: https://github.com/damienbod/WebSearchWithElasticsearch

Other tutorials:

Part 1: ElasticsearchCRUD introduction
Part 2: MVC application search with simple documents using autocomplete, jQuery and jTable
Part 3: MVC Elasticsearch CRUD with nested documents
Part 4: Data Transfer from MS SQL Server using Entity Framework to Elasticsearch
Part 5: MVC Elasticsearch with child, parent documents
Part 6: MVC application with Entity Framework and Elasticsearch
Part 7: Live Reindex in Elasticsearch
Part 8: CSV export using Elasticsearch and Web API
Part 9: Elasticsearch Parent, Child, Grandchild Documents and Routing
Part 10: Elasticsearch Type mappings with ElasticsearchCRUD
Part 11: Elasticsearch Synonym Analyzer using ElasticsearchCRUD
Part 12: Using Elasticsearch German Analyzer
Part 13: MVC google maps search using Elasticsearch
Part 14: Search Queries and Filters with ElasticsearchCRUD
Part 15: Elasticsearch Bulk Insert
Part 16: Elasticsearch Aggregations With ElasticsearchCRUD
Part 17: Searching Multiple Indices and Types in Elasticsearch
Part 18: MVC searching with Elasticsearch Highlighting
Part 19: Index Warmers with ElasticsearchCRUD

Search for documents in Elasticsearch

The application uses ElasticsearchCRUD to access Elasticsearch. The API can use any DTO or entity class and this is automatically mapped to an Elasticsearch index and type using the class Type. The default index is the type of the class pluralized and all characters are converted to lower case characters. The default type is the type name in lower case characters. This can easily be changed if a different mapping is required for Elasticsearch. For example, classes which are saved as child documents in Elasticsearch need a mapping for its index, ie. use the index where the parent document is stored.

A Skill class is used in this example. This class is saved in the search engine with the index ‘skills’ and the type ‘skill’.

public class Skill
{
  [Required]
  [Range(1, long.MaxValue)]
  public long Id { get; set; }
		
  [Required]
  public string Name { get; set; }
		
  [Required]
  public string Description { get; set; }
		
  public DateTimeOffset Created { get; set; }
		
  public DateTimeOffset Updated { get; set; }
}

The ElasticsearchCRUD Search method is used to do the full text search using QueryString. The method takes the term(s) from the SearchController and does a QueryString search in the engine. This search is case insensitive.

You can do almost any search with context.Search method, not just a query string search. Highlighted results are not supported and multiple index or multiple type searches are not supported. All you need to do is build a JSON query for the search and a collection of types is returned from the hits. The Query is built using the Search class. The Query property defines a Query string search which can use wildcards.

public IEnumerable<Skill> QueryString(string term)
{
	var results = _context.Search<Skill>(BuildQueryStringSearch(term));
        return results.PayloadResult.Hits.HitsResult.Select(t => t.Source);
}

private ElasticsearchCRUD.Model.SearchModel.Search BuildQueryStringSearch(string term)
{
	var names = "";
	if (term != null)
	{
		names = term.Replace("+", " OR *");
	}

	var search = new ElasticsearchCRUD.Model.SearchModel.Search
	{
		Query = new Query(new QueryStringQuery(names + "*"))
	};

	return search;
}

The search method is used in the MVC SearchController. Json is returned because the action method is used by the jQuery Autocomplete control.

public JsonResult Search(string term)
{			
  return Json(_searchProvider.QueryString(term), "Skills", JsonRequestBehavior.AllowGet);
}

The Autocomplete (jquery-ui) source method uses the action method search from the SearchController. This saves the Object array to the autocomplete control. This control requires both a label and a value property in each item for the control. The select method is used to choose a result from the search. The selected item is then added to other html controls on the page for update or delete HTTP requests.

<fieldset class="form">
    <legend>SEARCH for a document in the search engine</legend>
    <table width="500">
        <tr>
            <th></th>
        </tr>
        <tr>
            <td>
                <label for="autocomplete">Search: </label>
            </td>
        </tr>
        <tr>
            <td>
                <input id="autocomplete" type="text" style="width:500px" />
            </td>
        </tr>
    </table>
</fieldset>

@section scripts
{
    <script type="text/javascript">
        var items;
        $(document).ready(function() {
            $("input#autocomplete").autocomplete({
                source: function(request, response) {
                    $.ajax({
                        url: "search",
                        dataType: "json",
                        data: {
                            term: request.term,
                        },
                        success: function(data) {
                            var itemArray = new Array();
                            for (i = 0; i < data.length; i++) {
                                itemArray[i] = { label: data[i].Name, value: data[i].Name, data: data[i] }
                            }
 
                            console.log(itemArray);
                            response(itemArray);
                        },
                        error: function(data, type) {
                            console.log(type);
                        }
                    });
                },
                select: function (event, ui) {
                    $("#spanupdateId").text(ui.item.data.Id);
                    $("#spanupdateCreated").text(new Date(parseInt(ui.item.data.Created.substr(6))));
                    $("#spanupdateUpdated").text(new Date(parseInt(ui.item.data.Updated.substr(6))));

                    $("#updateName").text(ui.item.data.Name);
                    $("#updateDescription").text(ui.item.data.Description);
                    $("#updateName").val(ui.item.data.Name);
                    $("#updateDescription").val(ui.item.data.Description);

                    $("#updateId").val(ui.item.data.Id);
                    $("#updateCreated").val(ui.item.data.Created);
                    $("#updateUpdated").val(ui.item.data.Updated);

                    $("#spandeleteId").text(ui.item.data.Id);
                    $("#deleteId").val(ui.item.data.Id);
                    $("#deleteName").text(ui.item.data.Name);

                    console.log(ui.item);
                }
            });
        });
</script>
}

The control can then be used as follows:
FullTextSearch_01

Elasticsearch Create, Update and Delete

To make it easy to do searches and provide some data, Create, Update and Delete implementations have been added using ElasticsearchCRUD. The Provider uses the context from ElasticsearchCRUD and executes all pending changes in one single bulk request to Elasticsearch. Multiple indexes, types can be executed in one single context, bulk request. The HttpClient class is used under the hub of the context. This works well when adding or changing large amounts of data in Elasticsearch.


private const string ConnectionString = "http://localhost:9200/";
private readonly IElasticsearchMappingResolver _elasticsearchMappingResolver = new ElasticsearchMappingResolver();

public void AddUpdateEntity(Skill skill)
{
   using (var context = new ElasticsearchContext(ConnectionString, _elasticsearchMappingResolver))
   {
       context.AddUpdateDocument(skill, skill.Id);
       context.SaveChanges();
   }
}

public void UpdateSkill(long updateId, string updateName, string updateDescription)
{
  using (var context = new ElasticsearchContext(ConnectionString, _elasticsearchMappingResolver))
  {
      var skill = context.GetDocument<Skill>(updateId);
      skill.Updated = DateTime.UtcNow;
      skill.Name = updateName;
      skill.Description = updateDescription;
      context.AddUpdateDocument(skill, skill.Id);
      context.SaveChanges();
  }
}

public void DeleteSkill(long deleteId)
{
   using (var context = new ElasticsearchContext(ConnectionString, _elasticsearchMappingResolver))
   {
       context.DeleteDocument<Skill>(deleteId);
       context.SaveChanges();
   }
}

This can then be used in the SearchController. Validation has been implemented for the Create skill document in the HTTP Post Index action method. This calls the AddUpdateDocument method of ElasticsearchCRUD. If a document with the same Id already exists in the search engine, the document is overwritten. Usually the entities come from a primary database and are saved the elasticsearch for optimal searches. The primary database manages the Ids of the entity so this is not a problem for this use case.

readonly ISearchProvider _searchProvider = new ElasticSearchProvider();

[HttpGet]
public ActionResult Index()
{
   return View();
}

[HttpPost]
public ActionResult Index(Skill model)
{
  if (ModelState.IsValid)
  {
    model.Created = DateTime.UtcNow;
    model.Updated = DateTime.UtcNow;
    _searchProvider.AddUpdateDocument(model);

    return Redirect("Search/Index");
  }

  return View("Index", model);
}

[HttpPost]
public ActionResult Update(long updateId, string updateName, string updateDescription)
{
   _searchProvider.UpdateSkill(updateId, updateName, updateDescription);
   return Redirect("Index");
}

[HttpPost]
public ActionResult Delete(long deleteId)
{
   _searchProvider.DeleteSkill(deleteId);
   return Redirect("Index");
}

The controller can then be used in the view. The view has 3 different forms: Create, Delete and Update. The Delete and Update forms are updated, when a search result is selected from the Autocomplete control.


<form name="input" action="update" method="post">
    <fieldset class="form">
        <legend>UPDATE an existing document in the search engine</legend>
        <table width="500">
            <tr>
                <th></th>
                <th></th>
            </tr>
            <tr>
                <td>
                    <span>Id:</span>
                </td>
                <td>
                    <span id="spanupdateId">-</span>
                    <input id="updateId" name="updateId" type="hidden" />
                </td>
            </tr>
            <tr>
                <td>
                    <span>Name:</span>
                </td>
                <td>
                    <input id="updateName" name="updateName" type="text" />
                </td>
            </tr>
            <tr>
                <td>
                    <span>Description:</span>
                </td>
                <td>
                    <input id="updateDescription" name="updateDescription" type="text" />
                </td>
            </tr>
            <tr>
                <td>
                    <span>Created:</span>
                </td>
                <td>
                    <span id="spanupdateCreated">-</span>
                    <input id="updateCreated" name="updateCreated" type="hidden" />
                </td>
            </tr>
            <tr>
                <td>
                    <span>Updated:</span>
                </td>
                <td>
                    <span id="spanupdateUpdated">-</span>
                    <input id="updateUpdated" name="updateUpdated" type="hidden" />
                </td>
            </tr>
            <tr>
                <td>
                    <br />
                    <input type="submit" value="Update Skill" style="width: 200px" />
                </td>
                <td></td>
            </tr>
        </table>
    </fieldset>
</form>

<form name="input" action="delete" method="post">
    <fieldset class="form">
        <legend>DELETE an existing document in the search engine</legend>
        <table width="500">
            <tr>
                <th></th>
                <th></th>
            </tr>
            <tr>
                <td>
                    <span id="deleteName">-</span>
                </td>
                <td>
                    <span id="spandeleteId">-</span>
                    <input id="deleteId" name="deleteId" type="hidden" />
                </td>
            </tr>

            <tr>
                <td>
                    <br />
                    <input type="submit" value="Delete Skill" style="width: 200px" />
                </td>
                <td></td>
            </tr>
        </table>
    </fieldset>
</form>

@using (Html.BeginForm("Index", "Search"))
{
    @Html.ValidationSummary(true)

    <fieldset class="form">
        <legend>CREATE a new document in the search engine</legend>
        <table width="500">
            <tr>
                <th></th>
                <th></th>
            </tr>
            <tr>
                <td>
                    @Html.Label("Id:")
                </td>
                <td>
                    @Html.EditorFor(model => model.Id)
                    @Html.ValidationMessageFor(model => model.Id)
                </td>
            </tr>
            <tr>
                <td>
                    @Html.Label("Name:")
                </td>
                <td>
                    @Html.EditorFor(model => model.Name)
                    @Html.ValidationMessageFor(model => model.Name)
                </td>
            </tr>
            <tr>
                <td>
                    @Html.Label("Description:")
                </td>
                <td>
                    @Html.EditorFor(model => model.Description)
                    @Html.ValidationMessageFor(model => model.Description)
                </td>
            </tr>
            <tr>
                <td>
                    <br />
                    <input type="submit" value="Add Skill" style="width:200px" />
                </td>
                <td></td>
            </tr>
        </table>
    </fieldset>
}

The application can then be used in the browser as follows:
FullTextSearch_02

Conclusion

As you can see, it is very easy to do a full text search using ASP.NET MVC with Elasticsearch. This could very easily be changed to use Web API with Angular JS. As these scale well, and have great performance, you could use this for almost any application.

Links:

https://www.nuget.org/packages/ElasticsearchCRUD/

http://www.elasticsearch.org/blog/introducing-elasticsearch-net-nest-1-0-0-beta1/

https://github.com/CenturyLinkCloud/ElasticLINQ

http://www.elasticsearch.org/

https://github.com/elasticsearch/elasticsearch-net

http://nest.azurewebsites.net/

http://jqueryui.com/autocomplete/

http://joelabrahamsson.com/extending-aspnet-mvc-music-store-with-elasticsearch/

http://joelabrahamsson.com/elasticsearch-101/

3 comments

  1. Do you have source code files?

    1. yes

      I always add the src to gitHub

      https://github.com/damienbod/WebSearchWithElasticsearch

      greetings Damien

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: