Web API OData V4 Batching Part 10

This article demonstrates how batching can be used with a Web API OData V4 service and an OData C# client. This blog is part 10 of the Web API OData series. Batching can be used to optimize network usage. Batching makes it possible to send many HTTP requests as a single multiple mixed request. The client/server packs/unpacks the different batch requests.

Note: Part 8 of this series explains how to setup an OData C# client.

Part 1 Getting started with Web API and OData V4 Part 1.
Part 2 Web API and OData V4 Queries, Functions and Attribute Routing Part 2
Part 3 Web API and OData V4 CRUD and Actions Part 3
Part 4 Web API OData V4 Using enum with Functions and Entities Part 4
Part 5 Web API OData V4 Using Unity IoC, SQLite with EF6 and OData Model Aliasing Part 5
Part 6 Web API OData V4 Using Contained Models Part 6
Part 7 Web API OData V4 Using a Singleton Part 7
Part 8 Web API OData V4 Using an OData T4 generated client Part 8
Part 9 Web API OData V4 Caching Part 9
Part 10 Web API OData V4 Batching Part 10
Part 11 Web API OData V4 Keys, Composite Keys and Functions Part 11

Code: https://github.com/damienbod/WebAPIODataV4SQLite

Web API 2 OData V4 Batching Server

To implement an OData batch handler in Web API, an ODataBatchHandler handler needs to be added. Message quotas can be configured as required for the batch handler. The batch handler underneath can be accessed using /odata/$batch

ODataBatchHandler odataBatchHandler = 
      new DefaultODataBatchHandler(GlobalConfiguration.DefaultServer);
odataBatchHandler.MessageQuotas.MaxOperationsPerChangeset = 10;
odataBatchHandler.MessageQuotas.MaxPartsPerBatch = 10;

config.MapODataServiceRoute(
  "odata", 
  "odata", 
  model: GetModel(), 
  batchHandler: odataBatchHandler
);

Now that the batch handler is configured, you should define a ContainerName in the model. This will be used in the OData client.

public static IEdmModel GetModel()
{
 ODataModelBuilder builder = new ODataConventionModelBuilder();
 builder.Namespace = "damienbod";
 builder.ContainerName = "SqliteContext";
 builder.EntitySet<AnimalType>("AnimalType");
 builder.EntitySet<EventData>("EventData");

 builder.EntitySet<Player>("Player");
 builder.EntityType<PlayerStats>();

 SingletonConfiguration<SkillLevels> skillLevels = 
          builder.Singleton<SkillLevels>("SkillLevels");
 builder.EntityType<SkillLevel>();

 builder.EntitySet<AdmDto>("Adms");
 return builder.GetEdmModel();
}

If using the OData batch handler in Web API, you need to configure this in the GlobalConfiguration in the Application_Start method. It will not work per default inside the OWIN middlerware due to routing problems!

protected void Application_Start()
{
 // This is required because the OData batchHandler
 // does not work when hosted in a OWIN container
 GlobalConfiguration.Configure(WebApiConfig.Register);

Web API 2 OData V4 Batching Client

This is very easy to setup. This works like a normal OData client. The client DTOs and client code is auto-generated from the OData Web API service, using T4 templates. Only the SaveChanges method is different to a normal client. The following client selects data from the server. (not batching but it could be!). Then some data is updated and a new entity is created. This is sent to the server as a batch HTTP multiple mixed post request. The batch is sent with as a single change request. If one fails, all requests fails.

var context = new SqliteContext(new Uri("http://localhost.fiddler:59145/odata/"));
context.Format.UseJson();

// Call some basic Get
var eventDataItems = context.EventData.ToList();
var animalsItems = context.AnimalType.ToList();
var skillLevels = context.SkillLevels.Expand("Levels").GetValue();
var players = context.Player.Expand(c => c.PlayerStats)
    .Where(u => u.PlayerStats.SkillLevel == 2).ToList();

// Create a new entity
var newObjectEventData = new EventData
{
  AnimalTypeId = animalsItems.First().Key,
  Factor = 56,
  FixChange = 13.0,
  StringTestId = "testdatafromodataclient",
  AnimalType = animalsItems.First()
};

// Update a new entity
var dataToUpdate = eventDataItems.FirstOrDefault();
dataToUpdate.Factor = 99;
dataToUpdate.FixChange = 97;

context.AddToEventData(newObjectEventData);
//context.UpdateObject(dataToUpdate);
context.AddAndUpdateResponsePreference 
      = DataServiceResponsePreference.IncludeContent;

// Add the data to the server
DataServiceResponse response 
      = context.SaveChanges(SaveChangesOptions.BatchWithSingleChangeset);

The batch could also be sent so each request is executed independently.

// Add the data to the server
DataServiceResponse response
      = context.SaveChanges(SaveChangesOptions.BatchWithIndependentOperations);

Debugging with Fiddler

Fiddler can be used to view the HTTP request sent from the C# client. This can be very helpful when debugging a HTTP client or a server. Instead of sending the request direct to the IIS, you can send it to fiddler which will send it onto the IIS server.

For example:
http://localhost.fiddler:59145/odata/$batch in the application will send the request to fiddler which will send the request to http://localhost:59145/odata/$batch

var context = new SqliteContext(new Uri("http://localhost.fiddler:59145/odata/"));

Http batch Request:

POST http://localhost:59145/odata/$batch HTTP/1.1
OData-Version: 4.0;NetFx
OData-MaxVersion: 4.0;NetFx
Content-Type: multipart/mixed; boundary=batch_ebdc0b88-eeb1-4dd6-b170-74331f39bd03
Accept: multipart/mixed
Accept-Charset: UTF-8
User-Agent: Microsoft ADO.NET Data Services
Host: localhost:59145
Content-Length: 1663
Expect: 100-continue

--batch_ebdc0b88-eeb1-4dd6-b170-74331f39bd03
Content-Type: multipart/mixed; boundary=changeset_54ac09ec-f437-4b08-9925-fd42ed7bd58f

--changeset_54ac09ec-f437-4b08-9925-fd42ed7bd58f
Content-Type: application/http
Content-Transfer-Encoding: binary
Content-ID: 43

POST http://localhost.fiddler:59145/odata/EventData HTTP/1.1
OData-Version: 4.0;NetFx
OData-MaxVersion: 4.0;NetFx
Content-Type: application/json;odata.metadata=minimal
Prefer: return=representation
Accept: application/json;odata.metadata=minimal
Accept-Charset: UTF-8
User-Agent: Microsoft ADO.NET Data Services

{"@odata.type":"#WebAPIODataV4SQLite.DomainModel.EventData","AnimalTypeId":0,"EventDataId":0,"Factor":56,"FixChange":13.0,"StringTestId":"testdatafromodataclient"}
--changeset_54ac09ec-f437-4b08-9925-fd42ed7bd58f--
--batch_ebdc0b88-eeb1-4dd6-b170-74331f39bd03
Content-Type: multipart/mixed; boundary=changeset_2346da5e-88c9-4aa5-a837-5db7e1368147

--changeset_2346da5e-88c9-4aa5-a837-5db7e1368147
Content-Type: application/http
Content-Transfer-Encoding: binary
Content-ID: 44

PATCH http://localhost:59145/odata/EventData(1) HTTP/1.1
OData-Version: 4.0;NetFx
OData-MaxVersion: 4.0;NetFx
Content-Type: application/json;odata.metadata=minimal
Prefer: return=representation
Accept: application/json;odata.metadata=minimal
Accept-Charset: UTF-8
User-Agent: Microsoft ADO.NET Data Services

{"@odata.type":"#WebAPIODataV4SQLite.DomainModel.EventData","AnimalTypeId":1,"EventDataId":1,"Factor":99,"FixChange":97.0,"StringTestId":"PressureOnThePigMarket"}
--changeset_2346da5e-88c9-4aa5-a837-5db7e1368147--
--batch_ebdc0b88-eeb1-4dd6-b170-74331f39bd03--

Http batch Response

HTTP/1.1 200 OK
Cache-Control: no-cache
Pragma: no-cache
Content-Type: multipart/mixed; boundary=batchresponse_6f21f127-229a-44b0-ace8-99ec488084e8
Expires: -1
Server: Microsoft-IIS/8.0
OData-Version: 4.0
X-AspNet-Version: 4.0.30319
X-SourceFiles: =?UTF-8?B?QzpcZ2l0XGRhbWllbmJvZFxXZWJBUElPRGF0YVY0U1FMaXRlXFdlYkFQSU9EYXRhVjRTUUxpdGVcb2RhdGFcJGJhdGNo?=
X-Powered-By: ASP.NET
Date: Wed, 13 Aug 2014 14:25:32 GMT
Content-Length: 1437

--batchresponse_6f21f127-229a-44b0-ace8-99ec488084e8
Content-Type: multipart/mixed; boundary=changesetresponse_47587855-a09b-4afb-a926-a2c6c2913bf9

--changesetresponse_47587855-a09b-4afb-a926-a2c6c2913bf9
Content-Type: application/http
Content-Transfer-Encoding: binary
Content-ID: 43

HTTP/1.1 201 Created
Location: http://localhost.fiddler:59145/odata/EventData(46)
Content-Type: application/json; odata.metadata=minimal; charset=utf-8
OData-Version: 4.0

{
  "@odata.context":"http://localhost.fiddler:59145/odata/$metadata#EventData/$entity","EventDataId":46,"Factor":56,"StringTestId":"testdatafromodataclient","FixChange":13.0,"AnimalTypeId":0
}
--changesetresponse_47587855-a09b-4afb-a926-a2c6c2913bf9--
--batchresponse_6f21f127-229a-44b0-ace8-99ec488084e8
Content-Type: multipart/mixed; boundary=changesetresponse_8da9a570-ca14-4793-8173-d743e15e8e38

--changesetresponse_8da9a570-ca14-4793-8173-d743e15e8e38
Content-Type: application/http
Content-Transfer-Encoding: binary
Content-ID: 44

HTTP/1.1 200 OK
Content-Type: application/json; odata.metadata=minimal; charset=utf-8
OData-Version: 4.0

{
  "@odata.context":"http://localhost:59145/odata/$metadata#EventData/$entity","EventDataId":1,"Factor":99,"StringTestId":"PressureOnThePigMarket","FixChange":97.0,"AnimalTypeId":1
}
--changesetresponse_8da9a570-ca14-4793-8173-d743e15e8e38--
--batchresponse_6f21f127-229a-44b0-ace8-99ec488084e8--

Extending the batch server to support transaction rollbacks

OData batch requests do not support transactions for complete batches per default. If you want to have rollbacks for a complete batch, you need to implement it yourself. The OData client supports this per default: BatchWithSingleChangeset

This example is based on the asp.net example. Thanks to the OData team for this.
http://aspnet.codeplex.com/SourceControl/latest#Samples/WebApi/OData/v3/ODataEFBatchSample/ODataEFBatchSample/

We need to implement a new batch handler which uses one DBContext for all the Http requests in the batch Http request. When all requests use the same DBContext, this can be rollbacked for all, if any single transaction fails.

using System;
using System.Collections.Generic;
using System.Data.Entity;
using System.Linq;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
using System.Web.Http;
using System.Web.OData.Batch;
using WebAPIODataV4SQLite.DomainModel;

namespace WebAPIODataV4SQLite.App_Start
{
	public class ODataBatchHandlerSingleTransaction : DefaultODataBatchHandler
	{
		public ODataBatchHandlerSingleTransaction(HttpServer httpServer): base(httpServer)
        {
        }

		public async override Task<IList<ODataBatchResponseItem>> ExecuteRequestMessagesAsync(
		   IEnumerable<ODataBatchRequestItem> requests,
		   CancellationToken cancellation)
		{
			if (requests == null)
			{
				throw new ArgumentNullException("requests");
			}

			IList<ODataBatchResponseItem> responses = new List<ODataBatchResponseItem>();
			try
			{
				foreach (ODataBatchRequestItem request in requests)
				{
					var operation = request as OperationRequestItem;
					if (operation != null)
					{
						responses.Add(await request.SendRequestAsync(Invoker, cancellation));
					}
					else
					{
						await ExecuteChangeSet((ChangeSetRequestItem)request, responses, cancellation);
					}
				}
			}
			catch
			{
				foreach (ODataBatchResponseItem response in responses)
				{
					if (response != null)
					{
						response.Dispose();
					}
				}
				throw;
			}

			return responses;
		}

		private async Task ExecuteChangeSet(ChangeSetRequestItem changeSet, IList<ODataBatchResponseItem> responses, CancellationToken cancellation)
		{
			ChangeSetResponseItem changeSetResponse;

			using (var context = new SqliteContext())
			{
				foreach (HttpRequestMessage request in changeSet.Requests)
				{
					request.SetContext(context);
				}

				using (DbContextTransaction transaction = context.Database.BeginTransaction())
				{
					changeSetResponse = (ChangeSetResponseItem)await changeSet.SendRequestAsync(Invoker, cancellation);
					responses.Add(changeSetResponse);

					if (changeSetResponse.Responses.All(r => r.IsSuccessStatusCode))
					{
						transaction.Commit();
					}
					else
					{
						transaction.Rollback();
					}
				}
			}
		}
	}
}

This is then used in the Register method instead of the default batch handler.

ODataBatchHandler odataBatchHandler = new ODataBatchHandlerSingleTransaction(GlobalConfiguration.DefaultServer);

An extension method is required, so that our DBContext can be set for each request within the batch request.

using System.Net.Http;
using WebAPIODataV4SQLite.DomainModel;

namespace WebAPIODataV4SQLite.App_Start
{
	public static class HttpRequestMessageExtensions
	{
		private const string DbContext = "Batch_DbContext";

		public static void SetContext(this HttpRequestMessage request,SqliteContext context)
		{
			request.Properties[DbContext] = context;
		}

		
		public static SqliteContext GetContext(this HttpRequestMessage request)
		{
			object sqliteContext;
			if (request.Properties.TryGetValue(DbContext, out sqliteContext))
			{
				return (SqliteContext)sqliteContext;
			}
			else
			{
				var context = new SqliteContext();
				SetContext(request, context);

				request.RegisterForDispose(context);
				return context;
			}
		}
	}
}

This is then used in the EventDataController

Request.GetContext()

Now all requests for the EventDataController which are executed in a single batch request will be roll backed when any single request failed. If the client uses (SaveChangesOptions.BatchWithSingleChangeset)

Batching works well for OData V4. It is not as flexible as Web API batching. Web API batching is easy to extend. OData data batching is very comfortable to use and can be setup very quickly.

Links:

http://blogs.msdn.com/b/odatateam/archive/2014/03/11/how-to-use-odata-client-code-generator-to-generate-client-side-proxy-class.aspx

http://aspnet.codeplex.com/SourceControl/latest#Samples/WebApi/OData/v4/

http://www.asp.net/web-api/overview/releases/whats-new-in-aspnet-web-api-22#OData

http://visualstudiogallery.msdn.microsoft.com/9b786c0e-79d1-4a50-89a5-125e57475937

http://www.odata.org/libraries/

http://wp.sjkp.dk/generate-c-classes-from-odata-metadata/

http://odata.jenspinney.com/

http://blogs.msdn.com/b/mrtechnocal/archive/2013/11/08/odata-web-api-batching-with-actions.aspx

http://blogs.msdn.com/b/odatateam/archive/2014/07/09/odata-client-code-generator-2-0-0-release.aspx

http://blogs.msdn.com/b/odatateam/archive/2014/07/09/client-delayed-query.aspx

http://blogs.msdn.com/b/webdev/archive/2013/11/01/introducing-batch-support-in-web-api-and-web-api-odata.aspx

19 comments

  1. Thanks for the informative post. However, I am seeing an issue when submitting batch changes with SaveChangesOptions.BatchWithSingleChangeset – it isn’t behaving as advertised and isn’t rolling back successful items in the changeset when there are failed requests.

    I have seen an example for odata v3 (though I am having a hard time finding it) where the default odata batch handler had to be overridden to enable this transactional behavior. Have you actually tested your code with a bad request in the changeset? Any help would be appreciated

    1. Thanks for your comment. SaveChangesOptions.BatchWithSingleChangeset does work as required, the problem is on the server, it needs to be implemented… Per default this does not rollback like you said. Thanks for this info. I’ve updated the blog and added an example how to do this.

      Hope this helps

      cheers Damien

      1. Perfect – thank you.

  2. You stated, “If using the OData batch handler in Web API, you need to configure this in the GlobalConfiguration in the Application_Start method. It will not work per default inside the OWIN middlerware due to routing problems!” Do you know if there is a workaround for this? I am currently configuring my OData service to ride on OWIN and would like to continue doing so, if at all possible.

    1. Hi Steve
      Thanks for the comment and sorry for the slow reply. At present, I don’t know of any workaround. The problem is in the routing, so with some effort, you could rewrite the route mappings and it will work, but I haven’t found a solution for this yet.

      greetings Damien

      1. Hi Damien,
        No problem. Thank you for getting back with me. If you find a solution, please let me know. And, I will do the same.
        Take Care,
        Steve

  3. […] This article demonstrates how batching can be used with a Web API OData V4 service and an OData C# client. This blog is part 10 of the Web API OData series. Batching can be used to optimize network…  […]

  4. […] This article demonstrates how batching can be used with a Web API OData V4 service and an OData C# client. This blog is part 10 of the Web API OData series. Batching can be used to optimize network…  […]

  5. […] This article demonstrates how batching can be used with a Web API OData V4 service and an OData C# client. This blog is part 10 of the Web API OData series. Batching can be used to optimize network…  […]

  6. Brad Lindberg · · Reply

    I was wondering if there has been a fix for the issue above. I’m using asp.net web api odata v4 hosted with OWIN and the transaction does not roll back.

  7. Brad Lindberg · · Reply

    UPDATE: This was an issue on my side, the transaction does rollback when the process is hosted with OWIN.

  8. @Brad : Could you share some guidelines on OWIN and batch processing ? I’m having hard times getting this to work😦

  9. Brad Lindberg · · Reply

    @ErPe: My issue was each call to an entity controller was using different instance of the DbContext, therefore the DbContext was unable to track and rollback the changes. I solved my problem by creating a base controller class that exposed the same instance of the DbContext for each batch operation, therefore entity framework change tracker was able to track all the changes for a batch and roll them back on error. What specifically is the behavior you are experiencing so I can better help you out with your issues?

  10. If using the OData batch handler in Web API, you need to configure this in the GlobalConfiguration in the Application_Start method. It will not work per default inside the OWIN middlerware due to routing problems!

    Fixed ?
    any update about it ?

  11. Hi,

    You can overcome this problem by using same HttpServer for batch handler and owin:

    You can see in this post here on stackoverflow:

    http://stackoverflow.com/questions/12008686/webapi-batching-and-delegating-handlers

    // Configure batch handler
    HttpServer webApiServer = new HttpServer(configuration);
    var batchHandler = new DefaultODataBatchHandler(webApiServer);
    webApiConfig.Routes.MapODataServiceRoute(“ODataRoute”,
    “odata”,
    BuildEdmModel(),
    new DefaultODataPathHandler(),
    ODataRoutingConventions.CreateDefault(),
    batchHandler);

    app.UseWebApi(webApiServer);

    When registering OWIN you use the same server. and then batching will work.

    @Damien: Maybe you can update article so people find this fix more quickly.

    Regards,

    Mihai

  12. Alexey Auslender · · Reply

    I would like to do a deep insert of a new Person entity along with two Trip child entities. Please, find below the client code:

    private static void AddPersonAndTrip(DefaultContainer dc)
    {
    var person = new Person
    {
    FirstName = “John”,
    LastName = “Doe”,
    Concurrency = 1,
    UserName = “JohnDoe”,
    Emails = new ObservableCollection(new[] { “johndoe@gmail.com” }),
    };
    dc.AddToPeople(person);
    // ReSharper disable once UnusedVariable
    var people = new DataServiceCollection(dc, new[] { person }, TrackingMode.AutoChangeTracking, “People”, null, null);
    person.Trips.Add(new Trip
    {
    Name = “Town N”,
    Budget = 1000,
    Description = “Trip to Town N”,
    StartsAt = DateTime.Now.AddDays(5),
    EndsAt = DateTime.Now.AddDays(10),
    Tags = new ObservableCollection(new[] { “town N” })
    });

    dc.SaveChanges(SaveChangesOptions.BatchWithSingleChangeset);
    }
    However, this one sends a single $batch request with 2 embedded POST requests that get dispatched to the 2 controller actions – POST for the new Person and 1 POST for the new Trips.

    I tried all the different options to the SaveChanges, none result in a deep insert. It is always 2 requests – either batched or not.

    So, my question is it possible to do a deep insert with the auto generated OData C# client proxy code?

    1. Brad Lindberg · · Reply

      Hello Alexey,
      Entity Framework does not support bulk inserts, every insert will generate a new database call. So the issue is not with OData but an underlying design within Entity Framework. FYI I think Entity Framework 7 does support batching as a new feature.

  13. Took me a while to figure out how to get the is to work with MS SQL MSDTC but here is what I did.
    1.) Make sure you have MSDTC working with MS SQL.
    2.) Remove the HttpRequestMessageExtensions class.
    3.) The ExecuteChangeSet method should look like the following:

    private async Task ExecuteChangeSet(ChangeSetRequestItem changeSet,
    IList responses, CancellationToken cancellation)
    {
    ChangeSetResponseItem changeSetResponse;

    using (TransactionScope transaction =
    new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
    {
    changeSetResponse = (ChangeSetResponseItem)await changeSet.SendRequestAsync(Invoker, cancellation);
    responses.Add(changeSetResponse);

    if (changeSetResponse.Responses.All(r => r.IsSuccessStatusCode))
    {
    transaction.Complete();
    }
    }
    }

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: