This article demonstrates how batching can be used with a Web API OData V4 service and an OData C# client. This blog is part 10 of the Web API OData series. Batching can be used to optimize network usage. Batching makes it possible to send many HTTP requests as a single multiple mixed request. The client/server packs/unpacks the different batch requests.
Note: Part 8 of this series explains how to setup an OData C# client.
Part 1 Getting started with Web API and OData V4 Part 1.
Part 2 Web API and OData V4 Queries, Functions and Attribute Routing Part 2
Part 3 Web API and OData V4 CRUD and Actions Part 3
Part 4 Web API OData V4 Using enum with Functions and Entities Part 4
Part 5 Web API OData V4 Using Unity IoC, SQLite with EF6 and OData Model Aliasing Part 5
Part 6 Web API OData V4 Using Contained Models Part 6
Part 7 Web API OData V4 Using a Singleton Part 7
Part 8 Web API OData V4 Using an OData T4 generated client Part 8
Part 9 Web API OData V4 Caching Part 9
Part 10 Web API OData V4 Batching Part 10
Part 11 Web API OData V4 Keys, Composite Keys and Functions Part 11
Code: https://github.com/damienbod/WebAPIODataV4SQLite
Web API 2 OData V4 Batching Server
To implement an OData batch handler in Web API, an ODataBatchHandler handler needs to be added. Message quotas can be configured as required for the batch handler. The batch handler underneath can be accessed using /odata/$batch
ODataBatchHandler odataBatchHandler = new DefaultODataBatchHandler(GlobalConfiguration.DefaultServer); odataBatchHandler.MessageQuotas.MaxOperationsPerChangeset = 10; odataBatchHandler.MessageQuotas.MaxPartsPerBatch = 10; config.MapODataServiceRoute( "odata", "odata", model: GetModel(), batchHandler: odataBatchHandler );
Now that the batch handler is configured, you should define a ContainerName in the model. This will be used in the OData client.
public static IEdmModel GetModel() { ODataModelBuilder builder = new ODataConventionModelBuilder(); builder.Namespace = "damienbod"; builder.ContainerName = "SqliteContext"; builder.EntitySet<AnimalType>("AnimalType"); builder.EntitySet<EventData>("EventData"); builder.EntitySet<Player>("Player"); builder.EntityType<PlayerStats>(); SingletonConfiguration<SkillLevels> skillLevels = builder.Singleton<SkillLevels>("SkillLevels"); builder.EntityType<SkillLevel>(); builder.EntitySet<AdmDto>("Adms"); return builder.GetEdmModel(); }
If using the OData batch handler in Web API, you need to configure this in the GlobalConfiguration in the Application_Start method. It will not work per default inside the OWIN middlerware due to routing problems!
protected void Application_Start() { // This is required because the OData batchHandler // does not work when hosted in a OWIN container GlobalConfiguration.Configure(WebApiConfig.Register);
Web API 2 OData V4 Batching Client
This is very easy to setup. This works like a normal OData client. The client DTOs and client code is auto-generated from the OData Web API service, using T4 templates. Only the SaveChanges method is different to a normal client. The following client selects data from the server. (not batching but it could be!). Then some data is updated and a new entity is created. This is sent to the server as a batch HTTP multiple mixed post request. The batch is sent with as a single change request. If one fails, all requests fails.
var context = new SqliteContext(new Uri("http://localhost.fiddler:59145/odata/")); context.Format.UseJson(); // Call some basic Get var eventDataItems = context.EventData.ToList(); var animalsItems = context.AnimalType.ToList(); var skillLevels = context.SkillLevels.Expand("Levels").GetValue(); var players = context.Player.Expand(c => c.PlayerStats) .Where(u => u.PlayerStats.SkillLevel == 2).ToList(); // Create a new entity var newObjectEventData = new EventData { AnimalTypeId = animalsItems.First().Key, Factor = 56, FixChange = 13.0, StringTestId = "testdatafromodataclient", AnimalType = animalsItems.First() }; // Update a new entity var dataToUpdate = eventDataItems.FirstOrDefault(); dataToUpdate.Factor = 99; dataToUpdate.FixChange = 97; context.AddToEventData(newObjectEventData); //context.UpdateObject(dataToUpdate); context.AddAndUpdateResponsePreference = DataServiceResponsePreference.IncludeContent; // Add the data to the server DataServiceResponse response = context.SaveChanges(SaveChangesOptions.BatchWithSingleChangeset);
The batch could also be sent so each request is executed independently.
// Add the data to the server DataServiceResponse response = context.SaveChanges(SaveChangesOptions.BatchWithIndependentOperations);
Debugging with Fiddler
Fiddler can be used to view the HTTP request sent from the C# client. This can be very helpful when debugging a HTTP client or a server. Instead of sending the request direct to the IIS, you can send it to fiddler which will send it onto the IIS server.
For example:
http://localhost.fiddler:59145/odata/$batch in the application will send the request to fiddler which will send the request to http://localhost:59145/odata/$batch
var context = new SqliteContext(new Uri("http://localhost.fiddler:59145/odata/"));
Http batch Request:
POST http://localhost:59145/odata/$batch HTTP/1.1 OData-Version: 4.0;NetFx OData-MaxVersion: 4.0;NetFx Content-Type: multipart/mixed; boundary=batch_ebdc0b88-eeb1-4dd6-b170-74331f39bd03 Accept: multipart/mixed Accept-Charset: UTF-8 User-Agent: Microsoft ADO.NET Data Services Host: localhost:59145 Content-Length: 1663 Expect: 100-continue --batch_ebdc0b88-eeb1-4dd6-b170-74331f39bd03 Content-Type: multipart/mixed; boundary=changeset_54ac09ec-f437-4b08-9925-fd42ed7bd58f --changeset_54ac09ec-f437-4b08-9925-fd42ed7bd58f Content-Type: application/http Content-Transfer-Encoding: binary Content-ID: 43 POST http://localhost.fiddler:59145/odata/EventData HTTP/1.1 OData-Version: 4.0;NetFx OData-MaxVersion: 4.0;NetFx Content-Type: application/json;odata.metadata=minimal Prefer: return=representation Accept: application/json;odata.metadata=minimal Accept-Charset: UTF-8 User-Agent: Microsoft ADO.NET Data Services {"@odata.type":"#WebAPIODataV4SQLite.DomainModel.EventData","AnimalTypeId":0,"EventDataId":0,"Factor":56,"FixChange":13.0,"StringTestId":"testdatafromodataclient"} --changeset_54ac09ec-f437-4b08-9925-fd42ed7bd58f-- --batch_ebdc0b88-eeb1-4dd6-b170-74331f39bd03 Content-Type: multipart/mixed; boundary=changeset_2346da5e-88c9-4aa5-a837-5db7e1368147 --changeset_2346da5e-88c9-4aa5-a837-5db7e1368147 Content-Type: application/http Content-Transfer-Encoding: binary Content-ID: 44 PATCH http://localhost:59145/odata/EventData(1) HTTP/1.1 OData-Version: 4.0;NetFx OData-MaxVersion: 4.0;NetFx Content-Type: application/json;odata.metadata=minimal Prefer: return=representation Accept: application/json;odata.metadata=minimal Accept-Charset: UTF-8 User-Agent: Microsoft ADO.NET Data Services {"@odata.type":"#WebAPIODataV4SQLite.DomainModel.EventData","AnimalTypeId":1,"EventDataId":1,"Factor":99,"FixChange":97.0,"StringTestId":"PressureOnThePigMarket"} --changeset_2346da5e-88c9-4aa5-a837-5db7e1368147-- --batch_ebdc0b88-eeb1-4dd6-b170-74331f39bd03--
Http batch Response
HTTP/1.1 200 OK Cache-Control: no-cache Pragma: no-cache Content-Type: multipart/mixed; boundary=batchresponse_6f21f127-229a-44b0-ace8-99ec488084e8 Expires: -1 Server: Microsoft-IIS/8.0 OData-Version: 4.0 X-AspNet-Version: 4.0.30319 X-SourceFiles: =?UTF-8?B?QzpcZ2l0XGRhbWllbmJvZFxXZWJBUElPRGF0YVY0U1FMaXRlXFdlYkFQSU9EYXRhVjRTUUxpdGVcb2RhdGFcJGJhdGNo?= X-Powered-By: ASP.NET Date: Wed, 13 Aug 2014 14:25:32 GMT Content-Length: 1437 --batchresponse_6f21f127-229a-44b0-ace8-99ec488084e8 Content-Type: multipart/mixed; boundary=changesetresponse_47587855-a09b-4afb-a926-a2c6c2913bf9 --changesetresponse_47587855-a09b-4afb-a926-a2c6c2913bf9 Content-Type: application/http Content-Transfer-Encoding: binary Content-ID: 43 HTTP/1.1 201 Created Location: http://localhost.fiddler:59145/odata/EventData(46) Content-Type: application/json; odata.metadata=minimal; charset=utf-8 OData-Version: 4.0 { "@odata.context":"http://localhost.fiddler:59145/odata/$metadata#EventData/$entity","EventDataId":46,"Factor":56,"StringTestId":"testdatafromodataclient","FixChange":13.0,"AnimalTypeId":0 } --changesetresponse_47587855-a09b-4afb-a926-a2c6c2913bf9-- --batchresponse_6f21f127-229a-44b0-ace8-99ec488084e8 Content-Type: multipart/mixed; boundary=changesetresponse_8da9a570-ca14-4793-8173-d743e15e8e38 --changesetresponse_8da9a570-ca14-4793-8173-d743e15e8e38 Content-Type: application/http Content-Transfer-Encoding: binary Content-ID: 44 HTTP/1.1 200 OK Content-Type: application/json; odata.metadata=minimal; charset=utf-8 OData-Version: 4.0 { "@odata.context":"http://localhost:59145/odata/$metadata#EventData/$entity","EventDataId":1,"Factor":99,"StringTestId":"PressureOnThePigMarket","FixChange":97.0,"AnimalTypeId":1 } --changesetresponse_8da9a570-ca14-4793-8173-d743e15e8e38-- --batchresponse_6f21f127-229a-44b0-ace8-99ec488084e8--
Extending the batch server to support transaction rollbacks
OData batch requests do not support transactions for complete batches per default. If you want to have rollbacks for a complete batch, you need to implement it yourself. The OData client supports this per default: BatchWithSingleChangeset
This example is based on the asp.net example. Thanks to the OData team for this.
http://aspnet.codeplex.com/SourceControl/latest#Samples/WebApi/OData/v3/ODataEFBatchSample/ODataEFBatchSample/
We need to implement a new batch handler which uses one DBContext for all the Http requests in the batch Http request. When all requests use the same DBContext, this can be rollbacked for all, if any single transaction fails.
using System; using System.Collections.Generic; using System.Data.Entity; using System.Linq; using System.Net.Http; using System.Threading; using System.Threading.Tasks; using System.Web.Http; using System.Web.OData.Batch; using WebAPIODataV4SQLite.DomainModel; namespace WebAPIODataV4SQLite.App_Start { public class ODataBatchHandlerSingleTransaction : DefaultODataBatchHandler { public ODataBatchHandlerSingleTransaction(HttpServer httpServer): base(httpServer) { } public async override Task<IList<ODataBatchResponseItem>> ExecuteRequestMessagesAsync( IEnumerable<ODataBatchRequestItem> requests, CancellationToken cancellation) { if (requests == null) { throw new ArgumentNullException("requests"); } IList<ODataBatchResponseItem> responses = new List<ODataBatchResponseItem>(); try { foreach (ODataBatchRequestItem request in requests) { var operation = request as OperationRequestItem; if (operation != null) { responses.Add(await request.SendRequestAsync(Invoker, cancellation)); } else { await ExecuteChangeSet((ChangeSetRequestItem)request, responses, cancellation); } } } catch { foreach (ODataBatchResponseItem response in responses) { if (response != null) { response.Dispose(); } } throw; } return responses; } private async Task ExecuteChangeSet(ChangeSetRequestItem changeSet, IList<ODataBatchResponseItem> responses, CancellationToken cancellation) { ChangeSetResponseItem changeSetResponse; using (var context = new SqliteContext()) { foreach (HttpRequestMessage request in changeSet.Requests) { request.SetContext(context); } using (DbContextTransaction transaction = context.Database.BeginTransaction()) { changeSetResponse = (ChangeSetResponseItem)await changeSet.SendRequestAsync(Invoker, cancellation); responses.Add(changeSetResponse); if (changeSetResponse.Responses.All(r => r.IsSuccessStatusCode)) { transaction.Commit(); } else { transaction.Rollback(); } } } } } }
This is then used in the Register method instead of the default batch handler.
ODataBatchHandler odataBatchHandler = new ODataBatchHandlerSingleTransaction(GlobalConfiguration.DefaultServer);
An extension method is required, so that our DBContext can be set for each request within the batch request.
using System.Net.Http; using WebAPIODataV4SQLite.DomainModel; namespace WebAPIODataV4SQLite.App_Start { public static class HttpRequestMessageExtensions { private const string DbContext = "Batch_DbContext"; public static void SetContext(this HttpRequestMessage request,SqliteContext context) { request.Properties[DbContext] = context; } public static SqliteContext GetContext(this HttpRequestMessage request) { object sqliteContext; if (request.Properties.TryGetValue(DbContext, out sqliteContext)) { return (SqliteContext)sqliteContext; } else { var context = new SqliteContext(); SetContext(request, context); request.RegisterForDispose(context); return context; } } } }
This is then used in the EventDataController
Request.GetContext()
Now all requests for the EventDataController which are executed in a single batch request will be roll backed when any single request failed. If the client uses (SaveChangesOptions.BatchWithSingleChangeset)
Batching works well for OData V4. It is not as flexible as Web API batching. Web API batching is easy to extend. OData data batching is very comfortable to use and can be setup very quickly.
Links:
http://aspnet.codeplex.com/SourceControl/latest#Samples/WebApi/OData/v4/
http://www.asp.net/web-api/overview/releases/whats-new-in-aspnet-web-api-22#OData
http://visualstudiogallery.msdn.microsoft.com/9b786c0e-79d1-4a50-89a5-125e57475937
http://www.odata.org/libraries/
http://wp.sjkp.dk/generate-c-classes-from-odata-metadata/
http://blogs.msdn.com/b/mrtechnocal/archive/2013/11/08/odata-web-api-batching-with-actions.aspx
http://blogs.msdn.com/b/odatateam/archive/2014/07/09/odata-client-code-generator-2-0-0-release.aspx
http://blogs.msdn.com/b/odatateam/archive/2014/07/09/client-delayed-query.aspx
Reblogged this on Dinesh Ram Kali..
Thanks for the informative post. However, I am seeing an issue when submitting batch changes with SaveChangesOptions.BatchWithSingleChangeset – it isn’t behaving as advertised and isn’t rolling back successful items in the changeset when there are failed requests.
I have seen an example for odata v3 (though I am having a hard time finding it) where the default odata batch handler had to be overridden to enable this transactional behavior. Have you actually tested your code with a bad request in the changeset? Any help would be appreciated
Thanks for your comment. SaveChangesOptions.BatchWithSingleChangeset does work as required, the problem is on the server, it needs to be implemented… Per default this does not rollback like you said. Thanks for this info. I’ve updated the blog and added an example how to do this.
Hope this helps
cheers Damien
Perfect – thank you.
You stated, “If using the OData batch handler in Web API, you need to configure this in the GlobalConfiguration in the Application_Start method. It will not work per default inside the OWIN middlerware due to routing problems!” Do you know if there is a workaround for this? I am currently configuring my OData service to ride on OWIN and would like to continue doing so, if at all possible.
Hi Steve
Thanks for the comment and sorry for the slow reply. At present, I don’t know of any workaround. The problem is in the routing, so with some effort, you could rewrite the route mappings and it will work, but I haven’t found a solution for this yet.
greetings Damien
Hi Damien,
No problem. Thank you for getting back with me. If you find a solution, please let me know. And, I will do the same.
Take Care,
Steve
[…] This article demonstrates how batching can be used with a Web API OData V4 service and an OData C# client. This blog is part 10 of the Web API OData series. Batching can be used to optimize network… […]
[…] This article demonstrates how batching can be used with a Web API OData V4 service and an OData C# client. This blog is part 10 of the Web API OData series. Batching can be used to optimize network… […]
[…] This article demonstrates how batching can be used with a Web API OData V4 service and an OData C# client. This blog is part 10 of the Web API OData series. Batching can be used to optimize network… […]
I was wondering if there has been a fix for the issue above. I’m using asp.net web api odata v4 hosted with OWIN and the transaction does not roll back.
UPDATE: This was an issue on my side, the transaction does rollback when the process is hosted with OWIN.
@Brad : Could you share some guidelines on OWIN and batch processing ? I’m having hard times getting this to work 😦
@ErPe: My issue was each call to an entity controller was using different instance of the DbContext, therefore the DbContext was unable to track and rollback the changes. I solved my problem by creating a base controller class that exposed the same instance of the DbContext for each batch operation, therefore entity framework change tracker was able to track all the changes for a batch and roll them back on error. What specifically is the behavior you are experiencing so I can better help you out with your issues?
If using the OData batch handler in Web API, you need to configure this in the GlobalConfiguration in the Application_Start method. It will not work per default inside the OWIN middlerware due to routing problems!
Fixed ?
any update about it ?
Hi,
You can overcome this problem by using same HttpServer for batch handler and owin:
You can see in this post here on stackoverflow:
http://stackoverflow.com/questions/12008686/webapi-batching-and-delegating-handlers
// Configure batch handler
HttpServer webApiServer = new HttpServer(configuration);
var batchHandler = new DefaultODataBatchHandler(webApiServer);
webApiConfig.Routes.MapODataServiceRoute(“ODataRoute”,
“odata”,
BuildEdmModel(),
new DefaultODataPathHandler(),
ODataRoutingConventions.CreateDefault(),
batchHandler);
app.UseWebApi(webApiServer);
When registering OWIN you use the same server. and then batching will work.
@Damien: Maybe you can update article so people find this fix more quickly.
Regards,
Mihai
I would like to do a deep insert of a new Person entity along with two Trip child entities. Please, find below the client code:
private static void AddPersonAndTrip(DefaultContainer dc)
{
var person = new Person
{
FirstName = “John”,
LastName = “Doe”,
Concurrency = 1,
UserName = “JohnDoe”,
Emails = new ObservableCollection(new[] { “johndoe@gmail.com” }),
};
dc.AddToPeople(person);
// ReSharper disable once UnusedVariable
var people = new DataServiceCollection(dc, new[] { person }, TrackingMode.AutoChangeTracking, “People”, null, null);
person.Trips.Add(new Trip
{
Name = “Town N”,
Budget = 1000,
Description = “Trip to Town N”,
StartsAt = DateTime.Now.AddDays(5),
EndsAt = DateTime.Now.AddDays(10),
Tags = new ObservableCollection(new[] { “town N” })
});
dc.SaveChanges(SaveChangesOptions.BatchWithSingleChangeset);
}
However, this one sends a single $batch request with 2 embedded POST requests that get dispatched to the 2 controller actions – POST for the new Person and 1 POST for the new Trips.
I tried all the different options to the SaveChanges, none result in a deep insert. It is always 2 requests – either batched or not.
So, my question is it possible to do a deep insert with the auto generated OData C# client proxy code?
Hello Alexey,
Entity Framework does not support bulk inserts, every insert will generate a new database call. So the issue is not with OData but an underlying design within Entity Framework. FYI I think Entity Framework 7 does support batching as a new feature.
Took me a while to figure out how to get the is to work with MS SQL MSDTC but here is what I did.
1.) Make sure you have MSDTC working with MS SQL.
2.) Remove the HttpRequestMessageExtensions class.
3.) The ExecuteChangeSet method should look like the following:
private async Task ExecuteChangeSet(ChangeSetRequestItem changeSet,
IList responses, CancellationToken cancellation)
{
ChangeSetResponseItem changeSetResponse;
using (TransactionScope transaction =
new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
changeSetResponse = (ChangeSetResponseItem)await changeSet.SendRequestAsync(Invoker, cancellation);
responses.Add(changeSetResponse);
if (changeSetResponse.Responses.All(r => r.IsSuccessStatusCode))
{
transaction.Complete();
}
}
}
odataBatchHandler.MessageQuotas.MaxOperationsPerChangeset = 100000;
odataBatchHandler.MessageQuotas.MaxPartsPerBatch = 10000;
this two lines does not work.
When I requests over 100 queries,The odata service will return 500.
@Anthony,
I also have the same problem. When I send a request to insert more than 1000 records, I got following exception ,
{“Message”:”An error has occurred.”,”ExceptionMessage”:”The current change set contains too many operations. A maximum number of ‘1000’ operations are allowed in a change set.”,”ExceptionType”:”Microsoft.OData.ODataException”,”StackTrace”:” at Microsoft.OData.ODataBatchReader.IncreaseChangeSetSize()\r\n at Microsoft.OData.ODataBatchReader.SkipToNextPartAndReadHeaders()\r\n at Microsoft.OData.ODataBatchReader.ReadImplementation()\r\n at Microsoft.OData.ODataBatchReader.ReadSynchronously()\r\n at Microsoft.OData.ODataBatchReader.InterceptException[T](Func`1 action)\r\n at Microsoft.OData.ODataBatchReader.Read()\r\n at System.Web.OData.Batch.ODataBatchReaderExtensions.d__0.MoveNext()\r\n— End of stack trace from previous location where exception was thrown —\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.OData.Batch.DefaultODataBatchHandler.d__f.MoveNext()\r\n— End of stack trace from previous location where exception was thrown —\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.OData.Batch.DefaultODataBatchHandler.d__0.MoveNext()\r\n— End of stack trace from previous location where exception was thrown —\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.Http.Batch.HttpBatchHandler.d__0.MoveNext()”}
Did you find any solution?
Look at Microsoft.OData.Core.ODataMessageQuotas – this is where you can configure these limits.
I have 404 error in the batch call, while all the URI are valid on its own. Is this the OWIN issue? Thanks!
–batchresponse_f3df4b26-086f-4b5d-b53f-ff20299e76d2
Content-Type: application/http
Content-Transfer-Encoding: binary
HTTP/1.1 404 Not Found
Content-Type: application/json; charset=utf-8
{“Message”:”No HTTP resource was found that matches the request URI ‘http://localhost:59822/odata/Lookups’.”,”MessageDetail”:”No route data was found for this request.”}
–batchresponse_f3df4b26-086f-4b5d-b53f-ff20299e76d2
Content-Type: application/http
Content-Transfer-Encoding: binary
HTTP/1.1 404 Not Found
Content-Type: application/json; charset=utf-8
{“Message”:”No HTTP resource was found that matches the request URI ‘http://localhost:59822/odata/MTasks’.”,”MessageDetail”:”No route data was found for this request.”}
–batchresponse_f3df4b26-086f-4b5d-b53f-ff20299e76d2–
hi,
in our case (IIS 10), we had to do some modifications within our web.config:
After that, we get the response:
HTTP/1.1 200 OK
after calling POST: https://ourwebserver/odata/$batch
hi,
in our case (IIS 10), we had to do some modifications within our web.config:
-handlers-
-remove name=”ExtensionlessUrlHandler-Integrated-4.0″ /-
-add name=”ExtensionlessUrlHandler-Integrated-4.0″ path=”*.” verb=”*” type=”System.Web.Handlers.TransferRequestHandler”
resourceType=”Unspecified” requireAccess=”Script” preCondition=”integratedMode,runtimeVersionv4.0″ /-
-/handlers-
(just replace – with tag open/close)
After that, we get the response:
HTTP/1.1 200 OK
after calling POST: https://ourwebserver/odata/$batch
[…] https://damienbod.com/2014/08/14/web-api-odata-v4-batching-part-10/ […]