The recent preview release of OData support in Web API is very exciting (see the new nuget package and codeplex project). For the most part it is compatible with the previous [Queryable] support because it supports the same OData query options. That said there has been a little confusion about how [Queryable] works, what it works with and what its limitations are, both temporary and long term.
The rest of this post will outline what is currently supported, what limitations currently exist and which limitations are hopefully just temporary.
Current Support
Support for different ElementTypes
In the preview the [Queryable] attribute works with any IQueryable<> or IEnumerable<> data source (Entity Framework or otherwise), for which a model has been configured or can be inferred automatically.
Today this means that the element type (i.e. the T in IQueryable<T>) must be viewed as an EDM entity. This implies a few constraints:
- All properties you wish to expose must be exposed as CLR properties on your class.
- A key property (or properties) must be available
- The type of all properties must be either:
- a clr type that is mapped to an EDM primitive, i.e. System.String == Edm.String
- Or clr type that is mapped to another type in your model, be that a ComplexType or an EntityType
NOTE: using IEnumerable<> is recommended only for small amounts of data, because the options are only applied after everything has been pulled into memory.
Null Propagation
This feature takes a little explaining, so please bear with me. Imagine you have an action that looks like this:
[Queryable]
public IQueryable<Product> Get()
{
…
}
Now imagine someone issues this request:
GET ~/Products?$filter=startswith(Category/Name,’A’)
You might think the [Queryable] attribute will translate the request to something like this:
Get().Where(p => p.Category.Name.StartsWith(“A"));
But that might be very bad…
If your Get() method body looks like this:
return _db.Products; // i.e. Entity Framework.
It will work just fine. But if your Get() method looks like this:
return products.AsQueryable();
It means the LINQ provider being used is LINQ to Objects. L2O evaluates the where predicate in memory simply by calling the predicate. Which could easily null ref if either p.Category or p.Category.Name are null.
The [Queryable] attribute handles this automatically by injecting null guards into the code for certain IQueryable Providers. If you dig into the code for ODataQueryOptions you’ll see this code:
…
string queryProviderAssemblyName = query.Provider.GetType().Assembly.GetName().Name;
switch (queryProviderAssemblyName)
{
case EntityFrameworkQueryProviderAssemblyName:
handleNullPropagation = false;
break;
case Linq2SqlQueryProviderAssemblyName:
handleNullPropagation = false;
break;
case Linq2ObjectsQueryProviderAssemblyName:
handleNullPropagation = true;
break;
default:
handleNullPropagation = true;
break;
}
return ApplyTo(query, handleNullPropagation);
As you can see for Entity Framework and LINQ to SQL we don’t inject null guards (because SQL takes care of null guards/propagation automatically), but for L2O and all other query providers we inject null guards and propagate nulls.
If you don’t like this behavior you can override it by dropping down and callingODataQueryOptions.Filter.ApplyTo(..) directly.
Supported Query Options
In the preview the [Queryable] attribute supports only 4 of OData’s 8 built-in query options, namely $filter, $orderby, $skip and $top.
What about the 4 other query options? i.e. $select, $expand, $inlinecount and $skiptoken. Today you need to useODataQueryOptions rather than [Queryable], hopefully that will change overtime.
Dropping down to ODataQueryOptions
The first thing to understand is that this code:
[Queryable]
public IQueryable<Product> Get()
{
return _db.Products;
}
Is roughly equivalent to:
public IEnumerable<Product> Get(ODataQueryOptions options)
{
// TODO: we should add an override of ApplyTo that avoid all these casts!
return options.ApplyTo(_db.Products as IQueryable) as IEnumerable<Product>;
}
Which in turn is roughly equivalent to:
public IEnumerable<Product> Get(ODataQueryOptions options)
{
IQueryable results = _db.Products;
if (options.Filter != null)
results = options.Filter.ApplyTo(results);
if (options.OrderBy != null) // this is a slight over-simplification see this.
results = options.OrderBy.ApplyTo(results);
if (options.Skip != null)
results = options.Skip.ApplyTo(results);
if (options.Top != null)
results = options.Top.ApplyTo(results);
return results;
}
This means you can easily pick and choose which options to support. For example if your service doesn’t support$orderby you can assert that ODataQueryOptions.OrderBy is null.
ODataQueryOptions.RawValues
Once you’ve dropped down to the ODataQueryOptions you also get access to the RawValues property which gives you the raw string values of all 8 ODataQueryOptions… So in theory you can handle more query options.
ODataQueryOptions.Filter.QueryNode
The ApplyTo method assumes you have an IQueryable, but what if you backend has no IQueryable implementation?
Creating one from scratch is very hard, mainly because LINQ allows so much more than OData allows, and essentially obfuscates the intent of the query.
To avoid this complexity we provide ODataQueryOptions.Filter.QueryNode which is an AST that gives you a parsed metadata bound tree representing the $filter. The AST of course it tuned to allow only what OData supports, making it much simpler than a LINQ expression.
For example this test fragment illustrates the API:
var filter = new FilterQueryOption("Name eq 'MSFT'", context);
var node = filter.QueryNode;
Assert.Equal(QueryNodeKind.BinaryOperator, node.Expression.Kind);
var binaryNode = node.Expression as BinaryOperatorQueryNode;
Assert.Equal(BinaryOperatorKind.Equal, binaryNode.OperatorKind);
Assert.Equal(QueryNodeKind.Constant, binaryNode.Right.Kind);
Assert.Equal("MSFT", ((ConstantQueryNode)binaryNode.Right).Value);
Assert.Equal(QueryNodeKind.PropertyAccess, binaryNode.Left.Kind);
var propertyAccessNode = binaryNode.Left as PropertyAccessQueryNode;
Assert.Equal("Name", propertyAccessNode.Property.Name);
If you are interested in an example that converts one of these ASTs into another language take a look at theFilterBinder class. This class is used under the hood by ODataQueryOptions to convert the Filter AST into a LINQ Expression of the form Expression<Func<T,bool>>.
You could do something very similar to convert directly to SQL or whatever query language you need. Let me assure you doing this is MUCH easier than implementing IQueryable!
ODataQueryOptions.OrderBy.QueryNode
Likewise you can interrogate the ODataQueryOptions.OrderBy.Query for an AST representing the $orderbyquery option.
Possible Roadmap?
These are just ideas at this stage, really we want to hear what you want, that said, here is what we’ve been thinking about:
Support for $select and $expand
We hope to add support for both of these both as QueryNodes (like Filter and OrderBy), and natively by the[Queryable] attribute.
But first we need to work through some issues:
- The OData Uri Parser (part of ODataContrib) currently doesn’t support $select / $expand, and we need that first.
- Both $expand and $select essentially change the shape of the response. For example you are still returningIQueryable<T> from your action but:
- Each T might have properties that are not loaded. How would the formatter know which properties are not loaded?
- Each T might have relationships loaded, but simply touching an unloaded relationship might cause lazyloading, so the formatters can’t simply hit a relationship during serialization as this would perform terribly, they need to know what to try to format.
- There is no guarantee that you can ‘expand’ an IEnumerable or for that matter an IQueryable, so we would need a way to tell [Queryable] which options it is free to try to handle automatically.
Support for $inlinecount and $skiptoken
Again we hope to add support to [Queryable] for both of these.
That said today you can implement both of these by returning ODataResult<> from your action today.
Implementing $inlinecount is pretty simple:
public ODataResult<Product> Get(ODataQueryOptions options)
{
var results = (options.ApplyTo(_db.Products) as IQueryable<Product>);
var count = results.Count;
var limitedResults = results.Take(100).ToArray();
return new ODataResult<Product>(results,null,count);
}
However implementing server driven paging (i.e. $skiptoken) is more involved and easy to get wrong.
I’ll blog about how to do Server Driven Pages pretty soon.
Support for more Element Types.
We want to support both Complex Types (Complex Type are just like entities, except they don’t have a key and have no relationships) and primitive element types. For example both:
public IQueryable<string> Get(); – maps to say GET ~/Tags
and
public IQueryable<Address> Get(parentId); – maps to say GET ~/Person(6)/Addresses
where no key property has been configured or can be inferred for Address.
You might be asking yourself how do you query a collection of primitives using OData? Well in OData you use the$it implicit iteration variable like this:
GET ~/Tags?$filter=startswith($it,’A’)
Which gets all the Tags that start with ‘A’.
Virtual Properties and Open Types
Essentially virtual properties are things you want to expose as properties via your service that have no corresponding clr property. A good example might be where you to use methods to get and set a property value. This one is a little further out, but it is clearly useful.
Conclusion
As you can see [Queryable] is a work in progress that is layered above ODataQueryOptions, we are planning to improve both over time, and we have a number of ideas. But as always we’d love to hear what you think!