PDA

View Full Version : Nested object field filtering



Ytterate
Sep 03, 2020, 10:14 PM
Hello.

I have some very large objects server-side, some of them upwards of 1Mb due to many fields and multiple variations of images. However, any given view only displays a subset of these fields, for example:



<ext:Model runat="server" Name="Step_Model" IDProperty="Id">
<Fields>
<ext:ModelField Name="Id" />
<ext:ModelField Name="DisplayName" />
<ext:ModelField Name="BatchId" />
<ext:ModelField Name="Quantity" />
<ext:ModelField Name="ImageSource" />
</Fields>
</ext:Model>


In these cases I've found that using an `<ext:Sore>` with an `<asp:LinqDataSource>` actually filters the data server-side and passes only exactly those fields to the client, vastly reducing the amount of data transferred:



<asp:LinqDataSource runat="server" ID="UnassignedSteps_Source" OnSelecting="UnassignedSteps_Selecting" />
<ext:Store ID="UnassignedSteps_Store" runat="server" ModelName="Step_Model" DataSourceID="UnassignedSteps_Source" />





protected void UnassignedSteps_Selecting(object sender, LinqDataSourceSelectEventArgs e)
{
e.Result = GetAllSteps();
}


That is, unless the models have associations:



<ext:Model runat="server" Name="Batch_Model" IDProperty="Id">
<Fields>
<ext:ModelField Name="Id" />
<ext:ModelField Name="DisplayName" />
</Fields>
<Associations>
<ext:HasManyAssociation Model="Step_Model" Name="Steps" AssociationKey="StepIdsReal" />
</Associations>
</ext:Model>


With this model, a store with a datasource doesn't load ANY of the associated data. I've tried setting the `StoreConfig` property for the `ext:HasManyAssociation` to point to an existing datasource (this just crashes because the datasource specified doesn't exist in the temporary internal page created for that store). I've tried explicitly specifying foreign and primary keys. The closest I got was the initial data sent from the server contained ALL the data for the sub models, but the page couldn't see or render them.

Ideally, I want to be able to somehow load child data (`Step_Model` in this example) with the same server-side filtering as the parent object (`Batch_Model`) so only three fields for the outer object are sent - `Id`, `DisplayName`, and `Steps`, and within the `Steps` array the objects only contain the fields specified in their model.

Currently the only way I can display all the data is setting `.Data` in `Page_Load`, but that transfers vast amounts of data; plus I have to be very careful with the data that there are no recursive references because it will never resolve. With the `asp:LinqDataSource` method that doesn't matter, because the fields are filtered and unspecified ones are never traversed.

How can this be done? Even some solution involving multiple stores, multiple data sources, and cross-references between them to get the associated data for one model from another store would be amazing. Thank you.

Ytterate
Sep 03, 2020, 10:38 PM
I can of course just manually filter server-side and create new anonymous objects with only the fields that I want, but I'd rather not as then I have to specify the fields in two places instead of one and that becomes a higher maintenance burden.

fabricio.murta
Sep 10, 2020, 4:47 AM
Hello @Ytterate, and welcome to Ext.NET forums!

Well, if you use, like, 5 columns, and your data comprises 15, then chances are you may save 50-70% data transfer. But again, you have to weigh maintenability with performance.

Getting the extra columns of data you'll never use is spending network bandwidth and client-side memory. Even if fields are not used they will be available nevertheless (say, if the store is linked to something else like a combo box that uses other fields, is a case where the extra fields may be useful).

Usually filtering data from server side and providing just what it takes for the proper data to be displayed is the way to go. The server can process data much faster than clients can download + process them. Especially download, as this is something that often is not desired to be cached (and the server still can efficiently cache data queries and refresh them as the base changes).

That put aside, and assuming you need all those fields, just not all at once, I can point you to some partial data fetching examples from our examples explorer. They may not fit your use case but perhaps you can draw us a test case based in one of them; then we'll be able to tell you exactly why whatever you're trying fails and point you to the right direction to properly handle your data.

So, here are some examples pulling partial data from the server (by paging, filtering, or buffer-rendering):

- Grid Panel > Filter Header > Remote (https://examples4.ext.net/#/GridPanel/FilterHeader/Remote/) (server-side filtering)
- Grid Panel > Paging and Sorting > Local Paging with remote data (https://examples4.ext.net/#/GridPanel/Paging_and_Sorting/Local_Paging_with_Remote_Data/) (pages thru a subset of local data with a slider to pull in a "bigger page" from the server)
- Grid Panel > Plugins > Grid Filters Remote (https://examples4.ext.net/#/GridPanel/Plugins/GridFilters_Remote/) (the server method filters, sorts, and returns only the current page in this case -- you can make a single page big enough to allow users to scroll yet only fetch data from the server when they switch pages)

You may find more useful examples by browsing the examples explorer, but I believe these three above should give you a good overview of what you can do.

In case none of these helps I am afraid we would need a test case (with mock data, etc). You'll probably be able to base it off an example in the Examples Explorer. With a test case we can see exactly what you have and provide you a better answer.

Take some time to review our forum guidelines, they contain invaluable information on "how to help us help you", I am confident it will be worth the while. Here they are, for convenience:

- Tips for creating simplified code samples (http://forums.ext.net/showthread.php?61176-Tips-for-creating-simplified-code-samples)
- More Information Required (http://forums.ext.net/showthread.php?10205-More-Information-Required)
- Forum Guidelines (http://forums.ext.net/showthread.php?3440-Forum-Guidelines-For-Posting-New-Topics)

Hope this helps!

Ytterate
Sep 28, 2020, 3:22 PM
I don't need all the fields, I was asking how to filter them, but in a nested way.

So the top level object has 15 fields, but I only want 5. But one of those 5 is a child object that also has 20 fields of which I want just 4. The best I could manage was loading just 5 of the parent object, but all of the child. None of the default nested loading examples I could find applied this filtering recursively.

fabricio.murta
Oct 01, 2020, 9:37 PM
Hello @Ytterate!

You can just create a new object with just the fields you need (properties, actually, they are automatically mapped to client-side readable json), or use an anonymous object. We have several examples in Examples Explorer (https://examples5.ext.net/) using anonymous objects, object[], List<object> and actual classes. Just browse around the grid examples and you'll see what I'm talking about.

Hope this helps!