Best practice for returning custom paging data

[anup]

Hi,

I have an MVC controller that returns a StoreResult for a grid.

StoreResult sets the Data property as well as Total. So this is typically a Paging<T> for me.

I have a particular sub class of Store that has a LOT of separate Parameters as part of the query (maybe 20! we have a complex search engine!!). In addition, I actually want to return a subclass of Paging<T> that has some other properties in addition to data and total records (so that the UI has some further information to act on as part of a single request/response, e.g. I send down a list of error messages associated with each row that is displayed in a panel above the grid).

Finally, the rows of data I have are sparsely populated so I have a lot of rows and many columns but many values are null. If I am custom serializing with Json.NET I can ignore nulls easily. However, trying to return this.Store() from an MVC Controller action, I can't see a way to pass my own instance of JsonSerializationSettings through to be used by the call to JSON.Serialize() that is called inside StoreResult.

I am thinking to achieve this I might need to do the following:

• Subclass StoreRequestParameters to handle the additional "extraParams" and then make this new class the action parameter
• Subclass Paging<T> so I can have additional properties I want to return when I query my search engine
• Subclass StoreResults to take my custom Paging object that can hold additional properties to return*
• Subclass StoreResponse so that my additional properties in my Paging object can be passed in the response

* If I am to subclass of StoreResults I would need to override ExecuteResult() to use my own instance of StoreResponse. But this also means I can then use my own serialization settings as I would need to call JSON.Serialize myself.

Does that sound right or am I complicating things and is there a better way to go about this?

I am doing all this to then compare whether ASHX, MVC Controller or Web API Controller is better from a performance (and code maintenance) point of view, for my particular purpose!

I am quite comfortable with ASHX and already have that working, for a long time, without the need for various subclasses as I can control it all. But I'd like to see whether or not there is significant performance degradation going via an MVC controller or Web API controller and what the different coding considerations are from a maintenance point of view. As our app gets more and more complex with more controllers/handlers, it starts to look inconsistent (to new comers) why sometimes we use Controllers and other times ASHX handlers, etc...! But I just want to see if there is a performance method to my madness :)

Thanks!

[anup]

Also, I just realized that if I need to extend StoreResponseData and override the ToString method, it currently ends like this:

    sb.Append("}");

    return sb.ToString();

This means I can't add my own Json easily. I can of course work around this, but I wonder, would it be possible to slightly change it to something like this:

    this.AddAdditionalString(sb);  // new method to allow subclasses to add their own data into the serialization process

    sb.Append("}");

    return sb.ToString();
}

protected virtual void AddAdditionalString(StringBuilder sb)
{
     return;
}

(Maybe a better method name would be good!)

This way I can inject my additional code a bit more efficiently.

(Also, just out of curiosity is it not possible to serialize the whole thing via Json.NET? Maybe it is too big a change now in case it results in different JSON to what this code currently produces.)

[anup]

I've also just realized that the JSON utility class you have has a GlobalSettings property.

It looks like I could just add my own settings into there, e.g. to ignore null values, and I could probably do this in global.asax upon application start.

If I were to do this globally, do you think that would have any ramifications elsewhere for using Ext.NET, and would it be safer to just use my own instance of JsonSerializerSettings instead?

[Daniil]

Hi Anup,

Does that sound right or am I complicating things and is there a better way to go about this?

It sounds right for me. Maybe, it looks a bit complicated, but, I believe, it should be quite easy for you to implement all that stuff.

I am doing all this to then compare whether ASHX, MVC Controller or Web API Controller is better from a performance (and code maintenance) point of view, for my particular purpose!

I like your systematic approach a lot. I would much appreciate if you share the results of your investigation:)

Also, I just realized that if I need to extend StoreResponseData and override the ToString method, it currently ends like this:

    sb.Append("}");

    return sb.ToString();

This means I can't add my own Json easily. I can of course work around this, but I wonder, would it be possible to slightly change it to something like this:

    this.AddAdditionalString(sb);  // new method to allow subclasses to add their own data into the serialization process

    sb.Append("}");

    return sb.ToString();
}

protected virtual void AddAdditionalString(StringBuilder sb)
{
     return;
}

(Maybe a better method name would be good!)

This way I can inject my additional code a bit more efficiently.

I like your suggestion! Created an Issue.
https://github.com/extnet/Ext.NET/issues/590

I tried the following.

That is the end of the ToString method.

this.InjectToString(sb, comma);
sb.Append("}");

return sb.ToString();

That is the InjectToString method. What do you think about the name and docs?

/// <summary>
/// The template method to inject the custom serialization string into the result of .ToString() call.
/// </summary>
/// <param name="sb">The StringBuilder</param>
/// <param name="needsComma">True if the JSON string requires a leading comma</param>
protected virtual void InjectToString(StringBuilder sb, bool needsComma)
{
    return;
}

A test case.

<%@ Page Language="C#" %>

<script runat="server">
    public class MyStoreResponseData : StoreResponseData
    {
        protected override void InjectToString(StringBuilder sb, bool needsComma)
        {
            if (needsComma)
            {
                sb.Append(",");
            }

            sb.Append("\"someProp\": \"some value\"");
        }
    }

    protected void Page_Load(object sender, EventArgs e)
    {
        MyStoreResponseData srd = new MyStoreResponseData();
        srd.Success = false;
        X.Msg.Alert("MyStoreResponseData", srd.ToString()).Show();    
    }
</script>

<!DOCTYPE html>

<html>
<head runat="server">
    <title>Ext.NET v2 Example</title>
</head>
<body>
    <ext:ResourceManager runat="server" />
</body>
</html>

I have not committed anything yet, I need your approval first to ensure my commit suites your needs. Approval of naming, docs and one more thing.

The thing is this piece of code at the beginning of the ToString method.

if (this.Data.IsEmpty() && this.Success && this.Message.IsEmpty())
{
    return "";
}

For example, this will result in an empty string.

MyStoreResponseData srd = new MyStoreResponseData();
X.Msg.Alert("MyStoreResponseData", srd.ToString()).Show();

At the moment, I am not sure what to do with that. Are you OK with that condition or you need something to be returned by a .ToString() call even if this is true.

this.Data.IsEmpty() && this.Success && this.Message.IsEmpty()

(Also, just out of curiosity is it not possible to serialize the whole thing via Json.NET? Maybe it is too big a change now in case it results in different JSON to what this code currently produces.)

Probably, there is a way, but the current serialization is quite complex and I don't think there is a simple way to get exactly the same result with just a JSON.Serialize() call. Also I don't think that we are going to change a JSON structure of a produced response, because a client side Store expects a certain response structure. Yes, it all looks to be a too big change and, seems, it doesn't look reazonable enough to change. What do you think?

I've also just realized that the JSON utility class you have has a GlobalSettings property.

It looks like I could just add my own settings into there, e.g. to ignore null values, and I could probably do this in global.asax upon application start.

If I were to do this globally, do you think that would have any ramifications elsewhere for using Ext.NET, and would it be safer to just use my own instance of JsonSerializerSettings instead?

I think there could be some ramifications if you change GlobalSettings, because it affects to almost any serialization in Ext.NET. So, if you change some setting you should be absolutely sure that you need to change it globally.

Besides JSON.GlobalSettings, there is JSON.RequestSettings that affects on the current request only. I would recommend to consider using of that rather than GlobalSettings.

[anup]

Daniil,

Thanks for the detailed response. Very helpful.

Thanks for confirming that my general approach is the way to go.

I will share results when I get a chance. Hopefully if I get a good run today I may be able to do so by end of the day or tomorrow...

Regarding your method, InjectToString, that looks good (in addition to passing the comma parameter which I missed). About the method name, I am happy to defer to you and your team as to what is the best/appropriate name.

About this part:

if (this.Data.IsEmpty() && this.Success && this.Message.IsEmpty())
{
    return "";
}

Good point you make! I suppose, that if condition could be refactored into a protected virtual method, something like

protected virtual bool HasResponseData()
{
    return !this.Data.IsEmpty() || !this.Success || !this.Message.IsEmpty()
}

And then replace the original code with

if (!this.HasResponseData())
{
    return "";
}

That way, I can override that HasResponseData as needed. (In my particular case I don't think I need to change the logic, but it is perhaps useful if the class is to be inherited for other scenarios.)

(In my example, I inverted the logic, as I think it reads more logically. However, that is my opinion only and you could certainly do it the other way if you want.)

Hope that helps, and answers all your questions :)

[anup]

Oh, forgot to add - thanks for the note about JSON.RequestSettings. I will look into that, too. Thanks!

[Daniil]

I also thought about something like the HasResponseData method. So, we think together in the same way, it is definitely a way to.

(In my example, I inverted the logic, as I think it reads more logically. However, that is my opinion only and you could certainly do it the other way if you want.)

I agree, in the new context of the HasResponseData() method it looks more logical. I just changed .IsEmpty() to .IsNotEmpty().

Finally, I have committed the changes in the revision #6146. You can review.

Thank you for the suggestions!

[anup]

Works great. Thanks for the quick turnaround.

One follow up about JSON.RequestSettings.

In my code I was doing something like this:

            JSON.RequestSettings = JSON.CopyCurrentSettings;
            JSON.RequestSettings.NullValueHandling = NullValueHandling.Ignore;
            JSON.Converters.Add(new MyCustomConverter());

What I found happening on each request was that the Converters list just grew. I think that is because CopyCurrentSettings can copy from the global settings, and the list that is copied is (maybe understandably) a shallow copy. So when I am adding my custom converter this way it is really adding to the global defaults...

So, looking at the JSON class, I can see a Converters property but it is also static, so I don't think I can just get a new list of converters for this request and add mine to it, as I would again just be adding to a global instance.

So, what is the best way to add my converter for this particular request while keeping your defaults in place and not affecting those defaults for other requests and uses of JSON serialization? (I'd like to avoid doing a JSON.Converters.Contains because I'll have to keep hold of an instance somewhere to compare against and I'd like to also avoid doing a Contains on a List<T> as it gets more expensive the more converters there are. Of course, in the grand scheme of things the list is small and alternatively I can just keep track of a flag etc, but it would be nice to avoid all this if possible?)

The only way I can see reliably is to copy/paste your code for your base converters list when I am using my converter.

I suppose another way might in global.asax to add it to the global converters list at start up so it is only added once, even though it is then there for all other uses of serialization where it is not needed.

(Also I am not sure the Converters property is thread safe? For example, if, in my above code I set the converters to null before adding my own - which would bring in all your other defaults, there is a risk that multiple requests in parallel might step over each other as there isn't a lock around that code?)

[anup]

I was probably staring at this too long. Going home and having a break helped freshen the mind! I realized JSON.Serialize has an overload that takes convertors, so I used it in this way:

JSON.Serialize(Data, new List<JsonConverter>(JSON.RequestSettings.Converters) { new MyConverter() });

This builds a new list based on your Converter, adding my own convertor for this particular scenario. I think this thread could be closed now.

Thanks for your help. (Not got round to comparing ASHX/MVC/Api yet, but will let you know when I do!)

[Daniil]

(Also I am not sure the Converters property is thread safe? For example, if, in my above code I set the converters to null before adding my own - which would bring in all your other defaults, there is a risk that multiple requests in parallel might step over each other as there isn't a lock around that code?)

Thank you for the question! We are going to add a lock.

Created an Issue.
https://github.com/extnet/Ext.NET/issues/815