[OPEN] [#356] Gzipped extnet static resources not persistanced

Page 1 of 2 12 LastLast
  1. #1

    [OPEN] [#356] Gzipped extnet static resources not persistanced

    I guess this is more of a feature request, but it seems to me that the static ext.net resources call CompressionUtils.GZip on the static bytestream each time a resource is requested (when GZip="true"). It seems to me that it would be 20 times more efficient if the requested resources were already gzipped as a static file resource (part of build project) and transmitted directly instead of processed then transmitted. The second best option would allow the resources to persist to a writable physical location on the server the first time requested then transmitted. As extnet is already pretty heavy on the cpu given the number of transformations, this would significantly task the cpu less from a scalability standpoint.

    I realize once-each client has cached the resource, it doesn't need to repeatedly process and transfer. I also verified that the processed output is server-side cached to prevent the zip being reprocessed again. However, as these fragments can be fairly large, I'm finding that my server is blowing away these cached resources frequently due to restricted access to memory. In other words, the server-side cache is more-often rendered null and void and has to reprocess again-and-again. Moreover, every worker thread keeps its own cached reference in memory. A different cache-provider might help but also add its own performance weight. This issue hasn't been one on my development environment, but I noticed the issue when I ran a trace on my production server. The short term answer would be make more memory available on my production environment, but the BEST solution of all worlds would be my original request for the resources to be static persisted files or resources like the other static ext.net content.
    Last edited by Daniil; Oct 15, 2013 at 12:55 PM. Reason: [OPEN] [#356]
  2. #2
    I also looked into the fact that all resources are cached, compressed or not, which also may be part of the reason the server-cache is so eager to release memory.

    Also note the following code in ResourceHandler.cs can be improved.
                    try
                    {
                        this.sm = new ResourceManager();
                        this.compress = CompressionUtils.IsGZipSupported && this.sm.GZip;
    
    
                        this.SetWebResourceName(file);
    
    
                        this.stream = this.GetType().Assembly.GetManifestResourceStream(this.webResource);
                        string ext = this.webResource.RightOfRightmostOf('.');
                        this.compress = this.compress && !this.IsImage(ext);
    
    
                        switch (ext)
                        {
                            case "js":
                                this.WriteFile("text/javascript");
                                break;
                            case "css":
                                this.WriteFile("text/css");
                                break;
                            case "swf":
                                this.compress = false;
                                this.WriteImage("application/x-shockwave-flash");
                                break;
                            case "gif":
                                this.WriteImage("image/gif");
                                break;
                            case "png":
                                this.WriteImage("image/png");
                                break;
                            case "jpg":
                            case "jpeg":
                                this.WriteImage("image/jpg");
                                break;
                        }
                    }
                    catch (Exception e)
                    {
                        string s = this.IsDebugging ? e.ToString() : e.Message;
                        context.Response.StatusDescription = s.Substring(0, Math.Min(s.Length, 512));
                        this.context.Response.Redirect(Page.ClientScript.GetWebResourceUrl(this.sm.GetType(), this.webResource));
                    }
                    finally
                    {
                        if (this.stream != null)
                        {
                            this.stream.Close();
                        }
                    }
                }
    You're getting the Resource byte stream every time a resource is requested, even though you test later to see if you already have it in memory.
    Instead you should call this.GetCache() before this entire section of code instead of inside of each WriteFile/WriteImage. You can pass the content instead as an additional parameter to WriteFile/WriteImage.

    Revised:
                    try 
                    {
                        this.sm = new ResourceManager();
                        this.SetWebResourceName( file );
                        string ext = this.webResource.RightOfRightmostOf( '.' );
                        this.compress = this.sm.GZip && CompressionUtils.IsGZipSupported; // changed order to prevent IsGZipSupported test unless required
                        this.compress = this.compress && ( !this.IsImage( ext ) && ext != "swf" ); // moved swf test here to load correct cache ref
    
    
                        // Load stream only if not already available in server-cache
                        this.output = this.GetCache();
                        if( this.output == null )
                            this.stream = this.GetType().Assembly.GetManifestResourceStream( this.webResource );
    
    
                        switch( ext ) 
                        {
                        case "js":
                            this.WriteFile( "text/javascript" );
                            break;
                        case "css":
                            this.WriteFile( "text/css" );
                            break;
                        case "swf":
                            this.WriteImage( "application/x-shockwave-flash" );
                            break;
                        case "gif":
                            this.WriteImage( "image/gif"  );
                            break;
                        case "png":
                            this.WriteImage( "image/png"  );
                            break;
                        case "jpg":
                        case "jpeg":
                            this.WriteImage( "image/jpg"  );
                            break;
                        }
                    }
                    catch( Exception e ) 
                    {
                        string s = this.IsDebugging ? e.ToString() : e.Message;
                        context.Response.StatusDescription = s.Substring( 0, Math.Min( s.Length, 512 ) );
                        this.context.Response.Redirect( Page.ClientScript.GetWebResourceUrl( this.sm.GetType(), this.webResource ) );
                    }
                    finally 
                    {
                        if( this.stream != null ) 
                        {
                            this.stream.Close();
                        }
                    }
                }
    I also fixed some other small ordering issues.


    Also, had to pull GetCache call from WriteImage and WriteFile:
            private void WriteFile(string responseType)
            {
    
    
                if (this.output != null)
                {
                    this.Send(this.output, responseType);
                    
                    return;
                }
    
    
                this.sb = new StringBuilder(4096);
                ...
    
    
            private void WriteImage(string responseType)
            {
    
    
                if (this.output == null)
                {
                    this.length = Convert.ToInt32(this.stream.Length);
                    this.output = new Byte[this.length];
                    this.stream.Read(this.output, 0, this.length);
    
    
                    this.SetCache(this.output);
                }
    
    
                this.Send(this.output, responseType);
            }
    Last edited by michaeld; Sep 23, 2013 at 3:43 AM.
  3. #3
    ^^^
    Sorry edited till I got the bugs out.
  4. #4
    Hi @michaeld,

    Thank you for the suggestion! We will review.
  5. #5
    Quote Originally Posted by Daniil View Post
    Thank you for the suggestion! We will review.
    I haven't seen this hit the trunk yet. Might it make 2.3? It would be nice not having to load the file resources on every load. It's not a major issue, but some of the streams are very large and can benefit from this edit by a a few tens of milliseconds. I've been testing my branch of this edit on production and dev for the past week; it working reliably.
    Last edited by michaeld; Oct 08, 2013 at 1:12 AM.
  6. #6
    I don't think this revision has been made in Svn. For some reason I missed this thread when originally posted.

    We're going to try and implement this immediately and get into the 2.3.0 release.

    It seems to me that it would be 20 times more efficient if the requested resources were already gzipped as a static file resource (part of build project) and transmitted directly instead of processed then transmitted.
    This never crossed my mind before, but it makes perfect sense. The pre-compressed files will probably not make it into 2.3.0, but we will investigate.
    Geoffrey McGill
    Founder
  7. #7
    Quote Originally Posted by geoffrey.mcgill View Post
    I don't think this revision has been made in Svn. For some reason I missed this thread when originally posted.
    We're going to try and implement this immediately and get into the 2.3.0 release.
    Cool. I observed the latest SVN updates were suggesting you were probably days away from that posting which is why I have been reminding you guys on some of the topics like this (oh and the Ext.Net.Utilities stringutils.cs).

    Quote Originally Posted by geoffrey.mcgill View Post
    This never crossed my mind before, but it makes perfect sense. The pre-compressed files will probably not make it into 2.3.0, but we will investigate.
    Your current implementation is pretty sufficient. It creates the gzipped version to cache only once which puts you already ahead of most competitive libraries which barely do a fraction. If it weren't for caches being independent per worker thread (garden) and instance (farm), I wouldn't even have been worth mention. Though, I would understand if you didn't have the pre-compressed versions in resources on any but release minified versions given the number of permutations of debug and dev, but since you already have a good handle on builder extensions in VS, it might not be that difficult to do them all.
  8. #8
    Geoffrey, on the topic of subtle improvements in resource/performance deployments, you might also want to check in on this thread too.
    http://forums.ext.net/showthread.php?25367-SSL-CDN/
  9. #9
    It should be addressed in the SVN trunk revision #5391. Could you update and give it a try, please?
  10. #10
    I observed vladimir changed the section to prevent reload of resources. I'll give it a try once I've fixed my extjs cdn code.
Page 1 of 2 12 LastLast

Similar Threads

  1. Replies: 7
    Last Post: Oct 02, 2012, 9:05 PM
  2. [CLOSED] Runtime client error in /extnet/extnet-core-js/ext.axd?v=0
    By vadym.f in forum 1.x Legacy Premium Help
    Replies: 7
    Last Post: Aug 26, 2012, 8:13 AM
  3. updating to ExtNet 1.0 from 0.8.2
    By unaltro2 in forum 1.x Help
    Replies: 6
    Last Post: Jun 03, 2011, 6:14 PM
  4. Replies: 13
    Last Post: May 16, 2011, 2:26 PM
  5. [CLOSED] bypassing authentication for EXTJS EXTNET resources
    By webclouder in forum 1.x Legacy Premium Help
    Replies: 1
    Last Post: Apr 22, 2011, 3:57 PM

Posting Permissions