[CLOSED] Reload buffered grid with scroll position and row selection restore

  1. #1

    [CLOSED] Reload buffered grid with scroll position and row selection restore

    Hello support team,
    what I want to achieve is to scroll through the buffered grid to any position, reload the grid, and if any rows have been selected, re-select them and focus that selection. Here is a simplified scenario:

    @using Ext.Net;
    @using Ext.Net.MVC;
    
    @{
        ViewBag.Title = "Grid with buffered store";
        Layout = null;
    
        var X = Html.X();
    }
    
    <!DOCTYPE html>
    
    <html>
    <head>
        <title>Ext.NET MVC Test Case</title>
    
        <script type="text/javascript">
    
            var gridInit = function (grid) {
                var store = grid.getStore(),
                    view = grid.getView();
    
                // Focus on the first row after the initial store load:
                store.on({
                    load: {
                        fn: function (records, operation, success) {
                            if (store.getCount() > 0) grid.getSelectionModel().select(0);
                            view.focus();
                        },
                        scope: this,
                        single: true
                    },
                });
            }
    
            var getSelection = function (grid) {
                var s = grid.getSelectionModel().getSelection();
    
                s.currentPage = grid.getStore().currentPage;
    
                return s
            }
    
            var restoreSelection = function (grid, selection) {
                var s = [];
    
                if (selection.length > 0) {
                    grid.getView().restoreScrollState();
    
                    for (var i = 0; i < selection.length; i++) {
                        console.log("FIND_ID", selection[i].get("threadid"));
                        var record = grid.getStore().findRecord("threadid", selection[i].get("threadid"));
                        console.log("RECORD", record);
                        if (!Ext.isEmpty(record)) {
                            s.push(record);
                        }
                    }
    
                    grid.getSelectionModel().select(s);
                }
    
                return s;
            }
    
            var gridReload = function (grid) {
                var store = grid.getStore(),
                    view = grid.getView(),
                    selection = getSelection(grid);
    
                view.preserveScrollOnReload = true;
    
                //store.reload(function (records, operation, success) { console.log("NEVER_HAPPEN"); });
                store.reload({
                    callback: function (records, operation, success) {
                        console.log("RELOAD_CALLBACK", records[0].data);
                        s = restoreSelection(grid, selection);
                        if (s.length > 0) view.focusRow(s[s.length - 1], 10);
                    }
                });
            }
    
            var gotoLine = function (grid, row) {
                grid.view.bufferedRenderer.scrollTo(row - 1, true);
            };
    
        </script>
    </head>
    
    <body>
        @X.ResourceManager()
    
        @X.DisplayField().ID("version").ReadOnly(true).Margin(10).Width(200)
        
        @(
            X.GridPanel()
                .Title("Buffered Grid (6678 records)")
                .Margin(10)
                .Width(600)
                .Height(300)
                .MultiSelect(true)
                .Listeners(l => l.AfterRender.Handler = "gridInit(this)")
                .Store(X.Store()
                    .AutoLoad(true)
                    .Buffered(true)
                    .PageSize(50)
                    .LeadingBufferZone(10)
                    .TrailingBufferZone(20)
                    .Model(X.Model()
                        .IDProperty("threadid")
                        .Fields(
                            new ModelField("threadid"),
                            new ModelField("forumid"),
                            new ModelField("title"),
                            new ModelField("username"),
                            new ModelField("replycount", ModelFieldType.Int)
                        )
                    )
                    .Proxy(X.JsonPProxy()
                        .Url("https://www.sencha.com/forum/remote_topics/index.php")
                        .Reader(X.JsonReader()
                            .RootProperty("topics")
                            .TotalProperty("totalCount")
                        )
                    )
                )
                .ColumnModel(
                    X.RowNumbererColumn().Width(40),
                    X.Column().Text("Id").DataIndex("forumid").Width(30),
                    X.Column().Text("Title").DataIndex("title").Flex(1),
                    X.Column().Text("Username").DataIndex("username").Width(120),
                    X.Column().Text("#").DataIndex("replycount").Width(30)
                )
                .SelectionModel(X.RowSelectionModel().PruneRemoved(false))
                .TopBar(X.Toolbar()
                    .Items(
                        X.Button()
                            .Text("Reload")
                            .Listeners(l => l.Click.Handler = "gridReload(this.up('grid'))")
                    )
                )
                .BottomBar(X.Toolbar()
                    .Items(
                        X.NumberField()
                            .FieldLabel("Jump to row")
                            .MinValue(1)
                            .MaxValue(5000)
                            .AllowDecimals(false),
                        X.Button()
                            .Text("Go")
                            .Listeners(l => l.Click.Handler = "gotoLine(this.up('grid'), this.up('toolbar').down('numberfield').getValue())")
                    )
                )
        )
    </body>
    </html>
    
    <script type="text/javascript">
        Ext.onReady(function () {
            Ext.getCmp("version").setValue("Ext JS " + Ext.getVersion().version + " / " + "Ext.NET " + Ext.net.Version);
        });
    </script>
    The problem is that the behavior is completely unpredictable. Sometimes I can scroll down and up the grid, select one or more rows, reload the grid ... and everything works fine. But usually, I get an error right after the first reload (no scrolling, no selection, just open the application and click on the reload button), sometimes it works well for the first few reloads, and then an error occurs:

    Click image for larger version. 

Name:	console_log.jpg 
Views:	5 
Size:	86.4 KB 
ID:	25485

    When I jump to row 1000 and then reload the grid, the result is just empty grid.

    It is also confusing to see records passed to a reload callback, but the records cannot be found in the store using findRecord and it still crashes.

    This callback will also never be executed, although according to the documentation it should:

    store.reload(function(records, operation, success) {
        console.log('loaded records');
    });
    Such mistakes are sometimes difficult to describe, but hopefuly my intention is clear and the problems demonstrable.

    Any help is appreciated. Thank you for your assistance.

    Ext JS 7.3.1.27 / Ext.NET 5.3.0 / Chrome 87.0.4280.88

    Kind regards
    Dan
    Last edited by fabricio.murta; Jan 13, 2021 at 3:58 PM.
  2. #2
    Hello Dan!

    You can't reliably find which record you want to select because you have remote data. If you want to reliably scroll back to the record anywhere it may be you should load the full data. The grid can still be buffered, but the store cannot.

    You noticed how you get RECORD null when you scroll a selected entry out of the buffered zone and click "reload", right? That's because the entry is simply not in the store so it can never be selected back.

    You should be safe to keep tens of thousands records locally (as store data) if you keep the grid panel buffered.

    Solutions to that would involve either:
    - remember the row number the selected record was before reload and bring that row back into view on refresh (regardless of what's in there);
    - drop the store's paging and let the grid panel buffered subsystem work on its own; loading all data and still rendering (the heavy part) on demand.
    - remember the selected record -and- row. then once the person clicks "reload", scroll back to the page the selection belonged -and- check if then the record is present. There would still a chance the record would no longer be brought by that chunk of data.

    Hope this helps!
  3. #3
    Hello Fabrício,
    thank you for the quick reply. I apologize for a maybe too complicated test case and perhaps not a clear enough explanation of the problem.

    I understand your hints, but I can't fully agree with the claim that the entry isn't in store. It's there, just "later", and that's also why I got RECORD null irregularly. Please see the simplified test case:

    @using Ext.Net;
    @using Ext.Net.MVC;
    
    @{
        ViewBag.Title = "Grid with buffered store";
        Layout = null;
    
        var X = Html.X();
    }
    
    <!DOCTYPE html>
    
    <html>
    <head>
        <title>Ext.NET MVC Test Case</title>
    
        <script type="text/javascript">
    
            var gridReload = function (grid) {
                var selection = grid.getSelectionModel().getSelection();
    
                if (selection.length == 0) return;
    
                grid.getView().preserveScrollOnReload = true;
    
                //grid.getStore().reload(function (records, operation, success) { console.log("NEVER_HAPPEN"); }); => does not work according to the documentation
                grid.getStore().reload({
                    callback: function (records, operation, success) {
                        console.log("RELOAD (#" + grid.getStore().indexOf(records[0]) + ")", records[0].data);
                        var record = grid.getStore().findRecord("threadid", selection[0].get("threadid"));
                        console.log("FIND (" + selection[0].get("threadid") + ")", record);
                        Ext.defer(function () {
                            var record = grid.getStore().findRecord("threadid", selection[0].get("threadid"))
                            console.log("FIND_LATER (" + selection[0].get("threadid") + ")", record);
                            if (record) grid.getView().focusRow(record, 10);
                        }, 1000, this);
                    }
                });
            }
    
            var gotoLine = function (grid, row) {
                grid.view.bufferedRenderer.scrollTo(row - 1, true);
            };
    
        </script>
    </head>
    
    <body>
        @X.ResourceManager()
    
        @X.DisplayField().ID("version").ReadOnly(true).Margin(10).Width(200)
        
        @(
            X.GridPanel()
                .Title("Buffered Grid (6678 records)")
                .Margin(10)
                .Width(600)
                .Height(300)
                .MultiSelect(true)
                .Store(X.Store()
                    .AutoLoad(true)
                    .Buffered(true)
                    .PageSize(50)
                    .LeadingBufferZone(10)
                    .TrailingBufferZone(20)
                    .Listeners(l => l.Prefetch.Handler = "Ext.defer(function () { console.log('PREF_LATER (#' + store.indexOf(records[0]) + ')', records[0]); }, 1000, this)")
                    .Model(X.Model()
                        .IDProperty("threadid")
                        .Fields(
                            new ModelField("threadid"),
                            new ModelField("forumid"),
                            new ModelField("title"),
                            new ModelField("username"),
                            new ModelField("replycount", ModelFieldType.Int)
                        )
                    )
                    .Proxy(X.JsonPProxy()
                        .Url("https://www.sencha.com/forum/remote_topics/index.php")
                        .Reader(X.JsonReader()
                            .RootProperty("topics")
                            .TotalProperty("totalCount")
                        )
                    )
                )
                .ColumnModel(
                    X.RowNumbererColumn().Width(40),
                    X.Column().Text("Id").DataIndex("forumid").Width(30),
                    X.Column().Text("Title").DataIndex("title").Flex(1),
                    X.Column().Text("Username").DataIndex("username").Width(120),
                    X.Column().Text("#").DataIndex("replycount").Width(30)
                )
                .SelectionModel(X.RowSelectionModel().PruneRemoved(false))
                .TopBar(X.Toolbar()
                    .Items(
                        X.Button()
                            .Text("Reload")
                            .Listeners(l => l.Click.Handler = "gridReload(this.up('grid'))")
                    )
                )
                .BottomBar(X.Toolbar()
                    .Items(
                        X.NumberField()
                            .FieldLabel("Jump to row")
                            .MinValue(1)
                            .MaxValue(5000)
                            .AllowDecimals(false),
                        X.Button()
                            .Text("Go")
                            .Listeners(l => l.Click.Handler = "gotoLine(this.up('grid'), this.up('toolbar').down('numberfield').getValue())")
                    )
                )
        )
    </body>
    </html>
    
    <script type="text/javascript">
        Ext.onReady(function () {
            Ext.getCmp("version").setValue("Ext JS " + Ext.getVersion().version + " / " + "Ext.NET " + Ext.net.Version);
        });
    </script>
    Please follow the steps below:
    1. Open the application.
    2. Scroll down somewhere in the middle of the list.
    3. Select any row you see.
    4. Click the Reload button.

    Network log:
    Click image for larger version. 

Name:	log1.png 
Views:	6 
Size:	42.3 KB 
ID:	25487

    How do I read this protocol:
    1. One Reload initiates several data source calls (301 status requests do not return any data).
    2. First, two pages from the beginning of the list are loaded.
    3. Config preserveScrollOnReload = true is then taken into account.
    4. A new data source calls are initiated, in this case for three pages around the last scroll position (sometimes it's just two pages and in a different order - this is unpredictable).

    Console log:
    Click image for larger version. 

Name:	log2.png 
Views:	6 
Size:	93.0 KB 
ID:	25488

    How do I read this protocol:
    1. Reload callback is triggered only for the first two pages of the list (twice for each one - why?) and not for the subsequent loading of pages around the selection.
    2. Prefetch triggers correctly as many times as the data source is called (and once for each call).
    3. When Prefetch is invoked, the repository is "still not loaded", so the searched record cannot be found.
    4. However, if the record search is delayed (in both Reload callback and Prefetch event), the selected record is found and can be focused.

    This scenario is suitable for testing purposes, but is not applicable to real applications because:
    1. I can't rely on any delayed search for records.
    2. I also don't know if all the pages are already loaded, so I can't be sure if
    - the record was not found because not all pages have been loaded into the store yet
    - the record was not found because it may have been deleted from the data source

    I expected the Reload method to reload the store using the last options passed to the method-load method, so it would load the pages around the selection directly. I also expected that loading pages provoked by preserveScrollOnReload = true would also be caught by the Reload callback. And the highest expectation was that the callback is called until Reload is actually fully executed and that it works well in conjunction with preserveScrollOnReload config.

    My question: is there any reliable way to capture the end of the Reload method and focus on the lastly selected row, or if it is not found, be sure it is because the record has been removed from the data store (or changed in a way causing it is already not in requested pages)?

    Please let me know if my conclusions are wrong and if it is possible to achieve my intention.

    Thank you for your help.

    Kind regards
    Dan
  4. #4
    Hello @dan!

    Quote Originally Posted by NewLink
    I understand your hints, but I can't fully agree with the claim that the entry isn't in store. It's there, just "later", and that's also why I got RECORD null irregularly.
    Before I get into your follow up, can you do a little test on the sample you provided first?

    1) add .ID("GridPanel1") to the grid panel.
    2) reload the examnple and open developer tools
    3) Enter App.GridPanel1.getAt(810). Should return undefined.
    4) Fill the 'Jump to Row' field with 810 and click "Go".
    5) Repeat the command of step (3). It will return the record.

    So, whenever step (3) is in effect (you try to check the selected record against the rows in the grid to scroll to it), you are not going to get anything so scroll would fail. You may even fetch the right page but sorting or changes to the record would make it not reliable. To address this you can simply fetch all data and let just the grid rendering process be buffered (which is by default).

    When you scroll far enough, the records are kept in memory but, in case you perform a reload/refresh, they won't come back until they are paged into view again. If you save the page once you save the selection, you can then pull that page (or scroll to the row it was in) in hopes the record would be pulled. But you'll probably have to handle the case when the record is no longer to be found.

    Let us know if this clarifies our last post. I see your new test case uses the same data pulling mechanism so I'll hold up until you respond this to check it out.

    p.s.: I actually adressed some issues in your original example before I noticed this limitation, so even if we use full data in your original example it probably will still needs some changes; I just need to settle on this matter before the rest can be handled, no matter how we fix it, it is not going to work due to the remote data. The scenario that will always break is: (1) select an entry (2) scroll away (3) reload
    Fabrício Murta
    Developer & Support Expert
  5. #5
    Hello Fabrício,
    if I try to get an entry at 810 without first jumping or scrolling there (and thus initializing the store load), it is clear to me that the result is undefined.

    You may even fetch the right page but sorting or changes to the record would make it not reliable. To address this you can simply fetch all data and let just the grid rendering process be buffered (which is by default).
    I know that even if I fetch the right page, sorting, filtering or record deletion can make the record impossible to find.

    I use a buffered store precisely because I don't want to load all the data. I just want to load the block required by scrolling position. If I read all the data first, then it doesn't matter if the grid is paged or buffered - all the data is transferred and all the traffic I wanted to avoid already took place.

    The scenario that will always break is: (1) select an entry (2) scroll away (3) reload.
    I know that and it's correct. If I select record 1000, then scroll to 5000 and click reload ... I expect the list to maintain its position and therefore remain around 5000. Record 1000 cannot be found and the selection can be destroyed.

    My scenario is quite simple:
    scroll down (pages around the current scroll position are loaded) -> select a row -> reload the store (thanks to preserveScrollOnReload config the list maintains a scroll position and store loads pages around this position) -> check if selected record can be found => YES: reselect, NO: do nothing.

    The whole point is, that Reload method isn't behaving the way I expected. This does not mean that it necessarily works badly. It may behave as designed, but then - for certain scenarios - it cannot be used. Please read my previous post. The problem is, that:

    1. Reload callback is not called for pages loaded thanks to preserveScrollOnReload config.

    2. I don't know if the Reload is already fully done or if there are still some pages waiting for a load.

    3. Record search must be postponed for a "moment" because Reload callback (and also Prefetch event) is invoked too soon.

    Thank you for assistance.

    Kind regards
    Dan
  6. #6
    Hello Dan!

    Trying to understand your point. Thanks for clarifying you are aware of the limitations involved.

    Quote Originally Posted by NewLink
    1. Reload callback is not called for pages loaded thanks to preserveScrollOnReload config.
    It does not trigger because the callback is only fired during requests made by the reload() method. The preserveScrollOnReload setting forces the view to scroll down back into position after the reload happens, thus the fact it scrolled down (like ordinary scrolling after the grid is loaded) does not trigger the reload event to, in turn, issue the callback.

    In the previous post you commended:

    Quote Originally Posted by NewLink
    1. Reload callback is triggered only for the first two pages of the list (twice for each one - why?) and not for the subsequent loading of pages around the selection.
    That's because Sencha is redirecting traffic from www.sencha.com/forum to forum.sencha.com since they moved the forums. Otherwise we'd be getting 404's instead of 301 in the first request as HTTP response.

    Back to your last post:

    Quote Originally Posted by NewLink
    2. I don't know if the Reload is already fully done or if there are still some pages waiting for a load.
    You are passing a callback to the reload() method. It is going to be triggered for every returned request the method initiated. This is not the same as binding a load event listener.

    Furthermore I believe there's a gap here between the reload and layout concepts. The store may be reloaded and ready but the layout and/or the grid view would still be pending refresh. You're setting preserveScrollOnReload in the View, right? The store has no scroller at all. But you are relying in the reload callback pertaining the store.

    This may sound too overcomplicated, but this allows, for instance, having a single store serving data to a combo box, a grid, and a chart, all at once. There's even situations where a store serves another store, in case two grids uses the same data with independent filters.

    So back to the main question in your last quote:
    - The callback is being triggered when the request made to reload the data is done, so the store is populated with that given page during its main load
    - It triggers twice, as you already noticed, because two pages are fetched at first
    - This would be a different story if you were relying in the store's load event. If not, then I'd say we may have a bug
    - If you rely in the load event you may rest assured the data is loaded in store once the event triggers.
    - Data being loaded in the store does not mean its display is refreshed in the view -- there are other events you should be looking at here

    In fact, the first step to address your issues would probably involve moving the callback you are using to an event in the view, maybe the refresh event.

    Quote Originally Posted by NewLink
    3. Record search must be postponed for a "moment" because Reload callback (and also Prefetch event) is invoked too soon.
    This makes sense and you should avoid "delayed postpones" as much as you can. Sometimes we just can't avoid them, but most times when we rely on delays, we are prone to random issues, say, when the client computer is under heavy load elsewhere during the page load.

    This makes sense again because, as much as the record may be in store, it may not have been rendered in the view, so you simply cannot still tell in which row it would fall. In most cases you should be able to find a proper event to bind the change to.

    Now to the last affirmation (within the last quote),

    Quote Originally Posted by NewLink
    Reload callback (and also Prefetch event) is invoked too soon.
    It is too soon for the view as a matter of dependency. The view cannot deliver its contents until the store has data ready for it to work with. So again, a sign that you may be relying on the wrong event. Or that event alone does not fulfill the end result you want.

    Another bit of useful information I believe is, whenever you see an event and are not sure it comes after or before itself, you may look at the docs whether they have a corresponding before one. Sometimes (like change in form fields), they may be agnostic, as the relevant data (previous and new values) are provided to the event anyway. Usually there are either a before- and the after- is the event itself. In other words, events usually are triggered as side effects of what they correspond to, unless explicitly otherwise.

    But there are exceptions and we once had complaints about the change event switching its "call point" between version updates. In this specific case, during a change event, one can't rely on own component's getValue() method while the event receives the previous and new values as parameters.

    As a bottomline (and maybe this resumes this whole big post), if you rely in Ext.View.Table's refresh event you'd be in a much better place to attain your goal. But I still needed some delay to the focusRow() call, although it became much more reliable. I'm afraid this could even need to span across events of different components or several different events in the same component.

    Well, hope this enlightens some of your inquiries. I am not sure I may have overlooked important points for you so I apologize beforehand if that's the case and kindly ask you to point what's left unanswered or needing further clarification.
    Fabrício Murta
    Developer & Support Expert
  7. #7
    Hello Fabrício,
    thank you for your quick response and for clarifying how the reload method works. Just one note ...
    Furthermore I believe there's a gap here between the reload and layout concepts. The store may be reloaded and ready but the layout and/or the grid view would still be pending refresh. You're setting preserveScrollOnReload in the View, right? The store has no scroller at all. But you are relying in the reload callback pertaining the store.
    What confuses me is that the findRecord method belongs to the store, not to the view. So even though there really is a gap between reload and layout and the view is still waiting to be updated, the record should be found in the store. Which is not the case and it can only be found after a "while" (my guess is after the View refresh).

    However, as you suggested, moving the code to the view refresh event seems to have solved the problem with findRecord and the scenario now works as desired. I want to believe in the accuracy of refresh trigger and that firing an event only once is exactly what is enough:

            var gridReload = function (grid) {
                var selection = grid.getSelectionModel().getSelection();
    
                if (selection.length == 0) return;
    
                grid.getView().preserveScrollOnReload = true;
    
                grid.getView().on({
                    refresh: {
                        fn: function (view) {
                            var record = view.grid.getStore().findRecord("threadid", selection[0].get("threadid"))
                            console.log("FIND_RECORD (#" + view.store.indexOf(record) + ":" + selection[0].get("threadid") + ")", record);
                            if (record) view.focusRow(record, 10);
                        },
                        scope: this,
                        single: true
                    },
                });
    
                grid.getStore().reload();
            }
    Thanks once again for the extensive and comprehensive answer. Ext.NET support is one of the best I've ever met in my professional life.

    Ticket can be closed.

    Kind regards
    Dan
  8. #8
    Hello again, Dan! Glad you could come down to a solution that worked for you.

    Quote Originally Posted by NewLink
    What confuses me is that the findRecord method belongs to the store, not to the view. So even though there really is a gap between reload and layout and the view is still waiting to be updated, the record should be found in the store. Which is not the case and it can only be found after a "while" (my guess is after the View refresh).
    Yes, findRecord() belongs to the store. Whenever a record is ready in the store this method should be able to find it.

    Yet now, the reload callback won't be the same than the reload event. It is expected the callback to be called right after the request has completed and data returned. That is, the reload() call completed, not necessary the store being fully reloaded, as it should be when the reload event is involved.

    For instance, the reload callback could be useful to treat received raw data before it is bound to the store as model instances (actual records).

    Please, give a read at the documentation entry, I'm sure you are going to understand what's up with this strange callback:

    - Ext.data.Store.reload(); which points to Ext.data.Store.load() regarding callback info.

    As stated there,

    A function which is called when the response arrives.
    See here, how it differs from the load event: Ext.data.Store's load event

    Hope this helps!
    Fabrício Murta
    Developer & Support Expert
  9. #9
    Hello Fabrício,
    just one last note... With the Prefetch event (load event you mentioned) the story is the same as with the reload callback: it is able to find the record only after a "while".

    It seems that the only usable event in this case is View refresh.

    Thank you very much for your assistance.

    Ticket can be closed.

    Kind regards
    Dan
  10. #10
    Hello Dan,

    Thanks for the feedback, and glad you could have it working in the end.
    Fabrício Murta
    Developer & Support Expert

Similar Threads

  1. Replies: 6
    Last Post: Feb 25, 2017, 4:51 AM
  2. Replies: 5
    Last Post: Apr 27, 2016, 1:43 AM
  3. [CLOSED] Error in buffered grid
    By RCM in forum 2.x Legacy Premium Help
    Replies: 5
    Last Post: Nov 25, 2013, 3:31 PM
  4. [CLOSED] Error in buffered grid
    By RCM in forum 2.x Legacy Premium Help
    Replies: 5
    Last Post: Nov 22, 2013, 7:43 PM
  5. [CLOSED] Buffered Grid with filtering and editing
    By jchau in forum 2.x Legacy Premium Help
    Replies: 1
    Last Post: Jan 13, 2013, 7:39 AM

Tags for this Thread

Posting Permissions