FG Browser with search [CoreRPG, 5e, SWD, other]

[sirnoobsauce]

I did a little testing to see (at least in my compute environment) what the impact of the various settings are on this version. The first thing to note is that I am definitely going to remove the index caching, so leave the "Persist Campaign Index" setting off for now until I remove that. Too much potential there to cause issues with slow networks / slow disks (when I moved the campaign data directory to a platter drive... forget about it).

As for the performance of the other options, I did A/B testing for all the settings. The option to exclude reference materials seems to have no meaningful impact on performance in my tests, so I will probably remove that option since it doesn't seem to hurt anything. The other 3 do all have meaningful impacts on the performance. Disabling module indexing will offer the greatest speedup, but of course means that you can't search records from modules. The "Basic Indexing" option also greatly speeds up indexing, at the cost of not being as thorough. And disabling formatted text indexing offers a modest boost to indexing performance.

The times in this table are for a test campaign using 5e with the standard 5e PG and DMG and the 3 SRD modules (Bestiary, magic items, data). Obviously these performance data are specific to my system, so you could have slower/faster indexing on your computer. But theoretically the % change row-to-row should be vaguely similar. Note that this is how long it takes to build the index at campaign load, or when clicking the "Rebuilt search index" button. Modifications to records only have to re-index the modified record, and that only takes milliseconds, regardless of the options:

Basic Indexing	Index Formatted Text	Index Modules	Average Load Time
Off	On	On	3.65s
Off	Off	On	1.75s
On	N/A	On	0.95s
Off	On	Off	0.05s
Off	Off	Off	0.04s
On	N/A	Off	0.02s

Let me know if that is consistent with what you are seeing, or if you are having a different experience with the indexing behavior on the new 1.1.1 version.

[spencerg]

I will definitely test later today. Got your messages. Thanks for your hard work on this!

[Mattkilt]

I loaded this into my system, and when it is enabled none of my games will load at all. it just locks up at the title page after I tell it to load the campaign.

[scoot138]

Spencer was in a campaign where I was running FGBrowser, my CampaignRegistry.lua was at 40mb, i have nvme m2 drives, 32gb ram, i5-9600 @3.7GHz. I was getting intermittent hangup. I unloaded FGbrowser, and all was well. I had toyed around with the settings enough to get the registry to that size. I find that I can go from about 2MB to 500MB of data in the registry depending on what modules I have loaded when I rebuild the index.

Currently set my options to basic index and module index on, the rest off. Its been 4 minutes and still locked up. This was starting out with an empty CampaignRegistry.lua

To be fair, I am testing against A LOT of modules. Prior to this release, it would take a minute or two.

[spencerg]

So far my experience with todays new release has been stable.

It obviously will take a good 3-5 mins to load up the campaign - but I really I dont have any issue with that - the benefits FAR outweigh a slow load-up.

Since updating, I've had zero weird hang up's in the last 2 hours and that ive been hammering it, building maps.

Image of settings attached. I will let you know If I have any issues.

Turning OFF Persist index = slow loadup always, but no stability issues yet that I can tell . search is blazing fast now, even more-so than before.

What I noticed prior to the option to disable it writing to the campaign registry was very odd.

I'd be doing nothing, not changing anything, just reading...or tabbing out too the browser. I'd come back in, and it would be frozen (milky overlay) -- give it a few, and it would unstick.

It was like it was just struggling at random.

Mostly likely because FG is polling and periodically backing things up, and it was chocked on the index data.

Image attached.
Screenshot 2023-04-07 125950.jpg

[scoot138]

Ran a new search with Basic and modules on the rest off, I disabled all non world/campaign/bestiary information modules and the rebuild took 24 seconds.
I recall, when persist was default enabled, there were modifiers in the CampaignRegistry.lua for what modules were indexed, I liked that, having the ability to set false to the ones I don't want indexed, would be a great feature.
When bringing up a client, it takes an extremely long time to fully load up the background decal, player selection, etc, this being on a client on the same 1gb wired, same switch, internal network, cannot imagine how long it will take across the WAN, with multiple clients pulling data. This occurred regardless of the type of index I created.
In all, this tool is in my mind is the next best thing to sliced bread and butter in a tub, even if there are still bugs.
I am happy to test out anything you need testing performed on.

[sirnoobsauce]

Thanks for all the feedback. I don't understand why my index rebuilds are soo much faster, I have fewer modules loaded, but the difference between 3 seconds and many minutes is odd... I will try to do some testing and see if there is some secondary issue happening.

Either way, I think based on the feedback I will set it to default to basic indexing with no modules, to ensure that the OOB experience won't prevent people from loading their campaigns, and then add an interface to select modules to index. I will take a look at that later today when I have some time.

[Ludd_G]

Hi,

One thing I've just noticed is that with this extension loaded the same campaign was using over 4 gig of extra RAM, pushing my RAM usage into the 93+ per cent, a level which, on my old 8 gig of RAM laptop, would've caused these type of slow downs and freezing. Now with my newer 16 gig of RAM laptop, my RAM usage during a session, with all t'other stuff going on (zoom, edge etc. etc.) is usually in the 60%, at which point enabling this extension pushes it to the mid 90%. This may not be the most recent updated version though, I haven't run 'check for updates' today yet.

Cheers,

Simon

[sirnoobsauce]

Hi,

One thing I've just noticed is that with this extension loaded the same campaign was using over 4 gig of extra RAM, pushing my RAM usage into the 93+ per cent, a level which, on my old 8 gig of RAM laptop, would've caused these type of slow downs and freezing. Now with my newer 16 gig of RAM laptop, my RAM usage during a session, with all t'other stuff going on (zoom, edge etc. etc.) is usually in the 60%, at which point enabling this extension pushes it to the mid 90%. This may not be the most recent updated version though, I haven't run 'check for updates' today yet.

Cheers,

Simon

Yeah I only really tested the indexing on my PC, and I have a ton of RAM... The changes I'm doing now er on the side of making the index as light as possible by default, and then users can turn on indexing per-module and get it set in a way that works for them.

The FG scripting system wasn't intended for this sort of use, so it's a bit of a balancing act to get it working well.

[Ludd_G]

No worries, I'd rather have people trying to push what FGU can do, than have it go stale. The competition from other VTTs is going to get intense going forwards.

Cheers,

Simon