Memory hole in chat output?

[Valyar]

Owners of SSDs might not like GB of data (8.5 in my last session) being written to their drive for single sessions.

And memory leaks are always bad, because they cause unpredictable behavior in an application, like instabilities and slowdowns over time, the latter of which FGU very much suffers from.

I can't agree more here. One of the reasons I didn't use FGU during beta was the memory issue and the explosion caused in the SWAP file in addition to the memory. And I am with NVMe, definitely don't like retarded memory operations that cause useless overwrites on the precious device.

[Weissrolf]

Swapfile size is not equal to swapfile usage, though. So if my 100 gb swapfile is consists only of "reserved" space then it uses NTFS sparse files.

A sparse file has an attribute that causes the I/O subsystem to allocate only meaningful (nonzero) data. Nonzero data is allocated on disk, and non-meaningful data (large strings of data composed of zeros) is not. When a sparse file is read, allocated data is returned as it was stored; non-allocated data is returned, by default, as zeros.

I will check this and report back.

[Valyar]

The initial FGU versions caused very significant swap size and usage because of the memory leaks and else Now is much better, but your findings show still things to be improved.

[Weissrolf]

Unfortunately the page-file is written to and only released once FGU is closed, which can take a long time when the page-file has grown large. This means wear and tear on NVMe/SSD and multiple concurrent writes to log-files + page-file can slow down HDD access. Peak usage of 98% was reached when my page-file was about 100 GB (!) in size.



The main issue I am seeing is that FGU keeps allocating virtual memory, but only seldomly and reluctantly releases any when I fill the chat. It does free up private memory periodically, but it does not deallocate virtual memory for the same objects.

I also noticed memory not being released for images that have to be reloaded from disk anyway.

For example:

- Started an empty PF2 campaign, no modules, no extensions. 507 mb Virtual memory, 299 mb Working Set Private.



- Loaded a single large 300 mb JPG image that decodes to 1385 mb uncompressed bitmap. +8000 mb Virtual (5.8x 1385 mb), + 6057 mb WS Private (4.4x 1385 mb). That's a lot of memory usage for a much smaller file. For comparison, Photoshop only needs +3687 mb Virtual and +1859 mb WS Private to load the same image. It peaks higher while decoding the JPG, but quickly cleans up after itself once the image is displayed, while FGU apparently does not.



- FGU keeps images in memory for several minutes when you close them. This allows to re-open the image without delay. Only when an image is not re-opened for some time it needs to be reloaded from disk. FGU does not necessarily release memory though, despite the image having to be reloaded from disk with accompanying delay. This screenshot is 15 minutes after the image had been closed, some Virtual was released, but no WS Private.



When I then reopened the same image again I saw Virtual drop down under 5000 mb and WS Private drop down to under 3500 mb for a a few seconds and then climb up again (screenshot came a second late). At the same time Virtual increased again. So memory for the already unloaded image was not released (at least in part) by FGU until the same image was reloaded from disk anyway. Instead it should have been released minutes earlier.



The good news is that after reloading the image twice memory usage did not increase over loading it for the first time. So at least upon reloading memory is properly deallocated.

[Weissrolf]

FG Classic after 1 mio. /die rolls:

[Weissrolf]

Loading a single test image (500 mb decoded) into an empty campaign. Classic even started off using more memory before the image was loaded, but still came out very much on top once both loaded the single image.

Classic:

Unity:

[Weissrolf]

Thanks. This is helpful information.

Is this looked into? Do you need any more information? Should I stop watching this thread which became kind of a monologue?

[ddavison]

This is enough for us to look into.

[Weissrolf]

The memory leak of FGU's chat still does not seem to be solved:



Even returning to the Lobby does not release memory, only restarting FGU does.

Additionally FGU becomes sluggish when the chat window is (ab)used too much. For demonstration - as in reproduce an issue - I copied about 7000 kb text into the chat input = 100% CPU load and huge memory increase (multiple gigabytes). Then I hit enter = 100% CPU usage and more memory increase (less than before).

The text pasted was from FGU's chatlog: 145400 lines of dice rolls "<font color="##660066">GM: </font> [d20 = 14]<br />". FGU crashed after this number of rolls (using the Insta-Dice extension), but it did not crash when pasting the text in.

Afterwards all chat related operations became extremely sluggish, while the rest of the program remained usable. So opening a character sheet and double-clicking for a roll was fast, but the output of the roll's result took several seconds. Typing in the chat input was affected, too.



At the end I held down the Delete key.

[Weissrolf]

On a side-note: I did D20 rolls via /roll instead of using the Insta-Dice extension and FGU crashed after 142127 rolls again. I saw three crashes around that mark now, two using the extension and one without extension.

One more thing I noticed, while memory consumption kept creeping up over 10 gb (50 gb swapfile) about 1-2 minutes before crashing FGU managed to drop down to below 2 gb all of a sudden during a input pause and then ramped up consumption again starting from that lower value.