Page 1 of 4 123 ... Last
  1. #1

    4E Module Generator - Offline compendium > Fantasy Grounds

    Hello everyone,

    This post is about a set of scripts that will allow you to generate Fantasy Grounds modules from the Offline Compendium data.


    What is the Offline Compendium?
    For those who never heard about the Offline Compendium, it was a way to access the Online 4E Compendium and with it, all official 4e content ever created without internet access.
    It was great for us since we have a tradition of playing in a large farmhouse without wifi (+ it has way, way better research feature).
    Nowadays since the online compendium has been shut down by WotC, it became the only way to access all 4E data and many DM made the migration before the shutdown.
    Here is a video for more details https://www.youtube.com/watch?v=aNDze9Ok5fE).

    But you know, there is several compendium existing, how do I know this is the right module generator?
    Indeed, there is two compendium hosting 4e data these days. Or at least, two that I am aware of. These module-generating scripts will only work with the "Offline Compendium".
    If your compendium looks like this picture below, you are in the right place. If it's not, you might want to check my other thread here https://www.fantasygrounds.com/forum...read.php?60524.

    How does it work and which modules will I be able to create?
    The scripts I bring here will allow you to generate modules sufficient to launch any campaign of the fourth edition in Fantasy Grounds.
    We're talking about 5000+ NPCs for you DM and for your players of 9000+ Powers and 3000+ Feats ready to drag and drop in their character sheets.
    Moreover, this can also generate modules including all non-magical base items, gear ready to take and armors/weapons available to be customized into magic items.

    Installation & Usage

    Ok so now how can I make it work ?
    It should be easy, or I least I tried to make it easy.
    First things first you need to download and unzip the module generation scripts here : https://drive.google.com/file/d/1Obt...p8NuuK0mATvKq/

    And follow carefully the detailed steps written below.

    1. Copy the armor, feat, implement, item, monster, power and weapon folders from "PathYoYour4eCompendium/4e_database_files/".

    2. Then paste them inside the sources folder of the generator located at "4E Module Generator/sources/".

    (Or you could go the barbarian way here and copy all the folders from within the "4e_database_files" of your 4E Offline Compendium into the sources folder).

    3. Execute the "4E Module Generator/_run_all.bat" file (double clic should be fine). Then six executable will be launched and six consoles should appear (once per modules generated) with info of the parsing happening.

    4. Each consoles will ask you to press enter to close them once finished.

    5. Finally you will be able to find the generated modules in the "4E Module Generator/export" folder in each specific repositories (i.e. "4e_NPC.mod" in "4E Module Generator/export/npc")

    6. Just copy these 6 modules from their export folders into your FG modules folder (default should be ".../Fantasy Gounds/Data/modules") and load them in your campain!

    7. Don't forget to give access to your players to the basic items / feats / powers modules to ease character creation.

    And voilà you should be done and ready to start creating colorfull characters and dreadfull encounters!

    Last notes

    I'm curious, how does it works?
    Remember when you copied the folders at step one? Each of these folders cointain `dataX.js` files with X beeing a number always ranging from 0 to 19.
    These files are the source information used by the Offline Compendium and are in fact the only thing you need for this converter to work.
    Each individuals scripts (*.py files compiled as *.exe files) are called by the `_run_all.bat` executable to parse text (json objects) from the Offline Compendium into Fantasy Grounds Compatible data which is then exported (in xml) and bundled in a module (mod).

    Since it's only a handfull of python scripts (not compiled language), anyone can open these to see more how it works.
    Or even change them into other generators for those of you with enough motivation (licence availaple in COPYING.txt).
    Yet it's far from state of the art code since it was done quickly and also part of the first experimentation of a friend with coding. But hey, it works! (or should, or at least it worked here? Tell me if it does not!)

    Any known issues?
    - Sublass of items are swaped with Class and therfore automation for ranged weapons and armors does not work properly, it's an easy fix for a future update.
    - Recharge for monsters skills are not tracked, this is due to the source data not storing this info at the same place every time. An upgrade of the npc script should put things back in the right order.

    I sincerely hope this can help many of you the way it helped me setting up my last campain!
    Last edited by VegaFontana; July 31st, 2020 at 16:25.

  2. #2
    Oh dear but I'm missing all my duty, I just realized that I didn't test the tool on another computer than mine!

    I'll do that in the upcoming days to make ensure there is no problems with it. In the meantime I'll be around if any issues are encoutered

  3. #3
    Like others in the community, your work on this is appreciated!

    I've tried it and ran into an issue. For each of the 6 modules, it gives this:

    "python.exe System Error

    The code execution cannot proceed because python37.dll was not found. Reinstalling the program may fix this problem."

    I had a copy of python37.dll and put it in place to see what happened. The errors change to this:

    "python.exe Application Error

    The application was unable to start correctly (0xc000007b). Click 'OK' to close the application."

    I tried a handful of other things including installing the latest VS C++ Redistributable (17/19) as well as the x64 and 32 versions of Python. Same issue. Not quite sure what it might be on my end for the moment. (I have tried this on two separate Win10 computers but they are largely identical. Same issue though.)

  4. #4
    Ah yeah that's what I feared.

    I replaced the bat executing code using virtual environment with packages executables (.exe files).
    It's heavier and less clean but it does work for now (at least I managed to generate modules on my sencond computer that had encountered this dll error).

    Tell me if it's fixed on your side too!

    ( Link is updated on original post but you can get it here too: https://drive.google.com/file/d/1yAY...TQ6f6PaLF/view )

  5. #5
    I'm no longer getting the previous errors, but the windows that arise to process each module disappear quickly and none of the data is processed as expected. Feels like it's on the edge of working though!

    I don't think I'm doing anything wrong with the process, but I'll let you know when I can fiddle more and can try again on another machine or two. I could be user error!

    Thanks again!

  6. #6
    Quote Originally Posted by VegaFontana View Post
    Tell me if it's fixed on your side too!
    I've now tried on another system with the same result: the .bat hits each of the .exes but they arise and shut down almost immediately. Maybe this will give you some hints, as I ran the scripts individually and saw these conclusions:


  7. #7
    Quote Originally Posted by YellowAdder View Post
    Maybe this will give you some hints
    Yes it does indeed! From what I see, it looks like there might be different versions of the Offline Compendium and different ways to store data

    Let's make sure about that. Can you post the first and last parts of your data0.js of powers for instance?

    Mine looks like this:
    od.reader.jsonp_batch_data(20160803,"power",{"power11860":"<h1 class=encounterpower>Adamantine Blas .......... gon Magazine 377, page(s) 49.</p>"})
    Additionally if you could tell me how "long" in terms of number of characters the file is?

  8. #8
    Quote Originally Posted by VegaFontana View Post
    Can you post the first and last parts of your data0.js of powers for instance?
    Here's mine:

    od.reader.jsonp_batch_data(20160803,"power",{"power8000":"<h1 class=encounterpower><span class=level>Adept of Whispers ......... in Arcane Power, page(s) 115.</p>",})
    Quote Originally Posted by VegaFontana View Post
    Additionally if you could tell me how "long" in terms of number of characters the file is?
    VSC reports 90399 characters.


    Looks like it may be the appearance of spans that are different. Here's what I found for your entry:

    od.reader.jsonp_batch_data(20160803,"power",{"power11860":"<h1 class=encounterpower><span class=level>Adamant Instructor Attack 11</span>Adamantine Blast ........
    Last edited by YellowAdder; July 23rd, 2020 at 13:22.

  9. #9
    Oook so this is about to get interesting, the data files syntax and size change indeed (even slightly) depending on the Offline Compendium (of course it does. -_-).

    I will work on a fix for your version and make this as generic as I can so that it should avoid the isse from reproducing.
    (Yet, I hope there is not too many changes on source html because I use these tags to generate the data for modules. If there is other changes, you might encounter other errors in the extraction process but let's not get negative too quick).

    Good news, I might have a fix for you while I'll be reworking the scripts to be more generic.
    It's a longshot and it will be a bit tedious. You need to open each one of yours dataX.js files and change the end of the line from ,}) into }) (remove comma will make it becom "valid json").
    Maybe just start with the feats folder since it's one of the less complex script and if it works do it for all of them.
    Last edited by VegaFontana; July 23rd, 2020 at 14:51.

  10. #10
    Because of course it does. LOL.

    I attempted to remove the extra comma for feats and it *did* process before running into an issue with the imports:


    Still, it's moving in the right direction!

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Log in

Log in