Page 1 of 3 123 Last
  1. #1

    Special characters issue found in Crypt of the Sun Lord

    I loaded up Crypt of the Sun Lord yesterday (fully updated FGU on Windows 10), and noticed that the apostrophes (and perhaps other characters) are not recognized and instead show up as blank squares. I see that this issue was previously reported as FGU-670, but it is no longer on the list of known issues. Perhaps the issue needs to be reopened and looked at for a more universal solution?

  2. #2
    Doing some research into this; it appears that the FGC fonts actually ended up getting generated using windows-1252 (cp1252) encoding support. We have always told developers that we support iso-8859-1; and the problematic modules you are seeing are because the DLC developer chose to insert characters that are part of windows-1252, but not iso-8859-1. (character range 128-159 differ).

    What I have been doing to date is to change/remove the problematic characters from those modules as we come across the modules. I'll see if I can change the XML encoding in the content header so that the encoding engines handle automatically or not. If so, I may able to just change the stated encoding to fix those as they come up. Otherwise, I'll have to continue modifying the modules as they come up.

    Either way, there is no "automatic" solution unfortunately, since the XML library is a standard library and the problematic modules are technically using the wrong encoding in the xml encoding tag.

    Regards,
    JPG
    Last edited by Moon Wizard; January 30th, 2020 at 00:28.

  3. #3
    Just looked into this a bit more, and the "windows-1252" encoding is considered an "exotic" encoding that is not part of the .Net library that allows Unity to build across multiple platforms (Windows, Mac, Linux). So, for the foreseeable future, the solution is to change/remove those characters in the modules as they are found.

    Regards,
    JPG

  4. #4

    Join Date
    May 2016
    Location
    Jacksonville, FL
    Posts
    1,795
    Blog Entries
    7
    I still don't see how this is acceptable in the 21st century... Every OS can use UTF-8. Can't FG Classic be tweaked instead of keeping the brand new client 20+ years behind?

  5. #5
    It's the source DLC, not the engine. The source DLC claims it is ISO-8859-1 (which is what we always claimed that FG supports); but the DLC uses characters from an Windows-1252 code page, which are technically undisplayable "control characters" in ISO-8859-1. So, there is no magical conversion to bring it up. Just don't use characters 128-159 in DLC meant for FGC and FGU (which is everything right now). Welcome to backward compatibility...

    By the way, I've updated all the A01 modules to remedy the characters.

    Regards,
    JPG

  6. #6

    Join Date
    May 2016
    Location
    Jacksonville, FL
    Posts
    1,795
    Blog Entries
    7
    I guess my point is, since FGC already doesn't really give a crap, but FGU does... can it be tweaked or further broken to continue not giving a crap but let us build DLC encoding to UTF-8? Nearly every RPG textbook utilizes characters in the 128–159 range, and it's just amateurish looking to have to stick to 1987 (ISO-8859-1) conventions and ASCII art from the 80s.

  7. #7
    Nope, because then you break every DLC for FGC, since nothing in the FGC engine understands utf8. You’ll just have to wait until FGC gets to retire.

    Regards,
    JPG

  8. #8
    Great, thanks for addressing the topic.

  9. #9
    Mortar's Avatar
    Join Date
    May 2014
    Location
    New Brunswick, Canada
    Posts
    982
    Blog Entries
    18
    The problem is made even more glaring by the fact that the Classic client exports modules using the Windows 1252 characters. As much as I have been messing around in the Unity client, I haven't checked that yet.
    Ultimate License Holder
    List of completed DLC products

  10. #10
    Actually, the client doesn’t export 1252 exactly; it just exports character codes 32-255 without any filtering. The issue is that the font files are building to 1252 code page; so the extra symbols for that code page are available in the fonts generated in FGC, even though they are not in ISO-8859-1.

    As above, I already looked into being able to just change the encoding in the XML to Windows-1252; but it’s not supported in Unity version of .Net that is cross-platform compatible.

    So, as I mentioned above, we’ll have to fix as they are found.

    Regards,
    JPG

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

Log in

Log in