node.getChild(string) vs DB.getChild(node, string)

[MeAndUnique]

I'd really like to push back on deprecating any API currently attached to any `databasenode` userdata object.

Even a gradual deprecation process would have to be spread out over the course of years (not months). Maybe 3 or 4. This would not only impact rulesets, it would impact every single extension ever made for FGU that also leverages this API. As a ruleset dev myself, it is a huge refactor (as we do use that API quite a bit) with an extensive regression test process to make sure we don't slip and introduce any bugs.

If what I am suggesting previously is possible, and I do believe it is, a better refactor can be done that allows the `databasenode` userdata object to access an API exposed by one object maintained in memory, thus achieving the same, or at least nearly similar, level of optimization.

Also, if I am understanding Moon Wizard correctly, it seems that they may have found a way to do something like that, so that a total refactor is not needed (though may still yield a performance boost.)

I just want to state I (respectfully) disagree with deprecating these functions, I think it will be a huge mistake for the community.

While I am very sympathetic with the concerns, if indeed it turns out to be a system problem with databasenodes degrading performance, the best thing for the community is in fact to bite the bullet and deprecate the problematic functionality. While worth avoiding wherever possible, this is a common reality for the maturation of any coding framework and necessary to retaining long-term viability. Performance of FGU is already a notable pain-point for many users and use-cases, and one that if left untended will inevitably result in FGU becoming noncompetitive in the VTT arena.

[Moon Wizard]

As MeAndUnique mentioned, the syntax for the strings is identical under the hood for Lua, but only for functions tied to a variable type. [vartype.fn(var, ...) == var:fn(...)]

Unfortunately, the whole point of this process is to make the databasenodes into very simple objects with no functions tied to them. If we don't have to register an entire global space to them, then that is a lot of overhead saved multiplied by the number of database nodes (hundreds of thousands or more per campaign with modules loaded). So, I can't just "hide" them in another calling method, or you still have all the overhead.

To be clear, if the performance is sufficient, this will be a planned under a year process with migration needed.
The migration is almost all a very straight regex replacement except for possibly nested calls, which are slow performance by themselves (i.e. multiple calls vs. single call).

Regards,
JPG

[MeAndUnique]

If what I am suggesting previously is possible, and I do believe it is, a better refactor can be done that allows the `databasenode` userdata object to access an API exposed by one object maintained in memory, thus achieving the same, or at least nearly similar, level of optimization.

Also, if I am understanding Moon Wizard correctly, it seems that they may have found a way to do something like that, so that a total refactor is not needed (though may still yield a performance boost.)

Though to clarify, I wholly agree with the concept here if it is in anyway feasible. Taking it even a step further, any mechanism afforded by the Lua library FGU uses (or even one of the alternatives, being several candidates) that more closely aligns the object and memory management will itself yield numerous benefits to performance overall. As a simple example, the recent addition of a getChildList as an alternative to getChildren, if mapped to C# Dictionary-like synonyms would yield performance measures that differ only by passing a single additional reference, or in other words a handful of nanoseconds per loop. So considering the entire DB would still be way under a single second difference, complete noise in light of other performance concerns.

[seansps]

The example syntax here is actually 100% identical between the two under the hood in Lua. Arguably, leveraging this language feature could actually be a viable middle ground for database nodes, where they could support ":" notation that routes to DB API calls.

This is something I've considered as well.

It is also possible to achieve this using `.` notation if the underlying object is a Lua table. I wrote some C# code as a proof of concept that works when running locally.

[seansps]

Though to clarify, I wholly agree with the concept here if it is in anyway feasible. Taking it even a step further, any mechanism afforded by the Lua library FGU uses (or even one of the alternatives, being several candidates) that more closely aligns the object and memory management will itself yield numerous benefits to performance overall. As a simple example, the recent addition of a getChildList as an alternative to getChildren, if mapped to C# Dictionary-like synonyms would yield performance measures that differ only by passing a single additional reference, or in other words a handful of nanoseconds per loop. So considering the entire DB would still be way under a single second difference, complete noise in light of other performance concerns.

Yes, exactly.

       
var lua = new Lua();

// Expose the databasenode userdata object to Lua
var databasenode = new DatabaseNode();

// Create a Dictionary that wraps the databasenode methods
var databasenodeWithMethods = new Dictionary<string, Delegate>();
databasenodeWithMethods["getValue"] = () => {
    databasenode.getValue();
};
databasenodeWithMethods["setValue"] = (string val) => {
    databasenode.setValue(val);
};
databasenodeWithMethods["getPath"] = () => {
    databasenode.getPath();
};
lua["databasenode"] = databasenodeWithMethods;

The above code uses the NLua C# library. The `DatabaseNode` is a class that just has a string pointing to a XML file and a pointer to an XML element.
The method calls on the `DatabaseNode` use an `XmlWrapper` class powered by C# `System.XML` and it each `DatabaseNode` just gets a pointer to the single instance maintained by the application using `xmlWrapper = XmlWrapper.Instance` which is one instance. This should achieve only one instance of the underlying XML Library needed by the application without having each database node instance instantiate their own.

I can provide more of what I did locally as a POC if it helps.

Edit: And yes in the above I was just printing within the methods instead of returning the values needed, but it's possible to wire it up properly.

[Moon Wizard]

On the string.fn(var, ...) vs. var:fn(...) part of the discussion; this approach with databasenode objects would not smooth migration any more than migrating to DB. That is because all the var.fn(...) calls with databasenode vars would need to be converted to either databasenode.fn(var, ...) or var:fn(...). Not really much savings in coding effort in that approach.

The current recommended change is DB.fn(var, ...). How is that different than the former conversion above?

Also, I think that the level of work that is being assumed with this change is higher than it is, I was able to convert about 20 rulesets in a couple days, including Core which is larger than most of the other rulesets.

Regards,
JPG

[seansps]

On the string.fn(var, ...) vs. var:fn(...) part of the discussion; this approach with databasenode objects would not smooth migration any more than migrating to DB. That is because all the var.fn(...) calls with databasenode vars would need to be converted to either databasenode.fn(var, ...) or var:fn(...). Not really much savings in coding effort in that approach.

The current recommended change is DB.fn(var, ...). How is that different than the former conversion above?

Also, I think that the level of work that is being assumed with this change is higher than it is, I was able to convert about 20 rulesets in a couple days, including Core which is larger than most of the other rulesets.

Regards,
JPG

It’s not just converting the code, but also, the amount of time needed to do a full regression test of a ruleset to ensure no bugs get into production.

Additionally, all the extensions that would need conversion as well.

In any case, check my response above — there’s a way to maintain “.” Notation functionality and not embed the API into each userdata object.

[Moon Wizard]

Actually, there is not a way to maintain a dot notation in Lua without embedding the API into each object. The way that Lua works is that the dot notation is a Lua table lookup, so that name = fn entry must exist in the object in order for var.fn(...) to work.

As mentioned above, adding a "databasenode" "library" to support databasenode.fn(var, ...) or var:fn(...), is not that different than DB.fn(var, ...). The example you displayed above just creates a "databasenode library".

Regards,
JPG

[seansps]

Actually, there is not a way to maintain a dot notation in Lua without embedding the API into each object. The way that Lua works is that the dot notation is a Lua table lookup, so that name = fn entry must exist in the object in order for var.fn(...) to work.

As mentioned above, adding a "databasenode" "library" to support databasenode.fn(var, ...) or var:fn(...), is not that different than DB.fn(var, ...). The example you displayed above just creates a "databasenode library".

Regards,
JPG

I see, how about this?

using System;
using System.Xml;
using NLua;

class Program
{
    static void Main(string[] args)
    {
        var lua = new Lua();

        // Expose the databasenode userdata object to Lua
        var databasenode = new DatabaseNode();

        lua["databasenode"] = databasenode;

        // Run the Lua script
        lua.DoFile("script.lua");
    }
}

class DatabaseNode
{
    private XmlWrapper xmlWrapper;
    private XmlNode nodeElement;

    public DatabaseNode()
    {
        xmlWrapper = XmlWrapper.Instance;
        var xmlDoc = xmlWrapper.LoadXmlFile("database.xml");
        var children = xmlDoc.GetElementsByTagName("charsheet");
        nodeElement = children.Count > 0 ? children.Item(0) : null;
        getValue = this.GetValue;
        setValue = this.SetValue;
        getPath = this.GetPath;
    }

    public Delegate getValue;
    public Delegate setValue;
    public Delegate getPath;

    public string GetValue()
    {
        return nodeElement.InnerText;
    }

    public string SetValue(string value)
    {
        // TODO
        return "SET TO " + value; ;
    }

    public string GetPath()
    {
        return "database.xml";
    }

    public void GetChild()
    {
        Console.WriteLine("TODO");
    }

    public void GetChildren()
    {
        Console.WriteLine("TODO");
    }

    /// ... etc, other methods needed for DB Node
}

That should create a databasenode userdata object with properties that call the underlying instance methods. It uses a singleton to get to the XML Library.

[Moon Wizard]

You're still assigning this library of functions to "databasenode" as a variable. It's all the same thing you're hashing over.

Regards,
JPG