xp_craft plugin: summary

[NegInfinity]

Hello there.

I tried to take a look at xp_craft plugin in the past weeks from time to time, it would be nice
if someone with more experience with toolset/scripts took a look at it to confirm few things... or to investigate things.

Experience with toolset, setting up servers and module writing (and free time) are required... I could upload missing files somewhere, if you need them.

Here's what is needed:
1. Combine Kemo auctioneer and xp_craft modules into one (basically, put xp_crafter into kemo testing modules and make sure it works - when I did that, xp_crafter stopped working).
2. Track down rampant memory allocation that happens when someone tries to alter armor appearance for the first time. Judging from my test results, it occurs in when xp_craft scripts call "StoreCampaignObject" function for the first time. That needs confirmation.

Hidden: show

To be precise, allocation happens within nwnx_craft.nss, in function called XPCraft_StoreItemToCraft:

Code: Select all

int XPCraft_StoreItemToCraft(object oPC, object oItem, int bSafeStore = TRUE)
{
	int bStorageSuccess = FALSE;
	
	if((oPC!=OBJECT_INVALID) && (oItem != OBJECT_INVALID))
	{
		string sDataFileName = XPCraft_GetDatafileName(oPC);
		if(bSafeStore)
		{
			XPCraft_DestroyDataFile(oPC,sDataFileName); 
		}
		bStorageSuccess = StoreCampaignObject(sDataFileName,"NO_ACCOUNT_NEEDED",oItem);///<<----here
	}
	
	if(!bStorageSuccess)
	{
		XPCraft_Debug(oPC,"Unable to store " + GetName(oItem) + " as a campaign object.");
	}
	return bStorageSuccess;
}

3. If StoreCampaignObject is culprit, attempt to figure out when Kemo Auctioneer stops suffering from this problem. Basically.... when you store item onto auction for the first time, similar memory allocation happens with kemo auctioneer. However, eventually it stops happening. Or at least that's what my testing results told me.

Reason for asking:
The issue prevents xp_craft from being used there. I attempted to debug xp_craft dll, and found that allocation of huge memory block happens elsewhere (basically, I think it is on StoreCampaignObject call), and can be mitigated by altering settings in xp_bugfix ini.

Baiscally...
1. If there's a leak in xp_craft dll, then I most likely will be able to fix that...
2. If the issue is with StoreCampaingObject, an attempt should be made to figure out if ALL calls to StoreCampaignObject causes extra memory allocation, or not. This function is used by Kemo Auctioneer, and it is already on the server... so, basically, there's need to figure out whether adding xp_craft to bgtscc would increase memory usage or not. As far as I know, when StoreCampaignObject causes memory allocation on server for the first time, consequent calls do not increase memory usage further. Amount of memory being allocated can be reduced by adjusting settings in xp_bugfix.ini... ( setting DatabaseBufferCount to 1024 reduces amount of allocated memory to about 70 megabytes).

That is... if people are interesting in getting better appearance changer here.

----

Failing that, someone with account on sigil could ask their admin if they used suffer from the same problem and/or how they fixed those. Sigil recently has seen increased population, so forums looks more active than before - good chance that someone will reply or will be even willing to share their setup.

[Ivan38Rus]

We are not using Campaing variables, as far as I remember. It's been converted to store stuff in SQL database

[NegInfinity]

We are not using Campaing variables, as far as I remember. It's been converted to store stuff in SQL database

That is very interesting.

However, as far as I know, the only method to save item somewhere is "StoreCampaignObject".
In case of mysql, that function call is hooked by nwnx server's xp_mysql plugin, so data goes to mysql, and not wherever server stores it normally.

However, even when using mysql plugin and with properly configured mysql server running, calling storecampaignobject in xp_craft demo causes allocation of large memory chunk.

So... if you aren't talking about xp_mysql plugin that hooks StoreCampaignObejct and redirects data to database, and you actually rewrote portion of kemo auctioneer that handles storing/retrieving items, then I would like to see how you're doing that.

Auctioneer uses StoreCampaignObject too. If there's alternative to StoreCampaignObject, then perhaps xp_craft could be rewritten to use it.

[Ivan38Rus]

We are not using Campaing variables, as far as I remember. It's been converted to store stuff in SQL database

That is very interesting.

However, as far as I know, the only method to save item somewhere is "StoreCampaignObject".
In case of mysql, that function call is hooked by nwnx server's xp_mysql plugin, so data goes to mysql, and not wherever server stores it normally.

There is this, it passes hex data between game engine and plugin which then puts it into MySQL/sqlite

// Set oObject's persistent object with sVarName to sValue
// Optional parameters:
// iExpiration: Number of days the persistent variable should be kept in database (default: 0=forever)
// sTable: Name of the table where variable should be stored (default: pwobjdata)
void SetPersistentObject(object oObject, string sVarName, object oObject2, int iExpiration =
0, string sTable = "pwobjdata");

[NegInfinity]

We are not using Campaing variables, as far as I remember. It's been converted to store stuff in SQL database

That is very interesting.

However, as far as I know, the only method to save item somewhere is "StoreCampaignObject".
In case of mysql, that function call is hooked by nwnx server's xp_mysql plugin, so data goes to mysql, and not wherever server stores it normally.

There is this, it passes hex data between game engine and plugin which then puts it into MySQL/sqlite

// Set oObject's persistent object with sVarName to sValue
// Optional parameters:
// iExpiration: Number of days the persistent variable should be kept in database (default: 0=forever)
// sTable: Name of the table where variable should be stored (default: pwobjdata)
void SetPersistentObject(object oObject, string sVarName, object oObject2, int iExpiration =
0, string sTable = "pwobjdata");

This thread says those two functions are the same.

To make something like KEMO Auctioneer or xp_craft work you need a method to convert any item into something that can be put into database and back. For example, convert item into string and convert string to item.

So... did you modify storage method used by KEMO auctioneer here or not?

[Ivan38Rus]

No physical file is getting created, the data is passed through RAM and saved in a database, not in a bloated campaing variable format on HDD.

Yes, it has been altered.

What made you think Kemo auctioneer is a big perfomance drain?

[NegInfinity]

No physical file is getting created, the data is passed through RAM and saved in a database, not in a bloated campaing variable format on HDD.

Yes, it has been altered.

What made you think Kemo auctioneer is a big perfomance drain?

Memory, not performance.

The reason why there is no xp_craft appearance changer, is because it allocates large chunk of memory when used for the first time.

I tried to track it down, and allocation happens at call to "StoreCampaignObject".

KEMO Auctioneer uses the same function call to store items on auction. When I tested kemo for the first time, it actually did that too (allocate huge chunk of memory first time item was put on auction)... but eventually stopped doing that for unknown reason.

So... if anyone wants better appearance changer, the issue needs to be investigated further...

Basically, someone needs to check if adding xp_craft would actually increase memory usage(if kemo is unchanged), or share the method which is used to store items in kemo auctioneer (if you use altered version of auctioneer), or attempt to figure out why KEMO eventually stopped doing that memory allocation thing, or look for other ways to turn item into data that can be stored within dll.

[Luna]

nwnx4 xp_bugfix took care of the Kemo issue.
Skywing capped the games database allocation.

[NegInfinity]

nwnx4 xp_bugfix took care of the Kemo issue.
Skywing capped the games database allocation.

xp_craft uses the same database. And settings in xp_bugfix affect its memory allocation too. Basically, you can reduce memory usage to 30 megabytes or so, by setting buffer count to low values.