Minimizing Area erf files

Information and Discussion Relating to NWN2 Area Design

Moderators: Moderator, Developer, DM

User avatar
Darker_Thought
Posts: 490
Joined: Wed Mar 10, 2010 1:23 pm
Location: Sweden!

Minimizing Area erf files

Unread post by Darker_Thought »

Hi there,

I'm working on an area(Interior) currently which file size is a bit to large to be accepted into the server.

I have tested and come up with the following:

Things that do not take up much space:
-Environmental placables
-Special effects
-Placables


Things that take up a little more than the others above:
-Readable props/placables. However removing 37 of them only eliminated 0.5mb.
(Most readable props come from the library... Im thinking maybe one could do a Merchant that sells the readable placables as books to reduce file size?)

Things that do take up much space:
-The largest factor of the filze size seems to be the "Area Width and hight". Even if the tile is "not used" it takes up alot of space making the erf file larger. This means I Have to compress al rooms closer to eachother.


- Is there any tips and tools you can advice me to make this job easier in th Toolset?
- Can we move whole rooms/tiles with the props?
- How would you solve this problem most effectively?
- Any other tips you guys have for making the file size smaler?


Thank you on forehand! :)

Regards:
Darker
"NWNScript is what happens when you ask Satan to turn C into a scripting language." - DM Doom
User avatar
Blackman D
Retired Staff
Posts: 4818
Joined: Sat Sep 12, 2009 5:43 am
Location: IL

Re: Minimizing Area erf files

Unread post by Blackman D »

im not sure if there is a whole lot that can be done for that

if you have an area and items then i would try doing an .erf with just the area and anything the area needs to function only and then make a different one with all the items (might have to make a small 2x2 area to hold them), tho i dont imagine that will do that much unless you have a lot of items
everyone is evil till proven otherwise
User avatar
Aspect of Sorrow
Custom Content
Posts: 2699
Joined: Fri Mar 28, 2014 7:11 pm
Location: Reliquary

Re: Minimizing Area erf files

Unread post by Aspect of Sorrow »

Objects flagged environment maintain appearance and general meta data, nullifying most other properties where applicable which aids in their space constraints. All placeables that are not interactive with the player must be flagged environment.

You can decompress the file using zlib and copy elements into a properly sized area using Olydbg, or some other hex reader, or you can make prefabs of the positioned content in the area to paste into a new area and export only the new area in the ERF.
User avatar
LISA100595
Retired Staff
Posts: 5202
Joined: Mon Aug 22, 2011 12:23 pm

Re: Minimizing Area erf files

Unread post by LISA100595 »

Most Areas that are submitted to the server should do a complete rundown of this list we all put together and Tfunke wrote up so nicely: I hope this helps :)
tfunke wrote:Toolset Basics:
Hidden: show
First and foremost, switch off the auto-save feature in the options menu.
Second, save your module as a directory (which will save all the modules contents inside a folder). It is significantly more stable saving this way, as well as easier access to the files you are working on.
Tips to avoid crashing the toolset:
There is one placeable in the toolset that will instantly crash the toolset. It is a rune pillar or some such and even has in its title (do not select). Why it never got removed with the games updates, I will never know.
Ctrl+right mouse = pan camera or click hold middle mouse button
Shift+right mouse = rotate placeable etc (once selected).
Shift+mouse scroll up/down = raise lower placeable etc on the Z axis (once selected).
“+” & “-“ = raise lower placeables etc a small amount (once selected)
Ctrl+Shift+ move mouse forward or backwards = moves the camera up or down on the Z axis. Good for keeping the camera level with whatever terrain you are working on.
Arrow keys = move a selected object around on the X and Y axis.
Pro Tips:
Hidden: show
Never drag-select multiple objects quickly, particularly if you have things disabled from being selectable (in the menu up top). Also, NEVER drag-select objects when the camera is panned to be looking straight down, ie: a bird’s eye view of whatever you are trying to select. Doing these in my experience, is one of the fastest ways to crash the toolset.
If you are having trouble selecting something you have already placed down, consider using the tab up top to disable selection of certain things. This is useful in particular when working on an exterior and trying to select something that is inside the boundary of a tree (which have large boundary boxes for the most part). You can also do this with other things, but as mentioned above, be careful when using this that you don’t hastily drag-select things.
If you find the spheres surrounding the sounds and lights to be distracting, consider using the tab up top to disable showing them. Disable showing light spheres and sounds and it makes things easier to see.
Try to limit all light sources from overlapping more than twice. Especially if you do not have shadows disabled. This helps performance.
When placing down carpet/rugs or other placeables that are very thin and do not have much height, use the “+” and “-“ keys to move one of the objects up or down slightly on the Z plane axis. Doing this will stop the overlapping objects from flickering in game when someone pans their camera, which can be very distracting to players.
Secret doors made really simply in an interior area - http://bgtscc.net/viewtopic.php?f=141&t=39124
Frame Rates for areas:
Hidden: show
Up top in toolset, you have a button to toggle on/off = Frame Rate. Clicking it will bring up a bunch of data on your screen. What you want to focus on is the far right bottom number.
Everything you put in an area adds to the overall framerate of an area. Though regarding placeables and trees, you can put down multiples of the same placeable and it will not increase the overall framerate. Same goes with NPC guards etc.
As a rule of thumb, we've always tried to keep an area below 2000 frames, give or take. Keep in mind when adding NPC's to an area (which most guilds do), these also add quite a bit to the overall framerate. Me personally? I try to limit my exteriors to 1400 and my interiors to 2000 (after NPC's).
Things that add to the framerate include; every different model you use, NPC's, VFX's, terrain textures, grass, trees and light source (shadows). As for trees also, you want to try and use only 8 or so different tree types in an exterior area. So you would change the size and proportions of trees to make it less obvious you are using the same model repeatedly.
Plugins:
Hidden: show
Airbrush.dll plugin – saves hours! For example, I have q, w and e bound to airbrush, so that when I push down the e key and scroll the mouse, the outer rim of a paint brush will enlarge or shrink. If I press the w key and do the same, the inner paint brush will enlarge/shrink. Saves heaps of time when painting or doing heightmesh on an exterior, because you can change the size of the paint brush on the fly. - http://neverwintervault.net/project/nwn ... s-toolbars
Multibrush.dll.bak plugin - I have this disabled (.bak) unless I need to use it, as it can be a little unstable –but brilliant all the same. It takes a little practice to use, you can find instructions on the nwn2 vault if you download it. This plugin will save you HOURS when painting an exterior. - http://148.251.86.81/project/nwn2/other ... ush-plugin
TerraCoppa.dll plugin – great for moving whole sections of a map to another map or merging areas. - http://neverwintervault.net/project/nwn ... acoppa-v18
Terrain:
Hidden: show
Try to limit an exterior to 4 particular textures you wish to use across the whole area. What I do then, is select each of the four textures and turn the texture down to about 5% and then use that texture inside every box on the exterior. This way, you can still leave 2 texture spaces free to add an additional texture you might want to use somewhere on the map, such as a riverbed texture etc.
Grass should also be used sparingly. Grass significantly increases the size of an area file. I usually paint down grass at 20% pressure.
One of the most important parts of working on terrain is using the colour palette under the terrainterrain tab. Once you are finishing off an area, I HIGHLY recommend using the colour palette to give your area a more authentic feel. I usually use a dark gray colour set at about 15% and go around the map using it on contours/cliff edges etc. You can even use actual colours to spice up the area. Maybe some green colour thrown about the map, to make the grass really appear healthy in certain parts.
Sounds:
Hidden: show
Sounds are vital too. Think of the best area you may have ever seen and if you turn off the sounds in the area, it will suddenly feel bland. There are an insane amount of sound files you can use, not just the ones available under the sound tab. If you throw down a generic sound and go into the properties under the sound tab, you can browse the game for numerous different sounds.
The properties in the sound tab that are mostly used:
Interval (how long until the sound plays)
Interval variation (how long until the sound plays above, plus or minus this number)
Looping (I almost always turn this off)
Pitch variation: Changes the pitch of the sound slightly. Rarely used, but can be useful.
Random: I usually always have this ticked, so that one of the sounds I have attached to this particular sound object, will randomly be selected to play.
Sounds: The sound files that will play.
Maximum distance: Distance in which the sound can be heard. Usually default is 10, so I often increase it to at least 15 minimum.
Minimum distance: Good to increase this, if the sound file is fairly quiet when it plays.
Lighting:
Hidden: show
As a rule of thumb, all object light shadow sources should either be switched to 0, or at least under 0.4. This increases performance, particularly in an online environment. Heavy shadows are intensive.
Sls2 (scripted lighting system) comes with some nice premade light sources, that I almost always use. You only need the light blueprints from the module, the rest is useless. http://148.251.86.81/project/nwn2/scrip ... ion-2-sls2
COLOR:
Ambient color: not sure. Seemingly useless? Slight effect if anything.
Intensity: how bright the light is. Has no effect on shadows.
Diffuse color: the color of the light.
Specular color: not exactly sure how this works. Will get back to you on this. But for now, know that it is a very slight/negligible effect, so just leave it plain white.
Behavior:
Flicker: This is set to true or false. True means the light will flicker/fluctuate.
Flicker type: jumpy/bounce/linear. These are hard to explain in words, but play around with them to get a feeling for what they do. NOTE: "jumpy" flickers are how you get fireplace/torch effects.
Flicker rate: how quickly the light flickers. Higher number = faster (for good jumpy effects, you need a fairly high number).
Lerp: lerp if this is set to "true," then the light will shift between its main color and its "lerp" color (i.e. the lerp target color settings above).
Lerp period: how quickly the light changes from its main to its lerp color.

Placeables:
Hidden: show
99% of placeables in an area should be set to environmental objects.
Try to be creative with your use of placeables and refrain from putting in too many different ones that raise the overall frames of an area.
Make good use of the “scale” properties on your placeables. Increasing or decreasing or warping the placeable out of shape, can give you alternate uses for placeables. A bad example, but the only one I can think of right now, is if you wanted four pillars in a room. You could take a chair and make its scale 7,7,7 and then the legs of the chairs could look like pillars coming down from the ceiling.
If you are using a placeable that is attached to a VFX and light source, then it is recommended to switch that placeables “Cast Light” properties to – “Don’t case shadows”. For example, if you are using a brazier and have a light source on it, then switching off the shadows of the brazier, will stop the sickly looking shadows that can emanate from the placeable attached to a light source.
Tilesets:
Hidden: show
Each variation you use on a tileset, increases the overall frame rate of an area. So try not to go overboard on different tiles, because it will limit your overall use of placeables, VFX’s etc in an area, that all contribute to framerate.
Some tilesets will not connect with each other, even though they look like they should. If you are using two different tilesets that cut the walkmesh at the door, then use a walkmesh placeable over the part of the ground that is blocking the walkmesh and players will now be able to move from one tileset to another. This only happens with some variations.
Don’t forget to look at the properties tab of a tile you have placed down. In there you will see Ceiling, wall and floor textures. We have many variations that can be used with lots of different types of different tilesets. Keep in mind, each variation will contribute to framerate. Some of the textures may also change in the toolset, but when you jump IG, they are not showing up. You will have to test this out on your end and see what works and what doesn’t.
Creatures: Area Properties:
Hidden: show
Set the areas tag to all lowercase, all one word. Try to keep it below 12 or so letters. For example: bgeastgate.
Search on the nwn2 vault for day/night cycles. - Here you go - http://neverwintervault.org/project/nwn ... ng-presets
Gives you about 140 daynight cycles to choose from, which really saves time been able to test different ones to find the atmosphere you desire. Then tweak if need be.
Set your areas sound (if you want any) for day time and night time. Also set battle music, if you want it.
Add relevant variables to your area. http://bgtscc.net/viewtopic.php?f=141&t=27789
Lady Elvina Aira-S'efarro - The Order of the Silver Rose
Salaria - Bounty Hunter half-sister of Darius Brothers
Angelina Northstar - Holy Warrior of Tyr / Knight of the Silver Rose
Matilda Stonehold - Honorable Sheild Dwarf
Loriah Swift - Morninglord of Lathander
User avatar
Duster47
Retired Staff
Posts: 3727
Joined: Mon Nov 16, 2009 7:57 pm
Location: Florida, USA

Re: Minimizing Area erf files

Unread post by Duster47 »

You can use TerraCoppa to copy area tiles from one area into another of same type (Int or Ext). Unfortunately, I never got it to truly copy an interior tile (need to redraw them). For things within an area, you can drag-select a bunch or all for a whole tile and create a "Group" (right click on selected items). You can copy/paste the group (though it won't be a "group" if copy) into the new area. Be sure to fix "height lock" or else all of the objects will drop to ground-level of wherever you are pasting them.

IIRC, NPCs greatly add to an areas file size and frame rate. You can reduce area size by using the MPWC(?) waypoint spawner for NPCs. This way they are not part of the area file and only exist IG if a PC is in the area. NPCs should have reduced script sets too (another topic). Copies of same objects and NPCs does not change FR. VFXs and grass do add to FR. Grass greatly adds to file size (ext issue really).

The main thing about guild areas and resource utilization is that guild areas are for a select few whereas public areas are for all to enjoy. Therefore we tend to want public areas to get more bells and whistles than someone's private sandbox. Last, guild areas are notorious for not lasting. Once something is in the module, it is a bit of a hassle to get all traces removed. If there is less to begin with, then there will be less chaff remaining to junk up the module after the guild folds.

Since the topic is ERFs, you can greatly reduce an ERF's size by stripping out the common-use template PLC, non-custom-script, and other files that the in-Toolset ERF exporter adds to the ERF. Use NWPacker to edit the ERF. Often, only the five area files are truly needed.
PC1 = Nerys, Emissary and Skald of the Greyfox tribe, last seen roaming north near Secomber
PC2 = Valqis Sanejmeh; far away cartographer, Oracle of Nut at chaltin QulDaq, former navigator of the Sea Seeker, Reader of Candlekeep and sometime performer.
User avatar
Darker_Thought
Posts: 490
Joined: Wed Mar 10, 2010 1:23 pm
Location: Sweden!

Re: Minimizing Area erf files

Unread post by Darker_Thought »

Duster47 wrote: Since the topic is ERFs, you can greatly reduce an ERF's size by stripping out the common-use template PLC, non-custom-script, and other files that the in-Toolset ERF exporter adds to the ERF. Use NWPacker to edit the ERF. Often, only the five area files are truly needed.
By reducing the maps total width and hight, and moving the rooms closer to eachother i got it from 4,58mb to 3,32mb. A great start. And without destroying any "used tiles".

The quote below:
"common-use template PLC, non-custom-script, and other files that the in-Toolset ERF exporter adds to the ERF."
I would like to ask further about this. I've never done this before So I need a bit of help.

- Does this affect the area in any way?
- Does it affect the readable placables?
- And...Could you aid me in doing this? Never done this before so would greatly apreciate the aid. Feel a bit lost on this one. :)

//Darker
"NWNScript is what happens when you ask Satan to turn C into a scripting language." - DM Doom
User avatar
Duster47
Retired Staff
Posts: 3727
Joined: Mon Nov 16, 2009 7:57 pm
Location: Florida, USA

Re: Minimizing Area erf files

Unread post by Duster47 »

NWN2Packer is available on NWVault here:
https://neverwintervault.org/project/nw ... packer-v20

Instructions are on the link.

The rule-of-thumb we use for separating tiles is three blanks between different levels of the same building or different areas within a zone. You can see this IG if you go into BG area interiors and look at your minimap. Lesser blank tiles can lead to floating-chat and sound spilling over from one area to another.

As for overall area's tile-count, I do not believe there are many public areas greater than 24x24 tiles. I think it will go up to 32x32. If going that large you will probably have a difficult time meeting the guidelines Lisa shared above from TFunke.
PC1 = Nerys, Emissary and Skald of the Greyfox tribe, last seen roaming north near Secomber
PC2 = Valqis Sanejmeh; far away cartographer, Oracle of Nut at chaltin QulDaq, former navigator of the Sea Seeker, Reader of Candlekeep and sometime performer.
User avatar
Darker_Thought
Posts: 490
Joined: Wed Mar 10, 2010 1:23 pm
Location: Sweden!

Re: Minimizing Area erf files

Unread post by Darker_Thought »

Duster47 wrote:NWN2Packer is available on NWVault here:
https://neverwintervault.org/project/nw ... packer-v20

Instructions are on the link.

The rule-of-thumb we use for separating tiles is three blanks between different levels of the same building or different areas within a zone. You can see this IG if you go into BG area interiors and look at your minimap. Lesser blank tiles can lead to floating-chat and sound spilling over from one area to another.

As for overall area's tile-count, I do not believe there are many public areas greater than 24x24 tiles. I think it will go up to 32x32. If going that large you will probably have a difficult time meeting the guidelines Lisa shared above from TFunke.
Alright thank you. I will download the software and try. Though I'l admit i still think this sounds tricky.

About the area size I managed to get it down to something around 17x9. Going to see if I can compress it further.

Thank you for the effort of being active on this thread providing support(To al of you) :)
"NWNScript is what happens when you ask Satan to turn C into a scripting language." - DM Doom
User avatar
Darker_Thought
Posts: 490
Joined: Wed Mar 10, 2010 1:23 pm
Location: Sweden!

Re: Minimizing Area erf files

Unread post by Darker_Thought »

Lo again.

What about Kemo? I need to keep that for the script to work right?


This is what have left.

Image[/URL]

And this (The area within the red Square) is this the files that I need to keep?
Image
"NWNScript is what happens when you ask Satan to turn C into a scripting language." - DM Doom
tfunke
Retired Staff
Posts: 2305
Joined: Sat May 07, 2011 9:09 pm
Location: Australia- GMT+10

Re: Minimizing Area erf files

Unread post by tfunke »

Darker_Thought wrote:And this (The area within the red Square) is this the files that I need to keep?
Image
Yep. Those 5 files (.trx, .trn, .are, .git, .gic) are all you need. If you wanted someone on the development team to review your area, you simply just need to send them those 5 files.

Everything else should already be in the master module, so you don't need to keep the kemo script files, unless you want to test it out as you use it.

By the way, something that you may have misunderstood about the above. If you place down a single barrel in an area, it will add to the size of the area. But if you keep using the same barrel from then on, it will not add to the framerates or area size. The trick is to limit your variety of placeables that you use in an area, by reusing them. If they are tintable, then great, change their colour to make it less obvious the same object is being used etc..
Per aspera ad astra
User avatar
Aspect of Sorrow
Custom Content
Posts: 2699
Joined: Fri Mar 28, 2014 7:11 pm
Location: Reliquary

Re: Minimizing Area erf files

Unread post by Aspect of Sorrow »

tfunke wrote:By the way, something that you may have misunderstood about the above. If you place down a single barrel in an area, it will add to the size of the area. But if you keep using the same barrel from then on, it will not add to the framerates or area size. The trick is to limit your variety of placeables that you use in an area, by reusing them. If they are tintable, then great, change their colour to make it less obvious the same object is being used etc..
The only instancing subsystem in the renderer is speedtree.
User avatar
Duster47
Retired Staff
Posts: 3727
Joined: Mon Nov 16, 2009 7:57 pm
Location: Florida, USA

Re: Minimizing Area erf files

Unread post by Duster47 »

Never export/send a module.ifo file. Awful things can happen if it is (accidentally )merged into the master (e.g. revert to L-20-max :shock: )

All of those *.utp files can be left behind. Only the Build Team should be creating PLC templates for use in our Builders' Module. Script files (*.ncs and *.nss) are only needed if they are custom for the area.

Generally speaking, areas (especially guild areas) should only export the five area files TFunke outlined and the in-area NPC conversations (*.dlg). NPCs in an area will be part of the area files. Also, I do not recall ever receiving an NPC with all of its properties correct for BG. We always end up editing those. Some we can convert to templates (*.utc) and use the spawner system, especially if the NPC is interactive (i.e. wanders, is chatty, able to defend himself, etc., which depend on script set assigned to it).
PC1 = Nerys, Emissary and Skald of the Greyfox tribe, last seen roaming north near Secomber
PC2 = Valqis Sanejmeh; far away cartographer, Oracle of Nut at chaltin QulDaq, former navigator of the Sea Seeker, Reader of Candlekeep and sometime performer.
User avatar
Darker_Thought
Posts: 490
Joined: Wed Mar 10, 2010 1:23 pm
Location: Sweden!

Re: Minimizing Area erf files

Unread post by Darker_Thought »

Thank you everyone on this thread for the tips in the Toolset. Very apreciated, and the area is now going ig! ^^

//Darker
"NWNScript is what happens when you ask Satan to turn C into a scripting language." - DM Doom
User avatar
K'yon Oblodra
Recognized Donor
Posts: 1002
Joined: Wed Oct 28, 2009 3:38 am
Location: Berlin

Re: Minimizing Area erf files

Unread post by K'yon Oblodra »

Darker_Thought wrote:Thank you everyone on this thread for the tips in the Toolset. Very apreciated, and the area is now going ig! ^^

//Darker

Cool what's it gonna be?
K'yon Oblodra
Necromancer of the school of Necromancy
Silent seat for the school of Necromancy

Return to “NWN2 Area Design”