Possible errors in language 2das.

[NegInfinity]

Alright... as far as I can tell bunch of language 2das in bgtscc haks do not match bioware 2da file format.

"csl_lang_aquan.2da: Invalid number of columns at line 10. 3 expected, 2 received. Columns: 7;^f"
"csl_lang_assassincant.2da: Invalid number of columns at line 4. 3 expected, 4 received. Columns: 1;^a;*sighs,;sadly*"
"csl_lang_bothii.2da: Invalid number of columns at line 19. 3 expected, 2 received. Columns: 16;^p"
"csl_lang_cant.2da: Invalid number of columns at line 4. 3 expected, 4 received. Columns: 1;^a;*shields;eyes*"
"csl_lang_celestial.2da: Invalid number of columns at line 468. 3 expected, 2 received. Columns: 465;latin"
"csl_lang_chardic.2da: Invalid number of columns at line 44. 3 expected, 2 received. Columns: 41;^O"
"csl_lang_chessic.2da: Invalid number of columns at line 35. 3 expected, 2 received. Columns: 32;^F"
"csl_lang_chultan.2da: Invalid number of columns at line 32. 3 expected, 2 received. Columns: 29;^C"
"csl_lang_dambrathan.2da: Invalid number of columns at line 12. 3 expected, 2 received. Columns: 9;^i"
"csl_lang_drowsign.2da: Invalid number of columns at line 4. 3 expected, 8 received. Columns: 1;^a;*shields;left;thumb;with;right;hand*"
"csl_lang_dtarig.2da: Invalid number of columns at line 44. 3 expected, 2 received. Columns: 41;^O"
"csl_lang_durpari.2da: Invalid number of columns at line 51. 3 expected, 2 received. Columns: 48;^V"
"csl_lang_easting.2da: Invalid number of columns at line 44. 3 expected, 2 received. Columns: 41;^O"
"csl_lang_goblin.2da: Invalid number of columns at line 8. 3 expected, 2 received. Columns: 5;^C"
"csl_lang_halruaan.2da: Invalid number of columns at line 44. 3 expected, 2 received. Columns: 41;^O"
"csl_lang_han.2da: Invalid number of columns at line 19. 3 expected, 2 received. Columns: 16;^p"
"csl_lang_ignan.2da: Invalid number of columns at line 10. 3 expected, 4 received. Columns: 7;^g;k;k"
"csl_lang_illuskan.2da: Invalid number of columns at line 19. 3 expected, 2 received. Columns: 16;^p"
"csl_lang_illuski.2da: Invalid number of columns at line 19. 3 expected, 2 received. Columns: 16;^p"
"csl_lang_imaskar.2da: Invalid number of columns at line 36. 3 expected, 2 received. Columns: 33;^G"
"csl_lang_jotun.2da: Invalid number of columns at line 119. 3 expected, 2 received. Columns: 116;Jotunen"
"csl_lang_kozakuran.2da: Invalid number of columns at line 19. 3 expected, 2 received. Columns: 16;^p"
"csl_lang_lantanese.2da: Invalid number of columns at line 24. 3 expected, 2 received. Columns: 21;^u"
"csl_lang_logos.2da: Invalid number of columns at line 19. 3 expected, 2 received. Columns: 16;^p"
"csl_lang_loross.2da: Invalid number of columns at line 44. 3 expected, 2 received. Columns: 41;^O"
"csl_lang_maidentounge.2da: Invalid 2da format in D:/c++/projects/nwnchargen/nwnchargen/run/2da/csl_lang_maidentounge.2da - unrecognized magic line \"2DA\tV2.0 Edit this somehow so its different its chondath now\""
"csl_lang_mulhorandi.2da: Invalid number of columns at line 24. 3 expected, 2 received. Columns: 21;^u"
"csl_lang_netherese.2da: Invalid number of columns at line 44. 3 expected, 2 received. Columns: 41;^O"
"csl_lang_nexalan.2da: Invalid number of columns at line 19. 3 expected, 2 received. Columns: 16;^p"
"csl_lang_oillusk.2da: Invalid number of columns at line 44. 3 expected, 2 received. Columns: 41;^O"
"csl_lang_orc.2da: Invalid number of columns at line 10. 3 expected, 2 received. Columns: 7;^f"
"csl_lang_payit.2da: Invalid number of columns at line 19. 3 expected, 2 received. Columns: 16;^p"
"csl_lang_raumvira.2da: Invalid number of columns at line 20. 3 expected, 2 received. Columns: 17;^q"
"csl_lang_reghedjic.2da: Invalid number of columns at line 19. 3 expected, 2 received. Columns: 16;^p"
"csl_lang_roushoum.2da: Invalid number of columns at line 36. 3 expected, 2 received. Columns: 33;^G"
"csl_lang_ruathlek.2da: Invalid number of columns at line 44. 3 expected, 2 received. Columns: 41;^O"
"csl_lang_shou.2da: Invalid number of columns at line 19. 3 expected, 2 received. Columns: 16;^p"
"csl_lang_tabaxi.2da: Invalid number of columns at line 32. 3 expected, 2 received. Columns: 29;^C"
"csl_lang_telfir.2da: Invalid number of columns at line 48. 3 expected, 2 received. Columns: 45;^S"
"csl_lang_terran.2da: Invalid number of columns at line 56. 3 expected, 2 received. Columns: 53;danish"
"csl_lang_thayan.2da: Invalid number of columns at line 24. 3 expected, 2 received. Columns: 21;^u"
"csl_lang_thresk.2da: Invalid number of columns at line 35. 3 expected, 2 received. Columns: 32;^F"
"csl_lang_tradetongue.2da: Invalid number of columns at line 19. 3 expected, 2 received. Columns: 16;^p"
"csl_lang_truskan.2da: Invalid number of columns at line 19. 3 expected, 2 received. Columns: 16;^p"
"csl_lang_tulung.2da: Invalid number of columns at line 19. 3 expected, 2 received. Columns: 16;^p"
"csl_lang_turmic.2da: Invalid number of columns at line 56. 3 expected, 2 received. Columns: 53;swedish"
"csl_lang_uluik.2da: Invalid number of columns at line 115. 3 expected, 2 received. Columns: 112;finnish"
"csl_lang_waan.2da: Invalid number of columns at line 19. 3 expected, 2 received. Columns: 16;^p"
"csl_lang_yipyak.2da: Invalid number of columns at line 56. 3 expected, 2 received. Columns: 53;maltese"
"csl_lang_ysgard.2da: Invalid number of columns at line 36. 3 expected, 2 received. Columns: 33;^G"
"csl_lang_yuanti.2da: Invalid number of columns at line 58. 3 expected, 2 received. Columns: 55;catalan"

It might make sense to check if those language work as expected...

Basically, bioware 2da spec says that if a word contains spaces, it should be enclosed in double quotes, and it is forbidden to use tabs.
Printed lines go against this convention.

[NegInfinity]

Based on in-game observations, at least some of the languages on this list are broken in game and do not translate properly. (giant language, assassin's cant, etc)

It is a quick fix, I could probably do it myself, as long as someone is willing to plug the fixed languages in.

[NegInfinity]

Alright, so I've found at least four broken languages.

Those include cant, assassin cant, ingan, and drow sign. Oh and a "maindentongue".
Those definitely do not work in the game, and their formatting is horribly broken. They have too many columns, and in the game they do not work at the moment, as far as I know.

The other languages which *might* have problems are:
alambik, aquan, bothii, celestial, chardic, chessic, chultan, dambrathan, dtarig, durpari, easting, goblin, halruaan, han, illuskan, illuski, imaskar, jotun, kozakuran, lantanese, logos, loross, mulhorandi, netherese, nexalan, orc, payit. raumviram, reghejic, ruathlek, shou.

Those occasionally have a line with less columns, meaning someone either skipped a letter or wanted to indicate whitespace, but never bothered to write " ".

Either way.

Here are fixes for the "horribly broken" languages:
https://mega.nz/#!uxExwJhS!yrhr-36uhvHL ... ptDBBB_Kb4

Would be just great if someone tested and integrated this.

[Rasael]

Almost all the 2da language files are courtesy of PainOfDungeonEternal, from the old NWN-2 Bioware forums. My own work is that we are able to load them into language objects and translate english into whatever sentence you had selected.

We've had a few offers by people to tweak the dictionaries and finish them. But no one ever reported back: until now!

They are not horribly broken in the way you're assuming they are (it sounds too negative to me, atleast). If a line doesn't make sense to the parser it skips that line. They are simply loading very slowly and gradually. A long term goal is to implement the translation through Skywing's C# wrapper. That would make the whole process a lot quicker.

I'll ask Duster to implement the tweaked dictionaries. Very kind of you to take the time and contribute.

[NegInfinity]

They are not horribly broken in the way you're assuming they are (it sounds too negative to me, atleast). If a line doesn't make sense to the parser it skips that line. They are simply loading very slowly and gradually.

Well. Horribly broken means that in case of, say, assassin cant almost all lines were malformed.

A long term goal is to implement the translation through Skywing's C# wrapper. That would make the whole process a lot quicker.

Would you mind throwing some documentation about that?
C#-based translation would be quite trivial, but there's potential issue with memory use. C# doesn't have any decent mechanisms to control object lifetimes and uses mutable strings. Meaning if C# portion sits in the same process as the server (which is 32 bit and already pushes memory use to the limit), it is possible to end at the mercy of Garbage Collector. But then again those are just strings and C#'s GC by default might be much better than, say, GC in UnityEngine...

[Aspect of Sorrow]

NWNX4 C# Wrapper , Skywing's Blog regarding this

I'll chime in as I've been spearheading this with a framework for the team to follow. Disposal will be handled automatically in the dispose/finalizer for the managed code under our specific template.

which is 32 bit and already pushes memory use to the limit

I've started on work that supports launching injection at module start to a separate managed / CLR process.

[NegInfinity]

I've started on work that supports launching injection at module start to a separate managed / CLR process.

What are you gonna use for IPC?

[Aspect of Sorrow]

The convenient one, since we're not going to be firing this up in Linux/Solaris anytime soon. Considering SM but the bottleneck will never exist on this end even under named pipes.

[NegInfinity]

which is 32 bit and already pushes memory use to the limit

I've started on work that supports launching injection at module start to a separate managed / CLR process.

The convenient one, since we're not going to be firing this up in Linux/Solaris anytime soon. Considering SM but the bottleneck will never exist on this end even under named pipes.

Opinion:
I gave it some thought, and I'd advise against investing heavily into this... if your goal is "to improve the server". It is one of those things that are incredibly fun to tinker with then one month later you realize that you've run into some sort of issue you didn't foresee. There's plenty of things to fix before making IPC-dependent addons... although a C#-based script here and there probably wouldn't cause any issues.

[Aspect of Sorrow]

Pushing across a large sum of objects, and requesting their location in shared mem has opened up a multitude of incredible opportunities, one of which is reset surviving DM objects without needing to query against SQL. NPC spawn systems can be elaborate via this mechanism. We can't replace everything, but we can make a massive dent in nwserver's performance and limitations.

[NegInfinity]

Pushing across a large sum of objects, and requesting their location in shared mem has opened up a multitude of incredible opportunities, one of which is reset surviving DM objects without needing to query against SQL. NPC spawn systems can be elaborate via this mechanism. We can't replace everything, but we can make a massive dent in nwserver's performance and limitations.

I just see investing heavily into IPC-based solution a potentially high risk (that can open up can of worms of problems that were never seen before), while occasionally using C# script in the same process (what Rasael said) as a good idea. If the framework you mentioned were already completed and fully debugged, that would've been a different story.

That's based on my experience/practices I employ.

In the end it is your time to spend.

What I wrote is my opinion. Do what you want about it.