I think I broke something.

[BFG]

Decided to reinstall SMAC/X from scratch today, after running into some general computer problems.  Big mistake.  Now, every time a Secret Project movie plays, my game crashes at the conclusion, after returning to the map and before the Datalink window appears.

I'm using Scient 2.0 installed on top of the official patches.  (Haven't gotten Thinker installed yet.)  I am NOT using PRACX.  DirectDraw and ForceOldVoxelAlgorithm are both off.  Playing on a Windows 10 x64 machine with an original (2002 CD) edition of SMAX.  Before the reinstall the machine had no problems.

Things I've tried with no effect:
--Turning on ForceOldVoxelAlgorithm
--Playing in Windows XP SP3 Compatibility Mode
--Turning off NVidia GPU graphics enhancements such as AA and VSync
--Letting the movie play all the way through

Interestingly, the intro movie does not cause a crash.

Anyone have any ideas?  Or maybe there's a troubleshooting thread I missed?

[Buster's Uncle]

It's probably a W10 issue with the movies - that's all I know.

[bvanevery]

I use the latest GOG version on Windows 10.  It's had various patches put into it.  I don't know what all they are or what they do.  You could try to learn what they actually did.  Or when it goes on sale sometime, you could buy it for $1.50.  Not on sale, $6.

I have "DirectDraw=0" in my .ini file.  That's so I can use the native resolution of my laptop screen.

I have ForceOldVoxelAlgorithm=1, which is the default for GOG, and maybe for all versions.

Admittedly I've had Secret Project videos turned off for a few years now.  I suppose I'll turn it back on in order to help you out with diagnosis.  There were times in the past when I thought it caused instability.  However Windows 10 has had more releases now, so who knows, maybe they fixed something by now.

[BFG]

I appreciate the help!  Yeah I may end up throwing in the towel and buying from GOG.  In the meantime, if I can’t get it working I will just turn off the videos.  I’m hesitant to though as there is something strangely charming about their 20 year old CGI-ness.

I’m also wondering if remastering the videos in a higher definition would make a difference.  May try that on one or two if all else fails.

Interesting that ForceOldVoxelAlgorithms is on by default in GOG.  I’ll make that switch as well.

Strange that it always worked in Win10 till now...

[BFG]

Found the root cause.  The breakthrough came when I switched from winning Weather Paradigm to Human Genome Project.  The latter one works fine.

It's not a display issue at all.  Turning off videos doesn't solve it.  Instead, there are some major problems with the helpx file included in the Scient 2.0 executable.

I went through all of the topics in Datalinks, and trying to access any of the following will cause an instant CTD:
Construct Bunker
Construct Condenser
Construct Sensor Array
Construct Tidal Harness
Geosynchronous Survey Pod
Hybrid Forest
Longevity Vaccine
Naval Yard
Orbital Defense Pod
Perimeter Defense
Plant Fungus
Tachyon Field
Temple of Planet
Tree Farm
Weather Paradigm

...Any idea what's going on?  I assume it has something to do with the new $LINKs added to Helpx.  Attached for reference.  (If I knew how to come up with the numbers in each $LINK I could do some fixing.)

[BFG]

I think I figured it out...there's a character limit for each row of text - I assume 255 but haven't confirmed.
If you carriage return in helpx then the game will accept it.

Causes crash:

(click to show/hide)

Code: [Select]

Increases terraforming speed by 50% for all tasks except for $LINK<Remove Fungus=90010>. Your $LINK<Formers=30001> may build $LINK<Condensers=90012>, $LINK<Boreholes=90014>, and $LINK<Echelon Mirrors=90013>, and may $LINK<Raise=90016>/$LINK<Lower=90017> terrain or $LINK<Drill to aquifer=90015> even if the appropriate technologies have not yet been discovered.

Works:

Code: [Select]

Increases terraforming speed by 50% for all tasks except for $LINK<Remove Fungus=90010>. Your $LINK<Formers=30001> may build
$LINK<Condensers=90012>, $LINK<Boreholes=90014>, and $LINK<Echelon Mirrors=90013>, and may $LINK<Raise=90016>/$LINK<Lower=90017>
terrain or $LINK<Drill to aquifer=90015> even if the appropriate technologies have not yet been discovered.

[bvanevery]

Huh.  If you're correct, then it proves that applying unofficial patches isn't a free move.  You can get new bugs that way.  It's not a given that someone dinking around with ASM code or .txt files, is improving the bug situation overall.  Such things in commercial production require testing.  Well, unless the company just throws it over the fence to consumers and lets them report back, as Microsoft historically has done.  Anyways, I'm just saying one should probably be aware of the level of testing engineering that goes into a one person .exe patching effort.  Not that high, and in fairness, not that many people to test and vet the new stuff either.  Caveat Emptor, YMMV.

I would also note that your CODE section is seriously breaking the left-to-right formatting of my Firefox browser, due to the extremely long lines.  I have to scroll back and forth horizontally to type this up now, and to hit "Post".  If such lines can mess up modern web browsers and websites, don't call me shocked if they also mess up SMAC!  Firaxis probably knew what the limits were back in the day, during production.  As opposed to making something bulletproof that would withstand the test of time on arbitrarily large screens.

[BFG]

Yeah, sorry about the formatting issue...but it certainly gets the point across!

I replaced the "improved" helpx file with the original and all is well now.  I'm tempted to go try to fix Scient's version, but I've seen so many typos and superfluous information in the additions, I'm not particularly keen to do so.  (To Scient's credit, I believe it was actually done by someone else and just incorporated into the patch.)

[bvanevery]

I don't blame you for not getting into it.  I will say, I have a "hardcore" open source background.  Disciplinarianship is something that has to be done in open source, or various things simply never get done.  When it's only 1 person banging away on an .exe, and nobody else is following the work, or invested in its success, you really rise and fall on the discipline of that 1 person doing whatever they're doing.  And in the real world, that 1 person usually has a shelf life of a few years at best, before they burn out on their $0 project.  The internet is littered with the bones of thousands upon thousands of abandoned projects.  I've looked at resurrecting the bones of these occasionally, hoping maybe I'd find something useful to my purposes.  It has never, ever happened.  Even active, currently developed projects, I've had trouble seeing how they'd benefit me.  That could partly be about me, but I think it's mostly about the social process of code, as an act of engineering.

I've known how to package up Yitzi's patch for greater compatibility with other stuff.  I've declined to do that busywork.  From an open source standpoint, it would be some simple scripting in something like Python.  But it's still something someone has to actually do, actually test, actually maintain.  Who has the incentive?  I blew all of my incentive on modding the crap out of SMAC.  I knew it was going to be a struggle to have the discipline to get that done, let alone a random month or two of programming gruntwork on top of it.  I did triage about my plans and ambitions, and said, Yitzi's patch ain't worth enough for me to put any effort into it at all.  I ain't pickin' up that torch, I've got other fish to fry.

The important thing is to do something.  But I fully understand if someone doesn't want to be someone else's janitor.  I've been in that career situation before, and it ended badly.  I'd say "never again" is my response to that.  Don't need a person year of my life wasted again, doing good engineering, on something that ultimately doesn't matter to much of anyone.

[BFG]

I can certainly understand burnout, particularly on a project done in one’s own spare time for no reason other than one’s own interest...

On a related note, I decided I am willing to fix this up if people would actually use the revised Datalinks.  But I’d like to have people with more knowledge of the mechanics than I, review the file as it currently stands and tweak it by adding/deleting text where needed.  I will work through the crashing issues and minor stuff (typos).

The source of the crash is more intricate than I thought.  I created a new entry for Heavy Transport to use for testing, and even a 600 character line did not cause a crash.  Maybe it instead has to do with long rows which have links, or links that end up split across rows in-game?  If it’s the latter, that could be difficult to resolve, because different resolutions would cause different break points.

Of course it is only worth the bother if people will actually read the things.

[Buster's Uncle]

Okay; I can't take the broke width any longer.  Sorry for the edit...

[BFG]

Okay; I can't take the broke width any longer.  Sorry for the edit...

No worries, it WAS pretty annoying.  And now that we know width by itself isn't the problem, it's kind of a moot point.

[scient]

Well that's unfortunate. 

I tracked down where the issue is. There is a function that handles "popups" that renders Datalinks and other parts of UI called BasePop::start(). This parses through help/x.txt entry and fixes up internal links and any style options. This function calls another function parse_string(input, output) that takes input read from files and copies it to output with some other logic related to parsing it. I've actually already partially decompiled this particular function.

The crash comes into play because BasePop::start only allocates 256 bytes to the stack variable passed to parse_string output parameter. This in turn executes a memcpy call that can overwrite memory outside of BasePop::start stack. SMAC/X was coded in a time before _s functions. This essentially causes a stack overflow and crashes the program when function tries to return to original calling function (Popup::start).

BFG is correct, I had little to nothing to do with revision to most of the text files. The only edits I made are documented in readme related to bugs I came across while working on the binary. If I remember correctly, most were minor. All my patches I meticulously tested for bugs. As far as I know, there has only been one bug my patch has triggered and that I ended up rolling back related to hardcoded unit limits. That is also around the time I decided to scrap doing any more piecemeal patching and move on to full scale decompilation.

My suggestion would be to bump any text after 255 mark to a new line (255 + null). That way, when parse_string is called that the stack doesn't get corrupted. There might be other functions that call parse_string where this isn't an issue. I tried to see if a quick and dirty increase to the stack size would solve the issue but it would require more time to fix offset references in logic. I'm juggling a couple different things right now and my time is better spent trying to get the first iteration of OpenSMACX dll out the door.

[BFG]

Thanks scient!  That perfectly explains my test results.  The problematic function wasn’t called on my 600 character test string because there was nothing to render.

I am going through helpx right now to try to fix up various issues including this one and some typos.  Stay tuned.

[bvanevery]

This function calls another function parse_string(input, output) that takes input read from files and copies it to output with some other logic related to parsing it.

Good sleuthing.

my time is better spent trying to get the first iteration of OpenSMACX dll out the door.

Is that this GitHub project?  Looks like no commits in 4 years.