Home » MODDING HQ 1.13 » v1.13 Idea Incubation Lab  » Logical Bodytypes
Logical Bodytypes[message #257910] Fri, 30 July 2010 03:10 Go to next message
Bio

 
Messages:62
Registered:April 2009
Location: Rhineland-Palatinate/Germ...
Hi there,

It's been well over a year since I posted the last time. Sorry for the long absence.

I decided to create a new thread for this (the last one went a little astray). I hope that is OK. As you know, last year I presented my framework for batch rendering multi-layered, shadow-casting animations and for batchprocessing the rendered images into STI archives ready to deoploy (since the applying of the palette is in the mean time also done automatically, there is actually zero user input required, you simply start the script and let it do the heavy lifting). I was a little disappointed that there was so little interest in actually using said framework; I guess it is because 3dsmax6 is a prerequisite (though newer versions might work as-well). If any of you have access to 3dsmax and would be willing to invest some time, that would be very much appreciated.
After realizing last year, that multi-layered body types were not gonna be a reality anytime soon, I didn't find the motivation to continue animating my model (by then, I had invested way more time in the automatic rendering process of pixel-acurate acurate layered animations (including layered shadows), that the little time I had spend modelling/animating didn't seem relevant). And since I am not a graphic designer, modeller, animator or somesuch by any stretch of the imagination, but merely a programmer, I set myself the goal to implement multi-layering, before doing anymore modelling/animating (especially since I was still hopeful, that some other people would chime in to help on that front). I started analyzing the code, then I contacted BirdFlu, because he had done some preliminary work in that direction. He provided me with the stuff he had already done (in terms of externalizing the animation database). I din't end up using any of his code, though I did take over some of his ideas. However, shortly after I had set out to implement layering, my vacation was up and I had very little time (and even less interest) to keep on working on that stuff. I have a 1.5h drive to work + the usual massive overtime so I'm rarely getting home earlier than 9pm. Let's just say when I get home I'm way too tired to continue doing basically the same stuff I do in the office...). This week I had vacation and since I didn't have anything else on my queue I went back to the layering function. And I have progressed quite nicely and far enough to have something to show.

Since it is so hard to add a new physical bodytype to the game (since everything is hardcoded), I decided to instead implement what I termed logical body types. The idea is to let the current animation/surface model as-is. And implement something on-top of it. Whether this will be the implementation that will be used in the end, or not (if BirdFlu gets around doing his thing, which basically would be to externalize all animation data, requiring a substanial re-work of the current implementation. But would probably be cleaner and also faster). So, because my system is build on-top of the old one (and the integration in the rendering process, is a little, well..., let's just leave it at that). My datamodel is pretty much finished. I am completely by-passing the caching mechanism (to safe memory only a sub-set of the surfaces used for rendering the animations is kept in-memory). Instead I am loading all surfaces (until I have a good reason for not doing so). Currently the memory requirement of the game is just above 100MB (with all my surfaces in memory). Since it is not 1999 I thought it prudent to not spent much time on a feature, that might not be needed anymore). I had my doubts about the efficiency of my system, but as I said already, that seems to have been unnecessary. The CPU load for the JA2 process is at under 2% while rendering multiple mercs on my machine, with a couple of logical bodytypes, a number of layers and a number of filters defined.

OK, guess it's time for some details. There are 4 XML files that have to be altered to implement a logical body type.

AnimSurfaces.xml:
here the surfaces are defined. With animation specific data and links to the STI container for the animation and the struct type file.

Layers.xml:
contains the layer hierarchy. Currently there are 15 distinct layers defined (how many will be fully realizeded is basically merely a matter of modeling/animating and editing XML files. For instance it would be trivial to add different heights, simply scale the model and all props with it down and re-render the whole thing). For now I have implemented 3 layers (corresponding to what STI files I had ready from last year). Namely: the body (only in the rifle-bearing posture, since this is the one I had done last year), the vest and the right hand (or weapon). I will probably add a backback next. Because it is easy and to test how the layering concept copes with backpack + body + weapons layers simultaneously.

Filters:
this is the logical component of the logical body types. Mind you filter is probably not the right term, but it will do for now. Anyway, these filters are quite powerful. You can match against a ton of different stats. And there is a nice arsenal of operators at your disposal. A small except:
You can match against:
the physical bodytype, the merc's name, nickname, against his skills/expierence, his sex, his traits.
you can match agains team affiliation, against merc type and of course and most importantly against stuff he wears (for every single pocket).
Operations availabel are: and, or, not, equal, greater than, less than, between and in. This is all already implemented and should be ready tobe uses.

BodyTypes:
the final data file. Brings everything together. Here are all the logical body types defined. A logical body type consists of a filter (The system determines the first body type that matches and uses this for rendering) and of an arbitrary number of animation surface layer elements that belong to a certain animaton sate (WALKING, STANDING, aso.), to a certain layer, links to one of the animation surfaces defined in AnimSurfaces.xml and of course can have a filter assigned (for instance IsAk47TypeGun). As you see this system is extremely flexible. You can define new layers and bodytypes. By just editing the XML files.

The data model is pretty much completed (I have some minor issues on my TODO list, but it is working flawlessly and also much more efficient than I had hoped). The rendering part seem to work quite nicely. However I still have a problem with some artifacts (black pixels). Might be connected to the rendering of the shadows. However, it is not that bad and will certainly be ironed out soon.

Sorry for this unorganized write-up. I usually try do edit out all the typing errors and superfluous sentences that I'm prone to produce and I'm afraid this time it is especially bad - sorry for that. I have practically worked on this for 2 days str8 and find it quite hard to articulate a coherent sentence at the moment Razz (and to spell it correctly too - ugh!)

Here are some screenshots showing the layers in-game (sorry for not providing a video, but the animations are not exactly professional grade anyways Razz) and some of the stitched PNGs produced by the batch rendering framework (these PNG are than cut and cropped and added to the STI containers). If someone wants to try this stuff, I can provide a build with my additions (using the current tip revision) and the needed data files.

OK. The screenshots are not very enticing (and I probably should have made a gif animations to visualize the differences better). However, fully layered, flexible body-types are a reaility!

http://img64.imageshack.us/img64/7562/bild8rj.png

For kicks: the filter configuration used for these screenshots.



	
	
		
			
			MERC_TYPE__AIM_MERC
		
	
	
		
			0
		
	
	
		
			0
		
	
	
		
			606,616
			14,25,26,35,347,602,603,608,655,656,706,708,742
		
	
	
		
			795,797
			12,23,346,715,797,630,632,633,662,762,763,769,1066,1070,1072,1077
		
	
	
	
	
	
	
		
			MALE
			SOLDIER_CLASS_ARMY
		
	
	 
		
			FEMALE
			Len
			Ivan
		
		
			MERC_TYPE__AIM_MERC
			Bun
		
	




And the coresponding body type xml:



	
	
	
	
		
			
			
			
			
			
			
			
			
			
			
			
			
		
	



The following are just the monolithic stitched renders (with palettes applied). Don't know why only the preview for the middle one is being displayed. They are all the exact same size and uploaded the exact same way.



http://img43.imageshack.us/img43/859/stitchedweaponsak47.th.png




[Updated on: Fri, 30 July 2010 03:12] by Moderator

Report message to a moderator

Corporal
Re: Logical Bodytypes[message #257913] Fri, 30 July 2010 03:35 Go to previous messageGo to next message
Beka is currently offline Beka

 
Messages:86
Registered:November 2009
Location: Vault 13
A-W-E-S-O-M-E.

Report message to a moderator

Corporal 1st Class
Re: Logical Bodytypes[message #257915] Fri, 30 July 2010 04:19 Go to previous messageGo to next message
Slax is currently offline Slax

 
Messages:1411
Registered:July 2006
Location: People riding polar bears...
ALGH! Progress at last. I'm crying right now, the savior has finally returned!

http://img693.imageshack.us/img693/808/fuckyeahg.jpg

Kudos.

Report message to a moderator

Sergeant Major
Re: Logical Bodytypes[message #257921] Fri, 30 July 2010 07:18 Go to previous messageGo to next message
Kaerar is currently offline Kaerar

 
Messages:2022
Registered:January 2003
Location: Australia :D
That setup looks excellent. I think a merger of this and the sticky is needed Smile

Report message to a moderator

Lieutenant

Re: Logical Bodytypes[message #257926] Fri, 30 July 2010 09:53 Go to previous messageGo to next message
Zarax

 
Messages:54
Registered:February 2007
Location: Italy
YES!

Report message to a moderator

Corporal
Re: Logical Bodytypes[message #257941] Fri, 30 July 2010 14:30 Go to previous messageGo to next message
Shadow21 is currently offline Shadow21

 
Messages:328
Registered:November 2001
Location: on route to San Hermanos
wow - just wow nice work

Report message to a moderator

Master Sergeant
Re: Logical Bodytypes[message #257953] Fri, 30 July 2010 18:19 Go to previous messageGo to next message
wanne (aka RoWa21) is currently offline wanne (aka RoWa21)

 
Messages:1961
Registered:October 2005
Location: Austria
Wow fantastic, great step forward. If you have anything that can be included in the 1.13 development svn source trunk (because we don't want to loose anything) just contact me.

Again, nice work!

Report message to a moderator

Sergeant Major

Re: Logical Bodytypes[message #257954] Fri, 30 July 2010 18:30 Go to previous messageGo to next message
lockie is currently offline lockie

 
Messages:3721
Registered:February 2006
Location: Scotland
Wowsers !!
Great stuff . :bluecool:

Report message to a moderator

Captain

Re: Logical Bodytypes[message #257957] Fri, 30 July 2010 18:54 Go to previous messageGo to next message
P.Val is currently offline P.Val

 
Messages:72
Registered:May 2010
Wow, it's really nice! It will bring quite a lot of "fresh blood" to an aging game. I hope it will not be thwarted by organizational problems and a full, official, and single-click installable version will be created from all the new improvements. (currently the new traits system, new attachment system and new ballistics and items system seem to diverge on separate paths...)

Report message to a moderator

Corporal
Re: Logical Bodytypes[message #257958] Fri, 30 July 2010 19:22 Go to previous messageGo to next message
CptMoore

 
Messages:224
Registered:March 2009
What about overlaying current bodies? Those new ones don't fit the JA style.

Report message to a moderator

Sergeant 1st Class
Re: Logical Bodytypes[message #257969] Fri, 30 July 2010 22:18 Go to previous messageGo to next message
Bio

 
Messages:62
Registered:April 2009
Location: Rhineland-Palatinate/Germ...
Well the right way to go about that, would be to simply make a new model in the JA2 style (I heard they used some of the stock poser figures).

Mind you this would require that we have the full set of animations first. If we indeed would have all animations, the process to generated a complete new bodytype including all layers would be a piece of cake (requiring quite a bit of rendering time, but little actual work):

* You'd simply load up the scene with the animated skeleton (or however it is called, can't recall), the props, the lighting, surfaces and cameras needed for rendering. Import/add your new model. Rig it (attach it to the "skeleton"). Then you'd choose the props you like (fitting as many of the pre-existing ones to your model. Weapons for example are independent of the shape of the body) or add new ones. And at that point you would be ready to go. You'd press start rendering. And then would wait, which could be, depending on your CPU and the number of props, quite a while (see my next post for some musings about how long that while may actually be).
Of course you don't have to render everything in one go. In fact the framework allows to make incremental builds, no problem.

* Now that you'd have a set of the rendered images, you would run the batch script to build your set of STI files (I also intend to extend the framework to automatically create an XML file with the animation surface definitions to be copied over to AnimSurfaces.xml. Or better yet enable the inclusion of external XML files, something I wanted to di anyways.).

* Now all you would need to do would be to define the logical body type itself. But of course it would make sense to create this automatically as-well (you'd define which filters to use for which prop, so at the end all you'd have to do is make sure that the filters existed. And since filters are meant to be rather general and can be used across different bodytypes, chances are the filters for your props are already in place).

Of course it would be rather trivial to paint layers over the existing stock animations (either by refering to the stock STI's in your logical bodytype, or by implementing drawing the default animations first). But for this to look decent you'd have to hand edit every single frame (you pretty much need pixel accuracy). To say that that would be a tedious work would be the understatement of the century. Please refer to the post below to get an idea of how much work that would be, even with just one single prop.

Report message to a moderator

Corporal
Re: Logical Bodytypes[message #257972] Fri, 30 July 2010 23:07 Go to previous messageGo to next message
Bio

 
Messages:62
Registered:April 2009
Location: Rhineland-Palatinate/Germ...
OK, that got me thinking about how the rendering process and the memory requirements for a bodytype would scale with the number of props.

Let's make an estimation as to how long it might take. Let's take the optimal/extreme case were we have props for all currently laid out 15 layers. And let's assume we have an average of 4 layers (that is probably the maximum which would make any sense, since with the minute amount of pixels at our disposal, you'd be hard pressed to discern more than a couple of different variations, especially for the smaller layers. Unless we could color the stuff independently of the individual character's palettes, which is something on my list and which I'm currently researching). I think there are about 60 animations that are relevant. Let's say they have an average length of 16 frames. We need to render in 8 different directions. So that gives us 60 * 16 * 8 * 15 * 4 = 460800 seperate renders. Let's say you're machine takes 2 second to render one frame (there are two renders being done per frame, one with antialiasing but without shadow, one without AA and with the casted shadow). That would result in a total render time of over 10 days (that is, with one of the cores of my CPU doing the rendering, so there probably is a lot of room for improvement). Like stated before incremental rendering is no problem. But that doesn't change the net time needed.

Anyways, with close to half a million frames per bodytype and say .5 KB per frame, that would be about 250 MB of memory for just one bodytype. Since there should be at least say 4 different bodytypes, that scenario would require us to implement a caching strategy, since we would probably cross the 1 GB mark easily and there probably are still people out there who could not afford a process allocating that much memory. So let's say we'd limit ourselves to, say an 90 MB per bodytype (so we would stay easily below 512 MB with 4 bodytypes). That leaves us with about 22 different props (which is kind of a bummer, but that would be a sh*tload of possible combinations). The render time would be reduced accordingly.

So after this crude analysis, I think it would be prudent to set a mid-term goal of, say 8 props. Should we cross that limit we shall reevaluate and either find that there are sufficient reserves for more props, or implement caching). Limiting to 8 props of course means that only some of the layers will be realized. The most important ones are weapons I suppose. I propose the following configuration for phase 1 of the project:

Helmet: 1 prop for all variations of helmets (excluding non-armored hats)
Vest: 1 general vest prop.
Left/Right leg: 1 general prop rendered if anything is attached to either leg

That leaves only 5 for weapons:

* AK47 obviously
* AR15 obviously
* Bullpup designs
* Maybe one for a general designated marksman gun (and all 7.62 mm ARs) Or maybe render these as AR15 as-well.
* Sniper Rifle (+ rifles and ARs without a pistol grip)
* Machine gun
* MP5 (for all SMGs)

Nothing for now for pistols or for dual wielding any 2 handguns

That would violate the limit of 8 props, but of course 8 isn't a hard limit. Another idea I had was concentrate more on the attachments of the guns. So for instance two props one with a bipod one without but sued for all pistol grip rifles with a bipod attached (including AK, AR-15, all applicable 5.56 mm and all 7.62 mm ARs, sniper rifles with postol grip).

I don't know.

Thoughts?

Report message to a moderator

Corporal
Re: Logical Bodytypes[message #257977] Sat, 31 July 2010 00:15 Go to previous messageGo to next message
DepressivesBrot is currently offline DepressivesBrot

 
Messages:3658
Registered:July 2009
I'm wondering, how do you recognize an AK type gun? Do you need a new tag in items.xml ( or something like that?)
Also, is the difference between an AK and an AR15 really that pronounced at the pixel counts we are talking about? I can't really see attachments making a big difference either...

Anyway, here my proposal for weapons:
- a generic assault rifle (medium long weapons, maybe an extra category for bullpups later on)
- a generic MP /SMG (compact, used for everything from P90 over 'true' SMG (MP5), to short carbines (HK416 10"), shortest in this list)
- a generic sniper rifle (longest in the list, relatively 'thin')
- a generic machine gun (bulkier than the others, length somewhere between AR and SR)
- a rifle without pistol grip for shotguns, rifles and some SR (length about the same as for MG)

I think with the available space, it's more important to distinguish certain categories (AR/SR/MG) than to differentiate between 'styles' (AK/AR)
For that pupose, this selection should display a big chunk of our available arsenal reasonably accurate.

Report message to a moderator

Captain

Re: Logical Bodytypes[message #257980] Sat, 31 July 2010 01:25 Go to previous messageGo to next message
Bio

 
Messages:62
Registered:April 2009
Location: Rhineland-Palatinate/Germ...
Sounds reasonable.

And it corresponds to the weapon classes in-game.
As for distinguishing between different guns, take a look at the filter.xml I posted in my first post. Basically right now, you would be required to maintain a list with the item id's (or a range). I thought about taking the item name, but I don't think uniqueness is guaranteed, so I opted for the ids. If uniqueness is guaranteed I would prefer the name obviously (on the other hand, I might just issue a warning if a name is specified that isn't unique, that should suffice). But for what you proposed that wouldn't be necessary anyway (corresponds directly to the in-game classification). Also might add the cartridges type as a possible filter criterion type.

Btw: a backpack should be displayed if worn. Since that would actually serves a practical purpose. Would be a good reminder that you can't climb until taken off.

Report message to a moderator

Corporal
Re: Logical Bodytypes[message #257981] Sat, 31 July 2010 01:36 Go to previous messageGo to next message
Bio

 
Messages:62
Registered:April 2009
Location: Rhineland-Palatinate/Germ...
gonna add a check for armor class. Makes it easy to define a filter for vests and helmets for which the props should be displayed (as opposed to an officer's hat and a 'Deidranna rules' t-shirt).

Bio

Helmet: 1 prop for all variations of helmets (excluding non-armored hats)
Vest: 1 general vest prop.
Left/Right leg: 1 general prop rendered if anything is attached to either leg

Report message to a moderator

Corporal
Re: Logical Bodytypes[message #257985] Sat, 31 July 2010 02:27 Go to previous messageGo to next message
Mauser is currently offline Mauser

 
Messages:756
Registered:August 2006
Location: Bavaria - Germany
Hey Bio, great to hear from you again. even greater that you have kept working on your project!

having fully flexible multi-layered sprites indeed is a long time dream come true.

but do you think there would be a chance to convert your script to a format useable by open source rendering software too, so it would be possible for more people to actually use it and produce new content with it? because limited interest is only natural if the needed software prerequisites to use such a feature are too high, exclusive and costly for the average modder.

besides that, what kind of engine limitations have you encountered in case of the current 1.13 engine?
do you see any chance to more or less easily revamp and expand the current JA2 engine to allow for more flexible and high quality graphics or do you think, something like creating a dual engine on top of standard JA2 graphics engine (to be able to render both old and new graphics alternately) would be the way to go?

as i understand, your current multi-layered sprite approach on top of the old sprites would be mainly very memory intensive.

do you have any idea, how other games like Diablo or Fallout series have solved that problem?

and have you built your concept from the ground up or have you looked at other games with similar graphics first?

Report message to a moderator

First Sergeant
Re: Logical Bodytypes[message #257990] Sat, 31 July 2010 04:38 Go to previous messageGo to next message
Bio

 
Messages:62
Registered:April 2009
Location: Rhineland-Palatinate/Germ...
Mauser
but do you think there would be a chance to convert your script to a format useable by open source rendering software too, so it would be possible for more people to actually use it and produce new content with it? because limited interest is only natural if the needed software prerequisites to use such a feature are too high, exclusive and costly for the average modder.


I will have a look at the blender scripting system.

Quote:
besides that, what kind of engine limitations have you encountered in case of the current 1.13 engine?
do you see any chance to more or less easily revamp and expand the current JA2 engine to allow for more flexible and high quality graphics or do you think, something like creating a dual engine on top of standard JA2 graphics engine (to be able to render both old and new graphics alternately) would be the way to go?


I'm confident there are others that are much more qualified to answer that question than me. While I'm a software developer, I'm not a game developer and my expertise in that regard is rather limited.
Also I did not spend a lot of time analyzing the JA2 graphics engine. All I did was a proof of concept last year that my approach, which basically simply consisted of blitting n succesive surfaces instead of 1, would work. After I had established that, the main piece of work had to go into the datamodel basically and the logic for determining when to blit which surface. The required code changes to the rendering functions were rather trivial. But personally I'm convinced that adding a second graphics engine for rendering live in 3d would be quite achievable.

Quote:
as i understand, your current multi-layered sprite approach on top of the old sprites would be mainly very memory intensive.


Well, mainly because there is no caching implemented. Basically all it would take is only keeping surfaces in memory that are needed. So iterating over all soldier objects and keeping all surfaces that match (for all animation states), while not being very memory efficient, would lead to more constant memory requirements not directly depended on the number of props.
Another approach I had been thinking about, is loading the animations directly from the disk, either with caching the last n animations, or by implementing a usage counter like in the JA2 engine, or even without any caching whatsoever! Modern hard drives have very low seek times and the required throughput is laughably small (like 50 KB for one animationstate). I'm fairly confident that that would work very well (and probably even would have 12 years ago). Remeber that you would not have constant streaming. And only a small disk access (probably like 8-16 ms, so in all likelyhood not perceptible) and only when an animation state changes (like when you take the vest off, or change posture from standing to crouching). And since JA2 is turn based, who'd care anyways? I mean the stock animations have fixed frame rates of what 20 fps tops (probably even 15 or less)? I mean with 20 fps you have a fixed delay of 50 ms between redraws. You might even be able to preload the needed surfaces in-between rendering cycles.

Quote:

do you have any idea, how other games like Diablo or Fallout series have solved that problem?


Not really.

But that gave me another idea. The STI containers contain uncompressed 8 bit bitmaps with usually a pretty big amount of background color and shadow. Which should be compressable nicely even with something as simple as RLE. Just tried zipping some of the STIs and I got around a compression factor of about 2.5:1 which is not bad at all (albeit less than I would have thought, they might already be RLE compressed after all). So that would definately also work, to keep the STIs in memory and than you would only have to operate on the memory to get your surface ready.
However, that is probably completely unnecessary, because I'm now convinced, that directly loading from HDD seems like a viable approach for a game like JA2 today. Which it would not have been for Diablo2 I presume. Since first of, there are now 10 years of HDD development between the two games (and seek times have continualy lessened) and of course Diablo2 as a real-time game could actually not afford the odd delay of a couple dozen ms, because some background process was accesing the hard drive at the same time, which would be perfectly alright with JA2 (and probably would go unnoticed).
And btw JA2 would occasionally have to load from disk too on the very first usage and the first usage after a surface got bumped (because it hadn't been used often enough). And I can't say that I have ever noticed that (though the cache might be so big, that the second case practically never happens anyways).

Quote:
and have you built your concept from the ground up or have you looked at other games with similar graphics first?


Pretty much from scratch I'm afraid. My objective was to implement the functionality I needed with the least amount of work possible. My simple concept for rendering the layers was already in place since last year and the datamodel was gonna be completely JA2 specific anyways.

Report message to a moderator

Corporal
Re: Logical Bodytypes[message #257991] Sat, 31 July 2010 04:51 Go to previous messageGo to next message
Bio

 
Messages:62
Registered:April 2009
Location: Rhineland-Palatinate/Germ...
I wasn't thinking straight I'm afraid (it's almost 4am). Of course you would have to load the animation every time (so every fame) without a cache. That would be completely ludicrous of course. What I described would require a cache of size 1. Always holding the last surface for every soldier object. When that is implemented though it would be quite easy to also add a usage counter like the std animation surface handling to keep the n most often used surfaces for every soldier. Whatever that stuff should be really quite easy to implement. I will try to implement something like it this weekend, so no one has to worry about high memory consumption (or a limitation in props).

Report message to a moderator

Corporal
Re: Logical Bodytypes[message #257992] Sat, 31 July 2010 05:32 Go to previous messageGo to next message
Slax is currently offline Slax

 
Messages:1411
Registered:July 2006
Location: People riding polar bears...
Colored overlays for the guns would be nice but that's kind of obvious, I guess. Modern AK versus oldschool and such.

Report message to a moderator

Sergeant Major
Re: Logical Bodytypes[message #258023] Sat, 31 July 2010 16:21 Go to previous messageGo to next message
Toneone is currently offline Toneone

 
Messages:376
Registered:October 2008
Location: Germany
you can not belive how much i want those bodys for my game....AWESOME!

Report message to a moderator

Master Sergeant
Re: Logical Bodytypes[message #258046] Sat, 31 July 2010 20:11 Go to previous messageGo to next message
Mauser is currently offline Mauser

 
Messages:756
Registered:August 2006
Location: Bavaria - Germany
Bio
Well, mainly because there is no caching implemented. Basically all it would take is only keeping surfaces in memory that are needed. So iterating over all soldier objects and keeping all surfaces that match (for all animation states), while not being very memory efficient, would lead to more constant memory requirements not directly depended on the number of props.
Another approach I had been thinking about, is loading the animations directly from the disk, either with caching the last n animations, or by implementing a usage counter like in the JA2 engine, or even without any caching whatsoever! Modern hard drives have very low seek times and the required throughput is laughably small (like 50 KB for one animationstate). I'm fairly confident that that would work very well (and probably even would have 12 years ago). Remeber that you would not have constant streaming. And only a small disk access (probably like 8-16 ms, so in all likelyhood not perceptible) and only when an animation state changes (like when you take the vest off, or change posture from standing to crouching). And since JA2 is turn based, who'd care anyways? I mean the stock animations have fixed frame rates of what 20 fps tops (probably even 15 or less)? I mean with 20 fps you have a fixed delay of 50 ms between redraws. You might even be able to preload the needed surfaces in-between rendering cycles.


well, maybe that

Report message to a moderator

First Sergeant
Re: Logical Bodytypes[message #258048] Sat, 31 July 2010 21:29 Go to previous messageGo to next message
Bio

 
Messages:62
Registered:April 2009
Location: Rhineland-Palatinate/Germ...
Mauser

well, maybe that

Report message to a moderator

Corporal
Re: Logical Bodytypes[message #258059] Sun, 01 August 2010 01:40 Go to previous messageGo to next message
Yombie is currently offline Yombie
Messages:3
Registered:July 2010
Bio, amazing work. Keep it up, looking forward to this more than any other feature or mod.

Report message to a moderator

Civilian
Re: Logical Bodytypes[message #258119] Sun, 01 August 2010 21:04 Go to previous messageGo to next message
Mauser is currently offline Mauser

 
Messages:756
Registered:August 2006
Location: Bavaria - Germany
Bio
Last time I did not have any code to be released. The render framework I sent to lalienxx and to a couple of Russian guys he refered me to, that were also working on new models.

If someone would give me an SVN account, I could migrate from my local SVN server. Btw. my additions are all well contained in a saperate folder + namespace. And Changes to existing files are currently limited to init.cpp and the RenderTiles function.


well, that should be the very least of problems. Just contact RoWa21 and he

Report message to a moderator

First Sergeant
Re: Logical Bodytypes[message #258127] Sun, 01 August 2010 21:34 Go to previous messageGo to next message
Bio

 
Messages:62
Registered:April 2009
Location: Rhineland-Palatinate/Germ...
A short heads-up about what I managed to get done over the weekend.

The main changes:

* fixed the rendering problems (see my first post). This required to add some additional complexity in the object rendering loop, but it's still easily managable.

* implemented LRU caching (should be constant time, using a vector + a double linked list). Every logical bodytype has it's own cache with it's own size. That way bodytypes that are meant for single characters (or have fewer animation states or fewer props) can get a smaller cache as opposed to those meant for larger groups of different characters (more concurrent combinations). This should do nicely (at least equivalent to the JA2 animation caching).

With the new caching mechanism. There is no longer any need for restricting the number of props or bodytypes to some arbitrary limit (imo there can now be as many different bodytypes + props as the community is able to produce). And I'm close to calling the logical body type implementation finished.
There is one thing I'd still like to investigate. Namely CPU time. After my recent changes the CPU usage for the JA2 process hovers around 12%. I think to have noted that previously it was about 2% on my computer. Mind you this is only the case for a debug build. Release results in the expected 2%. It looks like some debugging specific function being called (but that is a huge jump for something like that). I did not find anything in the logfiles suggesting something logging excessively (and I didn't find anything in the code that could explain it). If someone could have a short look if their debug builds have a significantly higher CPU usage (tactical screen, no user input) than their release build. That would be quite helpful.
Other than that, there are still a couple of smaller features I'd like to add. But that is maybe 3 or 4 hours of work. After that I would like to release a test version for people to play around with (and uncovering potential bugs).

By the way. I just stumbled across kliFFotHx's thread. Has there been any communication with him since? It's unfortunate we never came in contact, because I could have optimized his editing process quite easily (he mentioned cropping by hand taking him a lot of time). Btw has he ever mentioned what his setup was, what he used for modelling/rendering? I also noted in the same thread Mauser's reference to an open source Fallout project and the potential use of some of their work. Am I correct in assuming that their models + animations have been done in Blender?

I had a look at Blender in the meantime. While it appears to be quite powerful, the time it would need to even get started on the basics would be extensive. I have a really hard time getting used to the controls and I would obviously need that at least to set up a basic scene.

Also I had a look at my old MAX render script. And the code is a complete mess (I adapted someone else's script to do what I needed, but the original script [and the general design of max scripts I venture to guess] are not easy to work with). So while what the script actually does is quite simple, getting it to run was quite a piece of work.

Btw. I previously stated, that one only had to click on render and all defined animations would be rendered automatically. This is currently not so! It would only render all defined props for ONE animation. So rendering a complete bodytype would be currently a process of at least 60 something seperate iterations.

The scrpt is still extremely useful in it's current form. And post processing still is completely automatic. Maybe one could adopt the use of a command line batch renderer for making it fully automatic. However in the lights of that (the bad code and the need to have multiple iterations), I wouldn't mind writing a python script (I love Python, should be much simpler than max script) for doing the batch rendering the way my post processing would need it. But I'm afraid the learning curve would be too steep with the amount of time I will have to spare in the next weeks/months. However if there was somebody proficient wih Blender and the scripting system and willing to do part of the work (especially the preliminary work/setting up the scene/providing an entry in the script), I would not mind taking part as well. Naturally I will put all my own feeble attempts at animating on hold (or maybe I do some more props for testing layering) until a concensus will be reached on a general process for animating (as I still would like to have one complete set of animations and use that for a multitude of diffent models) and rendering.


One afterthough I just had:

I think it would be a nice little project to create a logical bodytype for kliFFotHx's bodytype! For this to work he of course would have had (though it might work anyways) to use the same animation length's as the stock animations (as logical bodytypes depend on the animationstates of an underlying physical bodytype). Does anyone know if he has or not? And btw. is there another complete set of animations that could be used to realize a logical bodytype?

Report message to a moderator

Corporal
Re: Logical Bodytypes[message #258132] Sun, 01 August 2010 21:52 Go to previous messageGo to next message
Bio

 
Messages:62
Registered:April 2009
Location: Rhineland-Palatinate/Germ...
I forgot that there is one more open issue. As stated before for technical reasons my prop layer renders also have the shadow of the main body object (remeber that the main body has to obstruct the prop so that the background is rendered were the prop would be behind the main body object, so the object is not actually hidden in the scene and thus it casts a shadow and I have not found a trivial way to remove that shadow). Anyways, because of that the shadow of the body is rendered n times over. So it is pretty much black. See my screens in the first post. The shadow is supposed to be substantially lighter. There is a bunch of different ways how this could be rectified some of which would not require the actual implementation of the logical body types to change. Therefore it has low priority at the moment (and the almost black shadows are not so bad, if you wouldn't know that they are supposed to be lighter, you probably wouldn't notice).

Report message to a moderator

Corporal
Re: Logical Bodytypes[message #258142] Sun, 01 August 2010 23:05 Go to previous messageGo to next message
lockie is currently offline lockie

 
Messages:3721
Registered:February 2006
Location: Scotland
Bio , I admire the work you have already started , hope you have the will to carry on and finish !
Good luck :cheers:

Report message to a moderator

Captain

Re: Logical Bodytypes[message #258156] Mon, 02 August 2010 01:11 Go to previous messageGo to next message
UniversalWolf is currently offline UniversalWolf

 
Messages:140
Registered:June 2009
Location: United States
I salute you, Bio! Nice work.

I've been looking for an excuse to dive into Blender again, too... :headbanger:

Report message to a moderator

Sergeant
Re: Logical Bodytypes[message #258161] Mon, 02 August 2010 03:36 Go to previous messageGo to next message
Kaerar is currently offline Kaerar

 
Messages:2022
Registered:January 2003
Location: Australia :D
For an additional set of animations that are almost complete check out Renegade Republic. It has a bodytype with a cloak that replaces the BigMale bodytype.

Report message to a moderator

Lieutenant

Re: Logical Bodytypes[message #258176] Mon, 02 August 2010 11:19 Go to previous messageGo to next message
Logisteric

 
Messages:3199
Registered:December 2008
Location: B
we need a little fat lady for meltdown Smile

Report message to a moderator

Captain
Re: Logical Bodytypes[message #258185] Mon, 02 August 2010 14:36 Go to previous messageGo to next message
GeneralArthur is currently offline GeneralArthur

 
Messages:11
Registered:January 2008
Location: Potchefstroom, South Afri...
Bravo Bio Bravo!

Looking interesting so far!

Report message to a moderator

Private
Re: Logical Bodytypes[message #258206] Mon, 02 August 2010 18:46 Go to previous messageGo to next message
cdudau
Kaerar
For an additional set of animations that are almost complete check out Renegade Republic. It has a bodytype with a cloak that replaces the BigMale bodytype.
The only animation that is missing with that one is the one that goes from ground level to roof level and vice versa.

Report message to a moderator

Re: Logical Bodytypes[message #258207] Mon, 02 August 2010 19:27 Go to previous messageGo to next message
lockie is currently offline lockie

 
Messages:3721
Registered:February 2006
Location: Scotland
There are a few missing in RR , grenade throw and some death anims missing too .

Report message to a moderator

Captain

Re: Logical Bodytypes[message #258273] Tue, 03 August 2010 05:45 Go to previous messageGo to next message
Bio

 
Messages:62
Registered:April 2009
Location: Rhineland-Palatinate/Germ...
Everything is coming along great. It was a good thing that I decided to create logical bodytypes for one of the existing animation sets. I found and fixed a couple of bugs. And I added animsurface (that is the physical animation surface) as an alternative key to animstate. Now exactly one of those two has to be set for each logical surface. The problem is the randomized animations for certain animation states. If these were to differ in number of frames from the main surface for the animation state, that would lead to access violactions. So for now the animation surface should be taken as key. That of course means, that now each logical animation surface is directly dependend on a certain physical bodytype. I will leave animstate as an option though (in certain cases it could still make sense) and hope in the future to extend the system to make it even more flexible.
Apart from fixings those bugs/shortcomings, I also added logical bodytypes for all animation sets I could find. That is the Renegade Republic M_MERC replacement and the two S_MERC replacements from

This screenshot shows all physical and logical bodytypes side-by-side in one game. The 3 I added were the Renegae Republic one and the ones done by CheKa (the one on the left) and kL.

http://img828.imageshack.us/img828/9510/bild2x.png

They all work as good as they would as replacements (they are all basically incomplete, but the animations that exist and that I have tried were all being rendered). The Renegade Republic one is almost completely animated indeed. It renders slightly different than as a replacement for the big guy bodytype, insofar as whenvever an animation is missing, the body disappears completely. The alternative would have been to render the stock animations, whenever no surface for the body layer was defined. But in my opinion that is even more annoying than the body disappearing for the duration of the animation.

The last thing I did was to implement external entities for my XML loader baseclass. You can now externalize arbitrary blocks of XML from the configuration files used for the logical bodytype implementation. This makes a lot of sense for the AnimationSurfaces.xml and the Bodytypes.xml as these would increase dramatically in size with new layers/props. The result looks a bit strange at first, but it is valid XML:

http://img530.imageshack.us/img530/3965/bild3qd.png

I will try to add the other features (all concerning new filter criteria) and provide you with a small release package (with the exe and all the logical bodytypes).

Report message to a moderator

Corporal
Re: Logical Bodytypes[message #258281] Tue, 03 August 2010 13:12 Go to previous messageGo to next message
deknegt is currently offline deknegt

 
Messages:140
Registered:June 2007
Location: Doetinchem, Netherlands
Lookin good Bio.

I especially like the drawn gun animation. Looks a lot more... Awesome.

Report message to a moderator

Sergeant
Re: Logical Bodytypes[message #258283] Tue, 03 August 2010 13:19 Go to previous messageGo to next message
clbg

 
Messages:35
Registered:February 2007
Location: Moscow, Russia
Awesome work, Bio! Cant find words to explain my feeling Smile

Report message to a moderator

Private 1st Class
Re: Logical Bodytypes[message #258284] Tue, 03 August 2010 13:21 Go to previous messageGo to next message
Bio

 
Messages:62
Registered:April 2009
Location: Rhineland-Palatinate/Germ...
While I agree that it looks awesome, I can't take any credit for that as it is the work of kL. See his thread here http://www.ja-galaxy-forum.com/board/ubbthreads.php?ubb=showflat&Number=237683#Post237683
All I did was create a logical bodytype for his animations (for demonstration purposes). What a pitty though that kL didn't finish his animations, or at least released his renders uncropped.

Report message to a moderator

Corporal
Re: Logical Bodytypes[message #258321] Tue, 03 August 2010 19:44 Go to previous messageGo to next message
lockie is currently offline lockie

 
Messages:3721
Registered:February 2006
Location: Scotland
I mailed him in the hope he still visits , no answer as yet though .

Report message to a moderator

Captain

Re: Logical Bodytypes[message #258372] Wed, 04 August 2010 02:56 Go to previous messageGo to next message
Bio

 
Messages:62
Registered:April 2009
Location: Rhineland-Palatinate/Germ...
OK, this will be probably my last update for the week. My vacation is up and I have to concentrate on my office duties.

Today I added the promised new filter criteria:

* WEAPON_IN_HAND
* CALIBRE
* WEAPON_TYPE
* WEAPON_CLASS
* VEST_ARMOR_PROTECTION
* VEST_ARMOR_COVERAGE
* HELMET_ARMOR_PROTECTION
* HELMET_ARMOR_COVERAGE

With these the a filter setup for general ARs, AK47 types and AK47 types with wooden buttstocks and handguards. Can be realized like this:


	 
		
			1
			RIFLECLASS
			GUN_AS_RIFLE
			7,10 
			14,25,614,785,708,615,613,612,347,26 
		
	
	 
		
			1
			RIFLECLASS
			GUN_AS_RIFLE
			7,10 
			659,790,791 
		
	
	
		
			1
			RIFLECLASS
			GUN_AS_RIFLE
		
	



These filters would than have to be applied with the more specialized ones first. So that the general filter can catch all the rest. The example provided is complete; it would correctly apply one prop to all AK types with wooden stocks currently in the game. One to all other AK types. And one to all other assault rifles. There are still hard coded ID's here as you see. But they are coupled with qualifiers for weapon type/class and calibre. I decided against using the item names, primerily because I wasn't sure if they would be internationalized or not. Anyways, Id's are probably easier to maintain anyways.

The second thing I did was add the possibility to render a surface with the palette embedded in the STI archive instead of the soldier palette. This is primarily meant for props not for the main body layer (though for individual characters it would also make sense). It has currently one drawback, that no shading is applied. This I will correct, though it will require to either store a number of palettes for every surface or calculate them anew on every render cycle. Since the latter would be quite ridiculous, I will opt to store 3 or 4 shaded palettes in memory per layer in the cache. This adds about 3 KB per loaded surface, so I think we can afford that. I don't know how many levels of shading for the soldierpalette are implemented, but since it will be used primarily for props, a crude approximation to the body layer shading level will do.



	
	
]>

	
	
	
	
	
	
	
	
	
	
	
	
	&AnimSurfacesBgmRenegade;
	&AnimSurfacesRgmCheKa;
	&AnimSurfacesRgmkL;



An example (though not a very handsome one):
http://img148.imageshack.us/img148/1016/bild6i.png

Report message to a moderator

Corporal
Re: Logical Bodytypes[message #258733] Sun, 08 August 2010 10:01 Go to previous messageGo to previous message
Bio

 
Messages:62
Registered:April 2009
Location: Rhineland-Palatinate/Germ...
OK, did quite a bit of work the last couple of days. Mainly refactoring and giving a major overhaul to the datamodel.

Palettes are now defined in a seperate XML file. These palettes can than be used in a new LayerProp element in the logical bodytype definition XML. LayerProp elements are for grouping related logical animation surfaces according to filter and palette. This allows a much easier to maintain list of logical animation surfaces. For each configured palette a palette table object is created (whose class is a descendent of SOLDIEROBJECT). All the shading and effect tables for the palette are created and thus the surface can be rendered just as if it was one using the default palette (with all the entries for skin, hair and clothing coloring). As states before this feature is mainly there for props and foremost for weapons. Also might be used for clothing, but currently you would lose the ability to use the SOLDIERTYPE (for mercs) specific coloring scheme in that case. For bodytypes it currently only makes sense for unique characters (no skin/hair coloring). This I will change at some point to allow compound palettes and selectively colortables from the default SOLDIERTYPE palette setup.

Additionally, I have added the possibility to deactivate the rendering of shadows per layer per logical bodytype (each bodytype has the posibiltiy to override the default layer configuration). This was done to prevent the overlaying of prop and body shadows resulting in very dark shadows. TO achieve this I edited all the 8BPB to 16BPB blitters that are being used for merc objects. The 8BPB to 8BPB blitters I did not touch, but I'll assume, that in case of 8BPB it uses opaque anyways. So multiple shadows don't stack up (if it uses a raster, it might not work though). Btw. there is a 8BPP to 16BPP blitter in the 8BPP branch of the code. This is certainly a mistake (but never noted because the branch is never entered). I'm sure quite a few of the branches are actually never entered. And even if it will make diffing with previous version difficult, someone should really send the whole rendertiles function through a code beautifier (and configure for opening braces on the statement line). Makes the function a heck of a lot easier to read. Why they created such a monster instead of moving part of it to inline functions is beyond me.

I'm afraid, that because of these two changes, the alterations to renderworld are now a little more extensive (up until now I put emphasize on keeping my changes localized, but that aspiration has been dropped now).

I added something for showcasing the new palette feature. Put more time into this little demo than I care to admit (but I did fix a couple of bugs in the render script while making it). Btw. the render framework will need a major overhaul now (and I still intend to take a closer look at Blender scripting).

Oh and the render times are substantially shorter than I had remembered, about 7x as fast.

Short videoclip demonstrating logical bodytypes (and both layers with fixed and with variable palettes together):
http://www.mediafire.com/?gc8z35la4f8xtpk

As you can see, the face and hair colors are fixed (see above). But the whole model is being shaded correctly, whether a normal surface is blitted (using the merc specific palette table with specific color ranges) or fixed palettes.

I will try to archive everything today and also provide a build for testing purposed.
If someone would be so kind to help me get some basic documentation together, that would be great. I can provide all the technical specs and details.

[Updated on: Sun, 08 August 2010 20:37] by Moderator

Report message to a moderator

Corporal
Previous Topic: Delete MOBILE_MILITIA
Next Topic: Idea on fixing traits for rifles
Goto Forum:
  


Current Time: Fri Mar 29 12:24:55 GMT+2 2024

Total time taken to generate the page: 0.02304 seconds