Ticket #461

[matt minton]

So, this seemed like a simple thing to tackle. What a fool I am!

A number of questions: (for reference, the file under examination is /binaries/data/mods/public/art/meshes/props/shield/hex_b.pmd)

Looking at the Collada to PMD converter code (PMDConvert.cpp) I find a function, WritePMD, which seems to do all the heavy lifting. Looking at the first part:

output("PSMD", 4);  // magic number
write(output, (uint32)3); // version number
write(output, (uint32)(
	4 + 13*4*vertexCount + // vertices
	4 + 6*faceCount + // faces
	4 + 7*4*boneCount + // bones
	4 + propPointsSize // props
	)); // data size

And a corresponding hex dump from hex_b.pmd (little endian):

Hex:
44 4d 53 50
00 00 00 02
00 00 01 60
00 00 00 06
3e 9f 04 61
bf af e0 fa
bc 30 12 60
b4 22 b9 5c
b3 3b b4 cf 
3f 80 00 00
3e 89 b5 d9
3a 02 f3 f8
ff ff ff 00
ASCII:
DMSP <plus a bunch of unprintable characters>

If I understand what is going on here, I see the 4 bytes of "PSMD" at the start of the file, but after that is gets hazy. The next 4 bytes give you "2" (or "02000000" if you look at it big endian). Is this the version number? The next four gives "00 00 01 60", or 352 in decimal. Is this the 'data size' mentioned about? Is the sequence 'ff ff ff 00' a stop sequence? It appears regularly in the hex dump. Continuing on:

write<uint32>(output, (uint32)vertexCount);
for (size_t i = 0; i < vertexCount; ++i)
{
	output((char*)&position[i*3], 12);
	output((char*)&normal  [i*3], 12);
	output((char*)&texcoord[i*2],  8);
	if (boneCount)
		write(output, boneWeights[i]);
	else
		write(output, noBlend);
}

and the hex:

00 00 00 06
3e 9f 04 61
bf af e0 fa
bc 30 12 60
b4 22 b9 5c
b3 3b b4 cf
3f 80 00 00
3e 89 b5 d9
3a 02 f3 f8
ff ff ff 00

The first line seems to suggest 6 verteces, but there are 8 uint32 data chunks after that, with no obvious encoding scheme, followed by the sequence 'ff ff ff 00'. Next is:

		// Face data
		write(output, (uint32)faceCount);
		for (size_t i = 0; i < indexCount; ++i)
		{
			write(output, (uint16)indices[i]);
		}

00 00 00 00
00 00 00 00
00 00 00 00
00 00 00 00
bf 25 0f 84
b9 ef a8 c1
bc 30 12 bb
b4 22 c4 0c
b3 3b d2 88
3f 80 00 00
3f 7f df 45
3e fd a3 32
ff ff ff 00

And finally:

		// Bones data
		write(output, (uint32)boneCount);
		for (size_t i = 0; i < boneCount; ++i)
		{
			output((char*)&boneTransforms[i], 7*4);
		}

		// Prop points data
		write(output, (uint32)propPoints.size());
		for (size_t i = 0; i < propPoints.size(); ++i)
		{
			uint32 nameLen = (uint32)propPoints[i].name.length();
			write(output, nameLen);
			output(propPoints[i].name.c_str(), nameLen);
			write(output, propPoints[i].translation);
			write(output, propPoints[i].orientation);
			write(output, propPoints[i].bone);
		}

and

00 00 00 00 
00 00 00 00 
00 00 00 00 
00 00 00 00 

3f 25 0f 84 
b9 ef a8 c1 
bc 30 11 e9 
b4 22 c4 0c 
b3 3b d2 88 
3f 80 00 00 
3c 5b dd 4c 
3e fd a3 36 
ff ff ff 00 
00 00 00 00 
00 00 00 00 
00 00 00 00 
00 00 00 00 

be 9f 04 61 
3f af c3 05 
bc 30 12 44 
b4 22 b9 5d 
b3 3b b4 cf 
3f 80 00 00 
3f 3e 73 cc 
3f 7d 82 77 
ff ff ff 00 
00 00 00 00 
00 00 00 00 
00 00 00 00 
00 00 00 00 

3e 9f 04 61 
3f af c3 05 
bc 30 11 df 
b4 22 99 50 
b3 3b 5b a4 
3f 80 00 00 
3e 89 b5 eb 
3f 7d 82 78 
ff ff ff 00 
00 00 00 00 
00 00 00 00 
00 00 00 00 
00 00 00 00 

00 00 00 04 
00 01 00 00 
00 03 00 02 
00 02 00 00 
00 02 00 03 
00 03 00 04 
00 05 00 04 
00 00 00 00 
00 00 00 00

It seems it wouldn't be too difficult to write a Python converter, if I knew how this data was being represented. Is there a DAE version of this file to compare the hex dump to? Any other thoughts on this process? Thanks!

[Ykkrosh]

Have you seen the PMD documentation on the wiki? That might clarify a few things

If you run the game then look in your cache directory (~/.cache/0ad/ on Linux, %appdata%\0ad\cache\ on Windows), it'll have .pmd files corresponding to the .dae files that are in binaries/data/..., so you could compare them that way.

The 2 is indeed the version number there, and 352 the data length (the file is 364 bytes, with 12 byte header).

The '00 ff ff ff' (your hex dump seems to be reversing the byte order) is usually the 'u8 bones[4];' array (each vertex can be influenced by up to 4 bones and this one is influenced by bone 0x00 and the other three slots are unused).

Looks like there's 6 vertices but each is 52 bytes (when you include the VertexBlend). Most of the values are floats, so you need something like Python's struct module to unpack them into numbers.

[matt minton]

I had a feeling the struct module would come in handy one day:

Output so far:

PSMD
2
352
0.310580283403,-1.37405323982,-0.0107465684414
-1.51548590566e-07,-4.37037748213e-08,1.0
0.268965512514,0.000499546062201
(0, 255, 255, 255),(0.0, 0.0, 0.0, 0.0)
-0.310580283403,-1.37405323982,-0.010746662505
-1.51431990503e-07,-4.36226734735e-08,1.0
0.743953883648,0.000499487039633
(0, 255, 255, 255),(0.0, 0.0, 0.0, 0.0)
-0.644767999649,-0.000457113637822,-0.0107466531917
-1.51587471464e-07,-4.37308074197e-08,1.0
0.999500572681,0.495385706425
(0, 255, 255, 255),(0.0, 0.0, 0.0, 0.0)
0.644767999649,-0.000457113637822,-0.010746457614
-1.51587471464e-07,-4.37308074197e-08,1.0
0.0134194605052,0.495385825634
(0, 255, 255, 255),(0.0, 0.0, 0.0, 0.0)
-0.310580283403,1.37313902378,-0.0107465423644
-1.51548604777e-07,-4.37037748213e-08,1.0
0.74395442009,0.990271985531
(0, 255, 255, 255),(0.0, 0.0, 0.0, 0.0)
0.310580283403,1.37313902378,-0.0107464483008
-1.51432004714e-07,-4.36226770262e-08,1.0
0.268966048956,0.990272045135
(0, 255, 255, 255),(0.0, 0.0, 0.0, 0.0)
(0, 1, 2)
(3, 0, 2)
(3, 2, 4)
(3, 4, 5)

What seems to be missing is any material information...this would generate an un-mapped mesh in game. Also, the Collada schema is exceedingly heavyweight. How much of it is boilerplate? I glanced through the DAE file of a (I hope) static mesh (bull_statue.dae) and found plenty of material info. Where does that get pulled into a PMD?

[Ykkrosh]

The .pmd just has the mesh and UV texture mapping coordinates, then it gets referenced by an actor XML file like binaries/data/mods/public/art/actors/props/units/shields/hex_a_back.xml which also says what texture to apply to the mesh (as well as animations, and props (other actors that get attached to the mesh)). (The same mesh can be reused with many different textures, e.g. we have lots of different shield patterns if you look at the hele_round_spartiate_sp.xml actor.)

The material data in the .dae file just gets ignored. It's included because that's what the standard Collada exporter does, but it's never required - you could omit all of it (library_images, library_materials, library_effects, and the bind_material) when constructing a new .dae file.

[matt minton]

Will this do?

reference: hex_b.pmd

<COLLADA version="1.4.0" xmlns="http://www.collada.org/2005/11/COLLADASchema">
	<asset>
		<contributor>
			<author>PMD to Collada Coverter by Matthew Minton (c) 2010</author>
			<authoring_tool>PMD to Collada Converter.py, v1.0</authoring_tool>
			<comments></comments>
			<copyright></copyright>
			<source_data></source_data>
		</contributor>
		<created>2010/08/15T00:55:37</created>
		<modified>2010/08/15T00:55:37</modified>
		<unit meter="0.01" name="centimeter"/>
		<up_axis>Z_UP</up_axis>
	</asset>
	<library_geometries>
		<geometry id="hex_b" name="hex_b">
			<mesh>
				<source id="hex_b-Geometry-Position">
					<float_array count="6" id="hex_b-Geometry-Position-array>0.310580283403 -1.37405323982 -0.0107465684414 -0.310580283403 -1.37405323982 -0.010746662505 -0.644767999649 -0.000457113637822 -0.0107466531917 0.644767999649 -0.000457113637822 -0.010746457614 -0.310580283403 1.37313902378 -0.0107465423644 0.310580283403 1.37313902378 -0.0107464483008</float_array>
				</source>
				<source id="hex_b-Geometry-Normals">
						<float_array count="6" id="hex_b-Geometry-Normals-array>-1.51548590566e-07 -4.37037748213e-08 1.0 -1.51431990503e-07 -4.36226734735e-08 1.0 -1.51587471464e-07 -4.37308074197e-08 1.0 -1.51587471464e-07 -4.37308074197e-08 1.0 -1.51548604777e-07 -4.37037748213e-08 1.0 -1.51432004714e-07 -4.36226770262e-08 1.0</float_array>
				</source>
				<source id="hex_b-Geometry-UV">
						<float_array count="6" id="hex_b-Geometry-UV-array>0.268965512514 0.000499546062201 0.743953883648 0.000499487039633 0.999500572681 0.495385706425 0.0134194605052 0.495385825634 0.74395442009 0.990271985531 0.268966048956 0.990272045135</float_array>
				</source>
				<vertices id="hex_b-Vertex">
					<input semantic="POSITION" source="#hex_b-Position"/>
				</vertices>
				<triangles count="4">
					<input offset="0" semantic="VERTEX" source="#hex_b-Vertex"/>
					<input offset="1" semantic="NORMAL" source="#hex_b-Normals"/>
					<input offset="2" semantic="TEXCOORD" source="#hex_b-UV"/>
					<p>0 1 2 3 0 2 3 2 4 3 4 5</p>
				</triangles>
			</mesh>
		</geometry>
	</library_geometries>
</COLLADA>

Edited August 15, 2010 by matt minton

[Ykkrosh]

Looks largely reasonable to me . Have you tried successfully importing into a 3d modelling program (like Blender)? If you have xmllint, have you tried validating the file?

It's possible you need a <library_visual_scenes> (probably with no translate or scale or bind_material) and <scene> to say the mesh exists. (I'm not certain, though.)

It looks like the created/modified dates are wrong - I think they need "-" instead of "/".

The <float_array count="6" id="hex_b-Geometry-Position-array> seems to be missing a " character.

To save disk space, the floats should be shorter - six decimal places should be sufficient.

[matt minton]

Thanks! After some tweaking (adding the library_visual_scenes and scene verbage) I was able to validate the DAE (though for some reason the Collada schema I got from the webpage didn't compile...odd), and I got a successful import into Blender, but without any material information no mesh appeared on screen. ANy reference to instance_geometry causes an import fail in Blender with the following stack trace:

FEEDBACK: Illusoft Collada 1.4 Plugin v0.3.162 started
Delete everything from the scene..
Traceback (most recent call last):
  File "C:\Documents and Settings\Matt\Application Data\Blender Foundation\Blend
er\.blender\scripts\bpymodules\colladaImEx\cstartup.py", line 681, in ButtonEven
t
	onlyMainScene, applyModifiers)
  File "C:\Documents and Settings\Matt\Application Data\Blender Foundation\Blend
er\.blender\scripts\bpymodules\colladaImEx\translator.py", line 120, in __init__

	self.__Import(fileName)
  File "C:\Documents and Settings\Matt\Application Data\Blender Foundation\Blend
er\.blender\scripts\bpymodules\colladaImEx\translator.py", line 127, in __Import

	documentTranslator.Import(fileName)
  File "C:\Documents and Settings\Matt\Application Data\Blender Foundation\Blend
er\.blender\scripts\bpymodules\colladaImEx\translator.py", line 333, in Import
	self.sceneGraph.LoadFromCollada(self.colladaDocument.visualScenesLibrary.ite
ms, self.colladaDocument.scene)
  File "C:\Documents and Settings\Matt\Application Data\Blender Foundation\Blend
er\.blender\scripts\bpymodules\colladaImEx\translator.py", line 550, in LoadFrom
Collada
	ob = sceneNode.ObjectFromDae(daeNode)
  File "C:\Documents and Settings\Matt\Application Data\Blender Foundation\Blend
er\.blender\scripts\bpymodules\colladaImEx\translator.py", line 1910, in ObjectF
romDae
	dataObject = self.document.meshLibrary.FindObject(daeInstance,True)
  File "C:\Documents and Settings\Matt\Application Data\Blender Foundation\Blend
er\.blender\scripts\bpymodules\colladaImEx\translator.py", line 3568, in FindObj
ect
	return self.LoadFromDae(daeInstance, bObject)
  File "C:\Documents and Settings\Matt\Application Data\Blender Foundation\Blend
er\.blender\scripts\bpymodules\colladaImEx\translator.py", line 3672, in LoadFro
mDae
	bMesh = meshNode.LoadFromDae(daeGeometry)
  File "C:\Documents and Settings\Matt\Application Data\Blender Foundation\Blend
er\.blender\scripts\bpymodules\colladaImEx\translator.py", line 2600, in LoadFro
mDae
	bMesh2 = Blender.NMesh.New(self.document.CreateNameForObject(meshID,replaceN
ames,'mesh'))
  File "C:\Documents and Settings\Matt\Application Data\Blender Foundation\Blend
er\.blender\scripts\bpymodules\colladaImEx\translator.py", line 246, in CreateNa
meForObject
	mesh = Blender.Mesh.Get(name)
NameError: Mesh "hex_b" not found

That last part is confusing, b/c I looked over the file in Notepad++ and both names were the same. Some more code tweaking and attention to names fixed that, but having an empty bind_materials attribute causes an import fail. Commenting it out will import fine, and Blender recognizes there is a mesh there somewhere, but without materials data, you don't see anything. Here is the output DAE file:

<?xml version="1.0" encoding="utf-8"?>
<COLLADA version="1.4.0" xmlns="http://www.collada.org/2005/11/COLLADASchema">
	<asset>
		<contributor>
			<author>PMD to Collada Coverter by Matthew Minton (c) 2010</author>
			<authoring_tool>pmd2collada.py, v1.0</authoring_tool>
			<comments></comments>
			<copyright></copyright>
			<source_data></source_data>
		</contributor>
		<created>2010-08-15T10:35:35</created>
		<modified>2010-08-15T10:35:35</modified>
		<unit meter="0.01" name="centimeter"/>
		<up_axis>Z_UP</up_axis>
	</asset>
	<library_geometries>
		<geometry id="hex_b-Geometry" name="hex_b-Geometry">
			<mesh>
				<source id="hex_b-Geometry-Position">
					<float_array count="18" id="hex_b-Geometry-Position-array">0.31058 -1.374053 -0.010747 -0.31058 -1.374053 -0.010747 -0.644768 -0.000457 -0.010747 0.644768 -0.000457 -0.010746 -0.31058 1.373139 -0.010747 0.31058 1.373139 -0.010746</float_array>
				</source>
				<source id="hex_b-Geometry-Normals">
					<float_array count="18" id="hex_b-Geometry-Normals-array">-0.0 -0.0 1.0 -0.0 -0.0 1.0 -0.0 -0.0 1.0 -0.0 -0.0 1.0 -0.0 -0.0 1.0 -0.0 -0.0 1.0</float_array>
				</source>
				<source id="hex_b-Geometry-UV">
					<float_array count="12" id="hex_b-Geometry-UV-array">0.268966 0.0005 0.743954 0.000499 0.999501 0.495386 0.013419 0.495386 0.743954 0.990272 0.268966 0.990272</float_array>
				</source>
				<vertices id="hex_b-Vertex">
					<input semantic="POSITION" source="#hex_b-Geometry-Position"/>
				</vertices>
				<triangles count="4">
					<input offset="0" semantic="VERTEX" source="#hex_b-Geometry-Vertex"/>
					<input offset="1" semantic="NORMAL" source="#hex_b-Geometry-Normals"/>
					<input offset="2" semantic="TEXCOORD" source="#hex_b-Geometry-UV"/>
					<p>0 1 2 3 0 2 3 2 4 3 4 5</p>
				</triangles>
			</mesh>
		</geometry>
	</library_geometries>
	<library_visual_scenes>
		<visual_scene id="Scene" name="Scene">
			<node layer="L1" id="hex_b" name="hex_b">
				<translate sid="translate">0.00000 0.00000 0.00000</translate>
				<rotate sid="rotateZ">0 0 1 0.00000</rotate>
				<rotate sid="rotateY">0 1 0 0.00000</rotate>
				<rotate sid="rotateX">1 0 0 0.00000</rotate>
				<scale sid="scale">1.00000 1.00000 1.00000</scale>
				<instance_geometry url="#hex_b-Geometry">
				</instance_geometry>
			</node>
		</visual_scene>
	</library_visual_scenes>
	<scene>
		<instance_visual_scene url="#Scene" />
	</scene>
</COLLADA>

Thoughts?

[Ykkrosh]

If it doesn't work with no material information, have you tried adding some material information? Perhaps something simple like

  <library_materials>
	<material id="default-material">
	  <instance_effect url="#default-effect"/>
	</material>
  </library_materials>
  <library_effects>
	<effect id="default-effect"/>
  </library_effects>
  ...
		<triangles material="default-material" count="4">
  ...
		<bind_material>
		  <technique_common>
			<instance_material symbol="default-material" target="#default-material"/>
		  </technique_common>
		</bind_material>
  ...

would be sufficient?

[matt minton]

So I tried that...no good. I also exported a cube from Blender and copied the effects and materials stuff into the file...that didn't work either. I can't find a problem, unless there is an ordering problem with the faces:

<triangles count="4" material="default-material">
					<input offset="0" semantic="VERTEX" source="#hex_b-Geometry-Vertex"/>
					<input offset="1" semantic="NORMAL" source="#hex_b-Geometry-Normals"/>
					<input offset="2" semantic="TEXCOORD" source="#hex_b-Geometry-UV"/>
					<p>
						0 1 2 3 0 2 3 2 4 3 4 5
					</p>
				</triangles>

This is the order it was extracted from the PMD file, so I'm assuming it's correct.

[Ykkrosh]

Hmm, I looked at the example you posted in Blender (2.53) and found a few issues:

You don't have to actually define any materials, you can say <triangles count="4" material=""> and it seems to load and render correctly. (If you don't specify any material then it crashes.)

I think you need to have a <technique_common><accessor>... thing in each <source>.

In the <triangles> you are saying that each vertex has three inputs at separate offsets, which means the <p> needs 3 numbers per vertex (and 3 vertexes per triangle and 4 triangles in total) but you only have one per vertex. You should either repeat each number three times (for the position index, normal index, and texcoord index); or change all the <input>s to say offset="0" so they share the same number.

You say <vertices id="hex_b-Vertex"> but then refer to source="#hex_b-Geometry-Vertex" instead.

You don't need the translate/rotate/scale in the <node>.

[matt minton]

Success!

<?xml version="1.0" ?>
<COLLADA version="1.4.0" xmlns="http://www.collada.org/2005/11/COLLADASchema">
	<asset>
		<contributor>
			<author>
				PMD to Collada Coverter by Matthew Minton (c) 2010
			</author>
			<authoring_tool>
				pmd2collada.py, v1.0
			</authoring_tool>
			<comments/>
			<copyright/>
			<course_data/>
		</contributor>
		<created>
			2010-08-18T15:07:56
		</created>
		<modified>
			2010-08-18T15:07:56
		</modified>
		<unit meter="0.01" name="centimeter"/>
		<up_axis>
			Z_UP
		</up_axis>
	</asset>
	<library_geometries>
		<geometry id="hex_b-Geometry" name="hex_b-Geometry">
			<mesh>
				<source id="hex_b-Geometry-Position">
					<float_array count="18" id="hex_b-Geometry-Position-array">
						0.31058 -1.374053 -0.010747 -0.31058 -1.374053 -0.010747 -0.644768 -0.000457 -0.010747 0.644768 -0.000457 -0.010746 -0.31058 1.373139 -0.010747 0.31058 1.373139 -0.010746
					</float_array>
					<technique_common>
						<accessor count="6" source="hex_b-Geometry-Position-array" stride="3">
							<param name="X" type="float"/>
							<param name="Y" type="float"/>
							<param name="Z" type="float"/>
						</accessor>
					</technique_common>
				</source>
				<source id="hex_b-Geometry-Normals">
					<float_array count="18" id="hex_b-Geometry-Normal-array">
						-0.0 -0.0 1.0 -0.0 -0.0 1.0 -0.0 -0.0 1.0 -0.0 -0.0 1.0 -0.0 -0.0 1.0 -0.0 -0.0 1.0
					</float_array>
					<technique_common>
						<accessor count="6" source="hex_b-Geometry-Normal-array" stride="3">
							<param name="X" type="float"/>
							<param name="Y" type="float"/>
							<param name="Z" type="float"/>
						</accessor>
					</technique_common>
				</source>
				<source id="hex_b-Geometry-UV">
					<float_array count="12" id="hex_b-Geometry-UV-array">
						0.268966 0.0005 0.743954 0.000499 0.999501 0.495386 0.013419 0.495386 0.743954 0.990272 0.268966 0.990272
					</float_array>
					<technique_common>
						<accessor count="6" source="hex_b-Geometry-UV-array" stride="2">
							<param name="U" type="float"/>
							<param name="V" type="float"/>
						</accessor>
					</technique_common>
				</source>
				<vertices id="hex_b-Geometry-Vertex">
					<input semantic="POSITION" source="#hex_b-Geometry-Position"/>
				</vertices>
				<triangles count="4" material="">
					<input offset="0" semantic="VERTEX" source="#hex_b-Geometry-Vertex"/>
					<input offset="0" semantic="NORMAL" source="#hex_b-Geometry-Normals"/>
					<input offset="0" semantic="TEXCOORD" source="#hex_b-Geometry-UV"/>
					<p>
						0 1 2 3 0 2 3 2 4 3 4 5
					</p>
				</triangles>
			</mesh>
		</geometry>
	</library_geometries>
	<library_visual_scenes>
		<visual_scene id="Scene" name="Scene">
			<node id="hex_b" layer="L1" name="hex_b">
				<instance_geometry url="#hex_b-Geometry"/>
			</node>
		</visual_scene>
	</library_visual_scenes>
	<scene>
		<instance_visual_scene url="#Scene"/>
	</scene>
</COLLADA>

I glanced right over the accessor parts when I compared it with the export from Blender. This imports into Blender perfectly, and looks like a hex. Will try with some more complicated meshes.

EDIT:

Coverted celt_helmet_m.pmd, imported into Blender successfully:

Edited August 18, 2010 by matt minton

[Ykkrosh]

Looking good . Is the default orientation a bit off? Maybe the <up_axis> should be changed to Y_UP or something.

[Wijitmaker]

Congratulations - a very useful tool indeed! Is the plan to create multiple plugins specific to the 3d software packages or will this be a single stand alone tool that simply converts the file to a .dae file?

Is there a way to recover smoothing groups from the .pmd in addition to the texture coordinates?

[WhiteTreePaladin]

Ooh, imports into blender? Very nice indeed! I'm sure I will have a use for this.

[Ykkrosh]

Is the plan to create multiple plugins specific to the 3d software packages or will this be a single stand alone tool that simply converts the file to a .dae file?

The .dae files will be readable by any 3d software, so there shouldn't be any need to do anything software-specific. What I might like to do is use this tool to convert all the .pmd files that are currently in the game (and also convert .psa), and then we won't have any more .pmd files and can throw the tool away. Then the engine will only ever be loading .dae files, and we won't have to bother maintaining compatibility with the old file formats, which would make it a bit easier to change our graphics engine (to use a more efficient mesh/animation format or whatever).

(The tool would have to support skeletal meshes before we could convert everything, and that doesn't sound trivial. If the current one works well for static meshes then perhaps it could be uploaded (as a Trac attachment or something) for people who want to play with it now?)

Is there a way to recover smoothing groups from the .pmd in addition to the texture coordinates?

It'd be possible to approximate smoothing groups based on the normals: if two triangles share an edge, and have the same normals for the shared vertexes, then they may be in the same smoothing group, and then you search for the largest sets of triangles that may all be in the same group as each other. But I don't know if smoothing groups can be represented in Collada files. But can't 3ds Max do this automatically? From the documentation it sounds like the Collada importer has a smoothing group option that should recreate them.

[Mythos_Ruler]

The .dae files will be readable by any 3d software

Yet my Max 2009 fails to import DAEs every single time.

Edited August 19, 2010 by Mythos_Ruler

[Ykkrosh]

(By "will" I mean "really should". I don't see why it'd fail - if you could post the problems you get (maybe in a separate thread) it'd be good to try debugging that.)

[Mythos_Ruler]

Oh man. I just got them to import into Max. I was doing it wrong for years. lol

[matt minton]

Holy schnikeys, it's whole dev team I'm happy to help. I haven't had the chance to work much on my other game project recently, so this gave me a way to get my feet wet again.

Changing the up_axis value is trivial. Done.

As for smoothing groups, I'm only pulling the data that exists in the PMD file out and mapping the bare minimum to the necessary Collada elements. If you've got an algorithm I can apply to the data before exporting to .dae, that wouldn't be difficult (alas, I know little about the finer points of 3d programming)

I took a stab at the PSA stuff, using the PMD as a jumping off point. Biggest road block was a lack of PSA files never mind, I just found them. Looking at the DAE versions of some of the boned and rigged meshes it seems they exhibit an order of magnitude greater level of complexity than the static meshes. It's going to take some input to figure out what is needed and what is fluff.

Is there some place to post the pmd2collada.py file?

[Ykkrosh]

Oh man. I just got them to import into Max. I was doing it wrong for years. lol

Ah, glad it works now . Do we need some documentation to prevent other people doing it wrong?

Is there some place to post the pmd2collada.py file?

If you register on Trac then attach to the ticket, that'd probably be best.

Skeletal meshes and animations should be possible (I think), but not easy. One problem is that the skeleton hierarchy gets flattened in the PMD/PSA files - you'd probably need to check the <standard_skeleton>s in binaries/data/mods/public/art/skeletons/skeletons.xml for the hierarchical version, then figure out which skeleton each file uses (they're probably almost all "Standard biped" but some might differ and the files don't actually say what skeleton they are), then unwrap the bone data arrays back into the nested structure. Then we still need all the skin weights and bind pose stuff. If you check something like meshes/skeletal/f_tunic.dae then I think everything there (except the materials etc) is needed.

Hmm... Looks like there's only 39 skeletal .pmd files. Do we have all the original .max files, that someone could load and export as .dae? Maybe that'd be less effort than writing the conversion tool for non-static meshes. Similar question for .psa files, though I see 100 of those.

[matt minton]

I have posted a zip archive containing all the necessary files to Trac. I have the ticket currently in a worksforme state, but I'll repoen and close it out completely when someone confirms it worksforthem as well.

It wouldn't be too difficult to re-inflate a flattened skeleton. The xml.dom.minidom module for Python provides some nice methods for parsing xml and node traversal is trivial. If the assumption is that the data in the PSA file appears in the order the bones are defined in the skeletons.xml file, mapping them won't be hard. The problem is this: when I extract the data defined in the PSA format for inf_hoplite_idle_a.psa, for example, the data in PSA format only comes out to 11K. The file is 75K. What is the rest of that data, and what format is it in? Is it a PSA stacked on a PMD? If it is, that makes life easier.

[Ykkrosh]

I think this description is accurate - that .psa file has 29 bones, 97 frames, and BoneState is 7 floats, so that's 78764 bytes (plus 37 for the header) which seems to match the file. (There's no mesh data in a .psa file.)

[matt minton]

Ah, I get it: 1 BoneState (7f) * 29 bones (29 BoneStates) * 97 frames (2813 BoneStates).

Maybe something like this would better internally represent this:

class BoneState
{
	Vector3d translation;
	Quaterion rotation;
	}

class Skeleton
{
	string[] boneNames[29];
	BoneState[] boneStates[29];
	Dict<string,BoneState> bones = map(boneNames,boneStates);
	}

Skeleton[] frames[numFrames];

written appropriately in Python, of course

EDIT:

So, I sucked out all the data from inf_hoplite_idle_a, and the resulting XML file is LARGE, to say the least. A sample:

	<frames count="97" id="inf_hoplite_idle_a-Frames">
		<frame>
			<skeleton>
				<bone id="0" name="bone0">
					<translation>
						<float_array count="3">
							0.035899 1.86706 0.271771
						</float_array>
					</translation>
					<rotation>
						<float_array count="4">
							0.017498 0.805136 -0.012881 0.592692
						</float_array>
					</rotation>
				</bone>
				<bone id="1" name="bone1">
					<translation>
						<float_array count="3">
							0.035899 1.86706 0.271771
						</float_array>
					</translation>
					<rotation>
						<float_array count="4">
							-0.714103 -0.683725 0.103913 0.10853
						</float_array>
					</rotation>
				</bone>
...

This iterates through all 29 bones in each frame, and all the frames in the file, so as you might imagine, the file is BIG.

As I mentioned above, bone names would be easy to get, but if the ColladaToPSA converter made some transformation that flattened the data in other ways (such as skin weights and poses) we'd need to some how extract that from here, which I can't see, since the data only describes a translation and rotation for each bone in each frame of the animation.

Edited August 19, 2010 by matt minton