Internal Model Definition

Gorilla3D holds model, animation and material data inside of an internal structure for better instancing, abstraction and management.

The so-called DefTypes can be stored and loaded to/from *.G3D file format.

Structure

Take a look at the following schematic structure of a model definition.

-TModelDef
 	// holds a list of TMeshDef instances
 	-List::Meshes
 		-TMeshDef|TVertexGroupDef
 			// a reference to the owner model
 			-@Model::TModelDef
 		-/TMeshDef|TVertexGroupDef
 	-/List::Meshes

        // holds a list of TMaterialDef instances
        -List::Materials
 		-TMaterialDef
 			// a reference to the owner model
 			-@Model::TModelDef

 			// holds a list of textures
 			-List::Textures
 				-TTextureDef
 				-/TTextureDef
 			-/List::Textures

 			// holds a list of shader codes
 			-List::Shaders
 				-TShaderDef
 				-/TShaderDef
 			-/List::Shaders

 			// holds a list of sub materials for layered material sources
 			-List::Materials
 				-TMaterialDef
 				[...]
 				-/TMaterialDef
 			-/List::Materials
 		-/TMaterialDef
        -/List::Materials

        -List::Lights
          -TLightDef
        -/List::Lights

        -List::Cameras
          -TCameraDef
        -/List::Cameras

 	// holds a list of THumanoidDef instances
 	-List::Humanoids
 		-THumanoidDef
 			// a reference to the owner model
 			-@Model::TModelDef
 			// holds a list of TControllerDef references
 			-@List::Controllers

 			// represents a tree of TJointDef nodes
 			-Root::TJointDef
 				-TJointDef
 					-TJointDef
 						-TJointDef
 						-TJointDef
 					-/TJointDef
 				-/TJointDef
 			-/Root::TJointDef
 		-/THumanoidDef
 		-THumanoidDef
 			[...]
 		-/THumanoidDef
 	-/List::Humanoids

 	// holds a list of TControllerDef instances
 	-List::Controllers
 		-TControllerDef
 			// a reference to the mesh, the controller handles
 			-@Mesh::TMeshDef

 			// is a sub component of TSkinDef
 			-Skin::TSkinDef
 				-TSkinDef
 					// the owner controller of the skin definition
 					-@Controller::TControllerDef

 					// holds a list of TJointRefDef instances - these
 					// are referenced objects to TJointDef instances
 					-List::LinkedJoint
 						-TJointRefDef
 						-TJointRefDef
 						-TJointRefDef
 						-TJointRefDef
 						-TJointRefDef
 						[...]
 					-/List::LinkedJoints

  					// contains one or more joints as skeleton roots
  					-List::Skeletons
 						-TJointRefDef
 						-TJointRefDef
 						[...]
  					-/List::Skeletons
 				-/TSkinDef
 			-/Skin::TSkinDef
 		-/TControllerDef
 		-TControllerDef
 			[...]
 		-/TControllerDef
 	-/List::Controllers

 	// holds a list of TAnimationDef instances
 	-List::Animations
 		-TAnimationDef
 			// reference to the owner model
 			-@Model::TModelDef

 			// holds a list of TAnimationStageDef instances
 			-List::Stages
 				-TAnimationStageDef
 					-List::Interpolators
 						-TInterpolatorDef
 						-TInterpolatorDef
 						-TInterpolatorDef
 						[...]
 					-/List::Interpolators
 				-/TAnimationStageDef
 				-TAnimationStageDef
 					[...]
 				-/TAnimationStageDef
 			-/List::Stages
 		-/TAnimationDef
 		-TAnimationDef
 			[...]
 		-/TAnimationDef
 	-/List::Animations
 -/TModelDef

G3D File Format

The Gorilla3D file format is a representation of the internal model structure defintion. It allows different kinds of storage formats:

Format Notes
BSON binary json format: https://en.wikipedia.org/wiki/BSON [wikipedia]
JSON default json format: https://en.wikipedia.org/wiki/JSON [wikipedia]

In addition to those formats we provide different storage options:

Option Notes
None The save routine simply stores data plain as BSON or JSON
Zipped Store as zipped data stream. You can extend this option by the FastestCompression or MaxCompression option. If none of them is set, the exporter will use default compression.
Beautified Compatible with JSON format. Defines data will be exported with linebreaks and indents or not. This option has no influence on bson format.
FastestCompression If the Zipped-Option is set, this option will choose the fastest algorithm for packing the data stream.
MaxCompression If the Zipped-Option is set, this option will chose the maximum compression algorithm for packing the data stream.

Depending on your model data you can configure those options to optimize filesize. In most cases a zipped JSON format with MaxCompression provides the best results.

G3D_TEST : TGorillaG3DOptions = [TGorillaG3DOption.Zipped, TGorillaG3DOption.MaxCompression];

File Header

Every G3D file starts with some header information, where is defined which format and options are used. The header format is described below:

  TGorillaG3DFormat = (BSONFormat, JSONFormat);
 
  TGorillaG3DOption  = (
    None,
    Zipped,
    Beautified,
    FastestCompression,
    MaxCompression);
 
  TGorillaG3DOptions = Set Of TGorillaG3DOption;
 
  TGorillaG3DHeader = record
    /// DEFAULT-VALUE = "Gorilla3D "
    /// 10 characters identify the exporter tool
    Exporter : Array[0..9] of Byte;
    Version : Cardinal;
    /// Datetime when the file was generated. (8 byte float value)
    Timestamp : TDateTime;
    /// The data format the was stored (bson or json).
    Format  : TGorillaG3DFormat;
    /// enum with values above
    Options : TGorillaG3DOptions;
  end;

TModelDef

The TModelDef structure is representing a scene consisting of meshes, materials, lights, cameras and animations. It does not contain mesh information itself. Instead sub meshes are attached which hold those kind of information.

The model definition is the logical representation of a scenery. By this visual FMX instances are built and rendered.

Meshes

A mesh definition is the most important type inside of a model, which holds all necessary vertex data for rendering.

LMesh := TMeshDef.Create(Self);
LMesh.Id := 'Mesh1';
LMesh.Transform := TMatrix3D.CreateTranslation(TPoint3D.Create(-1, -2, 0.5));
// Self = TModelDef
Self.AddMesh(LMesh);

TMeshDef instances are able to manage further sub-meshes inside. If no vertex data is applied to them, the functionate as group. A working example for mesh groups is the TModelDef, which doesn't hold any vertex data, but meshes instead.

LSubMesh := TMeshDef.Create(LMesh);
LSubMesh.Id := 'SubMesh1';
LSubMesh.Transform := TMatrix3D.CreateTranslation(TPoint3D.Create(1, 2, -0.5));
LMesh.AddMesh(LSubMesh);

Static meshes

Static meshes pushing their vertex information once to the GPU and getting rendered only by switching to the specific MeshBuffer.

This increases performance enormously and allows rendering of models with huge number of vertices.

If “IsStatic” is deactivated, firemonkey pushes vertex data to the GPU on each frame, instead.

Warning: “IsStatic” can only be used for static and none-animated meshes. It's recommended to use it for environmental objects (houses, boxes, …) or other static objects in your scene. Do not use it for characters!

Remark: For animated elements Gorilla3D provides an animation-caching mechanism, taking advantage of that feature. Read more about animation-caching here: Animations

Remark: STL or OBJ import automatically set “IsStatic” to true.

Setup vertex data manually

There are a lot of applications need to setup vertex data manually. The mesh definition provides a simple to use method AddPolygon(). Have a look at the following example:

TUserMesh = class(TGorillaMesh)
    public
      constructor Create(AOwner : TComponent); override;
 
      procedure Build();
  end;
 
  [...]
 
constructor TUserMesh.Create(AOwner: TComponent);
begin
  inherited;
 
  FDef := TMeshDef.Create(nil);
end;
 
procedure TUserMesh.Build();
begin
  with TMeshDef(FDef) do
  begin
    AddPolygon([PointF(-0.5, -0.5), PointF(0.5, -0.5), PointF(0.5, 0.5), PointF(-0.5, 0.5)]);
    AddPolygon([PointF(-0.5, -1.5), PointF(0.5, -1.5), PointF(0.5, -0.75), PointF(-0.5, -0.75)]);
    AddPolygon([PointF(-1.5, -1.5), PointF(-0.75, -1.5), PointF(-0.75, -0.75), PointF(-1.5, -0.75)]);
 
    // after all polygon vertices were added
    // compute tex-coords, normals, tangents and binormals
    CalcTextureCoordinates();
    CalcFaceNormals();
    CalcTangentBinormals();
 
    // after vertex data was finally setup - set mesh to static and push data
    // to GPU
    IsStatic := true;
  end;
end;
 
[...]
 
// create a visual instance at runtime
FUserMesh := TUserMesh.Create(GorillaViewport1);
FUserMesh.Parent := GorillaViewport1;
FUserMesh.Position.Point := Point3D(3, -1, 3);
FUserMesh.RotationAngle.X := -45;
FUserMesh.Build();
FUserMesh.MaterialSource := GorillaLambertMaterialSource1;

Fast Ray-Casting

Gorilla3D implements an optimization for fast ray-casting onto triangles inside the model definition and its meshes. By default Firemonkey ray-casting, f.e. by click-detection, only the boundingbox or all triangles are getting scanned. This is extremely slow on large models.

Therefore bounding volume hierarchy (BVH) supported was introduced. To enable this functionality manually, use the following snippet.

Remark: By acquiring a bounding volume hierarchy, vertices are getting duplicated, which increases memory usage.

LModelDef.AcquireBVH();
try
  [...]
finally
  LModelDef.ReleaseBVH();
end;

If the vertex data changed meanwhile and you need to update BVH tree, use the following method:

LModelDef.UpdateBVH();

Use this functionality, f.e. to ray-cast the height of a terrain at a certain position to place an object onto.

VertexGroups

Since format version 2, TVertexGroupDef instances are supported. A vertex group allows to group a number of triangles, referred in an owner mesh, to a virtual mesh. By this you are able to render only this specific triangle group by a specific material.

In some cases this helps to improve memory usage. While declaring vertex data in a parent mesh, those virtual meshes can reuse the same vertices for their triangle rendering, instead of duplicating them.

Materials

A model manages multiple material definitions which can be referenced multiple times to reduce memory usage.

// Self = TModelDef structure, manages all materials
LMatDef := TMaterialDef.Create(Self, 'Material1');

Textures

For better texture management link an TGorillaAssetsPackage to your TModelDef. This allows optimized texture handling and removes duplicated textures. Otherwise you may create an texture-image multiple times, which increases memory-usage a lot.

var LAsset : TGorillaTextureAsset;
     LChannel : TColorChannelDef;
     LTexFile : String;
begin     
  LTexFile := 'textures\diffuse.jpg';
 
  // we want a texture for the diffuse color channel
  LChannel := TColorChannelDef.Diffuse;
 
  // Self = TModelDef
  if Assigned(Self.Package) then
  begin
    // try to read texture file from assets package
    // checks if it was already loaded
    LAsset := TGorillaAssetsPackage(Self.Package).GetOwnerAssetFromFile(
      Self.Asset as TGorillaAsset,
      GORILLA_ASSETS_TEXTURE, LTexFile) as TGorillaTextureAsset;
 
    // if we have found a texture asset, add it to the material
    if Assigned(LAsset) then
      LMatDef.AddTexture(LChannel, LAsset, false); // false == NOT standalone
  end
  else
  begin
      // we don't want to use an assets package
      LAsset := TGorillaTextureAsset.Create(nil);
      // import texture file manually
      LAsset.ImportFromFile(LTexFile);
      // afterwards add it to the material
      LMatDef.AddTexture(LChannel, LAsset, true); // true == STANDALONE
  end;
end;

Lights

Gorilla3D is also able to manage logical instances for lights. It is very common for 3D file formats to export lighting information.

// Self = TModelDef
LLightDef := TLightDef.Create(Self);
LLightDef.LightType := TLightType.Spot;
LLightDef.Direction := Point3D(0, 1, 0);
LLightDef.Diffuse := TAlphaColorF.Create(1, 0, 0, 1);
LLightDef.Ambient := TAlphaColorF.Create(0.5, 0.5, 0.5, 1);
LLightDef.Specular := TAlphaColorF.Create(0.15, 0.15, 0.15, 1);
LLightDef.Intensity := 1;
// attenuation values are configurable but have no effect on rendering yet, due to FMX limitations
LLightDef.ConstantAttenuation := 1.0;
LLightDef.LinearAttenuation := 0;
LLightDef.QuadraticAttenuation := 0;
LLightDef.SpotCutOff := 180;
LLightDef.SpotExponent := 0;
LLightDef.Transform := TMatrix3D.CreateTranslation(TPoint3D.Create(0, -50, -50));

Cameras

Gorilla3D is also able to manage logical instances for cameras. It is very common for 3D file formats to export camera information.

// Self = TModelDef
LCamDef := TCameraDef.Create(Self);
LCamDef._Type := TCameraType.PerspectiveCamera; // TCameraType.OrthographicCamera
LCamDef.Target := 'Mesh1';
LCamDef.FOV := 45;
LCamDef.AspectRatio := 16 / 9;
// near and far plane are configurable, but have no effect on rendering
// due to hardcoded constant values in FMX framework
LCamDef.NearPlane := 1;
LCamDef.FarPlane := 1000;
LCamDef.Transform := TMatrix3D.CreateTranslation(TPoint3D.Create(0, -5, -10));

Shaders

The TShaderDef class is already defined, but not productive yet. It is planned to be able to define shaders inside the logical G3D format. But further previous steps are necessary to allow this functionality.

Skin/Skeleton Animations

G3D format supports animation export and import of skin and skeleton animations by bone skin structures.

[Source: https://cgi.tutsplus.com/tutorials/building-a-basic-low-poly-character-rig-in-blender--cg-16955]

Skin/Skeleton

A skin and skeleton structure is necessary to animate meshes in modern way by vertex weights. Imagine your model contains a skeleton consisting of bones having certain influence on the vertices surrounding them. When moving a bone all connected vertices are moving with it.

To declare such a structure, a so called humanoid (or armature) definition (THumanoidDef) is defined, which contains a bone hierarchy. Each bone (or joint) is connected to a parent bone or to the root bone of the armature. At this point a humanoid/armature has no connection to a specific mesh.

To create a connection between a mesh and a humanoid, create a TControllerDef structure. It will connect mesh and bone information with each other to allow animation. The controller definition contains references to the humanoid/armature-bone structure and stores weights and affected vertices. This allows to reuse a humanoid/armature by another controller with different values or a different mesh.

Humanoid / Joints

A humanoid or also called armature is a hierarchy of bones or also called joints. This hierarchy allows to represent relations between bones and their dependency. For example, if you move the upper leg, the foot and lower leg should also move.

[Source: https://commons.wikimedia.org/wiki/File:Blender3D_Shuffle-Cycle.png]

A joint (or bone) is defined by a position relative to its parent and is used to define the influence on certain vertices.

The armature contains a root joint which is the starting point and top instance in the hierarchy. A single humanoid can be linked to multiple controller definitions, which define how to animate a humanoid.

When an animation is executed, all humanoids and their controllers are executed. In most cases the controller modifies the joint position by the given [key:value] pair of the animation (TAnimationDef | TAnimationStageDef | TInterpolatorDef).

While the joint is transformed, he knows by its controller information, how to modify the connected mesh and its vertices. And it will change vertex information based on its the weight values and a list of connected vertices.

// Self = TModelDef structure, which manages all humanoids
LHuman := THumanoidDef.Create(Self);
LHuman.Id := 'Armature1';
Self.AddHumanoid(LHuman);

Controllers

A controller definition is a structure to declare how a humanoid/armature interacts with a specific mesh and its vertices. You can create multiple controllers for a single humanoid, for example to modify different sub-meshes inside a model, but with the same armature behind.

When creating a controller, humanoid joints are linked as reference. You then have to define weights and affected vertices for each joint-reference. The references and mesh link is managed by the TSkinDef sub structure of the controller.

// LMesh = TMeshDef structure
LCtrl := TControllerDef.Create(LMesh);
LCtrl.Id := 'Armature1Controller';
 
// Self = TModelDef structure which manages all controllers
Self.AddController(LCtrl);
 
 // create a skin to manager joints
LCtrl.Skin := TSkinDef.Create(LCtrl);
LCtrl.Skin.Id := LCtrl.Id + 'Skin';

As mentioned above, we need to reference all humanoid joints by the LinkJoint() method. Besides all sub joints/bones we have to link our root bone manually.

// because it's the root joint, last argument needs to be TRUE (= standalone)
LCtrl.Skin.Skeleton := TJointRefDef.Create(LHuman.Root, LCtrl, true);
 
// afterwards we link the root joint, which will create another reference
LCtrl.Skin.LinkJoint(LHuman.Root);

In the following go through all humanoid joints and link those explicitly. In case you are writing an importer for a specific 3D file format, this may depend on the given data in your file. Because sometimes not all joints are linked, because they might not affect the mesh.

procedure LinkMyJoints(ASkin : TSkinDef; AJoint : TJointDef);
var LEnum : TJointDefMap.TPairEnumerator;
begin
  if not Assigned(AJoint.Joints) then
    Exit;
 
  LEnum := AJoint.Joints.GetEnumator();
  try
    while LEnum.MoveNext() do
    begin
      ASkin.LinkJoint(LEnum.Current.Value);
      LinkMyJoints(ASkin, LEnum.Current.Value);
    end;
  finally
    FreeAndNil(LEnum);
  end;
end;
 
[...]
LinkMyJoints(LCtrl.Skin, LHuman.Root);

Animations

TAnimationDef contains information about animating a model and its meshes by various modifiers or so called interpolators. Each animation definition (TAnimationDef) can contain multiple TAnimationStageDef instances, which contain modifiers / interpolators to change specific properties of a referenced object (mesh, joint, …).

// Self = TModelDef structure, which manages all animations
LAnimDef := TAnimationDef.Create(Self);
LAnimDef.Id := 'Walk-Animation';
Self.AddAnimation(LAnimDef);

AnimationStage

An animation stage is directly referred to a specific object (mesh or joint) inside the parent model structure. It is the container which holds all modifiers / interpolators.

LStage := TAnimationStageDef.Create(LAnimDef);
LStage.Id := 'Joint1';
// link the TJointDef (of the humanoid) to the stage, as element to be modified
LStage.Reference := LMyJointDef;
// finally add the stage to the animation
LAnimDef.AddStage(LStage);

Interpolators

Interpolators are modifiers, that are able to change various properties, f.e. joint transformation or vertex data.

Time-Value pairs

An interpolator holds a specific number of keys and their values at a certain point of time. During animation it computes missing values between two keys.

Each interpolator needs to store at least two key values to define start and endpoint of an animation. The key consists of a timestamp and a value depending on the used interpolator datatype. The timestamp values need to be normalized to a range between 0.0 and 1.0. While the length (in seconds) of the animation is stored in the “Duration” property.

Here is an example on how to setup an interpolator to move a joint by 5.0 unit upwards in 3.5 seconds. The values will be computed linearly, but various types are also possible:

TAnimationKeyInterpolationType = (Linear, Quadratic, Cubic, Quartic, Quintic, Sinusoidal, Exponential, Circular, Elastic, Back, Bounce);
var LTimes : TArray<Single>;
    LValues : TArray<TPoint3D>;
    LTypes : TAnimationKeyInterpolationDynArray;
 
// setup at least 2 keys
SetLength(LTimes, 2);
SetLength(LValues, 2);
SetLength(LTypes, 2);
 
LTimes[0] := 0.0;
LTimes[1] := 3.5; // == 3.5 seconds
 
LValues[0] := TPoint3D.Zero;
LValues[1] := TPoint3D.Create(0.0, -5.0, 0.0); // move upwards by 5.0 units
 
LTypes[0] := TAnimationKeyInterpolationType.Linear;
LTypes[1] := TAnimationKeyInterpolationType.Linear; // linear interpolation
 
// lets create a TPoint3D interpolator
LPt3DInter := TPoint3DInterpolatorDef.Create(LStage);
LPt3DInter.Id := 'Joint1PositionInterpolator';
LPt3DInter.Path := 'Position.Point';
LStage.AddInterpolator(LPt3DInter);
 
// add keys to our interpolator
LPt3DInter.AddKeys(LTimes, LValues, LTypes);
 
// this method will normalize LTimes value to a range of 0.0 - 1.0
// and stores 3.5 seconds in the "Duration" property
LPt3DInter.NormalizeKeyTimeValues();
Supported Datatypes

To allow different datatypes for property modification, the most common interpolators are already predefined and used by various import formats like DAE (Collada), glTF or FBX.

Point3D Interpolator

Allows to animate a TPoint3D property value.

LPt3DInter := TPoint3DInterpolatorDef.Create(LStage);
LPt3DInter.Id := 'Joint1PositionInterpolator';
LPt3DInter.Path := 'Position.Point';
LStage.AddInterpolator(LPt3DInter);
Vector3D Interpolator

Allows to animate a TVector3D property value.

LV3DInter := TVector3DInterpolatorDef.Create(LStage);
LV3DInter.Id := 'Joint1Position2Interpolator';
LV3DInter.Path := 'Position.Vector';
LStage.AddInterpolator(LV3DInter);
Quaternion3D Interpolator

Allows to animate a TQuaternion3D property value.

LQ3DInter := TQuaternion3DInterpolatorDef.Create(LStage);
LQ3DInter.Id := 'Joint1RotationInterpolator';
LQ3DInter.Path := 'Quaternion';
LStage.AddInterpolator(LQ3DInter);
Matrix3D Interpolator

Allows to animate a TMatrix3D property value. This is especially used for joint-transformation animations. A transformation matrix contains information about translation, rotation and scaling. It is recommended to use this interpolator instead of single position, scale and rotation interpolators.

LMat3DInter := TTransformationInterpolatorDef.Create(LStage);
LMat3DInter.Id := 'Joint1TransformationInterpolator';
LMat3DInter.Path := ''; // it automatically uses the "TransformMatrix" property
LStage.AddInterpolator(LMat3DInter);
MeshData Interpolator

Allows to animate an array of vertex positions. Vertex positions for each vertex inside the array value will be interpolated. CAUTION: A direct vertex manipulation is not managed by a joint/bone. So you have to link the stage to a specific mesh instance.

LVertexInter := TMeshInterpolatorDef.Create(LStage);
LVertexInter.Id := 'Mesh1VertexInterpolator';
LVertexInter.Path := 'Coordinates';
LStage.AddInterpolator(LVertexInter);

Optimization

Because many formats like X3D, FBX or DAE export separated interpolators for rotation, translation or scaling, runtime processing is getting very slow. Therefore Gorilla3D provides a helper method to merge those value to a single transformation interpolator. This increases performance and reduces structure complexity.

LAnimDef.UnifyInterpolators(
    [TUnifyInterpolatorOption.EulerXYZ],
    procedure(AStage : TAnimationStageDef; var AKey: Single; var AValue: TMatrix3D;
      var AInterpolation : TAnimationKeyInterpolationType)
    begin
      // allows to modify the merged transformation matrix before adding as key:value pair
      AValue := ConvertToFbxMatrix(AValue);
    end
    );

In some formats key time values are not normalized to a range between 0.0 - 1.0. Because Gorilla3D expect those timestamps to be normalized, you have to adjust them by:

LMat3DInter.NormalizeKeyTimeValues();

But carefully, this method modifies the “Duration” property of the specific interpolator.

TODO

  • TSamplerDef integration
  • TImageDef integration
  • TShaderDef integration

Changelog

  • Version = 1
    • first format introduction with undocumented modification during development process
  • Version = 2
    • Gorilla.DefTypes.TVertexGroupDef introduced to render an owner-mesh partially
    • Gorilla.DefTypes.TCameraDef introduced
    • Gorilla.DefTypes.TLightDef extended
    • “Type” in every node exporting the qualified delphi class name
    • TMaterialDefKind.mkVertexColor introduced for separation between Color and VertexColor rendering
    • TMaterialDefKind.mkCustom introduced with new properties: ShadingModel, UseTexturing, UseTexture0, UseLighting, UseVertexColor, UseSpecular
    • late resolving of humanoid controllers, because controllers were not registered before creating humanoids (circular reference)
    • on loading g3d files, the loader now adds the path of the g3d to texture paths, if they are relative
    • multiple skeletons per skindef allowed

Next step: Transparency