Interaction
In the following the implementation of mouse, keyboard and gesture interaction with default FMX methods will be described.
In case you'll need extended features, have a look at the InputController implementation: Input Polling
Mouse Interaction
Default FMX event handling may be confusing for some of you, by a bit misleading and unexpected behaviour. Especially the way when events getting called and which components receive those events.
At first you should decide which kind of mouse event handling you expect.
In Viewport
In case you want an overall mouse interaction over the complete scene in your viewport, like a rotation of camera around the scene center. This may be the behaviour you like.
The following code snippet shows how to rotate camera around the center, when left mouse button is down. A special scene setup is important. The GorillaCamera1 is attached as child of Dummy1 with an offset (Position.Z = -10). Also the GorillaCamera1.Target is set to Dummy1. This means the camera is always focusing the dummy object.
type TForm1 = class(TForm) GorillaViewport1: TGorillaViewport; Dummy1: TDummy; GorillaCamera1: TGorillaCamera; private FIsMoving : Boolean; FLastPoint : TPointF; procedure DoOnViewportMouseUp(ASender : TObject; AButton : TMouseButton; AShift : TShiftState; X, Y : Single); procedure DoOnViewportMouseDown(ASender : TObject; AButton : TMouseButton; AShift : TShiftState; X, Y : Single); procedure DoOnViewportMouseMove(ASender : TObject; AShiftState : TShiftState; X, Y : Single); end; [...] procedure TForm1.FormCreate(Sender: TObject); begin GorillaViewport1.OnMouseUp := DoOnViewportMouseUp; GorillaViewport1.OnMouseDown := DoOnViewportMouseDown; GorillaViewport1.OnMouseMove := DoOnViewportMouseMove; [...] end; procedure TForm1.DoOnViewportMouseUp(ASender : TObject; AButton : TMouseButton; AShift : TShiftState; X, Y : Single); begin FIsMoving := false; end; procedure TForm1.DoOnViewportMouseDown(ASender : TObject; AButton : TMouseButton; AShift : TShiftState; X, Y : Single); begin FIsMoving := true; FLastPoint := PointF(X, Y); end; procedure TForm1.DoOnViewportMouseMove(ASender : TObject; AShiftState : TShiftState; X, Y : Single); var LDiff : TPointF; begin if FIsMoving then begin if (ssLeft in AShiftState) then begin LDiff := PointF(X, Y) - FLastPoint; Dummy1.RotationAngle.Y := Dummy1.RotationAngle.Y + LDiff.X; end; FLastPoint := PointF(X, Y); end; end;
A mouse event checks all controls on your form to see if the mouse position is within one of these elements. So in case you have, for example a cube in your scene and the mouse hovers over this cube, FMX will call the MouseMove event of the cube, instead of the viewport.
So they above defined events will not be thrown and rotation will not be executed. To make your mouse move events available all over the viewport, you have to set HitTest of each object inside of the viewport to FALSE.
Another solution is to include rotation behaviour in each object related mouse-move event, or to use TGorillaInputController with global mouse hooks (Input Polling).
Classic Camera Mouse Movement
Very often users want to implement simple camera movement controlled by mouse. Here is a simple example how to move your camera in your scene.
It expects a simple component setup before it works.
- Put A TDummy onto your viewport
- Put A TCamera / TGorillaCamera inside this TDummy component
- Position the TCamera/TGorillaCamera component at a position like (0.0, -2.5, -10)
- Set TCamera/TGorillaCamera “Target” to the Dummy
- Do not forget to set the “Camera” as active in your viewport
- Do not forget to disable “UseDesignCamera”
Then you can create 4 events by the event editor in your IDE object inspector with the following code:
var FMove : Boolean; FLastPos : TPointF; FSpeed : Single = 0.1; procedure TGameWin.GorillaViewport1MouseDown(Sender: TObject; Button: TMouseButton; Shift: TShiftState; X, Y: Single); begin /// allow camera movement FMove := true; /// store the current mouse position to detect the pixel movement FLastPos := PointF(X, Y); end; procedure TGameWin.GorillaViewport1MouseMove(Sender: TObject; Shift: TShiftState; X, Y: Single); var LDiff : TPointF; LDir : TPoint3D; begin /// if mouse was not down - skip camera movement here if not FMove then begin FLastPos := PointF(X, Y); Exit; end; /// compute the offset to the last pixel position LDiff := PointF(X, Y) - FLastPos; if ssRight in Shift then begin /// rotate camera on right mouse button down Dummy1.RotationAngle.Y := Dummy1.RotationAngle.Y + LDiff.X; end else if ssLeft in Shift then begin /// move camera in view direction if left mouse button is down LDir := TPoint3D.Zero; /// at first we apply the left-side direction LDir := LDir + TPoint3D(GorillaCamera1.AbsoluteLeft) * (LDiff.X * FSpeed); /// then we apply the forward direction LDir := LDir + TPoint3D(GorillaCamera1.AbsoluteDirection) * (LDiff.Y * FSpeed); /// we move the parent dummy, not the camera itself! /// simply by adding our direction vector to the current dummy position Dummy1.Position.Point := Dummy1.Position.Point + LDir; end; FLastPos := PointF(X, Y); end; procedure TGameWin.GorillaViewport1MouseUp(Sender: TObject; Button: TMouseButton; Shift: TShiftState; X, Y: Single); begin FLastPos := PointF(X, Y); FMove := false; end; procedure TGameWin.GorillaViewport1MouseWheel(Sender: TObject; Shift: TShiftState; WheelDelta: Integer; var Handled: Boolean); begin if WheelDelta < 0 then GorillaCamera1.Position.Z := GorillaCamera1.Position.Z - 0.5 else GorillaCamera1.Position.Z := GorillaCamera1.Position.Z + 0.5; Handled := true; end;
On components
Like described above, it sometimes is necessary to detect mouse events on components itself. For TControl3D instances like a TGorillaCube or TSphere, different kind of events will be called.
- TMouseEvent3D
- TMouseMoveEvent3D
Those callback events have a different method signature and should be defined like that:
procedure TForm1.DoOnCubeMouseDown(Sender: TObject; Button: TMouseButton; Shift: TShiftState; X, Y: Single; RayPos, RayDir: TVector3D); begin FIsMoving := true; FLastPoint := PointF(X, Y); end; procedure TForm1.DoOnCubeMouseUp(Sender: TObject; Button: TMouseButton; Shift: TShiftState; X, Y: Single; RayPos, RayDir: TVector3D); begin FIsMoving := false; end; procedure TForm1.DoOnCubeMouseMove(Sender: TObject; Shift: TShiftState; X, Y: Single; RayPos, RayDir: TVector3D); var LDiff : TPointF; begin if FIsMoving then begin if (ssLeft in AShiftState) then begin LDiff := PointF(X, Y) - FLastPoint; Dummy1.RotationAngle.Y := Dummy1.RotationAngle.Y + LDiff.X; end; FLastPoint := PointF(X, Y); end; end;
HitTest Property
The HitTest property is a very important value for user interaction and for performance. Because Firemonkey checks all controls on the form each time the mouse moves, many operations are caused. Especially ray cast operations on 3D objects are very costly!
So our intention should be, to reduce those operations to a minimum. Any object that is not important for user interaction or ray-casting, should be disabled for intersection tests.
GorillaCube1.HitTest := false;
But remember, if you disable intersection tests, no mouse events on that object will be called anymore!
Bounding-Volume-History
Gorilla3D offers an optimization for fast raycast intersection tests. This optimization is only available for derived classes of TGorillaMesh with a valid “Def” (TMeshDef) property. Take a further read for Bounding volume hierarchy at: https://en.wikipedia.org/wiki/Bounding_volume_hierarchy
procedure TForm1.FormCreate(Sender: TObject); var LMeshDef : TMeshDef; begin [...] // setup bounding volume hierarchy LMeshDef := GorillaMesh1.Def as TMeshDef; LMeshDef.AcquireBVH(); end;
To destroy a bounding volume hierarchy again, we have to call ReleasBVH().
var LMeshDef : TMeshDef; begin // destroy bounding volume hierarchy LMeshDef := GorillaMesh1.Def as TMeshDef; LMeshDef.ReleaseBVH(); end;
Because setting up a bounding volume hierarchy causes some overhead data you should keep in mind to use this optimization only where it's useful.
Keyboard Interaction
In case you'll need keyboard interaction, best performance will be reached when key events are declared in the form itself. In the following example we setup a physics demo with a sphere. When cursor keys are down, an impulse will be applied to the sphere rigid body, which moves the object in space.
uses [...], Gorilla.Sphere, Gorilla.Physics; type TForm1 = class(TForm) GorillaSphere1: TGorillaSphere; GorillaPhysicsSystem1: TGorillaPhysicsSystem; procedure FormKeyDown(Sender: TObject; var Key: Word; var KeyChar: Char; Shift: TShiftState); [...] end; [...] procedure TForm1.FormKeyDown(Sender: TObject; var Key: Word; var KeyChar: Char; Shift: TShiftState); var LImpulse : TPoint3D; LBody : TQ3Body; begin if (Key = vkLEFT) then LImpulse := TPoint3D.Create(-1, 0, 0) else if (Key = vkRIGHT) then LImpulse := TPoint3D.Create(1, 0, 0) else if (Key = vkUP) then LImpulse := TPoint3D.Create(0, 0, 1) else if (Key = vkDOWN) then LImpulse := TPoint3D.Create(0, 0, -1); // apply impulse on sphere GorillaPhysicsSystem1.RemoteBodyImpulse(GorillaSphere1, LImpulse); end;
The usage of form key events is very rudimentary and may reach its limits very fast.
Gesture Interaction
Gesture handling is supported for Gorilla3D viewport. To allow gesture management you will need a TGestureManager component on your form and you have to link it to the TGorillaViewport component.
In the example below it is described, how to rotate and zoom a model. Therefor we use the interactive gestures: ZOOM and PAN. In the OnGesture event of our GorillaViewport1 we setup a callback procedure to handle gesture input. Simple check for the supplied interactive gesture event: igiZoom or igiPan and execute the very simple DoZoom and DoRotate sub-routines.
type TForm1 = class(TForm) GorillaViewport1: TGorillaViewport; GestureManager1: TGestureManager; GorillaModel1: TGorillaModel; procedure GorillaViewport1Gesture(Sender: TObject; const EventInfo: TGestureEventInfo; var Handled: Boolean); protected FLastPos : TPointF; [...] end; procedure TForm1.FormCreate(Sender : TObject); begin GorillaViewport1.Touch.GestureManager := GestureManager1; GorillaViewport1.Touch.InteractiveGestures := [Zoom, Pan]; GorillaViewport1.OnGesture := GorillaViewport1Gesture; [...] end; procedure TForm1.GorillaViewport1Gesture(Sender: TObject; const EventInfo: TGestureEventInfo; var Handled: Boolean); procedure DoZoom(); var LScale : Single; begin if not (TInteractiveGestureFlag.gfBegin in EventInfo.Flags) and not (TInteractiveGestureFlag.gfEnd in EventInfo.Flags) then begin LScale := EventInfo.Distance / 100; GorillaModel1.Scale.Point := Point3D(LScale, LScale, LScale); end; end; procedure DoRotate(); var LDist : Single; LDiff, LNew, LPos : TPointF; begin LNew := EventInfo.Location; if not Assigned(GorillaModel1) then begin FLastPos := LNew; Exit; end; if not (TInteractiveGestureFlag.gfBegin in EventInfo.Flags) and not (TInteractiveGestureFlag.gfEnd in EventInfo.Flags) then begin LPos := EventInfo.Location; LDiff := FLastPos - LPos; LDist := LDiff.X / 5; GorillaModel1.RotationAngle.Y := GorillaModel1.RotationAngle.Y + LDist; end; FLastPos := LNew; end; begin // just a dirty and quick solution for gesture handling case EventInfo.GestureID of igiZoom : begin // zoom DoZoom(); Handled := true; end; igiPan : begin // rotate model DoRotate(); Handled := true; end; else begin Handled := false; end; end; end;
GamePad Interaction
Interacting with gamepads in Firemonkey is not possible out of the box. But of course you can build your own component.
But before bothering, have a look at the TGorillaInputController of Gorilla3D, which already provides support for gamepads. It is very easy to use and allows combinations of mouse and keyboard input.
Read more: Input Polling
Input-Controller
Because most games and applications use mouse and keyboard in combination, it can be become complex very fast. It's also a bad idea to handle inputs in mainthread an reduce rendering performance by that. Besides that, it's very often a hassle to manage lots of hotkeys or input-sequences. Take a further read on the TGorillaInputController to learn how you could handle those things much easier.
Read more: Input Polling
Next step: Picking