Homam's Mind

Tuesday, September 27, 2011

CSSMatrix 3D Transformations

Well I’ve spent a good weekend figuring out CSS3 3D transformations. In short, now I think it is intelligently designed to fit web design needs, however coming from Direct3D background I was looking for a camera in W3C API.

In this post I show how to make a touch sensitive, interactive 3D model of the iPad using matrix transformations.

Although iPad is not a perfect cuboid, for simplicity we assume it is. Google and download pictures of the 6 sides of the iPad. Read this excellent Introduction to CSS 3-D Transforms on how to make 3D cuboids using CSS3. It is fairly easy and straight forward.

[caption id="attachment_255" align="aligncenter" width="122" caption="iPad layers before transformation; six sides stacked on top of each others"]Untransformed layers[/caption]

I want to rotate the model by touching the screen / moving the mouse. We’re not digging into the details of touch event handling here, check out Touching and Gesturing  on the iPhone for a nice discussion on this subject.

Obviously we move our fingers on the phones screen, or the mouse in 2D flat surfaces, but we want to rotate our iPad in 3D space. Luckily any rotation in 3D space can be decomposed to 3 elemental rotations around the axes of a coordinate system (frame of reference), using Euler angles; meaning that a user can rotate the model to a desired state by no more than 3 gestures.

Transformed iPad ModelCheck out the first draft.


It works; you can freely rotate the model in any direction. But something’s not quite right: the UI response to gestures is not intuitive. Sometimes when you move your fingers to the left, the model rotates upward, another time it rotates down-right…. The problem is that every time that you rotate the model you change the orientation of its axes. You can leave your program as it is, it is sellable and in fact I've bought programs with this bug before. The rest of this post explains a solution to this problem.

Linear Transformations


All CSS3 transformations (rotate, scale, skew) are reversible, you can rotate an object 40 degrees clockwise, scale it to 2x bigger, then shrink it to half and rotate it 40 degrees counter clockwise and you end up with the object in its initial state. Another interesting feature of CSS3 transformations is that although an image can be distorted by a transformation, straight lines don’t curve or bend and remain straight under any transformation. Each point of the original image is always mapped to one and only one point of the transformed image. These are the characteristics of linear maps.

Any linear map can be represented by a transformation matrix. In our case, in a 3-dimensional space, it is a 3x3 matrix. I won’t dig into the technical details of matrices, simply because we don’t need to know those details. Check out your old analytic geometry textbook.

Given a transformation matrix M, any point P of our object will be transformed by this matrix product:
P’ = M * P

(Here we use the fact that points can be represented by their position vectors, hence column matrices)

There’s a little thing about translation transformation. In good old geometry, a translation can be represented by a vector (when you translate an object, you move it along a path that has a direction and length), and you find the translated coordinates of a point P by summing up its original coordinates with the translation vector: P’ = P + T.  There’s a trick to combine translation with other forms of linear transformations using 4x4 matrices.

If M is a 4x4 transformation matrix and P is a 4x1 column vector representing the coordinate of a point in space (the last row of the vector is set to 0), then:
P’ = M * P

P’ (the matrix product of M and P) is the coordinate of our point after the object has been transformed by M.

M can represent any state of escalation, rotation, translation, or skewness.

CSSMatrix


CSS gives us the option to define our desired transformation by a 4x4 transformation matrix. This method is virtually useless in CSS declarative way, for a 3D transformation you have to calculate the matrix elements and pass 16 parameters to matrix3d() property function (for an example to see how obscure the code might become, check out rotate3d() definition in W3C’s CSS 3d transforms draft). But it is easy and very convenient to use this matrix in JavaScript code, thanks to DOM’s CSSMatrix interface. Currently (Sep. 2011) WebKit implements this interface by WebKitCSSMatrix type.

We initialize an instance of a WebKitCSSMatrix by passing a correct string value of -webkit-transform CSS property. So one can construct it by something like this:
new WebKitCSSMatrix("scale3d(1,2,1)")
or
new WebKitCSSMatrix("scale3d(1,2,1) rotate3d(0,0,1, 45deg) translate3d(100px, 0, -20px)")

Here’s where this window object’s little useful function comes handy: 'window.getComputedStyle()'. getComputedStyle() takes a DOM Element and returns an instance of CSSStyleDecleration that is a representation of all the style properties currently set for the element. It is also a dictionary. You can get the current transform value by: window.getComputedStyle(element)["-webkit-transform"] or by window.getComputedStyle(element).webkitTransform property. Its value is in form of matrix() or matrix3d(). To get the current CSSMatrix that is applied to an element use:
m = new WebKitCSSMatrix(window.getComputedStyle(element).webkitTransform)

CSSMatrix is indeed a 4x4 matrix (its properties are named m11 to m44), its toString() method returns its CSS representation (in matrix() or matrix3d() form).

It also provides a handful of useful functions for matrix manipulation. These functions don’t mutate the object; they return a new instance of CSSMatrix:

  • multiply

  • inverse

  • translate

  • scale

  • rotate

  • rotateAxisAngle

  • skewX

  • skewY


Check out Apple’s documentation.

I learnt it in a hard way that multiply() function doesn’t exactly work as I understand from the documentations. The text says, and I naturally expected that, given matrices A and B, A.multipy(B) must be equal to A * B in math notation. But it turned out that it is actually equal to B * A.

Back to our original problem, let’s differentiate between the model's frame of reference and the world (device viewport) frame of reference. Your view port (computer’s screen) has a static frame of reference (for our purpose). Viewport axes: Up (Y), Right (X) and Facing you (Z) are attached to the device; they don’t change with respect to the device. But the directions of your 3D model’s axes (X’,Y’,Z’) change as you rotate it inside the viewport. Our UI inconsistent response problem happened because when we move our fingers upward on the device, we expect the model to rotate around device X axis, but rotate3d(1,0,0,#deg) actually rotates the model around its own X’ axis.

Viewport vs Model frame of reference


Luckily rotate3d(z,y,z,#deg) function can rotate the model around any arbitrary axis (defined by vector [x,y,z] here). So the problem boils down to finding, which axis of the rotated object is parallel to the device X and Y axes, after an arbitrary rotation.

We know that rotation can be represented by a linear transformation matrix. If V’ is an arbitrary axis on the model, that was parallel to V (an axis in device frame of reference) prior to the rotation, then we can find what axis on object is now parallel to V after the rotation, by:
V’’ = M * V’

(where M is the transformation matrix that defines the rotation)

If the model is not transformed (when window.getComputedStyle(element).webkitTransform is the identity matrix) V’’ is parallel to V’ parallel to V.

Note that we represent an axis by a vector parallel to it, so the column vector:  represents X axis, [0,1,0] represents Y and [0,0,1] represent Z. A rotation linear transformation can be represented by a 3x3 matrix
M =Transformation matrix (M)

There you go. You can extract CSSMatrix m11..m33 elements and write a little bit of JavaScript to produce V’’. Or use CSSMatrix.multiply() function that takes a CSSMatrix as its argument; then you have to construct a 4x4 representation of axes by just padding the column vector and setting all the other elements to 0.

In JavaScript:
var deviceXAxis = new WebKitCSSMatrix("matrix3d(1,0,0,0, 0,0,0,0, 0,0,0,0, 0,0,0,0)");
var deviceYAxis = new WebKitCSSMatrix("matrix3d(0,0,0,0, 1,0,0,0, 0,0,0,0, 0,0,0,0)");

The result of deviceXAxis.multiply(transformation) is the object’s X’’ axis that is currently parallel to device X axis.

The following function rotates the model around device X and Y axes (resulting in a natural user experience):
function rotateModel (xRot, yRot) {
    // get the current transformation matrix:
var m = new WebKitCSSMatrix(window.getComputedStyle(cube).webkitTransform);
    // Model Y’ axis that is now parallel to device Y axis:
var yAxis = ipad.deviceYAxis.multiply(m);
    // Rotate around Y’:
var m1 = m.rotateAxisAngle(yAxis.m11, yAxis.m21, yAxis.m31, yRot);
    // Model X’ axis that is now parallel to device X axis:
var xAxis = ipad.deviceXAxis.multiply(m1);
    // Rotate around X’:
var m2 = m1.rotateAxisAngle(xAxis.m11, xAxis.m21, xAxis.m31, xRot);
    // Apply the final rotation matrix to the model:
cube.style.webkitTransform = m2.toString();
}

Check out the final product here.
Sunday, July 10, 2011

HTML5 mobile games

For everybody who got here looking for selling their games to my company, you can catch me in twitter @homam, GoogleWindows Live (whichever you like :)).