doc: explain the stencil texture coordination transformations

Author: mardy

doc/src/opengx.tex

@@ -432,8 +432,82 @@ \subsubsection{Lit and textured render}
 
 Note that we'll have to make the Z buffer operate per fragment and not per vertex (by setting \lstinline{GX_SetZCompLoc(GX_DISABLE)}) and set the alpha compare function (\fname{GX\_SetAlphaCompare}) to exclude all fragments having an alpha value of zero: this is important so that the discarded fragments won't update the Z-buffer.
 
+\paragraph{Generating texture coordinates}
+
 The next problem we have to solve is setting up a texture coordinate generation that, once the stencil texture is loaded in our TEV stage, would allow us to read its pixels using screen coordinates; in other words, we want to make it so that for every fragment processed in this stage, its texel would coincide with a screen pixel. This can be achieved by setting up a texture coordinate generation matrix that multiplies the primitive's vector's \emph{position} and transforms that to the exact x and y coordinates that this vertex will occupy on the screen. Such a matrix can be built by concatenating the movel-view matrix with the projection matrix, but we must take into account that such a matrix will transform vertex coordinates into the \lstinline{[-1,1]x[-1,1]} range whereas the TEV expects texture coordinates to be in the \lstinline{[0,1]x[0,1]} range, so we have to concatenate an additional matrix to translate and scale the coordinates by half.
 
+This is relatively simple to do when the projection is an orthographic one, because in that case we set the texture coordinate generator engine to work on 2x4 matrices (since we only deal with affine transformations), so we can use such a matrix to map the \lstinline{[-1,1]x[-1,1]} range to \lstinline{[0,1]x[0,1]}:
+
+\begin{figure}[ht]
+\centering
+$ \begin{pmatrix}
+	0.5 & 0    & 0 & 0.5\\
+	0   & -0.5 & 0 & 0.5\\
+\end{pmatrix} $
+\caption{Matrix to translate the texture coordinates to be in the \lstinline{[0,1]x[0,1]} range: it's composed by two operations: scale by half and translate by half.}
+\label{fig:transortho}
+\end{figure}
+
+In the case of perspective projection, the $(x, y, z)$ vertex coordinates get divided by the fourth $w$ coordinate before rendering, and $w$ is computed by the fourth row of the perspective matrix (see Figure~\ref{fig:transperspective}), which, being fixed to $(0, 0, -1, 0)$, always renders $w = -z$. The texture coordinate generator in the TEV does not support 4x4 matrices, but only 2x4 and 3x4 ones, so the problem of computing the $w$ coordinate and dividing the $x$, $y$ and $x$ coordinates by it is not trivial.
+
+\begin{figure}[ht]
+\centering
+\subfigure[The generic OpenGL perspective projection matrix] {
+$ \begin{pmatrix}
+    a_{1,1} & 0       & a_{1,3} & 0      \\
+    0       & a_{2,2} & a_{2,3} & 0      \\
+    0       & 0       & a_{3,3} & a_{3,4}\\
+    0       & 0       & -1      & 0      \\
+\end{pmatrix} $
+}
+\hspace{1em}
+\subfigure[The scale and translation matrix we set on the TEV when a perspective projection is used.] {
+$ \begin{pmatrix}
+    -0.5 & 0   & 0.5 & 0 \\
+    0    & 0.5 & 0.5 & 0 \\
+    0    & 0   & 1   & 0 \\
+\end{pmatrix} $
+}
+\caption{Mapping of coordinates when using a perspective projection.}
+\label{fig:transperspective}
+\end{figure}
+
+Luckily, the texture coordinate generator in the TEV already performs a very similar operation, which we can reuse for our purposes: when fed with a 3x4 matrix, it will generate a vector made of three elements, $(s, t, u)$, of which the last is used to divide the first two in order to obtain the usual $s$ and $t$ texture coordinates (that is, the texture coordinates will effectively be $(s/u, t/u)$). Given that $w = -z$ and that $u$ in our transformation takes the value of $z$, we just need to take care of inverting the sign of our $s$ and $t$ values, which can be easily done by inverting the sign of the $a_{1,1}$ and $a_{2,2}$ elements of our transformation matrix. Unfortunately, there's still another problem: since the division by $u$ is the very final operation that gets performed, we cannot just encode the translation by $0.5$ (which we need in order to remap the vertex coordinates to \lstinline{[0,1]x[0,1]}) as we do in the orthographic case, because this would also get divided by $q$ and produce an incorrect result. The solution to this problem is using a different transformation matrix, which would take into account the fact that the translation factor will also get divided by $q$: if we move the $0.5$ translation elements to the third column, instead of storing them on the fourth one, they will get multiplied by the $z$ coordinate, and this will compensate the division by $q$, being $q = z$:
+
+$$ \begin{pmatrix}
+    a_{1,1} & 0   & 0.5 & 0 \\
+    0    & a_{2,2} & 0.5 & 0 \\
+    0    & 0   & 1   & 0 \\
+\end{pmatrix}
+\begin{pmatrix}
+    x\\
+    y\\
+    z\\
+    1\\
+\end{pmatrix}
+/ q
+=
+\begin{pmatrix}
+    a_{1,1}x + 0.5z\\
+    a_{2,2}y + 0.5z\\
+    z\\
+    1\\
+\end{pmatrix}
+/ q
+\stackrel{q=z}{=}
+\begin{pmatrix}
+    \frac{a_{1,1}}{q}x + 0.5\\
+    \frac{a_{2,2}}{q}y + 0.5\\
+    1\\
+    1\\
+\end{pmatrix}
+$$
+
+This allows us to translate the texture coordinates by half towards the positive direction of the axes.
+
+
+\paragraph{Comparing stencil texels}
+
 Another issue is how to actually implement the comparison, since the OpenGL specification supports all kinds of arithmetical comparisons, whereas the TEV only supports comparing for equality (\lstinline{GX_TEV_COMP_A8_EQ}) and strict "greater than" (\lstinline{GX_TEV_COMP_A8_GT}); however, since we know that we are operating on integer values, most of this operations can be emulated by inverting the order of the operands in the TEV, or by altering the reference value by ±1, as shown in Figure~\ref{table:stencil1}.
 
 \begin{figure}[ht]