1) Build on your previous Project C results, where CquikMeshImg vertex
positions are described in world space, and pressing the 'V' key cycles the
viewpoint through at least two different viewing positions. I have also
made a series of 36 photos of a Scotch Tape dispenser, rotated in 10-degree
increments. Small versions (320x240 pixels) are available here:
lil2tape.zip.
Send me e-mail if you'd like larger versions: the originals
are 1280x960 pixels.
2) Add two new keyboard response functions in CquikGL, one for the 'F'
(fan) key, the other for the 'P' (picture) key. Users press the 'F' key to
cycle between one image, a fan of images, and a 3D object made from that fan of
images. Users press the 'P' key to cycle between seeing just one image in
the fan, or all of its images.
Create an integer 'm_fanState' member variable for CquikGL,
and increment it modulo-2 (e.g. 0,1,2,0,1,2,...) every time a user presses the F
key. Similarly, create an integer m_pictureState variable,
alternates between 0 and incremented modulo-N: (e.g.
0,1,0,2,0,3,0,4,0.5,...0,N,0,1,0,2,...) where N is the number of pictures loaded
as CquikMeshImg objects in the m_fan array.
The m_fanState value determines what your program will do:
m_fanState==0: display CquikGL's
m_vimg member on-screen. (e.g. don't show the image fan)..
m_fanState==1: Part 1 results,
described in step 9); you should see a matted version of m_vimg from a novel
camera position.
m_fanState==2: Part 2 results,
described in steps 13 and 14. Hide m_vimg, and show a fan of matted CquikMeshImg
objects that all intersect along the y axis. (Shows nothing if user hasn't
yet loaded a sequence of image files).
m_fanState==3; Part 3 result,
described in step 17.. Hide m_vimg, compute the fan of matted CquikMeshImg
objects from Part 2 with all CquikMeshImg vertices moved or 'inflated' to form a
3D shape determined by silhouettes in each picture. This may be
slow! Note that on leaving state 3 you will need to return vertices
to their original positions for Part2 results to work.
The m_pictureState value determines what parts of the image fan are visible
when m_fanState=2 or 3:
m_pictureState = 0; show all
images available in the m_fan array.
m_pictureState = M; show only picture
# M in the m_fan array.
3) Create new classes: a P3 camera class (CP3cam), a P3 plane class (CP3plane), and P3
point class (CP3point) using Matrx and Vec3 classes. You are not required
to make 'drawMe()' functions for these classes.
4) Add a new member to the CquikImgMesh class that holds a P3
camera-class object named m_P3camera. Recall that the extrinsic parameters of a
camera class define a camera position in Cartesian (R3) space, and a camera
coordinate system measured from that point.
5) Add a camera-setting member function CquikImgMesh::setCamera(theta,height)
that sets the m_P3camera member variable, and also moves all the
CquikImgMesh vertices. Recall that initially these mesh vertices are within a unit
square positioned on z=-1 plane and centered on the -z axis, and a view of
these vertices from the origin corresponds to the 'basic' camera P0
described in the class notes. If we translate that CquikImgMesh's camera
and all its vertices by (0,0,+1), the vertices will be in the z=0 plane, and the
camera center will be at (0,0,1). Your setCamera() function should construct
this translated 'basic' camera and vertex set if given theta=0.0,
height=0.0. The camera should always look towards the origin (e.g. the
principal axis must intersect the origin), and the distance from the camera
center to the origin must always be 1.0, but the x,y,z position of the camera
center is set by the function args (height,theta). While height=0 and
theta varies from 0 to 360, the camera center's x,z position traces out a
circle on the x,z plane; if height is not zero, then that circle will move away
from the x,z plane so that y=height. However, as the camera changes, the
vertices of the CquikImgMesh should also change to follow the camera; put
another way, the vertices do not move at all as measured from the camera's
coordinate system. No inverse camera matrices required--just figure out
how to move the vertices to follow the camera movements. Specifically, for each
vertex that was initially positioned at (x0,y0,z0,1) find a new vertex position
(x1,y1,z1, w1); when this new position is multipled by m_P3camera matrix
we get the old position (x0,y0,z0,1), (or a scaled version of it). Test your program by moving and rotating
the camera matrix to different positions with each press of the 'F' key.
6) Create your own CquikBMP::blueMatte(BYTE r_key, BYTE g_key, BYTE b_key,
float dmax) that can identify which image pixels show the object and
which pixels show the background whose nominal color is (r_key,g_key, b_key),
but contains lighting variations. For more details on the
'blue-screen' matte process and a much more robust solution, see Alvy Ray Smith,
James F. Blinn: Blue Screen Matting. SIGGRAPH96, pp. 259-268.
. I suggest this
simple hack for the images I supplied (the 'tape' image series):
-compute a chrominance-like 3-vector for the key color: let
keyL =
log10((r_key + g_key + b_key +1)/3.0); // a log-luminance-like
value, (add 1 to prevent log(0) calls)
keyC[0] =
log10(r_key +1) - keyLum;
// a chrominance-like value: log of the ratios (R/L,G/L,B/L)
keyC[1]=
log10(g_key +1) - keyLum;
// forms a 'colorfulness' vector, = 0 for grayscale values,
keyC[2] =
log10(b_key +1) - keyLum;
// but nonzero with increasing color saturation.
-next, find the normalized vector
keyCnorm[]. (e.g. divide by vector magnitude)
-compute the same normalized chrominance-like value for each pixel: for
(R,G,B) between 0,255, let
pixL = log10((R + G + B +1) /3.0);
pixC[R,G,B] = log10([R,G,B] +1) -
pixL;
pixCnorm[R,G,B[
= pixC[R,G,B] / || pixC[R,G,B] ||
-if vector distance from keyCnorm to pixCnorm is less than
user-supplied constant 'dmax', then this must be a background pixel. Label the
pixel as 'background' by setting its value to pure blue (0,0,255).
-For example, I chose a pixel value
(47,89,109) from a background of lil1tape01.bmp, found keyL = 1.9138, keyC=
[-0.232, +0.0404,+0.1276], giving keyCnorm=[-0.8662,+0.1508,+0.4764]. The
vector distance image ||pixCnorm - keyCnorm|| has values near 1.9 for foreground
and about 0.2 or less for background pixels; I set dmax=1.0 and got good
results.
7) Write a CquikBMP::matteOutline() function, called after BlueMatte() to find and mark all 'outline' pixels. These are all the background pixels (RGB = 0,0,255) that have at least one adjacent pixel value that is NOT (0,0,255). Change all outline pixel values to red: (255,0,0). A pair of pixels are adjacent if both their x and y addresses differ by no more than 1; every pixel has 8 adjacent neighbors, unless the pixel is on the outer edge of the image.
8) Write a CquikMeshImg::textureMatte() function that re-loads the texture-map image from its m_BMP object (easy if you use the m_isNewTexture member variable), but this time it sets texture map alpha (the opacity) to 0.0 for blue-matte pixels (0,0,255), and to 1.0 for all other pixels. After calling blue-matte, all background pixels become invisible on-screen. Be sure your CquikMeshImg objects have exactly one vertex per pixel; (m_Txsize,m_Tysize)==(m_Mxsize,m_Mysize).
9) F key: When the 'F' key state variable value is '1', load one of the 'tape' images included in the zip files, use setCamera() to move its vertices according to a non-trivial camera position of your own choosing, call blueMatte(), matteOutline(), and textureMatte() so that you have a red-outlined planar object on-screen.
10) Add a new CquikGL member variable 'm_fan' that holds an array of up to 36 CquikMeshImg objects. Also, add a new member variable to class CquikMeshImg; 'BOOL m_isVisible' that enables/disables openGL rendering of the object. Its initial value should be FALSE.
11) Modify CquikGL::onFileOpen() so that it will automatically open more than one file. If users enter a file name ending with two digits such as "lil2tape01", then FileOpen() should load a sequence of files into the 'm_fan' array of CquikMeshImg objects. The function loads "lil2tape01" into the first m_fan array element, then searches that same directory for the next-higher-numbered filename (e.g. "lil2tapeXX" where XX is >01 but <=99), and loads that into the next-higher element in the 'm_fan' array and enables it's 'm_isVisible' flag. It keeps loading files until either the m_fan array is full or no more files are found. (Hint: for debugging, at first put just 1 or 2 image files in the directory).
12)Make a CquikGL::makeFan(float camHeight) member function that sets
the m_P3camera positions for every 'm_fan' array element that was loaded
by onFileOpen(). The goal is to set camera positions that correspond to
those that took the pictures we use. To position the cameras:
-define a unit circle (radius=1) in world space centered
around the y-axis (or 'x2' axis in our earlier P2 projects), and in the plane
defined by y=camHeight.
-Place cameras at equally-spaced intervals, in
counter-clockwise order around the circle. Put the first camera in the +y,+z
plane (or x2,x3 plane), and initially use camHeight=0; we may change it later.
13) Modify CquikGL::onFileOpen() so that it performs matting on each image it loads into m_fan, and also calls MakeFan() to construct the displayed image.
14) Complete the 'P' keypress handling function to allow users to select one or all matted images from m_fan to display. Enable/disable display of m_fan array elements using the CquikMeshImg member 'm_isVisible'. At first, put just 3 or 4 images in the source directory so that the fan of images is simple to understand and debug.
15) Add a new member variable to CquikMeshImg class named 'm_pBounds' that is a pointer to a dynamically-allocated array of CP3plane objects.
16) Write a new member function CquikMeshImg::makePlanes() that:
finds all adjacent pairs of 'outline' pixels in the image,
and makes a CP3plane object that passes through those two points and that
CquikMeshImg's camera center.
Notes: a) be sure your CquikMeshImg
objects have exactly one vertex per pixel; (m_Txsize,m_Tysize)==(m_Mxsize,m_Mysize),
and b) Be sure that the normal vector direction of each plane you create
is pointed outwards, in the direction of blue pixels and not inwards.
16a) Optional; may help in testing: when CquikGL's m_fanState==4, draw a thin red line from a visible CquikMeshImg camera center to each of its boundary points.
17) INFLATE: For every CquikMeshImg object we have loaded into m_fan,
move all of its non-matte vertices (RGB != 0,0,255) or (255,0,0)) inwards
along a ray towards the camera's center, stopping at the closest position that
will not violate 3D silhouette constraints. Specifically, we know each
such vertex has a 3D location somewhere along the ray from its current location
through the camera center, but we also know that it cannot be outside the 3D
plane boundaries set by silhouette planes found in 16). Thus to place each
vertex,
--define a ray from the vertex position through the camera
center (set by the m_P3camera member of CquikMeshImg);
--find the intersection point of this ray with all the
boundary planes from all the CquikMeshImg objects in our fan of images, and keep
the intersection point that is nearest the camera center,
--place the vertex at that position.
--Remember, this can be VERY slow!
18) Experiment with different values for camHeight used in step 12. Notice that the image sequence I've given you doesn't guarantee that the axis of rotation for the tape object actually intersects the principal axis of the camera. Will this reduce the accuracy of the 3D shape you construct? How could you find the axis of rotation for the object measured in camera coordinates?
Turn in Your Project: Please follow the instructions found here: Homework Instructions
Example Images: (not done yet)
Last Modified: 05/28/2003 Jack Tumblin