Homework 5Computer Vision, Fall 2021Due Date: 10AM November 15, 2021Total Points: 18This homework contains one written assignment, one programming walkthroughand one programming challenge. All submissions are due at 10:00 a.m. onNovember 15, 2021. Follow the submission instructions in the document titled“Guidelines for Homework Assignments.” Note that there is no credit for latesubmissions. So please start working on the homework early.Written AssignmentsProblem 1: A Lambertian surface is illuminated simultaneously by two distant pointsources with equal intensity in the direction s1 and s2. Show that for all normals onthe surface that are visible to both sources, illumination can be viewed as comingfrom a single “effective” direction s3. How is s3 related to s1 and s2? Now, if the twodistant sources have unequal intensities I1 and I2, respectively, what is the directionand intensity of the “effective” source? (4 Points)Problem 2: The reflectance map can be parameterized in various ways. In class wehave concentrated on using the gradient (p, q) as a means of specifying surfaceorientation. In some cases, the Gaussian sphere is more suitable for this purpose.Each point on the Gaussian sphere corresponds to a particular direction, from thecenter of the sphere to that point. The orientation of a surface patch can be specifiedby giving the direction of its surface normal. Thus a given surface orientation can beidentified with a particular point on the Gaussian sphere. The reflectance map ismerely a means of associating brightness with orientation.a. What are the contours of constant brightness on the Gaussian sphere in thecase of a Lambertian surface illuminated by a point source? Hint: See Figure1a. (2 Points)b. Show that there are at most two surface orientations that give rise to a givenpair of brightness values when the photometric stereo method is applied to aLambertian surface. Assume that two different light sources are used. Hint:1See Figure 1b. (2 Points)Figure 1: The reflectance map can be plotted on the Gaussian sphere. (a) The contours of constantbrightness are particularly simple for a Lambertian surface illuminated by a point source. (b) Thismakes it easy to prove that there are at most two solutions to the two-source photometric stereoproblem in this case.Programming AssignmentsThis programming assignment has one challenge (with its own subset of milestonesor unit tests). Instructions and summary corresponding to these are given below.runHw5.m will be your main interface for executing and testing your code.Parameters for the different programs or unit tests can also be set in that file.Before submission, make sure you can run all your programs with the commandrunHw5(‘all’) with no errors.MATLAB is optimized for operations involving matrices and vectors. Avoidusing loops (e.g., for, while) in MATLAB whenever possible—looping can resultin long running code. Instead, you should “vectorize [1]” loops to optimize yourcode for performance. In many cases, vectorization also results in morecompact code (fewer lines to write!). If you are new to MATLAB, refer to thesearticles [1] [2] on techniques to optimize MATLAB code.2Challenge 1: Your task is to develop a vision system that recovers the shape, surfacenormal and reflectance of an object. For this purpose you will use photometricstereo.You will be given 5 images of an object taken using five different light sources. Yourtask is to compute the surface normal and albedo for the object. For this purpose,however, you will need to know the directions and intensities of the five lightsources. Thus, in the first part of the assignment, you will compute the light sourcesdirections and intensities from 5 images of a sphere and use this information in thesecond part to recover the orientation and reflectance.The 11 images, sphere0…sphere5, and vase1…vase5 are provided to you.Before you begin, pay attention to the following assumptions you can make aboutthe capture settings:● The surfaces of all objects (including the sphere) are Lambertian. This meansthere are only diffuse peaks in the reflectance maps (no specularcomponents).● For the images, assume orthographic projections.● Image files with the same indices are taken using the same light source. Forexample, sphere1 and vase1 are taken using light source number 1 only.● The objects maintain the same position, orientation and scale through thedifferent images – the only difference is the light source. For example, thesphere in sphere0…sphere5 has the same coordinates and the same radius.● The light sources are not in singular configuration, i.e., the S-matrix that youwill compute should not be singular.● You may NOT assume that the light sources are of equal intensities. Thismeans that you need to recover not only the directions of the light sourcesbut also their intensities.● The background in the image is black (0 pixel value) in all images.The task is divided into four parts, each corresponding to a program you need towrite and submit.a. First you need to find the locations of the sphere and its radius. For thispurpose you will use the image sphere0, which is taken using many lightsources (so that the entire front hemisphere is visible).3Write a program findSphere that locates the sphere in an image andcomputes its center and radius.[center, radius] = findSphere(input_img)Assuming an orthographic project, the sphere projects into a circle on theimage plane. Find the location of the circle by computing its centroid. Alsoestimate the area of the circle and from this, compute the radius of the circle.You may use the Image Processing Toolbox functions im2bw andregionprops.(1 points)b. Now you need to compute the directions and intensities of the light sources.For this purpose you should use the images sphere1…sphere5.Derive a formula to compute the normal vector to the sphere’s surface at agiven point, knowing the point’s coordinates (in the image coordinate frame),and the center and radius of the sphere’s projection onto the image plane(again, assume an orthographic projection). This formula should give you theresulting normal vector in a 3-D coordinate system, originating at thesphere’s center, having its x-axis and y-axis parallel respectively to the x-axisand the y-axis of the image, and z-axis chosen such as to form an orthonormalright-hand coordinate system. Don’t forget to include your formula in yourREADME file.Write a program computeLightDirections that uses this formula, alongwith the parameters computed in (a), to find the normal to the brightestsurface spot on the sphere in each of the 5 images. Assume that this is thedirection of the corresponding light source (Why is it safe to assume this?State this in the README).Finally, for the intensity of the light source, use the magnitude (brightness) ofthe brightness pixel found in the corresponding image. Scale the directionvector so that its length equals this value.light_dirs_5x3 = computeLightDirections(center, radius,img_cell)Center and radius are the resulted computed in (a). img_cell contains the5 images of the sphere. The resulting light_dirs_5x3 is a 5x3 matrix. Row i4of light_dirs_5x3 contains the x-, y-, and z-components of the vectorcomputed for light source i. (2 points)c. Write a program computeMask to compute a binary foreground mask for theobject. A pixel in the mask has a value 1 if it belongs to the object and 0 if itbelongs to the background. Distinguishing between the foreground andbackground is simple: if a pixel is zero in all 5 images, then it is background.Otherwise, it is foreground.mask = computeMask(img_cell)The img_cell contains the 5 images of an object and mask is the binaryimage mask. (1 points)d. Write a program computeNormals that, given 5 images of an object,computes the normals and albedo to that object’s surface.[normal, albedo_img] = computeNormals(light_dirs, img_cell,mask)You may want to use the formula given in the class lecture notes. Be carefulhere! Make sure to take into account the different intensities of the lightsource.Photometric stereo requires the object to be lit by at least 3 light sources.However, in our case, we have a total of 5 light sources. The lighting has beenarranged in such a way that all visible surface points on an object are lit by atleast 3 light sources. Therefore, while computing the surface normal at apoint, choose the 3 light sources for which the point appears brightest. Becareful – choosing the wrong light sources will result in erroneous surfacenormal. (You may also decide to choose more than 3 light sources to computethe surface normal. This results in an over-determined linear system and canprovide robust estimates. However, such a computation is not mandatory.)Do not compute the surface normal for the background. You can assume thatthe surface normal in this region is looking toward the camera. Use the maskgenerated in the previous program to identify whether a given pixelcorresponds to the object or the background.Scale the albedo up or down to fit in the range 0…1 and show them in theoutput image. Thus each pixel in the output image should be the pixel’salbedo scaled by a constant factor. (6 points)5At this point you can use the outputs of your program to reconstruct thesurface of the object. demoSurfaceReconstruction andreconstructSurf.m demonstrate how to use the Frankot-Chellappaalgorithm to compute the 3D shape from surface normals. You can use theMATLAB function surf to visualize the reconstructed surface. Surfacereconstruction is provided as a demo--no submission is required
...