Last update: Feb. 18, 2008
BACK
SkiP: Sketch
in Perspective
Future Steps
History
Bibliography
With SkiP, our intent is to provide a user with a system allowing
gesture-based interactions for rapid 3D reconstruction of complex
models/shapes/buildings from images showing strong perspective cues.
The emphasis in 3D reconstruction from multiple images
has been either:
- In Photogrammetry: high precision of
measurements (3D loci) with extensive and cumbersome human interactions
(mostly through point measures and correspondences).
- In Computer Vision: full automation with
less precision, but still through the matching of point-based
measurements (mostly), and some primitive-based measures (recent work
based on conics and similar regular shapes).
3D reconstruction from a single view has also been considered
to a lesser extent, in both fields, with similar paradigms. Working
with a single view has the disadvantage of providing less constraints
on object point loci, and less cues for 3D computations. However, there
are some practical advantages when working with a single view:
- Sometimes it is all we have available: the scene does not exist
anymore or is unreachable and all we have is this single snapshot
(think of an old photo).
- The scene of interest may be a drawing or painting of a scene
that may have only existed in the imagination of the artist.
- It is arguably easier for a human to work on one image at a
time.
Hence, our first ambition with SkiP was to explore interactive ways of
doing 3D reconstruction from single views. We have had some level of
success with this approach (cf. the first 2 versions of SkiP, below). Our approach is based on three
steps:
- Interactive extraction of Perspective cues, an object being seen
in 1, 2 or 3 Point Perspective, and of a Ground plane, in overlay to
the original single view (picture), to specify camera parameters
(Central projection, with no lens distortion, is assumed).
- Interactive placement of primitives from/onto the Ground plane,
in overlay of the picture, allowing for an eventual refinement of the
camera parameters (if the primitives do not fit well enough the
outlines in the picture).
- Continuation of the placement of 3D primitives & CSGs to
obtain as much geometric information as can be from the single view;
followed by texture mapping of some selected primitives using again the
picture.
Examples of reconstruction and use of Skip (v2):
- Ames room (in 1 PtP)
- Archeological site of Petra
in Jordan (1 or 2 PtP).
The extensions to SkiP shall tackle the following problems:
- Importing degrees of automation in the extraction of:
- calibration data, such as vanishing points;
- extraction of outlines of objects.
- Process multiple images.
- Permit the fusion with other useful sources of data, i.e.
previous "sketches" (seen now as more complex sets of "primitives"),
maps and floor plans.
- Free-form modeling in 2D & 3D.
- Use of texture gradients in natural scenes.
First version of the project: May
1998 - Starts as a final course project for CS224.
Second version: Oct. 1998 - Improvements
on the user interface & some rendering tools.
BACK
Page created & maintained by Frederic Leymarie,
1998-2008.