To take these sequences we drove in complete loops around each block, with the camera pointed out the left window. Thin strips from the center of each frame have been blended together with a Gaussian blend function. No image stabilazation or other processing has been done. The corners didn't come out well and are excluded. Hopefully if we stabilize the images and warp the strips according to horizontal optical flow, we will get much nicer results than these.
This sequence needs a lot of image stabilization, perhaps because I hadn't quite developed my technique for holding the tripod still when going over bumps.
This sequence is more stable, but there are a lot of places where distant background objects or close foreground objects are messed up due to motion paralax. Perhaps I can correct this by computing optical flow and warping the strips (coming soon). There are also a few cars passing in the opposite direction, which show up as extremely shortened and blurry objects due to their motion opposite us.
The corners tend to look incorrect without warping because of the large difference in optical flow. Also, objects on corners will often be extremely warped. Sometimes a corner contains both an object and its mirror image. The first corner is an intersection with several cars. One is backing-up, which makes it look elongated. The mirror images of this car and the one parked next to it are visible. Another car is driving through the intersection, which makes it appear extremely shortened.Yet another is waiting at the stop sign, probably mad at us for running our stop sign. It is strangely warped due to the viewing angle, its change in motion, and the change in perspective. The second corner is a much smoother turn and shows nothing but some grass, trees, and power poles.
Corner between Third and Fourth Strips
Corner between Fourth and Ending Strips
The following images demonstrate the effect of warping the strips according to optical flow in order to make them line up correctly, even for close objects, distant objects, and objects on corners. In cases where the optical flow is backwards, the strips are mirrored so even then they will line up correctly. Sometimes due to limitations of the optical flow calculation, some artifacts are introduced. However, they are not as noticable as the artifacts that the technique corrects.
Blurry cars before strip warping
Crisp cars after strip warping
Nasty corner before strip warping
Not so nasty corner after strip warping
Until now we've just been setting the tripod with the camera on it in the back seat of the car. This system has many disadvantages. Most annoying is that the camera bounces around if the car goes over bumps. The amount of motion is greater than that of the car, so the affect of going over bumps is magnified. Also, we wanted the camera higher than the car so that it could see over cars and other short objects on the side of the road. Furthermore, we wanted a system that could be used with just a driver and no assistant to operate the camera.
To solve all these problems we decided to fix the tripod to the roof of the vehicle. We used an ingenious system of three bar clamps and six hose clamps to accomplish this. Each tripod leg was wrapped in plastic to prevent scratching and then firmly attached to the bar of a bar clamp with two hose clamps. To mount the resulting modified tripod to the roof of the car, we simply open the sunroof and clamp each bar clamp to the roof.
The sytem works well. It stays firmly in place and is quite rigid. I drove the car at speeds of up to 70 mph with the tripod in place and the legs unextended with no difficulties (the camera was not on the tripod at the time). I wouldn't want to go that fast with the legs extended though, as it is not quite as rigid in this mode, and the air resistance would be greater.
The only major drawbacks to the roof-mounted tripod system are that the bar clamps form permanent dents in the roof of the car, and that the camera lens tends to get dirty being outside the vehicle.
Here are some images of the tripod and camera attached to the vehicle roof. The bright red objects are the bar-clamp handles.
Roof mount 1
Roof mount 2
Roof mount 3
Stabilization works by computing the vertical translations between frames using Lucas & Kanade, and keeping a running sum. The successive strips are offset vertically by this summed vertical drift. To prevent excessive drift, a small percentage of the drift total drift is subtracted before adding every strip.
We made a panorama from an entire block and wrapped it around a square with rounded corners using functionality we added to our software. The result has been scaled down so as to be not so huge. Note that we used no image stabilization in creating this panorama, so we wouldn't have to wait all day. Thus it's rather jumpy. There is also no strip-warping, but at this scale it's not really noticeable.
The resulting block map
A pair of stereo pairs were made by using strips with centers offset 20 pixels right or left of the frame center. The pairs were reduced to single channels, colored red and blue, and combined to form a stereo image that can be view with red-blue glasses. Red goes over the left eye, blue goes over the right.
We videotaped a 1.3 mile long freight train and created some panoramas of it. The only way to get the entire train in one image is to either scale everything way down, or squash everything length-wise. The first image shows only the very front of the train at full resolution.
Full resolution, beginning only
10/86 resolution, entire train
10/86 y, full resolution x, entire train
If each frame is only a single vertical column of pixels, and no other data is available, stabilization can still be done if the movement between frames is very small. To demonstrate this, I made a jittery handheld sweep of part of my room and tried to stabilize it. The results are not bad for a first try, but more work will be needed to get them better. One big advantage of this stabilization method is that it runs at a rate of almost a hundred frames per second, instead of around one frame per second as for full-sized frames.
After StabilizationSite Sponsor...