[4eyes] Updates from SIGGRAPH (Days 2 and 3)

[Pradeep Sen]

Hi everyone,

Here are just a few of the things of interest in SIGGRAPH Days 2 and 3:


*PAPER TALKS*

*Computational Image Processing*

Hu et al. "Exposure: A White-Box Photo Post-Processing Framework"
This paper attempts to learn the photo post-processing pipeline. They can
then automatically
retouch pictures to make them look good. It's interesting because I've been
talking about
something related, albeit in a different way for another application.

He et al. "Deep Examplar-Based Colorization"
Another paper along the lines of training for colorizing grayscale images,
but one thing that's
cool is that they use an example image as a prior to reconstruct the final
image's color
scheme. Results are pretty impressive!

Zhou et al. "Non-Stationary Texture Synthesis by Adversarial Expansion"
First texture synthesis work I have seen that handles spatially-varying
textures well without
the need of user guidance. The cons is that a separate network is needed
for each texture
learned. I think that this overall area has more room to explore,
especially for real-time
applications.

Zhou et al. "
Stereo Magnification: Learning View Synthesis Using Multiplane Images
"
Extrapolates view from a narrow baseline of cameras using a data structure
they
call
multiplane
images (MPI) (basically a set of alpha-blended planes at some known
depths)
and
they train it
using deep learning. The multiplane image structure is
differentiable so they
can train the system
end-to-end. Since they don't have labeled
samples
for real-world scenes

to directly train the MPI, they train them so they can
compute the real results at
different positions with known camera positions.  Here's
the part I thought was most clever:
to get their dataset (which is always a pain for
these kinds of things), they
used 7,000
real-estate
videos
from YouTube
which
have a camera
slowly panning over
scenes of
houses, etc.
Very
good idea to get
lots of data!

He et al. "
Gigapixel Panorama Video Loops
"

Very cool paper which stitches together a bunch of different videos and
turns them into a
navigatable
video panorama.  This idea could be applied, e.g., to VR. Has anyone done
even the standard video
panoramas for VR?  If not, this could be an interesting area to
explore as a collaboration between
MIRAGE Lab and 4Eyes lab?

Aberman et al.
"Neural Best-Buddies: Sparse Cross-Domain Correspondence"
This paper uses a hierarchy of features from a pre-trained deep CNN (VGG)
to find sparse
correspondences across images from vary different categories. Very useful
idea that could
potentially have many applications.



*Rendering*
Anyone working on rendering or using rendering in their applications (e.g.,
for AR relighting)
should probably check out these two new papers which talks about how to
efficiently
compute the integrals of spherical harmonics for lightling:

Belcour et al. "Integrating Clipped Spherical Harmonics Expansions"
Wang et al. "Analytic Spherical Harmonic Coefficients for Polygonal Area
Lights"

Marco et al. "Second-Order Occlusion-Aware Volumetric Radiance Caching"
This paper proposes a way to compute participating media derivatives that
account for
occlusion changes for interpolation.  They can produce some very nice
results for
participating media in complex scenes.

Gruson et al. "
Gradient-domain Volumetric Photon Density Estimation
"

Voumetric rendering in the gradient domain with path tracing, and they can
apply it to a
variety to different methods.
Maybe I am biased (no pun intended), but I love these gradient-
based methods.


*AR/VR*

Langbehn et al. "
In the Blink of an Eye: Leveraging Blink-Induced Suppression for
Imperceptible
Position and Orientation Redirection in Virtual Reality
"
Clever idea: slightly move the scene when the viewer is blinking so that
you can
re-orient
them
so that they can navigate a larger
virtual space in a limited physical location. Seems to
work well!

Sun et al. "Towards Virtual Reality Infinite Walking: Dynamic Saccadic
Redirection"
Kind of like the previous paper, except that the exploit our eyes' saccades
to do this. These
are all ideas we should pursue.


*Geometry/Modeling*

Atzmon et al. "
Point Convolutional Neural Networks by Extension Operators"
One of the challenges with creating CNNs for point-cloud geometry is that
it is difficult to
map convolutions to point-clouds because they are unordered, not on a grid,
etc. This paper
comes up with the nice idea to come up with a continuous function of the
surface from the
point cloud and then apply a continuous convolution operator to this
function. Finally, the
resutling function can be sampled at the end to give the final result. This
idea could be very
useful, e.g., for reconstructing 3D environments from structure-from-motion
data.

Zsolnai-Fehér  et al. "Gaussian Material Synthesis"
Good approach, great results! I think there is a great future in working in
automatic learning
for scene content creation, and material synthesis is a very important part
of this.


There were lots of other cool papers, but these were just some of the ones
that stood out to me in Days 2 and 3. Let me know if anyone wants to
discuss any of these or to brainstorm
potential extensions of this work.


*SIGGRAPH KEYNOTE*

Rob Bredow (Head of ILM) gave the keynote on Monday and talked about
strategies for
success and also gave some background on the VFX on the new "Solo," where
he was VFX
supervisor. It's interesting to hear how many of the effects were done
"old-school" with
physical screens on set.


*COMPANY ANNOUNCEMENTS*

NVIDIA had a big show on Monday afternoon where CEO Jensen Huang announced
their
new "Turing" class architecture Quadro RTX which can do real-time
ray-tracing.  This is very
exciting for those who work in photorealistic rendering or need it for
their applications (e.g.,
AR/VR, etc.)

I'll send out more info later.

Best,

-Pradeep






---
Pradeep Sen
Associate Professor
UCSB MIRAGE Lab
Dept. of Electrical & Computer Engineering
University of California, Santa Barbara
Santa Barbara, CA 93106-9560
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.cs.ucsb.edu/pipermail/ilab-users/attachments/20180815/f6cd0c76/attachment.html>