[4eyes] Updates from SIGGRAPH (Days 2 and 3)

[Yon Visell]

Pradeep,

Thank you for these very informative summaries!

Skimming the the gaussian material synthesis paper already suggests to me a
different approach to a problem we are working on related to constructing a
latent control/parameter space from haptic perception data.

Looking forward to reading the texture synthesis paper as soon as time
permits. Haptic textures share common causes with visual and auditory
textures, but the physics involved are quite different (a few people have
investigated transfer learning for textures with limited results). There’s
some interesting recent work on haptic textures (e.g. photometric data
driven texture synthesis) reflecting a need for new methods. On the
hardware side, emerging surface haptic displays have high dynamic range and
bandwidth for texture display (with caveats), but existing methods of
haptic texture rendering are largely based on simpler methods such as
direct measurement/sampling, simple procedural synthesis or manually
prescribed microgeometries.  There are some serious but interesting
complexities related to finger mechanics, a topic we have investigated in
recent work.

- Yon


On Wed, Aug 15, 2018 at 12:16 AM Pradeep Sen <psen at ece.ucsb.edu> wrote:

> Hi everyone,
>
> Here are just a few of the things of interest in SIGGRAPH Days 2 and 3:
>
>
> *PAPER TALKS*
>
> *Computational Image Processing*
>
> Hu et al. "Exposure: A White-Box Photo Post-Processing Framework"
> This paper attempts to learn the photo post-processing pipeline. They can
> then automatically
> retouch pictures to make them look good. It's interesting because I've
> been talking about
> something related, albeit in a different way for another application.
>
> He et al. "Deep Examplar-Based Colorization"
> Another paper along the lines of training for colorizing grayscale images,
> but one thing that's
> cool is that they use an example image as a prior to reconstruct the final
> image's color
> scheme. Results are pretty impressive!
>
> Zhou et al. "Non-Stationary Texture Synthesis by Adversarial Expansion"
> First texture synthesis work I have seen that handles spatially-varying
> textures well without
> the need of user guidance. The cons is that a separate network is needed
> for each texture
> learned. I think that this overall area has more room to explore,
> especially for real-time
> applications.
>
> Zhou et al. "
> Stereo Magnification: Learning View Synthesis Using Multiplane Images
> "
> Extrapolates view from a narrow baseline of cameras using a data structure
> they
> call
> multiplane
> images (MPI) (basically a set of alpha-blended planes at some known
> depths)
> and
> they train it
> using deep learning. The multiplane image structure is
> differentiable so they
> can train the system
> end-to-end. Since they don't have labeled
> samples
> for real-world scenes
>
> to directly train the MPI, they train them so they can
> compute the real results at
> different positions with known camera positions.  Here's
> the part I thought was most clever:
> to get their dataset (which is always a pain for
> these kinds of things), they
> used 7,000
> real-estate
> videos
> from YouTube
> which
> have a camera
> slowly panning over
> scenes of
> houses, etc.
> Very
> good idea to get
> lots of data!
>
> He et al. "
> Gigapixel Panorama Video Loops
> "
>
> Very cool paper which stitches together a bunch of different videos and
> turns them into a
> navigatable
> video panorama.  This idea could be applied, e.g., to VR. Has anyone done
> even the standard video
> panoramas for VR?  If not, this could be an interesting area to
> explore as a collaboration between
> MIRAGE Lab and 4Eyes lab?
>
> Aberman et al.
> "Neural Best-Buddies: Sparse Cross-Domain Correspondence"
> This paper uses a hierarchy of features from a pre-trained deep CNN (VGG)
> to find sparse
> correspondences across images from vary different categories. Very useful
> idea that could
> potentially have many applications.
>
>
>
> *Rendering*
> Anyone working on rendering or using rendering in their applications
> (e.g., for AR relighting)
> should probably check out these two new papers which talks about how to
> efficiently
> compute the integrals of spherical harmonics for lightling:
>
> Belcour et al. "Integrating Clipped Spherical Harmonics Expansions"
> Wang et al. "Analytic Spherical Harmonic Coefficients for Polygonal Area
> Lights"
>
> Marco et al. "Second-Order Occlusion-Aware Volumetric Radiance Caching"
> This paper proposes a way to compute participating media derivatives that
> account for
> occlusion changes for interpolation.  They can produce some very nice
> results for
> participating media in complex scenes.
>
> Gruson et al. "
> Gradient-domain Volumetric Photon Density Estimation
> "
>
> Voumetric rendering in the gradient domain with path tracing, and they can
> apply it to a
> variety to different methods.
> Maybe I am biased (no pun intended), but I love these gradient-
> based methods.
>
>
> *AR/VR*
>
> Langbehn et al. "
> In the Blink of an Eye: Leveraging Blink-Induced Suppression for
> Imperceptible
> Position and Orientation Redirection in Virtual Reality
> "
> Clever idea: slightly move the scene when the viewer is blinking so that
> you can
> re-orient
> them
> so that they can navigate a larger
> virtual space in a limited physical location. Seems to
> work well!
>
> Sun et al. "Towards Virtual Reality Infinite Walking: Dynamic Saccadic
> Redirection"
> Kind of like the previous paper, except that the exploit our eyes'
> saccades to do this. These
> are all ideas we should pursue.
>
>
> *Geometry/Modeling*
>
> Atzmon et al. "
> Point Convolutional Neural Networks by Extension Operators"
> One of the challenges with creating CNNs for point-cloud geometry is that
> it is difficult to
> map convolutions to point-clouds because they are unordered, not on a
> grid, etc. This paper
> comes up with the nice idea to come up with a continuous function of the
> surface from the
> point cloud and then apply a continuous convolution operator to this
> function. Finally, the
> resutling function can be sampled at the end to give the final result.
> This idea could be very
> useful, e.g., for reconstructing 3D environments from
> structure-from-motion data.
>
> Zsolnai-Fehér  et al. "Gaussian Material Synthesis"
> Good approach, great results! I think there is a great future in working
> in automatic learning
> for scene content creation, and material synthesis is a very important
> part of this.
>
>
> There were lots of other cool papers, but these were just some of the ones
> that stood out to me in Days 2 and 3. Let me know if anyone wants to
> discuss any of these or to brainstorm
> potential extensions of this work.
>
>
> *SIGGRAPH KEYNOTE*
>
> Rob Bredow (Head of ILM) gave the keynote on Monday and talked about
> strategies for
> success and also gave some background on the VFX on the new "Solo," where
> he was VFX
> supervisor. It's interesting to hear how many of the effects were done
> "old-school" with
> physical screens on set.
>
>
> *COMPANY ANNOUNCEMENTS*
>
> NVIDIA had a big show on Monday afternoon where CEO Jensen Huang announced
> their
> new "Turing" class architecture Quadro RTX which can do real-time
> ray-tracing.  This is very
> exciting for those who work in photorealistic rendering or need it for
> their applications (e.g.,
> AR/VR, etc.)
>
> I'll send out more info later.
>
> Best,
>
> -Pradeep
>
>
>
>
>
>
> ---
> Pradeep Sen
> Associate Professor
> UCSB MIRAGE Lab
> Dept. of Electrical & Computer Engineering
> University of California, Santa Barbara
> Santa Barbara, CA 93106-9560
> _______________________________________________
> Ilab-users mailing list
> Ilab-users at lists.cs.ucsb.edu
> https://lists.cs.ucsb.edu/mailman/listinfo/ilab-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.cs.ucsb.edu/pipermail/ilab-users/attachments/20180815/9440c2eb/attachment-0001.html>