Dear Fellow Scholars, this is Two MinuteÂ
Papers with Dr. Károly Zsolnai-Fehér. Everybody loves style transfer.Â
This is a task typically done  with neural networks where we haveÂ
two images, one for content, and one  for style, and the output is the content imageÂ
reimagined with this new style. The cool thing  is that the style can be a different photo,Â
a famous painting, or even, wooden patterns. Feast your eyes on these majestic images ofÂ
this cat reimagined with wooden parquetry  with these previous methods. And now, look atÂ
the result of this new technique, that looks  way nicer. Everything is in order here, except oneÂ
thing. And now, hold on to your papers, because  this is not style transfer. Not at all. This isÂ
not a synthetic photo made by a neural network,  this is a reproduction of this cat imageÂ
by cutting wood slabs into tiny pieces  and putting them together carefully. ThisÂ
is computational parquetry. And here,  the key requirement is that if we look from afar,Â
it looks like the target image, but if we zoom in,  it gets abundantly clear that the puzzleÂ
pieces here are indeed made of real wood.
And that is an excellent intuition for thisÂ
work. It is kind of like image stylization,  but done in the real world. Now that is extremely challenging. Why is that?Â
Well, first, there are lots of different kinds  of wood types. Second, if this piece was not aÂ
physical object but an image, this job would not  be that hard because we could add to it, clone it,Â
and do all kinds of pixel magic to it. However,  these are real, physical pieces of wood,Â
so we can do exactly none of that.
The only  thing we can do is take away from it,Â
and we have limitations even on that,  because we have to design it in a way thatÂ
a CNC device should be able to cut these  pieces. And third, you will see thatÂ
initially, nothing seems to work well.  However, this technique does this with flyingÂ
colors, so I wonder, how does this really work? First, we can take a photo of the wood panels thatÂ
we have our disposal, decide how and where to cut,  give these instructions to the CNCÂ
machine to perform the cutting,  and now, we have to assemble them in a way thatÂ
it resembles the target image. Well, still, that’s  easier said than done. For instance, imagineÂ
that we have this target image, and we have  these wood panels. This doesn’t look anything likeÂ
that, so how could we possibly approximate it? If we try to match the colors of the two, weÂ
get something that is too much in the middle,  and the colors don’t resemble anyÂ
of the original inputs.
Not good.  Instead, the authors opted to transform bothÂ
of them to grayscale, and match not the colors,  but the intensities of the colors instead. ThisÂ
seems a little more usable…until we realize  that we still don’t know whatÂ
pieces to use and where. Look. Here, on the left, you see how the imageÂ
is being reproduced with the wood pieces,  but we have to mind the fact that asÂ
soon as we cut out one piece of wood,  it is not available anymore, so it has to beÂ
subtracted from our wood panel repository here.  As our resources are constrained, depending onÂ
what order we put the pieces together, we may  get a completely different result. But look. ThereÂ
is still a problem…the left part of the suit gets  a lot of detail, while the right part, not soÂ
much. I cannot judge which solution is better,  less or more detail, but it needs to beÂ
a little more consistent over the image.  Now you see that whatever we do, nothingÂ
seems to work well in the general case.
Now, we could get a much better solutionÂ
if we would run the algorithm with  every possible starting point in the image, andÂ
with every possible ordering of the wood pieces,  but that would take longerÂ
than our lifetime to finish,  so what do we do? Well, the authors have twoÂ
really cool heuristics to address this problem.  First, we can start from the middle, that usuallyÂ
gives us a reasonably good solution, since the  object of interest is often in the middle of theÂ
image and the good pieces are still available for  it. Or, even better, if that does not workÂ
too well, we can look for salient regions,  these are the places where there is a lot goingÂ
on, and try to fill them in first. As you see,  both of these tricks seem to work quite wellÂ
most of the time. Finally, something that works. And if you have been holding on to your papers,Â
now squeeze that paper, because this technique  not only works, but provides us a great deal ofÂ
artistic control over the results. Look at that!  And that’s not all, we can evenÂ
control the resolution of the output,  or, we can create a hand-drawn geometryÂ
ourselves.
I love how the authors took  a really challenging problem,Â
where nothing really worked well,  and still, they didn’t stop until theyÂ
absolutely nailed the solution. Congratulations! Thanks for watching and for your generousÂ
support, and I'll see you next time!.