Abstract: Due to the availability and afford-ability of the stereoscopic equipment’s (e.g. stereo camera, lens, and display devices), stereoscopic image manipulation is receiving considerable research attention in recent years. In this paper, we present a semi-automatic, aesthetic-driven stereoscopic image recomposition approach that capacitates the change of the spatial position of the foreground object(s) in a given stereoscopic image for improving human visual aesthetics. Our algorithm recomposes both the left and right stereo images simultaneously using a global optimization algorithm. To maximize image aesthetics, our algorithm minimizes a set of aesthetic quality errors, that is derived from selected photographic composition rules. In addition, depth adaptation is applied to the resized objects and change in vertical disparity of the resulting stereo image pair is minimized to ensure pleasant 3D viewing experience. Our method can be used to perform stereoscopic image re-targeting and recomposition simultaneously by providing an additional the target image scale as the input. The conducted empirical evaluations illustrate the effectiveness of our approach in enhancing the aesthetics of stereoscopic 3D images. Notably, depth adaptation is shown to play an important role in aesthetics enhancement.
PDF at https://ieeexplore.ieee.org/document/8327910
Parsimony in signal representation is a topic of active research. Sparse signal processing and representation is the outcome of this line of research which has many applications in information processing and has shown significant improvement in real-world applications such as recovery, classification, clustering, super resolution, etc. This vast influence of sparse signal processing in real-world problems raises a significant need in developing novel sparse signal representation algorithms to obtain more robust systems. In such algorithms, a few open challenges remain in (a) efficiently posing sparsity on signals that can capture the structure of underlying signal and (b) the design of tractable algorithms that can recover signals under aforementioned sparse models.
In this dissertation, we try to view the signal recovery problem from these viewpoints. First, we address the sparse signal recovery problem from a Bayesian perspective where sparsity is enforced on reconstruction coefficients via probabilistic priors. In particular, we focus on a variant of spike and slab prior, which is known to be the gold standard to encourage sparsity. The optimization problem resulting from this model has broad applicability in recovery and regression problems and is known to be a hard non-convex problem whose existing solutions involve simplifying assumptions and/or relaxations. We propose an approach called Iterative Convex Refinement (ICR) that aims to solve the aforementioned optimization problem directly allowing for greater generality in the sparse structure. Essentially, ICR solves a sequence of convex optimization problems such that sequence of solutions converges to a sub-optimal solution of the original hard optimization problem. We propose two versions of our algorithm: (a) an unconstrained version, and (b) with a non-negativity constraint on sparse coefficients, which may be required in some real-world problems. Many signal processing problems in computer vision and recognition world can benefit from ICR. These include face recognition in surveillance applications, object detection and classification in the video, image compression and recovery, image quality enhancement etc.
On the other hand, one of the most signify cant challenges in image processing is the enhancement of image quality. To address this challenge we aim to recover signals by using the prior structural knowledge about them. In particular, we pose physically meaningful probabilistic priors to promote sparsity on reconstruction coefficients or design parameters of the problem. This has a variety of applications in signal and image processing including but not limited to regression, denoising, inverse problems, demosaicking and super-resolution. In particular, sparsity constrained single image super-resolution (SR) has been of much recent interest. A typical approach involves sparsely representing patches in a low-resolution input image via a dictionary of example LR patches, and then using the coefficients of this representation to generate the high-resolution output via an analogous HR dictionary. In this dissertation, we propose extension of the SR problem which is twofold: (1) extension of sparsity-based SR problems to multiple color channels by taking prior knowledge about the color information into account. Edge similarities amongst RGB color bands are exploited as cross-channel correlation constraints. These additional constraints lead to a new optimization problem, which is not easily solvable; however, a tractable solution is proposed to solve it efficiently. Moreover, to fully exploit the complementary information among color channels, a dictionary learning method is also proposed specifically to learn color dictionaries that encourage edge similarities (2) Tackle the super-resolution problem from a deep learning standpoint and provide deep network structures designed for superresolution. A step further in this line of research is to integrate sparsifying priors into deep networks and investigate their impact on the performance especially in absence of abundant training.
Abstract: This paper studies the composition in portrait paintings and develops an algorithm to improve the composition of portrait photographs based on example portrait paintings. A study of portrait paintings shows that the placement of the face and the figure is pose-related. Based on this observation, this paper develops an algorithm to improve the composition of a portrait photograph by learning the placement of the face and the figure from an example portrait painting. This example portrait painting is selected based on the similarity of its figure pose to that of the input photograph. This similarity measure is modeled as a graph matching problem. Finally, space cropping is performed using an optimization function to assign a similar location for each body part of the figure in the photograph with that of the figure in the example portrait painting. The experimental results demonstrate the effectiveness of the proposed method. A user study shows that the proposed pose-based composition improvement is preferred more than rule-based methods and learning-based methods.
PDF at https://ieeexplore.ieee.org/abstract/document/8319951
Abstract: Photo composition is an important factor affecting the aesthetics in photography. However, it is a highly challenging task to model the aesthetic properties of good compositions due to the lack of globally applicable rules to the wide variety of photographic styles. Inspired by the thinking process of photo taking, we formulate the photo composition problem as a view finding process which successively examines pairs of views and determines their aesthetic preferences. We further exploit the rich professional photographs on the web to mine unlimited high-quality ranking samples and demonstrate that an aesthetics-aware deep ranking network can be trained without explicitly modeling any photographic rules. The resulting model is simple and effective in terms of its architectural design and data sampling method. It is also generic since it naturally learns any photographic rules implicitly encoded in professional photographs. The experiments show that the proposed view finding network achieves state-of-the-art performance with sliding window search strategy on two image cropping datasets.
After MAC OS X Sierra, OpenCV2 became buggy to install on it. The best solutions were partially installed it without some packages. Now it is no longer supported by MAC OS X Sierra, and when you “brew install opencv”, it will install OpenCV3 no more OpenCV2 exists in homebrew.
Integrating Deep-learned Models and Photography Idea Retrieval
Farshid Farhat, Mohammad Kamani, Sahil Mishra, James Wang
[Click to get the pro portrait dataset]
ABSTRACT: Retrieving photography ideas corresponding to a given location facilitates the usage of smart cameras, where there is a high interest among amateurs and enthusiasts to take astonishing photos at anytime and in any location. Existing research captures some aesthetic techniques such as the rule of thirds, triangle, and perspectiveness, and retrieves useful feedbacks based on one technique. However, they are restricted to a particular technique and the retrieved results have room to improve as they can be limited to the quality of the query. There is a lack of a holistic framework to capture important aspects of a given scene and give a novice photographer informative feedback to take a better shot in his/her photography adventure. This work proposes an intelligent framework of portrait composition using our deep-learned models and image retrieval methods. A highly-rated web-crawled portrait dataset is exploited for retrieval purposes. Our framework detects and extracts ingredients of a given scene representing as a correlated hierarchical model. It then matches extracted semantics with the dataset of aesthetically composed photos to investigate a ranked list of photography ideas, and gradually optimizes the human pose and other artistic aspects of the composed scene supposed to be captured. The conducted user study demonstrates that our approach is more helpful than the other constructed feedback retrieval systems.
Computer Vision: A Modern Approach, 2e, is appropriate for upper-division undergraduate- and graduate-level courses in computer vision found in departments of Computer Science, Computer Engineering and Electrical Engineering.