Here's how the bokeh effect works in smartphones

Here's how the bokeh effect works in smartphones

What is the bokeh effect?

A fuzzy background directs the observer's gaze towards the subject in the foreground. This effect is called bokeh, from the Japanese word meaning blur. The bokeh mode is often called portrait mode, because it is particularly suitable for this use

iphone x pixel 2 bokeh2
Here is an image with a bokeh effect. AndroidPIT

Let's first explain the concepts of sharpness and blur. The larger the diameter of the lens of the lens, the faster the light beams move in the path of the beam. Objectives then no longer have a point shape, but become circles larger and larger as the diameter of the lens increases. As soon as one of these circles is larger than one pixel, the corresponding image becomes blurred.

bokeh schematic
Here, we have adopted an infinite distance of subject to simplify the representation: that is why the incoming light rays are parallel. The larger the goal, the lower the tolerance, until the pixel size (3) is larger. With a small objective (2), the objects remain a greater distance net than with a large objective (1). AndroidPIT

If the diameter of the lens is large, the focal length of the lens is also great. The higher this focal length is, the more the focus is selective. The data of smartphones (here 24 or 26 millimeters) do not indicate the true focal distance of the objectives. Here we indicate the equivalent focal length (35mm). This is the focal length that a lens needs with a small 36x24mm lens to have the same picture angle as the combination of lenses of the smartphone. The actual focal lengths of the wide-angle lenses of smartphones are over a range of 4.0 to 4.5 millimeters.

Smartphones, with their mini-lenses and tiny focal lengths, can hardly translate the depth of field. Knowing that there is a significant influence factor: the distance of the subject. The lower it is, the lower the depth of field is selective (an effect that most of you will have already observed on macro photos taken with a smartphone).

iphone x pixel 2 bokeh
LiPhone X and Pixel 2 each have their own approach to create a bokeh effect. AndroidPIT

The computing power to replace the size of the lens

What smartphones have to offer is great computing power, as well as other physical ways to mimic effects by telling images. Basically, the method is always the same: the camera diffrencie the foreground and larrire-plan, and scrambles what is larrire. The closer the subject and the foreground are, the more convincing the effect.

Some defects are recurrent with this kind of process, such as the difficulty of not blowing the end of the hair. The visible plane-gap through glasses is also not blurred properly, as would be the case with an optical bokeh effect.

oneplus 5 botched bokeh
Glasses glasses or fine details often pose problems for the modeling of the effect bokeh. AndroidPIT

The experiments to allow representation of the depth of field on a smartphone are not new. The HTC One (M8) and some Nokia device with Windows Phone (peace them me) were already tried. But the quality of the bokeh effect and the speed of processing was not sufficient, so that smartphones have only recently begun to offer convincing effects.

The HTC One (M8) and Nokia devices mentioned use different approaches, which still distinguish the majority of current smartphones. In the following paragraphs we discuss their difference and the advantages and disadvantages of each method. Apart from Sony, LG and HTC (the exception of the One M8), all major manufacturers now offer the bokeh function on their high-end devices.

AndroidPIT LG V30 iPhone 8 plus Galaxy Note 8 5T Plus HTC U11 Google Pixel 2 Huawei Mate 10Pro 5630
Five of these seven smartphones offer a bokeh effect? AndroidPIT

A double camera for the bokeh effect

Like the pioneer HTC and its One M8, many smartphones nowadays offer a dual purpose camera to calculate the depth of field of the scene photography. In the image of the brain and its two eyes, the smartphone uses the shift of the two lenses. The software then applies a blur to the image parts identified as a background-plane.

Identical focal lengths

The two cameras on the Huawei, Honor, Nokia and Motorola smartphones use the same focal lengths for both sensors. The Bokeh function is therefore available for wide angle shots. In less high-end models like the Honor 7X, the second camera has only 2 mgapixels. Its only goal is the creation of depth of field.

However, it is possible to find another goal for this sensor, such as the Huawei Mate 10 Pro which uses its second lens for high resolution black-and-white photographs to give additional brightness information to the RGB image of the main lens. This improves the quality of the image, especially with the digital zoom. The OnePlus 5T, on the other hand, uses the second lens of the camera for night shots.

AndroidPIT huawei mate 10 pro vs oneplus 5t 5876
The OnePlus 5T and the Huawei Mate 10 Pro offer two cameras with the same focal length, but different sensors. AndroidPIT

Different focal lengths

Other models, such as Asus ZenFone 4, iPhone or OnePlus 5, offer different focal lengths for both lenses. In this case, the bokeh effect can not be used for a wide-angle rendering. The main objective can not take the depth of field of the wide-angle lens. On the other hand, the reverse works: A picture taken with the standard lens can use the wide-angle lens for the depth of field.

This restriction should not be very serious in practice. For portrait photos, the focal lengths of the standard lens give better results anyway. The Galaxy Note 8 is a special case in the field: the wide angle sensor offers a dual-pixel autofocus, which should theoretically allow a bokeh effect without a second lens. But we will come back to it later.

AndroidPIT iPhone 8 Plus vs Samsung Galaxy Note 8 3396
The Samsung Galaxy Note 8 and iPhone 8 Plus each have a dual camera with different focal lengths. AndroidPIT

Special cases

With some more complex systems, the depth of field is not determined by the timing of both lenses. The front camera of the X-phone, for example, projects a pattern of infrared dots in the environment, allowing to recognize the face of the user but also to clean the image. Huawei is working on a similar system.

LAsus Zenfone AR or the Lenovo Phab 2 Pro also go a step further, with their integrated time-of-flight goal under the main objective. These smartphones scan the room using an infrared laser, which allows an increased door. This technology, piloted by Google as part of the Tango Project, is not yet developed. We have already talked about the Tango project and the ToF cameras.

AndroidPIT lenovo phab 2 pro tof camera 4479
The Phab 2 Pro's photo time of flight camera allows for dramatic increases in reality. AndroidPIT

Bokeh effect with a single lens

You know this effect: by keeping a closed eye, your perception of depth is limited. So, how do smartphones manage to distinguish the foreground and the map with a single objective?

For Windows Phone mentioned above, there was an application called Refocus. Smartphones took pictures with different focusing distances, so the computer could select blur areas. It worked pretty well, but the shooting was way too slow. Nowadays, camera software allows this effect much faster.

AndroidPIT Google Pixel 2 4574
Thanks to the dual pixel autofocus, the Google Pixel 2 only needs one lens to make the bokeh effect. AndroidPIT

The Google Pixel 2 offers a bokeh effect with one purpose. This is thanks to a feature offered by the image sensor IMX362, namely dual pixel autofocus. Each pixel in the image is divided here in two halves. As with the dual cameras, it is possible to generate two images slightly shifted. However, the implementation is more complex than with two sensors, since the offset corresponds to half the diameter of the lens. The current generation of pixels combines a series of photos to make the depth, while analyzing the image by artificial intelligence.

Samsung also uses a dual-pixel autofocus on the Galaxy S8 and S8 + (the Sony IMX333), but has for now refuse to offer a true bokeh function. The selective focus works only with macro photos, and without really being user-definable. With the Oreo update coming, both Samsung smartphones should offer a real bokeh effect. Other smartphones equipped with a dual pixel autofocus (such as the HTC U11 or Moto G5 Plus), have not yet access this feature. This could happen however with software update.

What do you think ?

There are many possibilities to make a bokeh effect, even with small smartphone goals. Which of these approaches would you prefer for your next smartphone? You can share your opinion in the comments on the advantages and disadvantages of each method

. (tagsToTranslate) bokeh (t) bokeh effect (t) portrait (t) portrait effect (t) portrait mode (t) android smartphones (t) google pixel 2 (t) bokeh effects