Recently, the U.S. Patent and Trademark Office officially granted Apple a patent related to future smart glasses. More specifically, the patent relates to the use of a new deformation sensor to ensure that the AR visuals presented to the user remain properly aligned in the event of a drop, a drastic change in the weather, and more.
In its patent background, Apple states that the case of an electronic device such as a headset is configured to be worn on the user's head. During the use of the headset, the components in the device may be misaligned due to the deformation of the case. For example, display components may be misaligned, which can adversely affect the user's ability to view the image.
Apple's patent, which has never been published before, covers a headset that may have a head-mounted support structure that can be worn on the user's head. The headset support structure can include bridges and temples that form a pair of glasses, or it can include enclosure walls and other housing structures that form the structure of goggles or other headsets.
Headsets can have stereo optics, such as left and right cameras or left and right display systems. Optics can have their own left-right directional vectors. For example, a camera might have a pointing vector that is associated with the direction in which the camera captures an image, while a display system might have a pointing vector that is associated with the direction in which the display system emits an image.
The left camera can capture an image along the left camera-pointing vector, and the right camera can capture the image along the right-camera-pointing vector. The left display system can have an output coupler or other display component that provides a left image to the left eye frame along the left display system pointing vector. The right display system can have output couplers or other display components that provide the right image to the right eyeframe along the right system pointing vector.
Deformation of the support structure due to the forces generated by mounting the device on the user's head, damage from drop events, thermal fluctuations, or other events may result in misalignment of the camera pointing vector or display system pointing vector. Sensor circuits, such as strain gauge circuits, can measure pointing vector deviation. The control circuitry in the device can use sensor measurements to compensate for changes in the direction of the pointing vectors relative to each other.
For example, the control circuit can process image data captured with the left and right cameras to compensate for changes in the direction of the left and right camera pointing vectors relative to each other, resulting in a satisfactory stereo captured image. To compensate for changes in the direction of the pointing vectors of the left and right display systems relative to each other, the control circuitry can adjust the image produced by the left and right display systems (for example, adjusting keying, cropping images, or adjusting other image attributes). In this way, the left and right images will blend correctly in the user's field of view and will not be distorted by the misalignment of the pointing vector.
In the example configuration, the headset support structure is configured so that the images of the left and right eye frames are satisfactorily aligned with each other even when the side parts of the headset support are partially deformed.
Figure 1 of Apple's patent shows a pair of smart glasses. Display System 14 can be opaque or transparent. During operation, the display system can be used to display images for the user. For example, images can include computer-generated images that contain text, computer-generated objects, and other virtual image content. If desired, device 10 (smart glasses) can be equipped with a front-facing camera, such as cameras 30L and 30R.
It may be necessary to configure the housing (12) so that the 12b section (e.g., the bridge in a pair of glasses) does not deform excessively. The distortion of Section 12b may result in a misalignment between the left and right pointing vectors of the camera and the monitor. Excessive misalignment of the displayed image can lead to user discomfort and poor stereoscopic image quality (e.g., the user cannot easily blend the left and right images into a satisfactory fused stereo image). Misalignment of the cameras 30L and 30R can adversely affect stereo image capture operations.
Smart glasses may deform in the following situations: in the event of an accidental drop; exposure to large temperature fluctuations, resulting in varying degrees of expansion or contact between different materials in the enclosure; The structure of the glasses expands and contracts due to aging; The user applies pressure to the glasses during use; or in other cases where pressure is applied to the enclosure structure.
One way to solve the problem of misalignment is to create a strong (e.g., rigid and inflexible) mechanical structure in the frame. However, the strength of glasses is limited without adversely affecting the weight, size, and comfort of the glasses.
In order to avoid creating an overly heavy and bulky support structure, sensors can be installed on the glasses to monitor deformation in the support structure. These sensors can be electrical, capacitive, resistive, magnetic, optical, acoustic, and other.
Figure 2 of the Apple patent is a rear view of glasses with a set of three strain gauges 40 used to measure optical pointing vector misalignment. Strain gauges can include sinuous sensor wires whose resistance varies with the amount of bending of the strain gauge.
Figure 4 of Apple's patent is an illustrative operation flow chart related to operating glasses. During the operation of block 46, the control circuit 20 can use sensor 16 to characterize the amount of deformation present in the spectacle housing.
As shown in Figure 5 above, is a top view of a portion of the illustrative headset, which may be deformed.