도서요약

Smartphone-Grade LiDAR Sees Objects Hidden from View

Cameras have traditionally recorded only the light directly in front of them, leaving spaces beyond walls and corners unseen. By combining consumer-grade LiDAR with computational imaging, faint traces of multiply reflected light can now be analyzed to reconstruct the positions and shapes of hidden objects in three dimensions. This development could expand autonomous driving, robotics, smartphones, and disaster-response technologies, showing how cameras are evolving from devices that merely see into systems that infer.

[Key Message]

* Cameras are evolving from devices that record visible scenes into systems that infer spaces hidden from direct view.

* Consumer-grade LiDAR can analyze faint traces of light reflected multiple times from walls and floors to reconstruct the positions and shapes of hidden objects in three dimensions.

* The central achievement of the research lies in combining motion and multiple measurements to overcome the limitations of inexpensive hardware rather than relying on costly sensors.

* Non-line-of-sight imaging could become a new safety technology that allows autonomous vehicles and robots to detect movement beyond corners and obstacles before an accident occurs.

* As technology for sensing hidden spaces enters everyday devices, standards governing privacy, data management, and the boundaries of surveillance will become increasingly important.

***

The Next Evolution of the Camera: Reconstructing Movements Beyond Walls and Corners in Three Dimensions

Cameras have long been a technology that extends human vision. They capture fleeting moments that are difficult for the human eye to grasp, magnify distant landscapes, and reveal scenes in darkness and worlds too small to see unaided. As lenses and image sensors have advanced, cameras have steadily expanded the range of what people can see. Yet even the most sophisticated cameras have faced a limitation that is difficult to overcome: they cannot see beyond a wall or obstacle blocking the space between the lens and the object.

For a conventional camera to create an image, light from an object must reach the lens. When that light is blocked by a wall or when an object is hidden around a corner, nothing appears in the camera?셲 image. A bicycle approaching from around a street corner, a child stepping into the road from behind a parked car, or a person trapped behind debris in a collapsed building can be difficult to detect until they enter the camera?셲 direct field of view.

A study published in Nature in May 2026 demonstrated the possibility of overcoming this longstanding limitation using consumer-grade LiDAR. The researchers used an inexpensive LiDAR system comparable to those that can be installed in smartphones to reconstruct objects hidden outside a camera?셲 direct line of sight in three dimensions. They restored the positions and shapes of concealed objects, tracked the movements of multiple objects, and even estimated the position of the sensor itself by using information obtained from objects that could not be seen directly.

Non-line-of-sight imaging technology had already been studied before this work. Most previous systems, however, required large and expensive laser equipment, highly sensitive detectors, and precisely controlled experimental conditions. The new study is significant because it achieved comparable functions with commercially available hardware costing less than 100 dollars. A technology that had remained confined to specialized laboratories has begun moving toward possible applications in everyday devices such as smartphones, compact robots, and automobiles.

This does not mean that a smartphone can now display a clear image of a person behind a wall or reveal the interior of a hidden room. The information that can currently be reconstructed remains limited, and numerous challenges must be solved before the technology can operate reliably in complex real-world environments. Nevertheless, the direction of change presented by the study is unmistakable. Cameras are evolving from devices that merely record light arriving from the visible scene into systems that calculate traces of faint light scattered through the surrounding environment and infer scenes that cannot be seen directly.

Spaces Cameras Could Not See

For people to see something, light reflected from that object must enter their eyes. When sunlight or artificial illumination strikes an object, the light is reflected in many directions. Some of that reflected light reaches the eyes, allowing people to recognize the object?셲 color and shape. Conventional cameras work according to the same principle. An image sensor receives light entering through the lens and converts information about brightness and color into a visible image.

The problem arises when an object is hidden behind a wall or around a corner. When light reflected from the object cannot travel directly to the lens, a conventional camera cannot record it. Regardless of how high the camera?셲 resolution may be or how much light its lens can capture, it cannot create an image of a space from which no light reaches the sensor. Competition in conventional camera technology has therefore focused largely on how clearly and accurately directly visible scenes can be recorded.

Non-line-of-sight imaging is a field that attempts to move beyond this limitation. Commonly abbreviated as NLOS imaging, it refers to technology that detects and reconstructs concealed targets when there is no direct line of sight between the sensor and the object. An NLOS system does not photograph the hidden object directly. Instead, it analyzes indirect traces of light left on visible surfaces such as walls, floors, and ceilings to estimate the position and shape of the object outside the sensor?셲 view.

Light is not reflected only by smooth surfaces such as mirrors. When light strikes a wall or floor that appears rough, it also scatters in many directions. When a LiDAR system directs light toward a visible wall, some of the light returns immediately to the sensor, but another portion may spread into the space beyond a corner. That light may strike a concealed object, return to the wall, reflect once again, and eventually reach the sensor.

Light that has undergone several reflections is extremely weak. It cannot be detected by the human eye and is easily buried beneath ambient illumination and electronic noise in an ordinary camera. Yet when the arrival time and reflection pattern of the returning light are measured precisely, the signal may still contain information about the concealed object.

The time taken for light to leave the sensor, travel to the wall, reach the hidden object, and return varies according to the total distance traveled. By repeatedly measuring this travel time at different points on the wall, researchers can estimate the direction and distance of the hidden object. When measurements obtained from multiple positions are combined, the structure of a space that cannot be seen directly can be reconstructed in reverse.

In this process, the wall serves as a kind of indirect mirror. Unlike an ordinary mirror, however, it does not produce a sharp reflection. Because the rough surface scatters light in every direction, the information received by the sensor is faint and incomplete. Reconstructing the concealed scene therefore requires complex calculations that identify weak signals, distinguish multiple reflection paths, and extract meaningful patterns from noise.

NLOS imaging is not simply a technology that extends a camera?셲 field of view by a small amount. Conventional cameras record strong and direct light, while NLOS imaging extracts spatial information from signals that have become extremely faint after multiple reflections. Its significance lies in a fundamental shift in the nature of imaging: rather than photographing a visible scene, the system calculates an invisible scene from traces left within the visible environment.

Reconstructing Objects from Reflected Light

LiDAR is a technology that uses light to measure distance. It emits laser light and measures how long the light takes to reflect from an object and return to the sensor. Light travels extremely quickly, but because its speed is constant, the distance between the sensor and the object can be calculated by measuring the round-trip travel time with sufficient precision.

Whereas a conventional camera records brightness and color, LiDAR records the distance to surrounding surfaces. It can distinguish how far nearby and distant objects are from the sensor even when they appear within the same scene. Because of this capability, LiDAR has been used for environmental perception in autonomous vehicles, obstacle avoidance in robots, three-dimensional surveying of buildings and terrain, and spatial scanning for augmented reality.

LiDAR installed in smartphones operates according to the same fundamental principle. The device emits infrared light that is invisible to the human eye and analyzes the returning signal to obtain distance information about the surrounding space. This information allows a camera to focus quickly even in dark locations, measure the dimensions of a room, and record furniture or indoor spaces in three dimensions. In augmented reality, LiDAR also helps place virtual objects naturally on real floors and tables.

Existing consumer-grade LiDAR systems, however, have generally been used to measure surfaces directly facing the sensor. When pointed at a wall, the sensor can determine the distance to that wall, but it cannot ordinarily reveal what lies behind the wall or beyond a corner. The new research focused on the time-resolved reflection signals recorded by consumer-grade LiDAR. Instead of examining only the strong light that returns directly from the wall, the researchers also analyzed weaker light that arrives slightly later after passing through several reflections.

Light returning from a concealed object is much weaker than direct reflections. As it travels from the sensor to the wall, from the wall to the hidden object, from the object back to the wall, and finally from the wall to the sensor, it is repeatedly scattered and partly absorbed by surfaces. The signal received by the sensor contains a mixture of direct wall reflections, ambient light, electronic noise, and information returning from multiple objects. Transforming this faint and complicated signal into the three-dimensional form of a concealed object is the most difficult challenge in NLOS imaging.

The researchers did not attempt to obtain all the necessary information from a single measurement. Instead, they collected measurements recorded over multiple moments and combined them to compensate for weak signals. The process resembles the way a smartphone camera captures several photographs in rapid succession in a dark environment and merges them into one brighter and clearer image. Information that is buried beneath noise in a single measurement may become distinguishable when multiple measurements reveal which signals appear repeatedly and which arise only by chance.

The movements of the sensor and the concealed objects were also used actively. In conventional photography, movement is usually treated as a source of blur and distortion that must be eliminated. In this research, however, changes in the measured signal caused by movement were transformed into a new source of information. When the sensor moves or the concealed object changes position, the system effectively observes the same space from slightly different perspectives.

Combining incomplete signals collected across multiple positions and moments can produce an effect similar to measuring the scene with a sensor much larger than the physical device. A small sensor gathers information as it moves, and the system combines the measurements as though they had been collected by a larger sensing surface. This approach enabled a consumer-grade LiDAR system with limited power and resolution to reconstruct the positions and shapes of concealed objects.

An important shift emerges at this point. Movement is no longer merely an error that must be removed. With an appropriate computational model, movement can provide new viewpoints that the sensor could not otherwise obtain. Rather than relying on expensive hardware to collect large quantities of information all at once, inexpensive sensors gather fragments of information while moving, and software combines those fragments. The central achievement of the study lies in using computation and movement to compensate for the limitations of the hardware.

From Laboratory Equipment to Smartphone-Grade Sensors

Research on NLOS imaging has been underway for years. Scientists have used ultrafast lasers, detectors capable of sensing light at the level of individual photons, and highly precise timing equipment to reconstruct objects concealed behind walls and corners. Numerous experiments have confirmed that the positions and shapes of hidden targets can be estimated when the travel time of light is measured with extreme precision.

Conventional research systems, however, have been large and expensive. They required powerful lasers, highly sensitive detectors, and precise optical equipment, while the positions of the sensor and the reflecting wall had to be adjusted carefully. Although relatively high-quality results could be obtained in controlled laboratory settings, the same equipment was difficult to install in cars, robots, or portable devices. Complicated calibration procedures and long measurement times also prevented everyday use.

Consumer-grade LiDAR is designed under entirely different conditions. Because it must fit inside a smartphone or another compact device, it needs to be small, inexpensive, energy-efficient, and thermally manageable. It must operate without complicated setup, and the power of its emitted light must be restricted to ensure safety for human eyes and skin. With strict limitations on size and cost, its spatial resolution and sensitivity are inevitably lower than those of laboratory equipment.

These characteristics place consumer-grade LiDAR at a disadvantage for NLOS imaging. Light returning from hidden objects is already extremely weak, and a consumer sensor may struggle to detect it clearly. Low spatial resolution makes it difficult to distinguish detailed points across a wall, while movement by the sensor or the object can cause signals to overlap. Using conventional methods alone, it has been difficult to reconstruct stable three-dimensional images from measurements collected by smartphone-grade LiDAR.

The significance of the new research does not lie in replacing the sensor with a more powerful one. It lies in reinterpreting the information that can be obtained from a limited sensor. By combining multiple measurements and incorporating the movements of both the sensor and the objects into the computational model, the researchers compensated for low laser power and insufficient spatial resolution. Rather than simply shrinking a laboratory system, they developed a reconstruction method suited to the characteristics of inexpensive sensors.

Without elaborate equipment installation or specialized calibration, the researchers used smartphone-grade LiDAR to achieve three-dimensional reconstruction of hidden objects and to track both single and multiple targets. They also demonstrated a function that estimated the sensor?셲 position using signals obtained from concealed objects. Achieving capabilities previously limited to laboratories with commercially available hardware costing less than 100 dollars significantly improves the accessibility of NLOS imaging technology.

The development of digital cameras followed a similar path. In the early stages, improving the physical performance of image sensors was the central route to better image quality. Later, computational technologies such as high dynamic range photography, night mode, artificial background blur, and image stabilization began to determine camera performance. A small smartphone camera can now produce high-quality photographs despite its limited lens and sensor because it does not complete an image through a single exposure.

A smartphone analyzes and combines information collected over multiple moments to produce the final image seen by the user. NLOS imaging is an extension of this broader movement toward computational imaging. It infers scenes that the sensor cannot obtain directly from indirect signals left in the surrounding environment. Rather than waiting until consumer hardware becomes powerful enough, the technology expands functionality by combining sensors that are already widely available with new computational methods.

This suggests that the competitiveness of future cameras will not be determined solely by lens specifications or image sensor performance. Even when the hardware remains unchanged, entirely different capabilities may emerge depending on which signals are collected, how multiple measurements are combined, and what meaning is extracted from noise. The camera is changing from a simple recording device into a system that calculates and interprets the surrounding space.

Three-Dimensional Reconstruction and Tracking of Hidden Objects

Detecting the existence of a concealed object and reconstructing its form in three dimensions are different challenges. Basic detection only needs to determine that something is moving around a corner. Three-dimensional reconstruction, however, requires estimating the direction of the object, its distance from the sensor, and its approximate size and shape.

The researchers combined multiple measurements recorded by consumer-grade LiDAR to reconstruct the three-dimensional structures of concealed objects. The results do not resemble ordinary camera images with clear colors and fine surface details. Instead, they represent the space occupied by the object, its approximate form, and its distance from the sensor as three-dimensional information. At the present stage, the technology is better understood as a form of spatial sensing that determines what kind of structure exists in an invisible area, rather than as a system that identifies the exact nature of an object in fine detail.

Tracking moving objects has a different kind of significance. For autonomous vehicles or robots, the precise appearance of a hidden object may be less important than its position and direction of movement. If a system can determine in advance which direction a person beyond a corner is moving, it may help prevent a collision and adjust speed even without obtaining a complete image.

The research also presented the ability to distinguish and track the movements of multiple concealed objects rather than just one. This is important because signals reflected from several targets arrive mixed together at a single sensor. The system estimated the respective positions and movements of those targets even under such conditions. Real roads and indoor spaces often contain people, vehicles, robots, and other moving objects at the same time.

The ability to estimate the sensor?셲 location using concealed objects is also noteworthy. Conventional positioning technologies calculate their own location by using directly visible features such as walls, doors, furniture, and road signs. When the environment is dark or lacks distinctive visible features, positioning can become unstable. Adding NLOS signals allows structures outside the direct field of view to serve as additional clues for determining the sensor?셲 location.

This capability could transform the way robots perceive space. Today, robots generally create maps and plan paths based on what their sensors can see directly. In the future, they may be able to choose routes while making limited estimates of structures and movements beyond corners. A robot traveling down a hallway might detect a person approaching from the opposite side before reaching an intersection, or a warehouse robot might identify the position of another robot moving behind a shelf.

However, results achieved in controlled environments cannot automatically be assumed to work in the same way in complex real-world settings. The material and color of a wall, the roughness of its surface, surrounding illumination, and the reflective properties of concealed objects all strongly affect measurement results. Black objects or materials that absorb light produce weaker signals, while glass and metal surfaces that reflect light strongly in particular directions can generate complex measurements.

In a space containing multiple walls and objects, light may travel through far more paths than expected before returning to the sensor. It becomes difficult to determine which object produced a particular signal, and an object may be reconstructed in the wrong location. Outdoors, strong sunlight can interfere with infrared signals, while rain, fog, and dust can also affect the movement and reflection of light.

Measurement range and processing speed are additional challenges. Autonomous vehicles must interpret rapidly changing road conditions in real time. If the system must collect a large number of measurements and perform lengthy calculations afterward, it will be difficult to use the result to avoid immediate danger. Smartphones and compact robots must also take battery consumption, computational load, and sensor heat into account.

Practical implementation will therefore require computational methods that produce fast and stable results from fewer measurements. The technology must maintain performance across different walls, floors, and lighting conditions, and it must reduce false detections in complex situations where multiple objects move simultaneously. Technologies that accurately determine the sensor?셲 own position and orientation while it is moving must also advance alongside NLOS reconstruction.

The present achievement should not be understood as a system that perfectly photographs an invisible space. It is better understood as evidence that meaningful information about such a space can be extracted even with consumer-grade sensors. Rather than expecting a clear photographic image, it is more accurate to view the technology as a new means of obtaining safety and spatial information that conventional cameras cannot provide.

The Next Evolution of the Camera Begins

Autonomous driving is considered one of the first areas in which NLOS imaging could be applied. Autonomous vehicles are equipped with several types of sensors, including cameras, radar, and LiDAR. Yet targets concealed by buildings, walls, or parked vehicles are difficult to detect until they enter the direct field of view. Pedestrians and bicycles that suddenly appear from alleys or intersections pose a major danger not only to human drivers but also to autonomous systems.

If a vehicle could analyze indirect light returning from buildings or the road surface and detect movement around a corner in advance, it would gain more time to respond. Even without reconstructing the detailed appearance of a concealed person, knowing the position and direction of an approaching object could allow the vehicle to slow down or stop. NLOS imaging is therefore more likely to complement existing cameras, radar, and direct-view LiDAR than to replace them, filling the blind spots those sensors cannot cover.

The range of applications is also broad in robotics. Mobile robots in logistics warehouses frequently lose sight of their surroundings because of shelves and boxes. Service robots in hospitals and hotels may unexpectedly encounter people at narrow hallway intersections. Household robots may fail to detect a pet or child moving behind furniture. If robots can sense motion in invisible areas in advance, they can adjust their speed and select safer routes.

The technology may have even more direct value in disaster response. Smoke and dust at fire scenes obstruct cameras, while debris from collapsed buildings makes it difficult for rescue workers to inspect interior spaces. If small robots or drones are equipped with NLOS sensing capabilities, they may be able to use indirect light reflected from walls and debris to search for human movement or identify the structure of concealed spaces.

It would be premature to claim that the current technology can accurately locate people buried deep beneath debris. It could, however, develop into an auxiliary tool that helps rescue workers identify traces of movement or the existence of empty spaces before entering a hazardous area. When combined with thermal sensors, acoustic systems, and radar, it could provide additional information for evaluating rescue possibilities.

Smartphone functions could also change. Smartphone LiDAR has so far been used mainly for spatial measurement, augmented reality, and camera assistance. If NLOS detection becomes possible, it could expand into pedestrian safety functions that alert users to approaching objects before they turn a corner, systems that detect movement in dark interiors, and technologies that determine location more precisely inside complicated buildings.

There is also potential for assistive technology for people with visual impairments. Existing mobility aids use cameras and ultrasonic sensors to detect obstacles ahead. If information about people or moving objects approaching from beyond a corner were added, users could become aware of surrounding conditions slightly earlier. For such systems to function as reliable assistive devices, however, false alarms and missed detections must be reduced substantially, and only essential information should be delivered clearly through vibration or audio.

In augmented reality, the scope of virtual space could expand. Current augmented reality devices create maps mainly from areas directly visible to their sensors. If NLOS information is added, a device may be able to estimate part of the surrounding space before the user has looked around the entire room or adjust virtual content according to objects moving in concealed areas.

If smart glasses and wearable devices become capable of understanding invisible movement in the surrounding environment, even to a limited extent, the relationship between physical and digital information could change. Such devices could move beyond simply overlaying information onto visible scenes and develop into sensory aids that provide advance warnings about hidden dangers or changes in nearby spaces.

Technological progress, however, brings privacy concerns. The ability to detect movement behind a wall or around a corner can support safety and rescue operations, but it also creates the possibility of monitoring people?셲 presence and behavior without their consent. Even when reconstructed images are not visually detailed, information about whether people are present in a particular space, how many people are moving, and in which direction they are traveling can be highly sensitive.

If NLOS sensing is incorporated into everyday devices such as smartphones and robots, standards governing operating range, data storage, user notification, and access rights will become as important as technical performance. Designers will need to determine under what circumstances the sensor may be activated, whether raw signals and reconstructed results should remain on the device or be transmitted to an external server, and how the system can be prevented from indiscriminately sensing private spaces belonging to others.

The history of the camera has been shaped by the pursuit of clearer images. Pixel counts increased, lenses became brighter, and cameras learned to record color and form even in darkness. Future change may move beyond a simple competition over image quality. Cameras may begin analyzing scattered light in the surrounding environment to calculate the structures and movements of spaces that cannot be seen directly.

The new research using consumer-grade LiDAR has not produced a completed camera capable of seeing through walls. There are clear limitations in reconstruction quality, measurement range, processing speed, and environmental adaptability. Sensors, computational methods, and safety standards must all advance before the technology can be applied to real vehicles, smartphones, and rescue equipment.

Even so, the direction of change is clear. In the past, saying that something could not be seen often implied that no usable information was available. Now, even when an object cannot be viewed directly, the timing and patterns of light reaching walls and floors can be analyzed to make limited inferences about what lies beyond. As a technology once confined to expensive research equipment moves to smartphone-grade sensors, NLOS detection is beginning to shift from a specialized experiment into a computational function that could be added to a wide range of devices.

The camera of the future may not remain merely an eye that records a scene as it appears. It could develop into an intelligent sensory system that gathers faint signals left in blind spots, combines information from multiple moments, and estimates what is moving within spaces that cannot be seen.

The possibilities demonstrated by smartphone-grade LiDAR suggest that cameras can move beyond simply seeing farther and recording more clearly. The next generation of cameras may evolve into technologies that not only photograph the world visible before them but also read the traces of hidden spaces and understand the surrounding environment more broadly and rapidly.

Reference

Siddharth Somasundaram, Aaron Young, and Ramesh Raskar, Nature, May 2026, ?쏧maging Hidden Objects with Consumer LiDAR via Motion-Induced Sampling??/div>


국내도서 요약

국내도서 프리뷰		해외도서 프리뷰

글로벌 트렌드		미디어 브리핑스