Image Sensors for a Smart EnvironmentFebruary 20, 2015 By: Piet De Moor, imec
The Internet of Things is a hot topic in the world of research and development. Everyone seems to have their own vision of the smart environment of the future where all objects have sensors and communicate with each other. Once such vision can be seen in a short YouTube video.
But what is certain, is that the IoT is not science fiction. Today, the building blocks are ready, in the form of compact and extremely energy-efficient sensor modules that send their data to each other or to the cloud.
Call it a professional deformation, but when I think of the IoT, I see image sensors and cameras that can be integrated everywhere and that make the world a lot smarter. Just think what cameras could do for your car. Today, cars may already use some cameras, for example, to assist you with parking. But imagine what else they could do.
Cameras pointed at the environment of the car, detecting all objects around you, their distance, and the speed with which they are coming at you. Cameras also inside the car to identify who is inside and adapt the car accordingly, like choice of music, temperature, position of safety belt, etc.
The ultimate car with sensors and cameras will be self-driving. When I could choose between driving a car to the south of France myself or leisurely being driven in my smart car, while reading a magazine behind the steering wheel, I know what to choose.
Compact and smarter cameras will also help us to manage traffic better. With cameras built into the traffic infrastructure and on every crossroad, we'll know exactly how many cars, bikes, and pedestrians are on the road at each moment and location. Send this data to a server for intelligent analysis, and it is possible to direct the traffic streams so that the traffic is flowing much more fluently than is the case now. The cameras we have today are not intelligent enough to do that—they are not connected to each other. And they are too few to make a real difference.
Also for our safety, compact and smart cameras will make a world of difference. Today already, cities use cameras to monitor safety risk zones. But there's much room for improvement. As cameras get smaller and cheaper, they can be integrated in more places. And as they get smarter, they can more easily detect real problems.
For me, a camera is smart if it can detect objects. Is it an animal or a person walking by? Is it someone living here or a passerby? Is this a normal action, or is something suspicious happening? It makes no sense to continuously stream all images to the cloud. The camera should be able to do some local processing and send only the relevant images. If not, we'll inevitably get issues with the quantity of data that has to be transmitted. So, also smart processing and the ability to only send relevant images is essential for a smart camera.
Also in our homes, we'll integrate more cameras, invisible in the walls or lighting armatures. That way, you can, for instance, check if your pets still have enough food while you are away for the weekend. And there will be cameras in your domestic appliances. A washing machine that identifies clothing colors, an oven that knows it has to warm lasagna or soup, or a dishwasher that recognizes if you load dirty plates or fingerprinted wineglasses. All of this will become possible thanks to compact, smart, and inexpensive image sensors and cameras.
Some of the previous may seem like luxury fads. But what do we think of a camera that allows elderly people to remain living longer in their own house? A local nurse, family member or emergency unit may be alerted when they remain in bed, have fallen, or when there isn't much movement in the house. Of course privacy is an issue, but when this technology helps elderly people stay at home, the choice is easily made.
The building blocks for all of these applications are ready—with room for improvement. I strongly believe in CMOS sensors not only because they are cheap, but more so because of their endless integration possibilities. It's fairly easy to integrate an image sensor and a processor into a compact system. Especially if you'd be able to use 3D technology so that you can process both components in a different technology, e.g., the image sensor in an established imager technology and the processor in today's most advanced CMOS technology. Both are then stacked and connected with extremely high data rates.
Also the readout of the pixels can still be improved. Now all pixels are read out (full frame transfer), but what if we let ourselves be inspired by the human eye? In comes asynchronous readout: only the pixels where the signal changes are read out. That way, you could make images using much less energy.
So there is still a lot to do, but we expect a bright future for image sensors. There are plenty of applications out there that will make our lives more comfortable and safe!
About the Author
Piet De Moor is imec's Program Manager, Space Technologies. Before that, he managed the Optical Sensors program. Piet has a PhD in Physics from the KU Leuven and started working for imec in 1998. First, he was involved with the research into innovative MEMS devices and MEMS packaging. Then he led a team working on 3D integration and advanced CMOS image sensors.
Most Read Articles