From Sensor Data to Usuable Information
November 1, 2005 By: Don Holley SensorsMany companies are drowning in a sea of sensor data collected from research, design, validation, and manufacturing operations. The engineers and scientists often lack the tools they need to efficiently capture, organize, and extract information from these data for general accessibility and usefulness.
![]() |
Computer-based data acquisition, control, and automated test systems can generate thousands of parameters every second. They stream raw data out to engineering and operations personnel who must interpret the information to ensure their products meet design specifications, along with quality and regulatory requirements. (Ever try to drink water from a fire hose?) This is information overload, but too much is better than too little. Still, how are you supposed to sift through all these data and make sense of them?
The ability to convert raw data into usable and timely information enables engineers and managers to better troubleshoot design and production problems, improve processes, and enhance efficiency and productivity. Real-time access to usable information can answer a variety of questions:
- 1. What's the overall yield and first-pass yield of product A supplied by our contract manufacturer last month?
- 2. How will changing component A affect product specifications and reliability?
- 3. How do variations in reactor temperature affect batch cycle times and product quality?
- 4. What key performance indicators are critical to maximize lot-to-lot yields?
The answers are out there somewhere, and the key to finding them lies in establishing a process methodology and acquiring the tools that can capture, organize, and provide secure access to the source data for analysis, visualization, and reporting (see Figure 1). Let's take a closer look.
![]() Figure 1. Turning sensor data into usable information requires a process methodology and tools for capturing, organizing, and providing secure access to the data whenever and wherever it's needed by authorized personnel. |
Capturing Data
The first step toward collecting and storing raw data will be a powerful, scalable, flexible repository. Industry-standard databases such as Microsoft SQL Server or Oracle are proven and reliable repositories. Storage requirements are application dependent and are determined by the types of data, sampling frequency, and length of time the data must be stored. Data from sensors and other measurement systems can consist of scalar data from analog measurements such as temperature and discrete signals from ON/OFF switches, ASCII strings from RFID or analyzers, binary data from vision and inspection systems, and data arrays from event-sequence recorders. Some applications need only short-term access to the stored data, but regulated industries among others require archiving for many years.
Securely and reliably capturing large volumes of real-time data requires additional considerations. Does your application require fail-safe data transfers to keep the information safe should a network, server, or application crash? Do you need data buffering to ensure that the data stream does not outpace the receiver? Do you need to manage data compression so that you can reduce network traffic and storage space?
You can develop your own data capture setup or buy one off the shelf that satisfies your particular requirements. One class of repositories, data historians, is commonly used in process, batch, and discrete manufacturing. They can aggregate real-time data from a variety of sensors, industrial networks, distributed control systems, programmable logic controllers, and supervisory control and data acquisition (SCADA) systems using industry standards such as OPC. Data historians are commonly based on proprietary databases optimized for storing and retrieving time-series data. They vary in size from small SCADA systems monitoring 100 points/s (e.g., temperature, flow, pressure, level), to large plantwide versions that are part of distributed control systems monitoring 100,000+ data points in oil refineries and chemical plants.
1 2 3
| sensorsonline: Do you subscribe to Sensors Weekly? If not, you might have missed these articles today http://t.co/YrbicBsi #sensorsweekly | |
| sensorsonline: Do you subscribe to Sensors Weekly? If not, you might have missed these articles today http://t.co/YrbicBsi #sensorsweekly | |
|
|
|
| sensorsonline: February R&D Round Up: This month's examples of clever and useful sensor research includes smart paint to monito... http://t.co/oTsU3EBx | |
| sensorsonline: February R&D Round Up: This month's examples of clever and useful sensor research includes smart paint to monito... http://t.co/oTsU3EBx | |
|
|
|
| sensorsonline: Will Tu: http://t.co/77KHZndE | |
| sensorsonline: Will Tu: http://t.co/77KHZndE | |
|
|
|
| sensorsonline: Quake Global Receives Certification for Q4000 Modem: The Q4000 is the first fully integrated, dual-mode modem fo... http://t.co/YzLgOtjg | |
| sensorsonline: Quake Global Receives Certification for Q4000 Modem: The Q4000 is the first fully integrated, dual-mode modem fo... http://t.co/YzLgOtjg | |
|
|
|
| sensorsonline: Tiny Black Box Detects Electric Power Crashes: New tiny electronic instrument detects and records power glitches... http://t.co/Pe6sbivL | |
| sensorsonline: Tiny Black Box Detects Electric Power Crashes: New tiny electronic instrument detects and records power glitches... http://t.co/Pe6sbivL | |
|
|
|
| sensorsonline: Libelium Offers e-Learning Platform for Sensor Networks: New e-Learning platform enables system integrators worl... http://t.co/TOCQclaS | |
| sensorsonline: Libelium Offers e-Learning Platform for Sensor Networks: New e-Learning platform enables system integrators worl... http://t.co/TOCQclaS | |
|
|
|
| sensorsonline: Schneider Electric To Expand Collaboration With Cisco: Introduces new solutions for comprehensive building and d... http://t.co/GoZvqLRT | |
| sensorsonline: Schneider Electric To Expand Collaboration With Cisco: Introduces new solutions for comprehensive building and d... http://t.co/GoZvqLRT | |
|
|
|
| sensorsonline: RT @SensorsExpo: #sensors12 digital brochure is here! download it at http://t.co/LFe7twTj OR register w/code 602K for the lowest rates h ... | |
| sensorsonline: RT @SensorsExpo: #sensors12 digital brochure is here! download it at http://t.co/LFe7twTj OR register w/code 602K for the lowest rates h ... | |
|
|
|
| sensorsonline: Seen and Heard: Articles and postings about Sensors Expo & Conference in blogs, news sites and publications read more http://t.co/rZCFxDoS | |
| sensorsonline: Seen and Heard: Articles and postings about Sensors Expo & Conference in blogs, news sites and publications read more http://t.co/rZCFxDoS | |
|
|
|
| sensorsonline: Sensor and System Design Track: Sensor and System Design Track read more http://t.co/Sz29WxXv | |
| sensorsonline: Sensor and System Design Track: Sensor and System Design Track read more http://t.co/Sz29WxXv | |
|
|
|
| sensorsonline: Soitec and Sumitomo Electric Announce GaN Milestone: The strategic joint developed program has successfully demo... http://t.co/DH4MFCBG | |
| sensorsonline: Soitec and Sumitomo Electric Announce GaN Milestone: The strategic joint developed program has successfully demo... http://t.co/DH4MFCBG | |
|
|
|
| sensorsonline: Imec, Genalyte Report Disposable Silicon Biosensor Chips: The disposable silicon photonics biosensor chips will ... http://t.co/Y22O3Bwf | |
| sensorsonline: Imec, Genalyte Report Disposable Silicon Biosensor Chips: The disposable silicon photonics biosensor chips will ... http://t.co/Y22O3Bwf | |
|
|
|
| sensorsonline: Peter Himes: Peter Himes has over 25 years' experience in helping startups and public companies establish their ... http://t.co/483WVNWS | |
| sensorsonline: Peter Himes: Peter Himes has over 25 years' experience in helping startups and public companies establish their ... http://t.co/483WVNWS | |
|
|
|
| sensorsonline: RT @MDT_Sensor_News: Announcement to our new magnetic switch #sensors reposted at @sensorsonline, http://t.co/0QoriRIo | |
| sensorsonline: RT @MDT_Sensor_News: Announcement to our new magnetic switch #sensors reposted at @sensorsonline, http://t.co/0QoriRIo | |
|
|
|
| sensorsonline: Catch the latest Sensors Prod Picks featuring @Kionix @BoschSensortec @Dynasonics @InvenSense http://t.co/vGQvTtyh #sensorsmag | |
| sensorsonline: Catch the latest Sensors Prod Picks featuring @Kionix @BoschSensortec @Dynasonics @InvenSense http://t.co/vGQvTtyh #sensorsmag | |
|
|
|
| sensorsonline: RT @SensorsExpo: #sensors12 welcomes these exhibitors @KamanIndustrial @Sika @InfinitePower @Xeno @MEDERelectronic @Interplex @HBM @Sens ... | |
| sensorsonline: RT @SensorsExpo: #sensors12 welcomes these exhibitors @KamanIndustrial @Sika @InfinitePower @Xeno @MEDERelectronic @Interplex @HBM @Sens ... | |
|
|
|
| sensorsonline: You Can’t Manage What You Can’t Measure: Turning Sensor Data into ROI: read more http://t.co/dPKhTgLZ | |
| sensorsonline: You Can’t Manage What You Can’t Measure: Turning Sensor Data into ROI: read more http://t.co/dPKhTgLZ | |
|
|
|
| sensorsonline: Speaker Marketing Tool Kit: Marketing Tool Kit for speakers selected to present at the 2012 event. read more http://t.co/TST8jMpA | |
| sensorsonline: Speaker Marketing Tool Kit: Marketing Tool Kit for speakers selected to present at the 2012 event. read more http://t.co/TST8jMpA | |
|
|
|
| sensorsonline: Pondering Parking: One of the promises of wireless sensor networking is its ability to add a level of sensing an... http://t.co/s12IH4Ha | |
| sensorsonline: Pondering Parking: One of the promises of wireless sensor networking is its ability to add a level of sensing an... http://t.co/s12IH4Ha | |
|
|
|
| sensorsonline: IMUs/AHRS from YEI Technology: The 3-Space Sensor family of intelligent IMUs and AHRS from YEI Technology, Ports... http://t.co/OqklJSKs | |
| sensorsonline: IMUs/AHRS from YEI Technology: The 3-Space Sensor family of intelligent IMUs and AHRS from YEI Technology, Ports... http://t.co/OqklJSKs | |
|
|
|
| sensorsonline: Photoelectric Sensors from Pepperl+Fuchs: Pepperl+Fuchs Inc., Twinsburg, OH, has expanded its ML4.2 Series of su... http://t.co/P6fvyrKf | |
| sensorsonline: Photoelectric Sensors from Pepperl+Fuchs: Pepperl+Fuchs Inc., Twinsburg, OH, has expanded its ML4.2 Series of su... http://t.co/P6fvyrKf | |
|
|
|
| sensorsonline: Wall Shear Stress Sensors from Lenterra: RealShear sensors from Lenterra Inc, Newark, NJ, are inline sensors tha... http://t.co/i3CUF2JR | |
| sensorsonline: Wall Shear Stress Sensors from Lenterra: RealShear sensors from Lenterra Inc, Newark, NJ, are inline sensors tha... http://t.co/i3CUF2JR | |
|
|
|
| sensorsonline: 3D Metrology Software from InnovMetric: InnovMetric Software Inc., Quebec City, PQ, Canada, offers its V12.0.3 P... http://t.co/lMoTasYo | |
| sensorsonline: 3D Metrology Software from InnovMetric: InnovMetric Software Inc., Quebec City, PQ, Canada, offers its V12.0.3 P... http://t.co/lMoTasYo | |
|
|
|
| sensorsonline: RT @SensorsExpo: Registration is now live for 2012 Sensors Expo & Conf. Register w/code 602K for the lowest 2012 rates: http://t.co/q0yR ... | |
| sensorsonline: RT @SensorsExpo: Registration is now live for 2012 Sensors Expo & Conf. Register w/code 602K for the lowest 2012 rates: http://t.co/q0yR ... | |
|
|
|
| sensorsonline: January R&D Round Up: Welcome to 2012! As we start the new year, how about some intriguing research breakthroug... http://t.co/48NNA24d | |
| sensorsonline: January R&D Round Up: Welcome to 2012! As we start the new year, how about some intriguing research breakthroug... http://t.co/48NNA24d | |
|
|
|
| sensorsonline: 01-Jan-2012: 01-Jan-2012 read more http://t.co/N7p9u6Fc | |
| sensorsonline: 01-Jan-2012: 01-Jan-2012 read more http://t.co/N7p9u6Fc | |
|
|
|
| sensorsonline: Molly Bakewell Chamberlin: Molly Bakewell Chamberlin is President of Embassy Global LLC. http://t.co/23PIOzFU | |
| sensorsonline: Molly Bakewell Chamberlin: Molly Bakewell Chamberlin is President of Embassy Global LLC. http://t.co/23PIOzFU | |
|
|
|
| sensorsonline: Happy New Years! Make one of your resolutions to stay on top of industry news w/ @sensorsonline - sign up today at http://t.co/LFI8n1zU | |
| sensorsonline: Happy New Years! Make one of your resolutions to stay on top of industry news w/ @sensorsonline - sign up today at http://t.co/LFI8n1zU | |
|
|
|
| sensorsonline: What's New at Sensors...System Solutions for Industrial Sensors/Field Transmitters...and more http://t.co/n7speoxH | |
| sensorsonline: What's New at Sensors...System Solutions for Industrial Sensors/Field Transmitters...and more http://t.co/n7speoxH | |
|
|
|
| sensorsonline: Happy Holidays from the Sensors Team to Yours http://t.co/taoAX06L #happyholidays #sensorsmag #sensorsexpo #sensors12 | |
| sensorsonline: Happy Holidays from the Sensors Team to Yours http://t.co/taoAX06L #happyholidays #sensorsmag #sensorsexpo #sensors12 | |
|
|
|
| sensorsonline: December Product Picks avail now at http://t.co/09v3hVAT. See New Sensors, Components, and Data Acquisition & Control Products #sensorsmag | |
| sensorsonline: December Product Picks avail now at http://t.co/09v3hVAT. See New Sensors, Components, and Data Acquisition & Control Products #sensorsmag | |
|
|
|
| sensorsonline: RT @sherbornesensor: Great article in @sensorsonline 'MEMS: The Future Looks Bright' http://t.co/x0fjxBhz | |
| sensorsonline: RT @sherbornesensor: Great article in @sensorsonline 'MEMS: The Future Looks Bright' http://t.co/x0fjxBhz | |
|
|
|
| sensorsonline: Sensors Weekly is available here: http://t.co/cZSkJcxr. Catch the latest R&D Round Up and sensors news #sensorsnews #sensorsmag | |
| sensorsonline: Sensors Weekly is available here: http://t.co/cZSkJcxr. Catch the latest R&D Round Up and sensors news #sensorsnews #sensorsmag | |
|
|
|
| sensorsonline: triaxial accelerometers, chlorine sensors, thermal imagers & more products can be found in this week's Product Picks http://t.co/WS50rlEX | |
| sensorsonline: triaxial accelerometers, chlorine sensors, thermal imagers & more products can be found in this week's Product Picks http://t.co/WS50rlEX | |
|
|
|
| sensorsonline: Oct issue of What's New at Sensors is now live at http://t.co/z89lKgKU #potentiometers #positionsensing #sensorsmag | |
| sensorsonline: Oct issue of What's New at Sensors is now live at http://t.co/z89lKgKU #potentiometers #positionsensing #sensorsmag | |
|
|
|




