From Sensor Data to Usuable InformationNovember 1, 2005 By: Don Holley Sensors
Many companies are drowning in a sea of sensor data collected from research, design, validation, and manufacturing operations. The engineers and scientists often lack the tools they need to efficiently capture, organize, and extract information from these data for general accessibility and usefulness.
Computer-based data acquisition, control, and automated test systems can generate thousands of parameters every second. They stream raw data out to engineering and operations personnel who must interpret the information to ensure their products meet design specifications, along with quality and regulatory requirements. (Ever try to drink water from a fire hose?) This is information overload, but too much is better than too little. Still, how are you supposed to sift through all these data and make sense of them?
The ability to convert raw data into usable and timely information enables engineers and managers to better troubleshoot design and production problems, improve processes, and enhance efficiency and productivity. Real-time access to usable information can answer a variety of questions:
- 1. What's the overall yield and first-pass yield of product A supplied by our contract manufacturer last month?
- 2. How will changing component A affect product specifications and reliability?
- 3. How do variations in reactor temperature affect batch cycle times and product quality?
- 4. What key performance indicators are critical to maximize lot-to-lot yields?
The answers are out there somewhere, and the key to finding them lies in establishing a process methodology and acquiring the tools that can capture, organize, and provide secure access to the source data for analysis, visualization, and reporting (see Figure 1). Let's take a closer look.
Figure 1. Turning sensor data into usable information requires a process methodology and tools for capturing, organizing, and providing secure access to the data whenever and wherever it's needed by authorized personnel.
The first step toward collecting and storing raw data will be a powerful, scalable, flexible repository. Industry-standard databases such as Microsoft SQL Server or Oracle are proven and reliable repositories. Storage requirements are application dependent and are determined by the types of data, sampling frequency, and length of time the data must be stored. Data from sensors and other measurement systems can consist of scalar data from analog measurements such as temperature and discrete signals from ON/OFF switches, ASCII strings from RFID or analyzers, binary data from vision and inspection systems, and data arrays from event-sequence recorders. Some applications need only short-term access to the stored data, but regulated industries among others require archiving for many years.
Securely and reliably capturing large volumes of real-time data requires additional considerations. Does your application require fail-safe data transfers to keep the information safe should a network, server, or application crash? Do you need data buffering to ensure that the data stream does not outpace the receiver? Do you need to manage data compression so that you can reduce network traffic and storage space?
You can develop your own data capture setup or buy one off the shelf that satisfies your particular requirements. One class of repositories, data historians, is commonly used in process, batch, and discrete manufacturing. They can aggregate real-time data from a variety of sensors, industrial networks, distributed control systems, programmable logic controllers, and supervisory control and data acquisition (SCADA) systems using industry standards such as OPC. Data historians are commonly based on proprietary databases optimized for storing and retrieving time-series data. They vary in size from small SCADA systems monitoring 100 points/s (e.g., temperature, flow, pressure, level), to large plantwide versions that are part of distributed control systems monitoring 100,000+ data points in oil refineries and chemical plants.
Most Read Articles