The data analysis Subsystem adopts general data collecting service, supports various form for data collection. The storage of off-line big data and the storage of relational database are supported. The platform offers general-purpose computing in real time. It provides external functional service in the form of micro-service. By the standard of Restful interface, the platform provides unified comprehensive operation interface, and supports business functional operation in all kinds of industries and scenarios.
Figure Interface Diagram of Intelligent Analysis Subsystem
Figure Architecture of Intelligent Analysis Subsystem
The main functions of the system are as follows:
1)Adaptive Collecting
Data collection service is available, including data file collection, relational database collection, and streaming data collection. The system supports inputting of different data sources, including variousfile reading, relational database reading, video-voice streaming media, wireless spectrum collection information and sensor acquisition information. It can pull media streaming from video and audio collection equipment for video and audio recording, storage and dispatch. Files and data in other HDFS file systems can be collected and imported into the local HDFS file system.
2) Data Storage
It mainly provides two permanent storage methods, HDFS file system and Hadoop database. In real-time computing process, Spark and Redis temporary fast storage methods are provided as needed.
For structured data, it can be written in the HBase database, such as offline frequency spectrum monitoring data and network operation data.
For structured data, it can be written in the HBase database, such as offline frequency spectrum monitoring data and network maintenance collection data.
3) Temporary Rapid Storage
In the real-time computing, Spark and Redis temporary fast storage methods are provided as needed. The former is in-memory database, which can quickly in-memory retrieve and index for database. The latter is a semi-permanent in-memory database, which can flexibly definite multiple data structure and catch data to HDFS file system. These two kinds of rapid storage are best suitable for different computing scenarios, and have high real-time read/write performance. All real-time and quasi-real-time applications with small amounts of data in the integrated platform should be deployed in this way.
4) Data Computing
It provides various computing and analysis functions, including offline big data computing, online quasi-real-time computing, and online real-time stream computing. In addition, the audio and video, various message data and network operation data in the emergency network are automatically calculated and analyzed.
5) Offline Big Data Computing
Structured massive data can be studied and computed on a large scale in Hadoop cluster. Based on the open source Mahout algorithm library, a number of standard algorithm flow are developed. It can be used to achieve the basic analytical capability, such as similarity analysis, principal component analysis (PCA), neural network learning, SVM learning, prediction algorithm, and classified algorithm process.
6) Intelligence Analysis System of AMP
Main Functions:
a)Completing the storage and management of multiple original reconnaissance data transmitted by 2 or more AMPs
b)Displaying in real time variety loading reconnaissance information transmitted by AMPs
c)Providing platform of interpreting, measuring and labelling to intelligent interpreter, and store and manage the extracted intelligence information
d)Producing intelligent products
e)Supporting online intelligent interpreting in real time as well as after the event
f)Supporting loading data interpreting such as EO video, CCD camera
Composition:
a)Intelligent processing clients
b)Data processing server
c)A set of intelligent processing software (including intelligent analysis software, data displaying software, integrated planning and situation displaying software, data citing software and intelligent data management software)