UGM2017 / 28. Real-time data management in iRODS - RENCI скачать в хорошем качестве

UGM2017 / 28. Real-time data management in iRODS - RENCI

Arcot Rajasekar - RENCI The iRODS middleware provides federation, virtualization, metadata integration and policy-oriented data management for static files. Real-time data streams (RTDS) from sensors and other sources pose a different challenge compared to static files. They are infinite in length and time line, consist of discrete packets of information which are time-specific, and the concept of byte-oriented i/o is not at all suited for accessing and managing real-time data streams. Moreover, because of the nature of the sources, the volume and velocity of the flow can be very low (temperature data) to very high (HDTV). They are also time-sensitive in two ways. One has to capture the data right away or it will be lost forever; and in many cases, the analysis has to be done immediately as the decision making can be time sensitive (eg. earthquake detection). We have developed new features in the iRODS system to capture, store and archive RTDS. Our model captures RTDS into a continuum of discrete files and archives them in a few different standardized formats. It also provides packet-based and time-oriented access for replay of sensor data. By folding in the management of RTDS into iRODS we have extended the four main functionalities - federation, virtualization, metadata integration and policy-oriented data management – for real-time data. June 15, 2017 - Utrecht, Netherlands https://irods.org/ugm2017