The information is prepared on the basis of data from the information-analytical system RSF, informative part is represented in the author's edition. All rights belong to the authors, the use or reprinting of materials is permitted only with the prior consent of the authors.

Project titleIntelligent automated complex for detecting and classifying in real time diagnostic indicators of long-term synchronous video-EEG monitoring of delayed cerebral ischemia after subarachnoid hemorrhage

AffiliationKotelnikov Institute of Radioengineering and Electronics of Russian Academy of Sciences,

Research area 09 - ENGINEERING SCIENCES, 09-609 - Automated complexes for biology and medicine

Keywordsautomated complex, real time, subarachnoid hemorrhage, delayed ischemia, indicators, detection, classification, prognosis, clinical trials

Annotation
Globally, stroke remains the second leading cause of death. Ischemic stroke accounted for 62.4% of all strokes, intracerebral hemorrhage accounted for 27%, and subarachnoid hemorrhage accounted for 9.7%. Delayed cerebral ischemia is one of the terrible consequences of hemorrhage in the subarachnoid space of the brain, it usually occurs 4-14 days after its onset. Early and operative diagnostics of cerebral ischemia at the very beginning of its development is the most important stage of intensive treatment of the patient after subarachnoid hemorrhage (SAH) since there are effective therapeutic and endovascular effects that allow you to stop and reverse this process. The scientific significance of the problem of the project lies in the research, development and testing in clinical conditions of the Department of Emergency Neurosurgery and the Department of Resuscitation and Intensive Care for Neurosurgical Patients of the N.V. Sklifosovsky Research Institute of Emergency Medicine experimental intelligent hardware and software complex of long-term continuous synchronous video-EEG monitoring of patients after hemorrhage in the subarachnoid space of the brain. This complex will include programs for automatic detection, classification of diagnostic indicators and prognosis of the development of delayed ischemia in real time of their manifestation, as well as detection of temporary fragments instrumental artifacts, artifacts of the patient's vital activity, and artifacts that arise when the patient is served by medical personnel. The problem of automated detection and classification of diagnostic and prognostic indicators of delayed cerebral ischemia after SAH in real time of their manifestation, including sporadic epileptiform discharges, lateralized rhythmic delta activity, lateralized periodic discharges or generalized periodic discharges is currently not resolved. Diagnostic patterns are supposed to be distinguished in the features of the time-frequency distribution of peaks and ridges of window Fourier spectra and wavelet spectra of EEG. The expediency of using peaks and ridges of spectra is due to their mathematical properties - in them the power spectral density and phase are equal to the amplitude and phase of the analytical signal of the EEG and only in them the derivative phase in time is equal to the frequency. This makes it easy to calculate the power spectral density, frequency, and phase of the EEG, analyze their dynamics in different channels and calculate the interchannel synchronization of diagnostic indicators. For automatic segmentation of video EEG fragments with artefacts, algorithms for their detection in video synchronous with EEG will be developed using methods for rapid calculation of optical flow and contour maps of the area of interest, as well object-oriented logic programming. The interdisciplinarity of research and development of the project is to combine knowledge, experience, and the ability to solve medical problems of neurophysiology and treatment in the intensive care unit of specialists in the intelligent processing and analysis of signals and images and specialists in the development and production of medical equipment for long-term video-EEG monitoring. The synergy of interdisciplinary research lies in the comprehensive development of a medical technique for long-term video-EEG monitoring of patients with subarachnoid hemorrhage, software and algorithmic support for the detection and classification of diagnostic and prognostic indicators of delayed ischemia in real time, the creation and testing in clinical conditions of an intelligent automated complex of long-term video-EEG monitoring patients.

Expected results
1. Results of complex medical studies and monitoring of video-EEG in the Research Institute of Emergency Care named after N.V. Sklifosovsky of at least 50 patients with delayed cerebral ischemia after subarachnoid hemorrhage. Educational samples of diagnostic and prognostic indicators of delayed ischemia in video-EEG monitoring data. 2. The space of features and diagnostic and prognostic indicators of synchronous video-EEG monitoring of patients with delayed cerebral ischemia because of hemorrhage in the subarachnoid space of the brain: amplitude, frequency, interchannel connectivity of EEG and dynamics of video objects. Methods and algorithms for extracting diagnostic features of delayed ischemia in long-term video-EEG monitoring data. 3. Methods, algorithms, and programs for detecting diagnostic and prognostic indicators in real time of their manifestation in the data of synchronous video-EEG monitoring of patients with delayed cerebral ischemia after subarachnoid hemorrhage in the presence of device artifacts and artifacts of patients' vital activity. 4. Methods, algorithms and classification programs for diagnostic and prognostic EEG and video indicators in real time of their manifestation in synchronous video-EEG monitoring data of patients with delayed cerebral ischemia after subarachnoid hemorrhage with an assessment of accuracy, sensitivity and specificity. 5. Experimental automatized software and hardware complex. The software of the complex allows to interaction with external programs using the streaming mechanism of the unified collection of measured data (Lab Stream Layer) and provide the ability to use programs not only in C ++, but also on the MATLAB software package and provide the ability to use applications for detecting and classifying events based on simultaneous analysis of synchronous video EEG monitoring data. Applications should be either directly embedded in the software of the complex or run from an external application in accordance with the agreed protocol of interaction. 6. Results of tests in clinical conditions of the Research Institute of Emergency Care named after N.V. Sklifosovsky experimental intelligent software and hardware complex of real time long-synchronous video-EEG monitoring of patients with delayed cerebral ischemia after subarachnoid hemorrhage, including detection diagnostic indicators and their classification in real time of their manifestation. 7. Methodological materials on the use of an intelligent automated software and hardware complex for long-term real-time video-EEG monitoring in the diagnosis of delayed ischemia after subarachnoid hemorrhage. Result No. 1 will be received by doctors - project executors from the Research Institute of Emergency Care named after N.V. Sklifosovsky. Results No. 2, No. 3 and No. 4 will be received by performers - specialists in the field of information technology from the Kotel'nikov Institute of Radioengineering and Electronics of the Russian Academy of Sciences and the Institute of Cybernetics and Educational Informatics of the Federal Research Center "Informatics and Control" of the Russian Academy of Sciences. The result No. 5 will be obtained by the partner organization - LLC research, production, and design firm "Medicom MTD" in Taganrog, Rostov region. Results No. 6 and No. 7 are general. Even though video-EEG is the "gold standard" of neurophysiological diagnostics, currently there are no methods and algorithms for joint processing and analysis of EEG and video, video is used only for visual analysis. The scientific significance of the expected results lies in the fact that for the first time it is planned to develop and test in clinical conditions an experimental intelligent software and hardware complex for processing and analyzing data of long-term synchronous video-EEG monitoring in real time, detection, and classification of diagnostic indicators in the process of their manifestation. and supporting the acceptance of diagnostic medical decisions in the study of patients with epilepsy, the development of epilepsy after traumatic brain injuries, the development of delayed cerebral ischemia, which has no analogues in the world. In addition to diagnostic studies of such patients, it is possible to study new diagnostic indicators of long-term video-EEG monitoring with the help of intelligent software and hardware complex. The interdisciplinarity of the approach lies in the fact that the achievement of the expected results in the project is possible only with the combined efforts of neurophysiologists, specialists in the field of information technology, data mining and specialists in the field of medical technology. The synergistic effect of the interdisciplinary approach is to create and test in clinical conditions an intelligent experimental software and hardware complex that allows detecting and classifying diagnostic indicators of delayed cerebral ischemia in real time of their manifestation in the process of long-term synchronous video-EEG monitoring of patients after subarachnoid hemorrhages. The public significance is obvious. First, the time and effort of neurologists in clinical diagnostic studies when planning operations and treatment of the above patients are significantly reduced. Secondly, the objectivity and efficiency of diagnostic and prognostic conclusions increases, which makes it possible to stop the development of delayed ischemia in time and treat patients after hemorrhage in the subarachnoid space of the brain in intensive care units. Thirdly, it is possible to use it in the diagnosis of other brain diseases by integrating into it the appropriate programs for processing and analyzing video-EEG. Given the fact that the algorithms and programs of the complex will be developed jointly with highly qualified neurophysiologists and extensive experience in offline visual analysis of long-term video-EEG monitoring data, its use in other clinics in Russia will contribute to the dissemination of this knowledge and experience. The development of algorithms for analyzing synchronous video-EEG monitoring data will be carried out jointly with employees of the Department of Emergency Neurosurgery and the Department of Resuscitation and Intensive Care for Neurosurgical Patients of the N.V. Sklifosovsky Research Institute of Emergency Medicine, there it is also planned to test the developed hardware and software complex. There are software systems for processing and analyzing data from EEG studies, the most famous of them are EEGLAB, PERSIST, etc. They provide many opportunities that help the neurophysiologist to process EEG data offline. A neurophysiologist must himself identify and classify diagnostic objects in these records using visual analysis of the EEG. Video analysis is currently carried out by a neurophysiologist visually. We do not know real-time systems for video EEG analysis. This indicates that the expected results correspond to the world level, and some, for example, algorithms and programs for the joint analysis of EEG and video, and the implementation of real-time mode, exceed it. Of course, the operating time of video EEG mining algorithms is longer than the sampling intervals of video EEG, if only because, obviously, reference points are not signs, but objects from reference points. Under the real time in this case, the time of detection and classification of the diagnostic object is considered. As a result, this is real time compared to offline analysis. The development and manufacture of an experimental sample of the real-time software and hardware complex will be carried out in the LLC research and production and design firm "Medicom MTD" in Taganrog, Rostov region, which has 25 years of experience in the development and production of high-tech medical equipment for functional diagnostics, neurophysiology, and rehabilitation. This will allow replicating the developed intellectual automatized the software and hardware complex and use it in the departments of neurophysiology, neurosurgery, and resuscitation of hospitals.

Annotation of the results obtained in 2022
A multi-day video of EEG monitoring (VEEGM) was conducted in 5 patients who underwent massive subarachnoid hemorrhage with a risk of developing delayed cerebral ischemia. In addition to VEEGM, all patients underwent daily clinical examination, computed tomography (CT), perfusion computed tomography (PCT), with the correlation of the results with each other to assess the severity of cerebral angiospasm. Taking into account the results of the daily clinical examination, a, computerтомографии (CT), perfusion computed tomography (PCT), Transcranial Doppler ultrasound (TCD), with the correlation of the results with each other to assess the severity of angiospasm of cerebral vessels, long-term EEG recordings for diagnostic fragments of delayed cerebral ischemia were performed by the traditional method of visual analysis of the EEG and with the formation of an initial a training sample of diagnostic and control fragments. Signs of delayed ischemia obtained by visual analysis of the EEG: rhythmics disordering, slowing of the delta rhythm, -decrease in the amplitude of the background rhythm, lateralization of the slowing of the delta rhythm in one of the hemispheres, rhythmic epileptiform discharges, representation Epileptiform discharges at different time intervals. An indicator of video data artifacts of video-EEG monitoring was found. The indicator is the degree of mobility of the area of interest - the part of the frame in which the patient is visible. A feature was found that determines the mobility of the area of interest - the total value of the optical flux, the calculated area of interest of the frames of the video sequence. Using the found feature, a threshold algorithm for fixing artifacts was developed. A pre-processing algorithm has been developed, which includes the operation of highlighting the area of interest, the operation of color reduction of frames (conversion of a color image to a halftone) and the operation of median filtering of the image in the frame. The pre-processing algorithm and the artifact capture algorithm are implemented in the MATLAB environment as a software module. For the first time, it has been shown that interchannel synchronization is characteristic of hyper rhythmic activity, which is pathological and can be considered as an analogue of epileptiform activity in patients with severe brain damage. There is reason to believe that interchannel synchronization characterizes the t focal violation of the functional state of the brain, reflecting the occurrence and end of delayed cerebral ischemia. The proposed algorithm allows you to automatically detect a new diagnostic indicator of delayed ischemia - interchannel synchronization of EEG. A study of frequency-time distributions of peaks of wavelet spectrograms was carried out using 2D AUC diagrams to isolate signs of delayed cerebral ischemia in EEG monitoring data. To search for patterns in EEG signals, AUC diagrams were used that display AUC values corresponding to certain ranges of burst parameters. It was found that on the fifth day of patient monitoring, the average number of frequency-time peaks per second in EEG signals increased dramatically compared to the first day. It was also found that the average number of frequency-time peaks per second in EEG signals when comparing the seventh and subsequent days of patient monitoring with the first day practically does not change - there is a certain stabilization. An approach to video surveillance of the actions of medical personnel that can cause the appearance of artifacts in the patient's EEG records, by methods of object-oriented logical programming in combination with neural network methods of image analysis, is proposed and tested on real video monitoring data. Neural network architectures based on human body models COCO 18 and MPI 15 (simplified architecture) provide performance of at least 25 fps. An approach to video surveillance of the patient's face and head movements that can cause the appearance of artifacts in the patient's EEG records, methods of object-oriented logical programming in combination with neural network methods of image analysis is proposed and tested on real video monitoring data. When using a graphics processor, the FaceDetectorYN and 8bit Quantized TensorFlow neural network architectures provide a performance of at least 25 fps. At the same time, however, more research is required on the impact of the observed instability of face coordinates on the recognition quality of these artifacts. To ensure reliable detection of a person's face in a supine position, it is first necessary to determine the angle of rotation of the image and make appropriate adjustments to the transmitted video image. For the first time, a new indicator of delayed ischemia - interchannel phase connectivity of EEG - was proposed and tested. The ability of this indicator to distinguish the dynamics of interchannel connections in the left and right hemispheres of the cerebral cortex has been demonstrated. It is shown that for epileptiform activity is characterized by the concentration of pairwise frequency-synchronized EEG reference points in a narrower frequency range, in contrast to electromyographic artifacts of a non-epileptic nature. To estimate the concentration on the plane of time-frequency synchronized reference points of the wavelet spectrogram ridges for each pair of leads, the average of arithmetic and root mean square deviation of the Euclidean distance in the time-frequency plane between synchronized points of the wavelet spectrogram ridge with values that differ significantly for epileptiform activity and artifacts. According to these 2 parameters, a binary classification of epileptiform activity from chewing artifacts was carried out. Options for interaction and exchange of information with external software applications and software packages for detecting and analyzing diagnostic information have been clarified.

Publications

1. Obukhov Yu.V., Kershner I.A., Sinkin M.V. Новый подход к автоматизированному обнаружению диагностических показателей отсроченной ишемии головного мозга после субарахноидального кровоизлияния в данных длительного ЭЭГ мониторинга Журнал Радиоэлектроники, №10, стр. 1-19. (year - 2022) https://doi.org/10.30898/1684-1719.2022.10.11

Annotation of the results obtained in 2023
1. Patients with subarachnoid hemorrhage (SAH), who are at risk of angiospasm and delayed cerebral ischemia due to subarachnoid hemorrhage, continued to be recruited at the Sklifosovsky Research Institute of Cerebral Hemorrhage. Over the past year, 20 patients (70% women) were included in the study, with a median age of 56.5 years. All patients underwent a standardized series of clinical examinations at intervals of each series of instrumental examinations: The initial clinical examination upon admission to the hospital included an assessment on the HKG, FOUR, NIHSS, WFNS, Hunt-Hess, and Fisher scales. Instrumental examinations of patients upon admission to the hospital were carried out: multispiral computed tomography (MSCT), transcranial Doppler sonography, selective cerebral angiography with determination of the size and localization of the aneurysm. All patients with non-traumatic SAH underwent neurosurgical treatment. On average, the operation was performed on the 3rd day after the onset of SAH. On the next day after surgical treatment, all patients underwent multispiral computed tomography, as well as perfusion CT to assess ischemia zones, and video-EEG monitoring began. Video-EEG monitoring was continued for at least 5 days (from 1 to 9 days, median 4) either until the patient's condition stabilized and was transferred from the intensive care unit, or until the development of a cerebral infarction or death. In total, in the studied group, electrographic signs of delayed cerebral ischemia (CT), confirmed by CT data, developed in 6 patients. Electrographic signs of OCI requiring intensive treatment, followed by EEG recovery, but no changes on MSCT in 6 patients. 2. For the detection of sporadic epileptiform discharges, a new approach was proposed and investigated based on the analysis of the mutual correlation of EEG signals with the clearest pattern of epileptiform peak-wave discharge of the order of a second, which is selected from some EEG recording. The algorithm for detecting sporadic epileptiform discharges is based on the formalization of visual characteristics of epileptiform activity in the form of a peak-wave discharge pattern and the analysis of mutual correlation of multichannel EEG signals with the selected pattern. Fragments of epileptiform activity in each pair of bipolar electrodes were determined from three conditions according to the description of the patterns of peak-wave discharges of epileptiform activity: 1) the value of the positive cross-correlation of the EEG with the acute peak of the selected pattern (peak in the peak-wave discharge) at the peak of the correlation function should be greater than 0.4 (Fig. 2); 2) a positive peak in the cross-correlation of the EEG signal should be followed by a peak with a negative correlation associated with the wave peak of the wave discharge; 3) the width of the peak of the negative cross-correlation must be greater than that of the preceding positive peak of the cross-correlation. As in the case of visual detection by a neurophysiologist of interchannel synchronization, epileptiform activity was selected simultaneously in a time interval of 50 milliseconds at the peaks of discharges in several bipolar electrodes. The time of operation of the program in the MATLAB system on a modern personal computer for processing the hourly recording of 16 bipolar signals is less than 10 seconds, so it can be used to calculate the hourly amount of epileptiform activity in almost real time of the manifestation of this indicator of delayed ischemia after subarachnoid hemorrhage. 3. A new type of AUC diagrams has been developed to analyze the interhemispheric asymmetry of the amplitude-frequency characteristics of electroencephalograms (EEG) in patients after subarachnoid hemorrhage. The use of the principal component method made it possible to identify the components of EEG signals characterized by correlated changes in signals on some groups of electrodes, including the separation of uncorrelated focal and regional EEG changes. It is shown that the first three components, the main components of the EEG, contain signs of interhemispheric asymmetry, cleared of EEG artifacts. An algorithm and a program in the MATLAB language have been developed for calculating AUC diagrams for analyzing the interhemispheric asymmetry of EEG amplitude-frequency characteristics of patients who have undergone subarachnoid hemorrhage. 4. Maps of interhemispheric EEG asymmetry have been developed, displaying the magnitude of amplitude asymmetry between the corresponding pairs of EEG leads on the left and right hemispheres. The magnitude of the interhemispheric asymmetry is numerically characterized by the coefficients of the principal EEG component under consideration. In hemispheric skewness maps, the magnitude of the skewness is displayed using a color ramp. Maps of interhemispheric asymmetry make it possible to detect changes in the amplitude and frequency of EEG oscillations in the neurophysiological frequency ranges delta, theta, alpha and beta. 5. An algorithm for detecting artifacts in long-term video-EEG monitoring data in the task of diagnosing cerebral ischemia after subarachnoid hemorrhage has been developed. The values of the algorithm parameters are determined and a method for improving the performance is proposed. It is shown that the proposed algorithm detected 98% of artifacts in clinical video recordings, while the value of the F1 measure was 0.97. An estimate of the performance of the algorithm for detecting motion artifacts on fragments of video recording lasting 600 seconds, amounting to 475 seconds, was obtained, confirming the ability of the algorithm to work in real time for the development of EEG indicators of delayed ischemia. 6. A new feature (indicator) of the mobility of the area of interest has been obtained. As a feature, the total value of the optical flux calculated from the contour maps obtained from the video frames is proposed. The new feature makes it possible to take into account the component of the optical flow caused only by moving objects and to exclude from the optical flux the component caused by changes in illumination, as well as noise components An algorithm for calculating the optical flow based on maps of the contours of the area of interest and a software module implementing this algorithm have been developed. A simplified algorithm for calculating the optical video stream based on the frame difference of the video sequence has been developed. 7. Methods and programs of object-oriented logical programming for the analysis of video sequences have been designed and implemented. The architecture of a low-level image processing virtual machine, extended by neural network methods of image analysis, is proposed. The experiments demonstrated the suitability of the developed methods and tools for analyzing video data of patients undergoing treatment in the intensive care unit. 8. Object-oriented logical algorithms for analyzing the movements of the patient and medical personnel, which can lead to the occurrence of artifacts in the patient's EEG recordings, have been proposed and tested on real video monitoring data. Neural network architectures were selected, which were implemented in the form of programs in the built-in class of the Actor Prolog VideoProcessingMachine 9. Specialized system software has been developed that provides access to real-time data via the LSL (Lab Stream Layer) protocol, providing synchronization of streaming data over the network and recording of this data in real time. (Received by a partner organization). 10. A software development kit (SDK) has been implemented, which ensures the transfer of physiological signal data to an external application using LSL streams. The option of obtaining video monitoring data from a video camera in real time directly in the process of research by an external specialized program has been formalized. A variant of the export file in the EDF+ (European Data Format+) format has been implemented, which is used for the exchange and storage of multichannel physiological signals. (Received by a partner organization.)

Publications

1. Morozov A.A., Sushkova O.S., Sinkin M.V., Okuneva I.V., Obukhov Y.V. Investigation and development of methods and algorithms for analyzing video-EEG monitoring of delayed cerebral ischemia after subarachnoid haemorrhage International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives 2023, XLVIII-2/W3-2023, pp. 179–185 (year - 2023) https://doi.org/10.5194/isprs-archives-XLVIII-2-W3-2023-179-2023

2. Murashov D.M., Obukhov Y.V., Kershner I.A., Sinkin M.V. An Algorithm for Detecting Artifacts in Video Recordings of Long-Term Video-EEG Monitoring Data for the Diagnostics of Delayed Cerebral Ischemia 2023 IX International Conference on Information Technology and Nanotechnology (ITNT), Pp. 1-5 (year - 2023) https://doi.org/10.1109/ITNT57377.2023.10139085

3. Murashov D.M., Obukhov Y.V., Kershner I.A., Sinkin M.V. An Algorithm for Automated Detection of Delayed Brain Ischemia Indicator from Video-EEG Monitoring Data International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives 2023, XLVIII-2/W3-2023, pp. 187-192 (year - 2023) https://doi.org/10.5194/isprs-archives-XLVIII-2-W3-2023-187-2023

4. Obukhov Y. V., Kershner I. A., Okuneva I.V., Sinkin M.V. Algorithm for detecting epileptiform EEG activity in delayed cerebral ischemia Radioelektronika, Nanosistemy, Informacionnye Tehnologii, 15(3), pp. 253-262 (year - 2023) https://doi.org/10.17725/rensit.2023.15.253

5. Sushkova O.S., Morozov A.A., Kershner I.A., Okuneva I.V., Sinkin M.V. Development of AUC diagrams for analysis of interhemispheric asymmetry of amplitude-frequency characteristics of EEG to detect delayed cerebral ischemia induced by non-traumatic subarachnoid hemorrhage Radioelektronika, Nanosistemy, Informacionnye Tehnologii, 15(1), pp. 139-152e (year - 2023) https://doi.org/10.17725/rensit.2023.15.139

6. Murashov D.M., Kershner I.A., Obukhov Y.V., Sinkin M.V. Алгоритм обнаружения артефактов в данных длительного видео-ЭЭГ мониторинга ИНФОРМАЦИОННЫЕ ТЕХНОЛОГИИ И НАНОТЕХНОЛОГИИ (ИТНТ-2023) сборник трудов по материалам IX Международной конференции и молодежной школы : в 6 т. Том 3. Самара, 2023, Том 3, стр. 32272 (year - 2023)

7. Obukhov Y.V., Kershner I.A., Okuneva I.V., Sinkin M.V. Применение хребтов вейвлет-спектрограмм для обнаружения диагностических показателей отложенной ишемии головного мозга после субарахноидального кровоизлияния в данных длительного мониторинга электроэнцефалограмм ИНФОРМАЦИОННЫЕ ТЕХНОЛОГИИ И НАНОТЕХНОЛОГИИ (ИТНТ-2023) сборник трудов по материалам IX Международной конференции и молодежной школы : в 6 т. Том 6. Самара, 2023, Том 6, стр. 60512 (year - 2023)