Microseismic event detection with Convolutional Neural Network
By: Hamed Ghazikhani
Presenting to Prof. Zhishan Guo
Microseismic events are very small scale earthquakes which occur under the ground as a consequence of human activities such as mining and hydraulic fracturing. Microseismic science is trying to catch these small earthquakes to find out that when and where did the microseismic events occur. Furthermore, It is important for us to know how big was the microseismic event. This event is happens when the volume of the rock underground is changed. This small replacement results in the release of energy in the form of seismic waves and so we call this phenomenon the microseismic events. When this event occurs, the energy will be released in the form of seismic waves. There are receivers that are installed to record the microseismic events and many ways to detect these events. One of the most popular is STA/LTA method. In this proposal, I am going to declare a new way to detect the microseismic event with the Machine learning state of the art method which is CNN (Convolutional Neural Network). CNN is a subfield of deep learning which yields from multi-layer perceptron feed-forward neural network. Many works in this domain show that this new way will enlighten to the future research.
Microseismic event monitoring has become the first method for analyzing the hydraulic fracturing of the non-traditional gas and oil extraction. There are much valuable information in the microseismic events that help the experts to find out where did the event occur and what type was the event. The receivers record the signals from both surfaces and near the surface of the ground. This signal has low amplitudes buried in noise and require advanced methods to provide an optimal processing (Zhu, Liu et al.). The better classification of microseismic events and noise yields the more accurate results (Zhao and Gross 2017). In this project, the purpose is to improve the detection of microseismic events with higher accuracy.
There are lots of different event detection algorithms in time and frequency domains (Akram and Eaton 2012). Conventional event detection algorithms were based on sliding window and threshold value (Zhao and Gross 2017). The short and long time average ratio (STA/LTA) method is widely used in this field and many modified versions (Akram and Eaton 2012). In this method if the ratio exceeds the user-defined threshold then we say that the event is detected.
In these years, researchers have applied the machine learning algorithms to detect the events more accurately. One the most powerful machine learning algorithms which are a binary classification method is SVM (Support Vector Machine). This algorithm has been widely used in different aspects of problems such as ECG (Electrocardiogram), EEG (Electroencephalograph), etc. Using SVM attains many successful results in the field of microseismic event detection (Zhao and Gross 2017). But, the accuracy needs to be more precise due to the processing on the ground and to reach the valuable information in the detection.
Some research followed the way of ANN (Artificial Neural Network) in many fields especially microseismic events. However, the ANN is sensitive to data and easily face to the overfitting. Because of noises and the complexity of the model we try to learn more data and this will put us in a tribble situation. Accordingly, scientists decided to move to a way which can handle the high complexity data that are growing more and more in these years. Hence, with the development of the systems the Deep ANN appeared. This approach at first helps us to manage the high dimensional data which have complex structures. Deep learning would be a great idea to not only automatically detect the microseismic events but also extract the appropriate features from the data by itself (Zheng, Lu et al. 2017).
3. Related works
As we mentioned, the most widely used algorithm to detect the microseismic event is based on the short-long time average energy ratio (STA/LTA), that first proposed by Stevenson (Stevenson 1976).This method is based on the differences in the signal and noise. STA/LTA method is highly effective for data in high SNR, however, it might not be good for data at low SNR. The main weakness of this method is the selection of the parameter by hand and it is not clear what should be the correct value, such as window size. There are some efforts to combine the STA/LTA method with other algorithms to improve the performance (Zheng, Lu et al. 2017). Some research tried to use the function feature to improve the accuracy of the detection. For instance, (Baer and Kradolfer 1987) proposed a new dynamic threshold method. According to their work the feature function would be achieved by a non-linear transformation. (Akram and Eaton 2012) provided a method to select the window size and dynamic threshold in Short Time Average/ Long Time Average (STA/LTA) and Akaike information criterion (AIC). Their algorithm is time efficient and fairly accurate which requires less user input. (Xiantai, Zhimin et al. 2011) combined the STA/LTA with Power Spectrum Envelope (PSE) algorithm. They have also proposed a dynamic threshold.
Because the microseismic events are a time-domain signal the feature functions are usually based on mathematical statistical methods. There are many usages of kurtosis and skewness in the statistics and hence these features might be more trustworthy when there is no signal in the arrival time. (Li and Song 2017) suggested a pre-processing technique for real-time microseismic event detection by using adaptive filtering and kurtosis to find out the right characterization of the signal. (Sabbione and Velis 2013) used three single trace picking from earthquake seismology to find the micro-seismic event. Then, by using the multi-channel technique, analyze the event which has automatically been detected.
The feature function can also be designed in the spectrum-domain. (Song, Kuleli et al. 2010) used an array stack correlation to detect the small magnitudes. Hence, the phase picking would be done by using the transformed spectrum.
In the recent years there have been a lot of approaches in the time-frequency methods used for the microseismic event detection (Zheng, Lu et al. 2017). (Iqbal, Al-Shuhail et al. 2017) proposed the first-break picking algorithm by using the concept of interferometry and time-frequency analysis. This method iteratively converges to find the real first-break picking. (Zhu, Feng et al. 2017) find the location of the microseismic events by using Hilbert-Huang transform (HHT) and intrinsic mode function (IMF) for signal filtering and reducing the noise, respectively. So, pick the first arrivals by the time window energy eigenvalue. After that, four-channel locating optimization has been used to locate the microseismic events.
Model oriented algorithms have been widely used in the recent research. The two algorithms which are the most among the other are autoregressive(AR) and neural network(NN) (Zheng, Lu et al. 2017). Artificial Neural Network (ANN) as one of the most popular technique has been used which has emitted astonishing results. (Akram, Ovcharenko et al. 2017) implemented and calculated the correct numbers of neurons and special features. These features are gained by a window with the same size from the low S/N waveforms. It might be good to gain the hyper parameter, but as we know this could be always a better number to be replaced. (Shang, Li et al. 2017) have proposed a model which is based on Principle Analysis Component (PCA) and ANN. According to their result, PCA shows significant improvement in their results in compare to those research that does not reduce the dimension. (Mousavi, Horton et al. 2016) used feature extraction in many signal domains and then use them as an input for classification by Logistic Regression (LR) and Artificial Neural Network (ANN). The main differences of the methods that were mentioned above are that the input data (features) of the neural network, were various as their input dimension. In theory, the higher complexity of a parameter for a model, the lesser generalization we achieve. On the other hand, when the model is trained with high complex parameters, this would readily encounter the over-fitting (Zheng, Lu et al. 2017). Ordinary neural networks could only handle the non-complex data structure with few features, then as we see in the recent developments, the data are provided and it is going to be more complex. Hence, traditional neural network would be affected by the curse of dimensionality. These sort of problems cannot be solved by using neural network with many layers which are going to be trained using backpropagation (BP). Theoretically, in normal NN the more layer we have the lesser impact of BP we would have on first layers. This could lead the network to be stuck in the local minima and accordingly, this is why we normally use two layers to classify or approximate every function (Va?eka, Prokop et al.).
Recent researches are focusing on learning from data in a deep neural network (DNN) structure. While shallow ones cannot handle the correct feature, DNNs are trying to learn from the data base on their representation. These models provide the correct features with unsupervised methods and perform a non-linear dimensionality reduction. Finally, the backpropagation method would be run to do the fine-tune operation on the whole network. (Zheng, Lu et al. 2017) proposed Deep Recurrent Neural Network (DRNN) to detect the microseismic event and their result are robust against the noise. The RNN that they used is based on Long Short Term Memory (LSTM) to which it has the ability to memorize the data. According to them, the more training data the better learning model will be gained. They have got above 80% accuracy without SNR and an approximately 70% accuracy at SNR -5dB. However, it is a big deal while the authors mentioned that their model has the weakness on the number of data and the features which the model could be achieved.
We need more reliable model to not only elicit the relevant features, but also do the classification perfectly. Some of the existing challenges that we may face in the field of microseismic would be explored in following part.
4. Existing challenges
The nature of the microseismic signals is full of noise. This because of the situation that the receivers get the data. While it has a lot of noise the traditional models are suitable for different aspects. The conventional arrival pick-up algorithms cannot avoid the manual modification of the parameters for the simultaneous identification of multiple events under different signal-to-noise ratios. Furthermore, the automatic selection and recognition of the arrival times of microseismic are significant to the realization of the automatic processing of massive microseismic. However, due to the variabilities of the stress waveforms, differences in the trigger source phases of the rupture sources, and the presence of various noise interference, the research studies regarding automatic recognition and selection methods remain challenging. Hence, Because of the small magnitude of microseismic events and noisy borehole or surface recording environment, the microseismic signals may often be neglected if no proper denoising algorithms or event detection techniques are applied. According to the above explains, we need an algorithm to handle the noise and the feature extraction while it is learning from the representation of the data.
5. Purposes of research
According to the challenges that we talked, one the appropriate way to solve them is to use deep learning methods. Recent development in the field of deep learning neural networks has opened new research possibilities regarding microseismic event detection. Deep learning as one of the most crucial machine Learning algorithms is promising in various domains which the model is learning a new representation of the data (LeCun, Bengio et al. 2015).
In this research we are exploring a new way to overcome the issues and the challenges which we have in field of microseismic event detection. We believe that deep learning would be a good suggestion to solve the problems.
6. Suggested solution
Convolutional Neural Network (CNN) is a subfield of deep learning which yields from multi-layer perceptron feed-forward neural network. CNNs have shown immense success in computer vision, natural language processing, speech recognition etc. The success of CNN in so many applications has inspired me to verify how effective this network is on learning features from sequential data. This deep learning structure is gained by the biological processes in the brain. The architecture of the CNN is similar to the other neural networks with some differences. In here, we also have an input layer (features), hidden layers, and the output layer. The hidden layers consist of different kinds of layers which are convolutional layers, pooling layers, fully connected layers. Convolutional layers apply the convolution operation to the input, passing the result to the next layer. Pooling layers are periodically inserted between convolutional layers and its objective is to reduce the parameters and therefore, avoid overfitting. Eventually, the fully connected layer is similar to the conventional type which every neuron in one layer is connected to another layer. As shown in Figure 1, given a set of features to the convolutional layer will apply the convolution operation to reach the net of each neuron and passing to the pooling layer.
Figure 1: The convolutional layer and pooling layer applied upon input features.
It is right that the CNN structure works fine with image and video data but because of its effectiveness for reducing the spectral variations, this can be used in sequential data as well (Kasfi, Hellicar et al. , Z?bik, Korytkowski et al. , Mittelman 2015, Cui, Chen et al. 2016, Zhang, Pezeshki et al. 2017). Zhang et al. 20 proposed an end-to-end speech recognition by combining the CNN and the Connectionist Temporal Classification (CTC) for sequence labeling. Zebik et al. 21 Used CNN and achieved the appropriate low- and high-level features and showed that the CNN can have accurate recognition in time-series data. Zhicheng Cui, Wenlin Chen and Yixin Chen 22 proposed multi-scale CNN (MCNN) which do the feature extraction and classification of sequential data in one framework and its accuracy compared to other is a leading one. Mittelman 23 introduced a fully convolutional network (FCN) architecture that uses causal filtering operations, and allows for the rate of the output signal to be the same as the input signal. Their results show important advantages compared to RNN with LSTM baseline. A research has done in the field of feature extraction from cattle behavior using the CNN 24.
To use the deep structure for the micro-seismic event detection needs a huge training data with lots of features that the deep architecture would work fine to reduce the noise and attain the proper features. As it is new to use the deep architecture for identifying the micro-seismic event, but it can introduce a new way on the detection of those crucial events. Deep RNN as previously described in the related work section, has been implemented to recognize the micro-seismic event but it has the problem with the number of data. According to the authors, they had to use the 80% of the data for the training phase and 20% of them for testing. Hence, they show that the number of data is an important factor while we tend to use a deep structure 4.
In micro-seismic detection, we have trouble with the number of data, while it is easy to understand that deep structure requires high amounts of them. So, we have to consider this as one of the biggest problems that we may face and should pay more attention to this phenomenon. One solution to this problem is to use the Sampling methods. In this method, we try to reproduce the data from the same distribution. Hence, while the data is accumulated then we can use the deep methods. To conclude, deep learning as a subfield of Machine learning is promising in different fields. CNN architecture has been proposed to be used as a micro-seismic event detection and classification. CNN shows powerful results in image and video processing, but it can also be used in sequential data (i.e. time series). Furthermore, it is obvious that CNN structure is impressive to find the proper features while it can do the classification. Many works in this domain show that this new way will enlighten to the future research.
A classifier based on a CNN seems to be a good approach for microseismic event classification as the signal to detect contains a lot of variations. The interest of the CNN is to directly classify the raw signal and to integrate the signal processing functions within the discriminant steps. Indeed, it is not always possible to know the type of features to extract. It is better to let the network extract the most discriminant features by constructing high- level features throughout the propagation step. According to the above explains, we would certainly say that the CNN would perfectly deal with the microseismic data.Bibliography
Akram, J. and D. Eaton (2012). Adaptive microseismic event detection and automatic time picking. 2012 CSEG Annual Convention.
Akram, J., O. Ovcharenko and D. Peter (2017). “A robust neural network-based approach for microseismic event detection.” SEG Technical Program Expanded Abstracts 2017.
Baer, M. and U. Kradolfer (1987). “An automatic phase picker for local and teleseismic events.” Bulletin of the Seismological Society of America 77(4): 1437-1445.
Cui, Z., W. Chen and Y. Chen (2016). “Multi-scale convolutional neural networks for time series classification.” arXiv preprint arXiv:1603.06995.
Iqbal, N., A. A. Al-Shuhail, S. I. Kaka, E. Liu, A. G. Raj and J. H. McClellan (2017). “Iterative interferometry-based method for picking microseismic events.” Journal of Applied Geophysics 140: 52-61.
Kasfi, K. T., A. Hellicar and A. Rahman Convolutional Neural Network for Time Series Cattle Behaviour Classification, New York, New York, USA, ACM Press.
LeCun, Y., Y. Bengio and G. Hinton (2015). “Deep learning.” Nature 521(7553): 436-444.
Li, F. and W. Song (2017). Automatic arrival identification system for real-time microseismic event location. SEG Technical Program Expanded Abstracts 2017, Society of Exploration Geophysicists: 2934-2939.
Mittelman, R. (2015). “Time-series modeling with undecimated fully convolutional neural networks.” arXiv preprint arXiv:1508.00317.
Mousavi, S. M., S. P. Horton, C. A. Langston and B. Samei (2016). “Seismic features and automatic discrimination of deep and shallow induced-microearthquakes using neural network and logistic regression.” Geophysical Journal International 207(1): 29-46.
Sabbione, J. I. and D. R. Velis (2013). “A robust method for microseismic event detection based on automatic phase pickers.” Journal of Applied Geophysics 99: 42-50.
Shang, X., X. Li, A. Morales-Esteban and G. Chen (2017). “Improving microseismic event and quarry blast classification using Artificial Neural Networks based on Principal Component Analysis.” Soil Dynamics and Earthquake Engineering 99: 142-149.
Song, F., H. S. Kuleli, M. N. Toksöz, E. Ay and H. Zhang (2010). “An improved method for hydrofracture-induced microseismic event detection and phase picking.” Geophysics 75(6): A47-A52.
Stevenson, P. R. (1976). “Microearthquakes at Flathead Lake, Montana: A study using automatic earthquake processing.” Bulletin of the Seismological Society of America 66(1): 61-80.
Va?eka, L., T. Prokop, R. Mou?ek, P. Mautner and J. Št?beták Application of Stacked Autoencoders to P300 Experimental Data, Springer.
Xiantai, G., L. Zhimin, Q. Na and J. Weidong (2011). “Adaptive picking of microseismic event arrival using a power spectrum envelope.” Computers & geosciences 37(2): 158-164.
Z?bik, M., M. Korytkowski, R. Angryk and R. Scherer Convolutional Neural Networks for Time Series Classification, Springer.
Zhang, Y., M. Pezeshki, P. Brakel, S. Zhang, C. L. Y. Bengio and A. Courville (2017). “Towards end-to-end speech recognition with deep convolutional neural networks.” arXiv preprint arXiv:1701.02720.
Zhao, Z. and L. Gross (2017). Using supervised machine learning to distinguish microseismic from noise events. 2017 SEG International Exposition and Annual Meeting, Society of Exploration Geophysicists.
Zheng, J., J. Lu, S. Peng and T. Jiang (2017). “An automatic microseismic or acoustic emission arrival identification scheme with deep recurrent neural networks.” Geophysical Journal International 212(2): 1389-1397.
Zhu, L., E. Liu, J. McClellan and A. Al-Shuhail “Microseismic Event Monitoring.”
Zhu, Q., Y. Feng, M. Cai, J. Liu and H. Wang (2017). “Interpretation of the extent of hydraulic fracturing for rockburst prevention using microseismic monitoring data.” Journal of Natural Gas Science and Engineering 38: 107-119.