| Sumario: | Over the last few decades, much research has focused on the development and optimization of video codecs for media distribution to end-users via the Internet, broadcasts or mobile networks, but also for videoconferencing and for the recording on optical disks for media distribution. Most of the video coding standards for delivery are characterized by using a high efficiency hybrid schema, based on inter-prediction coding for temporal picture decorrelation, and intra-prediction coding for spatial picture decorrelation. As is well known, this schema achieves a high performance at the expense of several drawbacks, such as a high coding latency, low stream robustness, and a high coding complexity due to the motion-estimation process of the inter-prediction, among others. Nevertheless, high efficiency video and image codecs are also required in other fields with quite different requirements, such as still-picture photography storage, live TV interviews where low latency is needed for natural communication between the interlocutors, and also for professional edition and post-production tasks commonly used in the TV and cinema industries, where high quality and fast access to the individual pictures are required. These production codecs, named mezzanine codecs, are highly relevant for the audio-visual industry, which demands two things: very high compression efficiency and a low computational burden. A high video coding performance allows a reduction in the storage space for archiving applications, and also a long recording capability on small physical supports, i.e. optical disks and solid state memories. Regarding low computational complexity, this is mainly motived by the fact that these mezzanine codecs are used on portable devices, mainly camcorders, and thus one wishes that the codec can be implemented on low cost processors, and just as importantly, with low power consumption, allowing a long operational autonomy. The latest video coding standard was approved in January of 2013, and it is the result of the collaboration between the ITU-T and the ISO/IEC international organizations grouped into the Joint Collaborative Team Video Coding group (JCT-VC). This standard is officially called Recommendation ITU-T H.265 and ISO/IEC 23008-2, and informally known as HEVC (High Efficiency Video Coding). Exhaustive quality assessments have proved that HEVC can reduce the bit rate of its predecessor, namely H.264/AVC, by half for the same perceptual quality. The high compression efficiency of HEVC has led to this standard being adopted for future TV services using the new UHD format by the two worldwide broadcasting organisations, Digital Video Broadcasting (DVB) and the Advanced Television Systems Committee (ATSC). It has also been adopted in other application fields, such as by the Blu-ray Disc Association (BDA) for the next generation of Ultra-HD Blu-ray. This fact will probably accelerate HEVC's expansion and development, prompting a rapid replacement of the H.264/AVC standard in the consumer markets and also in massive media services such as video streaming, VoD and OTT (Over The Top). HEVC is also considered by the industry as the best candidate to replace the current mezzanine compression codecs, due to the high performance of the novel HEVC intra-prediction coding. This schema significatively improves H.264/AVC's performance, mostly by using a high density of angular predictors. For this reason, the HEVC standard has approved a set of specific profiles which exclusively use the intra-prediction schema, known as ¿Main still Picture¿ and the ¿Main Intra¿ profiles, with support for different bit-depths and chroma sampling formats. However, the high compression performance of HEVC comes at the expense of a huge computational complexity increase compared with other codecs, which is hindering the rapid adoption of HEVC by the professional market. The speeding up of the intra-prediction coding can be achieved by applying advanced techniques that allow the taking of decisions that are needed in the different stages of intra-prediction with low complexity. This approach constitutes the basis of this thesis, which addresses the complexity reduction of HEVC intra-coding by using non-traditional techniques used in the video coding standards, such as Machine Learning and image processing algorithms for texture orientation detection. 2. RESEARCH OBJECTIVES In this dissertation four main objectives have been proposed for the computational reduction of the real time implementation of the intra-prediction coding in HEVC, and these are the following: Study of state-of-the-art of HEVC and fast intra-prediction approaches. As an initial aim we have defined an in-depth study of the HEVC architecture and the new coding tools introduced in this standard, paying special attention to the performance differences between H.264/AVC and HEVC. One of the main objectives is the detailed study of the different algorithms that comprises the intra-prediction in HEVC, and the analysis of the computational complexities required for these. Then, we will conduct a detailed study of the most relevant approaches proposed in the literature for fast partitioning and mode decision in intra-prediction, with particular attention to the algorithms already adopted in the Rate Distortion Optimization (RDO) stage of the test model proposed for the HEVC reference software. The conclusion of this study will constitute the basis for the design of the efficient low complexity algorithms proposed in this thesis. Development and evaluation of a fast partitioning decision algorithm. Being aware that the partitioning of the coding units is the most critical decision that the encoder has to take, in terms of computational burden but also regarding the impact on quality, the second objective proposed in this thesis is the complexity reduction of the partitioning decision. With the aim of tackling this task with high efficiency, the use of Machine Learning techniques for the design of a decision tree will be studied. Special efforts will be made in the training stage of the decision trees, selecting the optimal number and type of attributes, allowing a high precision classifier with the minimum computational cost. It required that the architecture of the proposed approach permit a scalable implementation with different decision nodes, in order to achieve different levels of speed-up and different levels of performance reductions, compared with the HEVC reference software. Development and evaluation of a fast mode decision algorithm. With the aim of exploiting the strong correlation between the texture orientation of the image and the angular modes defined in HEVC, the development and evaluation of a fast mode decision algorithm based on texture orientation will be studied. In order to achieve this objective, an in-depth study of the different texture and edge detection techniques published in the literature will be carried out. The proposal will be highly efficient for the detection of the dominant gradient in all the range of coding units, from 64x64 to 4x4 pixels, with a low computation complexity. The aim of the approach will be to achieve a significant speed-up of intra-prediction coding by reducing the number of modes to be evaluated, with a very low performance penalty. Combination of both proposals in a full fast intra-prediction algorithm. As a last aim, the integration of both fast decision approaches in a unified architecture is proposed. The analysis and performance evaluation of the combined proposal will allow us to know the best performance by speeding up both decisions. The analysis of the results will also show the mutual inference of the wrong classification decisions of both algorithms in terms of bit coding efficiency degradation. That is, how a wrong mode decision affects the global encoding performance when a wrong partitioning decision is taken, or vice versa, how a wrong partitioning decision affects the global encoding performance when a wrong mode decision is taken. 3. CONCLUSIONS The major conclusions obtained in this thesis are summarized in the following lines: Regarding HEVC Intra-prediction encoding. Nowadays, multimedia services are widely delivered through wireless and broadcast networks to a broad range of devices with very diverse computational capabilities. With the aim of achieving high compression efficiency, these services commonly use a long GOP encoding structure, which applies an inter-picture prediction scheme, but intra-prediction encoding also has to be used at least in one picture of each GOP. Intra-prediction is currently an active research topic due to the practical application of the intra encoding scheme in several fields, such as: professional media production, where high quality and random access to the individual pictures are required; very low delay communications, in which intra-coding appears to be the only available solution; and also delivery through networks with transmission error, typically packet loss, because the non-temporal dependency between pictures avoids temporal error propagation. Nowadays, the high efficiency of HEVC and its performance supremacy over previous video coding standards, such as H.264/AVC, is unquestionable. However, the extremely high computational burden demanded by the HEVC algorithms makes HEVC encoding a challenge for developers, and also for the research community. In the last few years, wireless and portable devices have experienced an incredible evolution, their multimedia capabilities increasing with very high resolution picture and video cameras for real time personal communications. The evolution of wireless networks has followed a similar trend, especially the cellular networks supporting 3G, 4G and the forthcoming 5G technologies, which allow huge bandwidths of tens, and soon hundreds, of Mbps, which were unthinkable a few years ago. However, video communication using these new multimedia functionalities imposes a huge computational burden that becomes a severe problem for battery-powered portable devices. The novel approaches presented in this thesis prove that the non-conventional techniques used in the video coding field, such as Machine Learning and image processing tools, make it possible to achieve a considerable complexity reduction when they are efficiently applied, which can be regarded as a solution for the above mentioned scenarios under computational or consumption constraints. The results reported in Chapter 5 show that the combination of both approaches can obtain a wide range of speed-ups, from 30% to nearly 70%, in a trade-off with slight performance losses in the range of 0.4% to 4%, when it is compared with the HM reference software. This noteworthy complexity reduction for HEVC intra-prediction coding will encourage the quick introduction and development of the new generation of video coding standards in a new generation of multimedia devices such as smartphones, tablets and smart TVs. Regarding a fast partitioning decision for HEVC intra-prediction encoding There is no doubt that the optimal CTU partitioning decision is the most computationally intensive process in intra-prediction, because of the high number of available PU sizes evaluated by the RDO stage. The proposal presented in this thesis addresses this problem by introducing a fast PU classifier which has been designed using Machine Learning models. ML techniques are commonly used to solve big data problems where a big number, which can be tens or sometimes hundreds, of attributes are known. These techniques make it possible to carry out a high accuracy data classification or infer a behavior with high reliability. However, the application of ML techniques to the intra-prediction scenario, where initial features of the CTU are unavailable or cannot be gathered from other stages of the encoder, constitutes one of the most challenging aspects addressed in this thesis due to the critical balance that has to be reached between the number of attributes to be computed, and the cost of computing these attributes. The approach presented in this thesis is based on decision trees whose rules were obtained by training a model using ML techniques supported by the WEKA tool. It is worth highlighting that the size of the training data set and the type of data set constituted the key elements in achieving a high accuracy classifier. Regarding the selection of the data set, this was assisted by the SI and TI metrics, which have proven to be very effective criteria for image complexity characterization. While attending to these parameters, the data type was selected by covering the full range of possible CU patterns, from very low textured images to high complexity images. As for the size of the data set, just 8 frames were used for the training stage, which is 0.081% of the total frames (9,780 frames belonging to the 22 test sequences) used for the validation test. The results obtained confirm that a high accuracy classifier can be achieved by training decision trees with a small data set, as long as a high quality data type selection is performed. Regarding the attributes used by the classifier, it has been found that first order statistics such as variance and the mean of PUs are not sufficient to classify a PU as Split or Not-Split with high precision. However, by introducing as an attribute the variance of these statistics of mean and variance but computed over the four sub-PUs, classifier accuracy is significantly increased. In addition, these statistics can be computed with a very low computational burden, thus the balance between the number and effectiveness of attributes and their cost is very profitable. Results for the training and non-training sequences have demonstrated that a significant speed-up, of over 52%, of the CTU partitioning decision can be achieved in intra-prediction encoding, with only a slight bit rate increase of less than 2%, favoring real-time software and hardware implementation. Finally, a comparative study with the most prominent fast CTU partitioning decision algorithms proposed in the literature has been conducted. The results confirm that the approach presented in this thesis achieves better performance for the full node implementation, with a higher complexity reduction and a lower bit rate penalty. Regarding fast mode decision for HEVC intra-prediction encoding The optimal mode decision constitutes the second critical parameter that has to be selected for intra-prediction, in terms of computational burden. In this thesis, the correlation between the optimal mode decision and the edge and texture orientation is exploited. Based on that observation, a novel texture orientation detection approach has been proposed which computes the MDV along a set of co-lines with rational slopes. The first accomplishment reached in this proposal is the definition of the reduced number of orientations, just 12 directions, which can estimate a set of modes close to the optimal mode in-between the 33 angular intra modes of HEVC. A noteworthy feature of this approach is that these orientations with rational slopes are exclusively defined in an integer position of the discrete lattice ¿, thus no pixel interpolation is required, significatively reducing the computational burden. The second key point of this proposal has been the demonstration that the pixel correlation is maximal in the dominant texture orientation, and consequently the variance computed in that direction obtains the lowest value, compared with the variance computed in other directions. This observation constitutes the basis of the proposed MDV metric, which presents several advantages such as the low complexity computation of the variance, and a scalability implementation in co-segments, allowing the computation of the all PU sizes in just one pass. The next achievement has been the improvement of the MDV metric by introducing the concept of Sliding Window. In this new approach, the directional variance computation for an NxN PU is expanded to an (N+1)x(N+1) window, thus the neighbouring pixels used as reference samples for the construction of the predictor are also included in the calculation of the directional variance. MDV-SW makes it possible to reduce the bit rate penalty of the MDV proposal by half. The results show that the proposed fast mode decision achieves a significant time saving of 30% with a negligible impact on the encoding performance, obtaining an average BD-rate of just 0.4%. Finally, a comparison with the 5 most noteworthy fast mode decision algorithms proposed in the literature has been presented. The results show that the MDV-SW proposal obtains the best performance with the lowest ¿ Rate¿¿ Time ratio. Regarding the combination of fast partitioning and mode decision approaches for HEVC intra-prediction encoding Finally, both proposals presented in this thesis, the fast partitioning and fast mode decision, are combined, creating an overall fast intra-prediction algorithm, called FPMD, which achieves a very high overall performance. The key point of the combined approach is that both decisions are taken by computing low complexity algorithms, first using decision trees (if-else statements) for the partitioning decision, and then optimal orientation is estimated by computing the MDV in 12 directions. The combined approach is a flexible solution because it can also be implemented using any combination of nodes including the fast mode decision, and thus better performance is achieved compared with the standalone partitioning approach. The simulation results for FPMD show a wide range of speed-ups, from 44% for the N64+MDV-SW to 67% for the overall implementation, namely N64+N32+N16+MDV-SW, at the expense of slight penalties in terms of BD-rate, from 1.1% to 4.6%, respectively.
|
|---|
