1*4882a593SmuzhiyunMPP (Media Process Platform) design (2016.10.12) 2*4882a593Smuzhiyun================================================================================ 3*4882a593Smuzhiyun 4*4882a593SmuzhiyunThe mpp is a middleware library for Rockchip SoC's cross platform media process. 5*4882a593SmuzhiyunThe main purpose of mpp is to provide very high performance, high flexibility 6*4882a593Smuzhiyunand expansibility on multimedia (mainly video and image) process. 7*4882a593Smuzhiyun 8*4882a593SmuzhiyunThe design target of mpp is to connect different Rockchip hardware kernel driver 9*4882a593Smuzhiyunand different userspace application. 10*4882a593Smuzhiyun 11*4882a593SmuzhiyunRockchip has two sets of hardware kernel driver. 12*4882a593Smuzhiyun 13*4882a593SmuzhiyunThe first one is vcodec_service/vpu_service/mpp_service which is a high 14*4882a593Smuzhiyunperformance stateless frame base hardware kernel driver. This driver supports 15*4882a593Smuzhiyunall available codecs that hardware can provide. This driver is used on Android/ 16*4882a593SmuzhiyunLinux. 17*4882a593Smuzhiyun 18*4882a593SmuzhiyunThe second one is v4l2 driver which is developed for ChromeOS. It currently 19*4882a593Smuzhiyunsupports H.264/H.265/vp8/vp9. This driver is used on ChomeOS/Linux. 20*4882a593Smuzhiyun 21*4882a593SmuzhiyunMpp plans to support serval userspace applications including OpenMax, FFmpeg, 22*4882a593Smuzhiyungstreamer, libva. 23*4882a593Smuzhiyun 24*4882a593Smuzhiyun 25*4882a593SmuzhiyunFeature explaination 26*4882a593Smuzhiyun================================================================================ 27*4882a593Smuzhiyun 28*4882a593Smuzhiyun1. Cross Platform 29*4882a593SmuzhiyunThe target OS platform including Android, Linux, ChromeOS and windows. Mpp uses 30*4882a593Smuzhiyuncmake to compile on different platform. 31*4882a593Smuzhiyun 32*4882a593Smuzhiyun2. High Performance 33*4882a593SmuzhiyunMpp supports sync / async interface to reduce the time blocked in interface. And 34*4882a593Smuzhiyunmpp internally make hardware and software run parallelly. When hardware is 35*4882a593Smuzhiyunrunning sofware will prepare next hardware task at the same time. 36*4882a593Smuzhiyun 37*4882a593Smuzhiyun3. High Flexibility 38*4882a593Smuzhiyunmpi (Media Process Interface) is easy to extend by different control function. 39*4882a593SmuzhiyunThe input/output element packet/frame/buffer is easy to extend different 40*4882a593Smuzhiyuncomponents. 41*4882a593Smuzhiyun 42*4882a593Smuzhiyun 43*4882a593SmuzhiyunSystem Diagram 44*4882a593Smuzhiyun================================================================================ 45*4882a593Smuzhiyun 46*4882a593Smuzhiyun +---------------------------------------+ 47*4882a593Smuzhiyun | | 48*4882a593Smuzhiyun | ffmpeg / OpenMax / gstreamer / libva | 49*4882a593Smuzhiyun | | 50*4882a593Smuzhiyun +---------------------------------------+ 51*4882a593Smuzhiyun 52*4882a593Smuzhiyun +-------------------- MPP ----------------------+ 53*4882a593Smuzhiyun | | 54*4882a593Smuzhiyun | +-------------------------+ +--------+ | 55*4882a593Smuzhiyun | | | | | | 56*4882a593Smuzhiyun | | MPI / MPP | | | | 57*4882a593Smuzhiyun | | buffer queue manage | | | | 58*4882a593Smuzhiyun | | | | | | 59*4882a593Smuzhiyun | +-------------------------+ | | | 60*4882a593Smuzhiyun | | | | 61*4882a593Smuzhiyun | +-------------------------+ | | | 62*4882a593Smuzhiyun | | | | | | 63*4882a593Smuzhiyun | | codec | | OSAL | | 64*4882a593Smuzhiyun | | decoder / encoder | | | | 65*4882a593Smuzhiyun | | | | | | 66*4882a593Smuzhiyun | +-------------------------+ | | | 67*4882a593Smuzhiyun | | | | 68*4882a593Smuzhiyun | +-----------+ +-----------+ | | | 69*4882a593Smuzhiyun | | | | | | | | 70*4882a593Smuzhiyun | | parser | | HAL | | | | 71*4882a593Smuzhiyun | | control | | reg_gen | | | | 72*4882a593Smuzhiyun | | | | | | | | 73*4882a593Smuzhiyun | +-----------+ +-----------+ +--------| | 74*4882a593Smuzhiyun | | 75*4882a593Smuzhiyun +-------------------- MPP ----------------------+ 76*4882a593Smuzhiyun 77*4882a593Smuzhiyun +---------------------------------------+ 78*4882a593Smuzhiyun | | 79*4882a593Smuzhiyun | kernel | 80*4882a593Smuzhiyun | RK vcodec_service / v4l2 | 81*4882a593Smuzhiyun | | 82*4882a593Smuzhiyun +---------------------------------------+ 83*4882a593Smuzhiyun 84*4882a593SmuzhiyunMpp is composed of four main sub-modules: 85*4882a593Smuzhiyun 86*4882a593SmuzhiyunOSAL (Operation System Abstraction Layer) 87*4882a593SmuzhiyunThis module shutters the differences between different operation systems and 88*4882a593Smuzhiyunprovide basic components including memory, time, thread, log and hardware memory 89*4882a593Smuzhiyunallocator. 90*4882a593Smuzhiyun 91*4882a593SmuzhiyunMPI (Media Process Interface) / MPP 92*4882a593SmuzhiyunThis module is on charge of interaction with external user. Mpi layer has two 93*4882a593Smuzhiyunways for user. The simple way - User can use put/get packet/frame function set. 94*4882a593SmuzhiyunThe advanced way - User has to config MppTask and use dequeue/enqueue funciton 95*4882a593Smuzhiyunset to communicate with mpp. MppTask can carry different meta data and complete 96*4882a593Smuzhiyuncomplex work. 97*4882a593Smuzhiyun 98*4882a593SmuzhiyunCodec (encoder / decoder) 99*4882a593SmuzhiyunThis module implements the high efficiency internal work flow. The codec module 100*4882a593Smuzhiyunprovides a general call flow for different video format. Software process will 101*4882a593Smuzhiyunbe separated from hardware specified process. The software will communicate with 102*4882a593Smuzhiyunhardware with a common task interface which combines the buffer information and 103*4882a593Smuzhiyuncodec specified infomation. 104*4882a593Smuzhiyun 105*4882a593SmuzhiyunParser/Controller and hal (Hardware Abstraction Layer) 106*4882a593SmuzhiyunThis layer provides the implement function call of different video format and 107*4882a593Smuzhiyundifferent hardware. For decoder parser provide the video stream parse function 108*4882a593Smuzhiyunand output format related syntax structure to hal. The hal will translate the 109*4882a593Smuzhiyunsyntax structure to register set on different hardware. Current hal supports 110*4882a593Smuzhiyunvcodec_service kernel driver and plan to support v4l2 driver later. 111*4882a593Smuzhiyun 112*4882a593Smuzhiyun 113