MediaMuxer가 때때로 검은 색 비디오로 나타납니다.

Question

AndroidEncoder를 사용하여 RTMP 네트워크에서 보내고 MediaMuxer를 사용하여 MP4 파일에 로컬로 저장할 수있는 AAC 오디오가 포함 된 H264 비디오를 생성합니다.

방송은 RTMP보다 좋게 작동하지만 로컬로 저장하면 때로는 결과 비디오가 괜찮을 때도 있고 때로는 사운드가있는 검은 색 프레임 (또는 정보 블록이있는 첫 번째 부분 프레임)이 있고 때때로 오디오를 재생합니다 , 임의의 FPS 속도로 동영상을 재생합니다.

이

내 먹서 래퍼 클래스와 MPEG4Writer SDK 클래스에 의해 표시되는 출력 : 로컬로 저장 비디오 괜찮 아니지만

02-28 11:57:38.521 6420-6475/com.myapp.broadcast W/AndroidMuxer: addTrack: Requested adding of track to class AndroidMuxer of type AUDIO 
02-28 11:57:38.522 6420-6475/com.myapp.broadcast W/AndroidMuxer: addTrack: Adding track for audio/mp4a-latm 
02-28 11:57:38.522 6420-6475/com.myapp.broadcast W/AndroidMuxer: addTrack: Track index to class AndroidMuxer of type AUDIO, track saved for future use! 
02-28 11:57:38.621 6420-6470/com.myapp.broadcast W/AndroidMuxer: addTrack: Requested adding of track to class AndroidMuxer of type VIDEO 
02-28 11:57:38.621 6420-6470/com.myapp.broadcast W/AndroidMuxer: addTrack: Adding track for video/avc 
02-28 11:57:38.621 6420-6470/com.myapp.broadcast W/AndroidMuxer: addTrack: All tracks added, marking Local muxer as ready! 
02-28 11:57:38.622 6420-6470/com.myapp.broadcast W/AndroidMuxer: addTrack: Track index to class AndroidMuxer of type VIDEO, track saved for future use! 
02-28 11:57:52.050 6420-6420/com.myapp.broadcast I/AndroidMuxer: init: Added audio track widh id 0 
02-28 11:57:52.050 6420-6420/com.myapp.broadcast I/AndroidMuxer: init: Added video track widh id 1 
02-28 11:57:52.050 6420-6420/com.myapp.broadcast W/AndroidMuxer: init: Muxer was successfully created! Starting to record 
02-28 11:57:52.056 6420-6420/com.myapp.broadcast I/MPEG4Writer: limits: 4294967295/0 bytes/us, bit rate: -1 bps and the estimated moov size 3195 bytes 
02-28 11:57:52.057 6420-6420/com.myapp.broadcast W/AndroidMuxer: start: Muxer started! 
02-28 11:57:52.070 6420-6965/com.myapp.broadcast I/MPEG4Writer: setStartTimestampUs: 0 
02-28 11:57:52.070 6420-6965/com.myapp.broadcast I/MPEG4Writer: Earliest track starting time: 0 
02-28 11:57:52.083 6420-6966/com.myapp.broadcast I/MPEG4Writer: setStartTimestampUs: 0 
02-28 11:58:02.617 6420-6475/com.myapp.broadcast W/AndroidMuxer: writeSampleData: All tracks finished! Calling Stop 
02-28 11:58:02.617 6420-6475/com.myapp.broadcast W/AndroidMuxer: stop: Calling stop to the muxer 
02-28 11:58:02.617 6420-6475/com.myapp.broadcast D/MPEG4Writer: Audio track stopping 
02-28 11:58:02.617 6420-6475/com.myapp.broadcast D/MPEG4Writer: Audio track source stopping 
02-28 11:58:02.617 6420-6475/com.myapp.broadcast D/MPEG4Writer: Audio track source stopped 
02-28 11:58:02.617 6420-6965/com.myapp.broadcast I/MPEG4Writer: Received total/0-length (452/0) buffers and encoded 452 frames. - Audio 
02-28 11:58:02.617 6420-6965/com.myapp.broadcast I/MPEG4Writer: Audio track drift time: 0 us 
02-28 11:58:02.617 6420-6475/com.myapp.broadcast D/MPEG4Writer: Audio track stopped 
02-28 11:58:02.618 6420-6475/com.myapp.broadcast D/MPEG4Writer: Video track stopping 
02-28 11:58:02.618 6420-6475/com.myapp.broadcast D/MPEG4Writer: Video track source stopping 
02-28 11:58:02.618 6420-6475/com.myapp.broadcast D/MPEG4Writer: Video track source stopped 
02-28 11:58:02.618 6420-6966/com.myapp.broadcast I/MPEG4Writer: Received total/0-length (315/0) buffers and encoded 315 frames. - Video 
02-28 11:58:02.618 6420-6475/com.myapp.broadcast D/MPEG4Writer: Video track stopped 
02-28 11:58:02.618 6420-6475/com.myapp.broadcast D/MPEG4Writer: Duration from tracks range is [10143021, 10495420] us 
02-28 11:58:02.618 6420-6475/com.myapp.broadcast D/MPEG4Writer: Stopping writer thread 
02-28 11:58:02.620 6420-6964/com.myapp.broadcast D/MPEG4Writer: 0 chunks are written in the last batch 
02-28 11:58:02.620 6420-6475/com.myapp.broadcast D/MPEG4Writer: Writer thread stopped 
02-28 11:58:02.621 6420-6475/com.myapp.broadcast I/MPEG4Writer: The mp4 file will not be streamable. 
02-28 11:58:02.621 6420-6475/com.myapp.broadcast W/AndroidMuxer: stop: muxer stopped! 

이 같은 영상의 라이브 스트림 (동일한 인한하지 않는 것이

인코딩 프로세스) 괜찮아요, 문제는 muxer 자체에 있어야합니다 (또는 더 정확하게, 내가 처리하는 방식으로).

이

은 (크게 bigflake의 mediacodec samples에서 영감을) 내 먹서 래퍼입니다 :

나는 미디어 정보를 저장하고 사용하는 이유 RTMP의 먹서 동시에 작업을 시작하지 않을 수 있습니다 현지 먹서이 먹으 렴하는 것으로

import android.media.MediaCodec; 
import android.media.MediaFormat; 
import android.media.MediaMuxer; 
import android.util.Log; 

import java.io.IOException; 
import java.nio.ByteBuffer; 

/** 
* @hide 
*/ 
public class AndroidMuxer extends Muxer { 
    private static final String TAG = "AndroidMuxer"; 
    private static final boolean VERBOSE = false; 

    private MediaMuxer mMuxer; 
    private boolean mStarted = false; 
    private boolean mStopped = false; 
    private boolean mReadyToStart; 
    private MediaFormat mVideoTrackFormat; 
    private MediaFormat mAudioTrackFormat; 

    private long mStartTimeUs = 0; 
    @Override 
    public void reset() { 
     Log.w(TAG, "reset: Reset is called on LOCAL MUXER!"); 

      mStarted = false; 
      mReadyToStart = false; 
      mNumTracks = 0; 
      mNumTracksFinished = 0; 
      Log.w(TAG, "reset: Reset worked on the Muxer"); 

    } 

    /** 
    * This will only initialize the fields, we only want the real muxer to be initialized 
    * when we go live. 
    * @param outputFile outputPath of the file 
    * @param format Format of the muxer (will always be MPEG4 TBH) 
    */ 
    public AndroidMuxer(String outputFile, FORMAT format) { 
     super(outputFile, format); 
     mStarted = false; 
    } 

    @Override 
    public void init() { 
     if(!mReadyToStart) { 
      return; 
     } 

     if(mStarted){ 
      Log.i(TAG, "init: Muxer already created! Skipping!"); 
      return; 
     } 


     try { 
      switch (mFormat) { 
       case MPEG4: 
        mMuxer = new MediaMuxer(mOutputPath, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4); 
        int audioTrack = mMuxer.addTrack(mAudioTrackFormat); 
        Log.i(TAG, "init: Added audio track widh id " + audioTrack); 
        mAudioTrackIndex = audioTrack; 

        int videoTrack = mMuxer.addTrack(mVideoTrackFormat); 
        Log.i(TAG, "init: Added video track widh id " + videoTrack); 
        mVideoTrackIndex = videoTrack; 
        Log.w(TAG, "init: Muxer was successfully created! Starting to record"); 
        start(); 
        break; 
       default: 
        throw new IllegalArgumentException("Unrecognized format!"); 
      } 
     } catch (IOException e) { 
      throw new RuntimeException("MediaMuxer creation failed", e); 
     } 
    } 

    public static AndroidMuxer create(String outputFile, FORMAT format) { 
     return new AndroidMuxer(outputFile, format); 
    } 

    @Override 
    public int addTrack(MediaFormat trackFormat, AndroidEncoder.TrackType trackType) { 
     Log.w(TAG, "addTrack: Requested adding of track to class " + getClass().getSimpleName() + " of type " + trackType); 
     //Super method just keeps track of amount of tracks added 
     super.addTrack(trackFormat, trackType); 
     Log.w(TAG, "addTrack: Adding track for " + trackFormat.getString(MediaFormat.KEY_MIME)); 
     if (mStarted) 
      throw new RuntimeException("format changed twice"); 

     //What we do is store the media info for later use once the muxer is created. 
     //This is done to prevent file creation until the event goes live. 

     if(trackType == AndroidEncoder.TrackType.VIDEO){ 
      mVideoTrackFormat = trackFormat; 
     } 
     else if(trackType == AndroidEncoder.TrackType.AUDIO){ 
      mAudioTrackFormat = trackFormat; 
     } 



     if (allTracksAdded()) { 
      Log.w(TAG, "addTrack: All tracks added, marking Local muxer as ready!"); 
      markReadyToStart(); 
     } 
     Log.w(TAG, "addTrack: Track index to class " + getClass().getSimpleName() + " of type " + trackType + ", track saved for future use!"); 
     //Return dummy 
     return 1; 
    } 


    /** 
    * 
    * Marking the Muxer as ready to be started but not yet! 
    * 
    */ 
    private void markReadyToStart() { 
     mReadyToStart = true; 
    } 



    private void start() { 
     if(mStarted){ 
      Log.i(TAG, "start: Skipped start due to muxer already started"); 
     } 
     mStartTimeUs = 0; 
     mMuxer.start(); 
     Log.w(TAG, "start: Muxer started!"); 
     mStarted = true; 
    } 

    protected void stop() { 
     if(mStarted) { 
      Log.w(TAG, "stop: Calling stop to the muxer"); 
      mMuxer.stop(); 
      Log.w(TAG, "stop: muxer stopped!"); 
      mStarted = false; 
      mStopped = true; 
      mReadyToStart = false; 
     } 
    } 

    @Override 
    public void release() { 
     super.release(); 
     if(mStopped) { 
      mMuxer.release(); 
      mStopped = false; 
      mStarted = false; 
     } 
    } 

    @Override 
    public boolean isStarted() { 
     return mStarted; 
    } 

    @Override 
    public void writeSampleData(MediaCodec encoder, int trackIndex, int bufferIndex, ByteBuffer encodedData, MediaCodec.BufferInfo bufferInfo) { 
     if(!mStarted) { 

      return; 
     } 

     super.writeSampleData(encoder, trackIndex, bufferIndex, encodedData, bufferInfo); 
     if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) != 0) { 
      // MediaMuxer gets the codec config info via the addTrack command 
      if (VERBOSE) Log.d(TAG, "ignoring BUFFER_FLAG_CODEC_CONFIG"); 
      return; 
     } 

     if (bufferInfo.size == 0) { 
      if (VERBOSE) Log.d(TAG, "ignoring zero size buffer"); 
      return; 
     } 

     if (!mStarted) { 
      Log.w(TAG, "writeSampleData called before muxer started. Ignoring packet. Track index: " + trackIndex + " tracks added: " + mNumTracks); 
//   encoder.releaseOutputBuffer(bufferIndex, false); 
      return; 
     } 

//  bufferInfo.presentationTimeUs = getNextRelativePts(bufferInfo.presentationTimeUs, trackIndex); 


     if(mStartTimeUs == 0) { 
       mStartTimeUs = bufferInfo.presentationTimeUs; 
      } 
     bufferInfo.presentationTimeUs -= mStartTimeUs; 
     if(bufferInfo.presentationTimeUs < 0) { 
      bufferInfo.presentationTimeUs = 0; 
     } 
     bufferInfo.presentationTimeUs = getSafePts(bufferInfo.presentationTimeUs, trackIndex); 


     mMuxer.writeSampleData(trackIndex, encodedData, bufferInfo); 

//  encoder.releaseOutputBuffer(bufferIndex, false); 

     if (allTracksFinished()) { 
      Log.w(TAG, "writeSampleData: All tracks finished! Calling Stop"); 
      stop(); 
     } 
    } 

    @Override 
    public void forceStop() { 
     Log.w(TAG, "forceStop: ForceStop called!"); 
     stop(); 
    } 
} 

그것은 사용자가 muxer가 시작되도록 요청할 때만 가능합니다.

Answer 1

로컬 muxer가 준비되지 않은 상태에서 패킷 작성을 건너 뛰면 키 프레임 작성을 건너 뛸 수 있습니다. 나중에 키 프레임이 아닌 비디오 프레임은 다음 키 프레임에 도달 할 때까지 디코딩 할 수 없습니다 (키 프레임이 아닌 파일로 시작하는 경우 비디오 플레이어가 혼란 스러울 수 있음을 알고 있습니까?). 키 프레임 사이에 간격이 길면 적당한 시간 내에 키 프레임을 얻을 수 없습니다. (RTMP를 통해 스트리밍하는 경우에도 정기적으로 키 프레임이 있습니다.)

제 조언은 로컬 멀티플렉서가 준비되기 전에 생성 된 모든 패킷을 저장하고 해당 패킷을 작성하는 것입니다 일단 준비가되면 로컬 멀티플렉서로 보내거나 다음 키 프레임에 도달 할 때까지 패킷을 건너 뛰십시오.