MKV with AAC encoding does not play

[rgabbro]

I'm able to reproduce same issue of @ldexterldesign in Windows 10 using Vivaldi 3.3.2022.45. I'm trying to watch a media from my Emby server (the closed source upstream of Jellyfin, very similar) but I get no audio. Also I noticed that if I open another tab in Vivaldi while playing video, this is paused until Emby's tab is again in focus (this does't occur in other videos). The same media plays regularly on Firefox 74.0.1 and Edge 44.17763.831.0. This occurs with every file on my library that is both MKV and with AAC LC audio codec.

I filled also a bug report: VB-71716

Here are some info from my source file obtained using MediaInfo:

General
Unique ID                                : 222087178845798151663228484415287159673 (0xA7146F4F13911FF6A28BCB4674436B79)
Format                                   : Matroska
Format version                           : Version 4
File size                                : 1.65 GiB
Duration                                 : 3 h 8 min
Overall bit rate                         : 1 252 kb/s
Encoded date                             : UTC 2020-08-25 11:22:36
Writing application                      : mkvmerge v44.0.0 ('Domino') 64-bit
Writing library                          : libebml v1.3.10 + libmatroska v1.5.2

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : [email protected]
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 4 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 3 h 8 min
Bit rate                                 : 1 090 kb/s
Width                                    : 1 280 pixels
Height                                   : 528 pixels
Display aspect ratio                     : 2.40:1
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.067
Stream size                              : 1.44 GiB (87%)
Writing library                          : x264 core 142 r2479 dd79a61
Encoding settings                        : cabac=1 / ref=2 / deblock=1:-1:-1 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 / psy_rd=0.40:0.00 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=3 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=8 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=240 / keyint_min=24 / scenecut=40 / intra_refresh=0 / rc_lookahead=60 / rc=2pass / mbtree=1 / bitrate=1090 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=62500 / vbv_bufsize=78125 / nal_hrd=none / filler=0 / ip_ratio=1.40 / aq=1:1.00
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Audio
ID                                       : 2
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : A_AAC-2
Duration                                 : 3 h 8 min
Bit rate                                 : 160 kb/s
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 48.0 kHz
Frame rate                               : 46.875 FPS (1024 SPF)
Compression mode                         : Lossy
Delay relative to video                  : -43 ms
Stream size                              : 216 MiB (13%)
Title                                    : Stereo
Language                                 : Italian
Default                                  : Yes
Forced                                   : No

Text
ID                                       : 3
Format                                   : UTF-8
Codec ID                                 : S_TEXT/UTF8
Codec ID/Info                            : UTF-8 Plain Text
Duration                                 : 3 h 0 min
Bit rate                                 : 77 b/s
Count of elements                        : 2752
Stream size                              : 103 KiB (0%)
Title                                    : english
Language                                 : English
Default                                  : No
Forced                                   : No

Menu
00:00:00.000                             : :Prologue
00:06:04.447                             : :One
00:12:58.819                             : :82% Chance of Rain
00:43:22.474                             : :Light Showers
00:56:30.387                             : :Momentum
01:17:18.383                             : :Carmen
01:28:06.281                             : :Family Picnic
01:41:53.148                             : :One on One
02:02:59.789                             : :Wise Up
02:36:52.277                             : :The Kiss
02:45:21.244                             : :Rain
02:52:22.540                             : :So Now Then

modedit split from Mac forum

[rgabbro]

@Gwen-Dragon Here is the paste from vivaldi://media-internals:

{
  "15:0": {
    "id": "15:0",
    "properties": {
      "render_id": 15,
      "player_id": 0,
      "origin_url": "http://192.168.1.185:8096/",
      "kFrameUrl": "http://192.168.1.185:8096/web/index.html#!/item?id=9337&serverId=9836c92eeb7c43a59d16f5323d5eca03",
      "kFrameTitle": "Emby",
      "url": "http://192.168.1.185:8096/emby/videos/9337/stream.mkv?DeviceId=d0e6a1ba-9dd3-4b60-a047-d8c5a45dbd7a&MediaSourceId=1ca2aafbbd23452d6dd326bf1db8ae1e&Static=true&PlaySessionId=e5cdf7d941864e07bf210a6aa49be460&api_key=c0ee2496f0014c11a5e9a05614903cca",
      "kTotalBytes": 1771625278,
      "kIsStreaming": false,
      "kIsSingleOrigin": true,
      "kIsRangeHeaderSupported": true,
      "pipeline_state": "kPlaying",
      "info": "Selected MojoVideoDecoder for video decoding, config: codec: h264, profile: h264 high, level: not available, alpha_mode: is_opaque, coded size: [1280,528], visible rect: [0,0,1280,528], natural size: [1280,528], has extra data: true, encryption scheme: Unencrypted, rotation: 0°, flipped: 0, color space: {primaries:SMPTE170M, transfer:SMPTE170M, matrix:SMPTE170M, range:LIMITED}",
      "debug": "Warning, FFmpegDemuxer failed to create a valid/supported audio decoder configuration from muxed stream, config:codec: aac, profile: unknown, bytes_per_channel: 0, channel_layout: STEREO, channels: 2, samples_per_second: 48000, sample_format: Unknown sample format, bytes_per_frame: 0, seek_preroll: 0us, codec_delay: 0, has extra data: true, encryption scheme: Unencrypted, discard decoder delay: true",
      "kAudioTracks": [],
      "kVideoTracks": [
        {
          "alpha mode": "is_opaque",
          "codec": "h264",
          "coded size": "1280x528",
          "color space": "{primaries:SMPTE170M, transfer:SMPTE170M, matrix:SMPTE170M, range:LIMITED}",
          "encryption scheme": "Unencrypted",
          "has extra data": true,
          "hdr metadata": "unset",
          "natural size": "1280x528",
          "orientation": "0°",
          "profile": "h264 high",
          "visible rect": "0,0 1280x528"
        }
      ],
      "kMaxDuration": 11319.701,
      "kStartTime": 0.04,
      "kBitrate": 1252065,
      "event": "kPause",
      "kIsVideoDecryptingDemuxerStream": false,
      "kVideoDecoderName": "MojoVideoDecoder",
      "kIsPlatformVideoDecoder": true,
      "dimensions": "1280x528",
      "kResolution": "1280x528",
      "video_buffering_state": {
        "state": "BUFFERING_HAVE_ENOUGH"
      },
      "pipeline_buffering_state": {
        "for_suspended_start": false,
        "state": "BUFFERING_HAVE_ENOUGH"
      },
      "duration": 11319.701
    },
    "allEvents": [
      {
        "time": 0,
        "key": "origin_url",
        "value": "http://192.168.1.185:8096/"
      },
      {
        "time": 0.03800000000046566,
        "key": "kFrameUrl",
        "value": "http://192.168.1.185:8096/web/index.html#!/item?id=9337&serverId=9836c92eeb7c43a59d16f5323d5eca03"
      },
      {
        "time": 0.04800000000977889,
        "key": "kFrameTitle",
        "value": "Emby"
      },
      {
        "time": 0.44400000001769513,
        "key": "url",
        "value": "http://192.168.1.185:8096/emby/videos/9337/stream.mkv?DeviceId=d0e6a1ba-9dd3-4b60-a047-d8c5a45dbd7a&MediaSourceId=1ca2aafbbd23452d6dd326bf1db8ae1e&Static=true&PlaySessionId=e5cdf7d941864e07bf210a6aa49be460&api_key=c0ee2496f0014c11a5e9a05614903cca"
      },
      {
        "time": 48.820999999996275,
        "key": "kTotalBytes",
        "value": 1771625278
      },
      {
        "time": 48.837999999988824,
        "key": "kIsStreaming",
        "value": false
      },
      {
        "time": 48.8410000000149,
        "key": "kIsSingleOrigin",
        "value": true
      },
      {
        "time": 48.84200000000419,
        "key": "kIsRangeHeaderSupported",
        "value": true
      },
      {
        "time": 52.57500000001164,
        "key": "pipeline_state",
        "value": "kStarting"
      },
      {
        "time": 93.81199999997625,
        "key": "info",
        "value": "FFmpegDemuxer: created video stream, config codec: h264, profile: h264 high, level: not available, alpha_mode: is_opaque, coded size: [1280,528], visible rect: [0,0,1280,528], natural size: [1280,528], has extra data: true, encryption scheme: Unencrypted, rotation: 0°, flipped: 0, color space: {primaries:SMPTE170M, transfer:SMPTE170M, matrix:SMPTE170M, range:LIMITED}"
      },
      {
        "time": 93.93900000001304,
        "key": "debug",
        "value": "Warning, FFmpegDemuxer failed to create a valid/supported audio decoder configuration from muxed stream, config:codec: aac, profile: unknown, bytes_per_channel: 0, channel_layout: STEREO, channels: 2, samples_per_second: 48000, sample_format: Unknown sample format, bytes_per_frame: 0, seek_preroll: 0us, codec_delay: 0, has extra data: true, encryption scheme: Unencrypted, discard decoder delay: true"
      },
      {
        "time": 93.94699999998556,
        "key": "info",
        "value": "FFmpegDemuxer: skipping invalid or unsupported audio track"
      },
      {
        "time": 93.99499999999534,
        "key": "kAudioTracks",
        "value": []
      },
      {
        "time": 94.00599999999395,
        "key": "kVideoTracks",
        "value": [
          {
            "alpha mode": "is_opaque",
            "codec": "h264",
            "coded size": "1280x528",
            "color space": "{primaries:SMPTE170M, transfer:SMPTE170M, matrix:SMPTE170M, range:LIMITED}",
            "encryption scheme": "Unencrypted",
            "has extra data": true,
            "hdr metadata": "unset",
            "natural size": "1280x528",
            "orientation": "0°",
            "profile": "h264 high",
            "visible rect": "0,0 1280x528"
          }
        ]
      },
      {
        "time": 94.04800000000978,
        "key": "kMaxDuration",
        "value": 11319.701
      },
      {
        "time": 94.05099999997765,
        "key": "kStartTime",
        "value": 0.04
      },
      {
        "time": 94.05200000002515,
        "key": "kBitrate",
        "value": 1252065
      },
      {
        "time": 94.77000000001863,
        "key": "info",
        "value": "Failed to initialize DecryptingVideoDecoder"
      },
      {
        "time": 97.7960000000312,
        "key": "info",
        "value": "Effective playback rate changed from 0 to 1"
      },
      {
        "time": 98.08500000002095,
        "key": "event",
        "value": "kPlay"
      },
      {
        "time": 100.78100000001723,
        "key": "info",
        "value": "Starting Initialization of DXVAVDA"
      },
      {
        "time": 104.49300000001676,
        "key": "info",
        "value": "Using D3D11 device for DXVA"
      },
      {
        "time": 115.68699999997625,
        "key": "kIsVideoDecryptingDemuxerStream",
        "value": false
      },
      {
        "time": 115.69800000003306,
        "key": "kVideoDecoderName",
        "value": "MojoVideoDecoder"
      },
      {
        "time": 115.70100000000093,
        "key": "kIsPlatformVideoDecoder",
        "value": true
      },
      {
        "time": 115.74099999997998,
        "key": "info",
        "value": "Selected MojoVideoDecoder for video decoding, config: codec: h264, profile: h264 high, level: not available, alpha_mode: is_opaque, coded size: [1280,528], visible rect: [0,0,1280,528], natural size: [1280,528], has extra data: true, encryption scheme: Unencrypted, rotation: 0°, flipped: 0, color space: {primaries:SMPTE170M, transfer:SMPTE170M, matrix:SMPTE170M, range:LIMITED}"
      },
      {
        "time": 115.92499999998836,
        "key": "pipeline_state",
        "value": "kPlaying"
      },
      {
        "time": 538.5080000000307,
        "key": "dimensions",
        "value": "1280x528"
      },
      {
        "time": 538.5260000000126,
        "key": "kResolution",
        "value": "1280x528"
      },
      {
        "time": 559.0540000000037,
        "key": "video_buffering_state",
        "value": {
          "state": "BUFFERING_HAVE_ENOUGH"
        }
      },
      {
        "time": 559.130999999994,
        "key": "pipeline_buffering_state",
        "value": {
          "for_suspended_start": false,
          "state": "BUFFERING_HAVE_ENOUGH"
        }
      },
      {
        "time": 93.96299999998882,
        "key": "duration",
        "value": 11319.701
      },
      {
        "time": 2694.2420000000275,
        "key": "event",
        "value": "kPause"
      },
      {
        "time": 7820.258000000031,
        "key": "event",
        "value": "kPlay"
      },
      {
        "time": 14007.233000000007,
        "key": "event",
        "value": "kPause"
      }
    ],
    "lastRendered": 0,
    "firstTimestamp_": 349414.37
  }
}

And no, the video is not encoded, it's played in DirectPlay mode (here are the info):

Playback Info
Player:
Web Video Player
Play method:
DirectPlay
Protocol:
http
Stream type:
Video
Video Info
Video resolution:
1280x528
Dropped frames:
0
Corrupted frames:
0
Original Media Info
Container:
mkv
Bitrate:
1.3 Mbps
Video codec:
H264 High
Video bitrate:
1.3 Mbps
Video framerate:
23.976025 fps
Audio codec:
AAC LC
Audio bitrate:
192 kbps
Audio channels:
2
Audio sample rate:
48000 Hz

[ldexterldesign]

For context, this is my post that @rgabbro is referring to (forgive me if I've missed it but unsure why you didn't include this in your OP?)

Sincerely

[Pathduck]

From doing some basic testing here, Vivaldi simply does not support playback of AAC-LC in a MKV container, even when the video type is set to video/mp4 (as x-matroska is not supported by any current browser).

Plays audio in Chromium 85, Opera 70, even old Edge 44 (!)
Firefox 80 fails audio+video, probably because it's stricter with MIME types (as it should be).

Chromium log:

"FFmpegDemuxer: created video stream, config codec: h264, profile: h264 high, level: not available, alpha_mode: is_opaque, coded size: [852,480], visible rect: [0,0,852,480], natural size: [852,480], has extra data: true, encryption scheme: Unencrypted, rotation: 0°, flipped: 0, color space: {primaries:SMPTE170M, transfer:SMPTE170M, matrix:SMPTE170M, range:LIMITED}"
"FFmpegDemuxer: created audio stream, config codec: aac, profile: unknown, bytes_per_channel: 4, channel_layout: STEREO, channels: 2, samples_per_second: 48000, sample_format: Float 32-bit planar, bytes_per_frame: 8, seek_preroll: 0us, codec_delay: 0, has extra data: true, encryption scheme: Unencrypted, discard decoder delay: true"

Vivaldi log:

"FFmpegDemuxer: created video stream, config codec: h264, profile: h264 high, level: not available, alpha_mode: is_opaque, coded size: [852,480], visible rect: [0,0,852,480], natural size: [852,480], has extra data: true, encryption scheme: Unencrypted, rotation: 0°, flipped: 0, color space: {primaries:SMPTE170M, transfer:SMPTE170M, matrix:SMPTE170M, range:LIMITED}"
"Warning, FFmpegDemuxer failed to create a valid/supported audio decoder configuration from muxed stream, config:codec: aac, profile: unknown, bytes_per_channel: 0, channel_layout: STEREO, channels: 2, samples_per_second: 48000, sample_format: Unknown sample format, bytes_per_frame: 0, seek_preroll: 0us, codec_delay: 0, has extra data: true, encryption scheme: Unencrypted, discard decoder delay: true"
"FFmpegDemuxer: skipping invalid or unsupported audio track"

Honestly, we're always told that this is because Vivaldi does not support "paid codecs". I do not believe this as the other browsers use FFmpeg for decoding, and last I checked this was free and open-source. Maybe Vivaldi just needs to update the FFmpeg library embedded in vivaldi.dll?

[Pathduck]

@Gwen-Dragon said in MKV with AAC encoding does not play:

AAC encoded audio is currently not supported because codec license/issue.

Like I said, I don't believe that this a codec/license issue. If it was the case, then logically Vivaldi would not be able to playback AAC in other containers like MP4. These tests would not work but they do, at least for me:
https://www2.iis.fraunhofer.de/AAC/multichannel.html

Here are the files I tested with:
https://fex.net/s/fvrtyn3
Note audio playback in mp4+aac and mkv+mp3, but not mkv+aac.

[Pathduck]

@Gwen-Dragon You're welcome
Actually, I don't think this is such a big issue. But it's interesting to investigate.

Honestly, I think Firefox is actually doing the correct thing here, by refusing to play MKV with an incorrect type. The other browsers might work by sheer luck from the robustness of FFmpeg.

MKV is not really a "web format"; browser support is shoddy at best and relying on workarounds like setting an incorrect MIME type. People trying to stream MKV are better off converting to MP4 or WEBM to ensure support in browsers.

But I guess it would be good if Vivaldi at least behaved like Chromium as this is what users expect.

[rgabbro]

So I'm assuming that MKV container is not officially supported and this is an Emby issue?

[Pathduck]

@rgabbro Best you can do at this point is to reencode the file as MP4+AAC or convert during streaming if Emby supports it.

I'm starting to think I should report it as a bug as Chromium clearly is able to playback MKV+AAC and possibly Vivaldi just needs to update its FFmpeg decoder.

EDIT: Reported as VB-72011. @Gwen-Dragon Are you able to confirm?