type
status
date
slug
summary
tags
category
icon
password
Property
Mar 18, 2023 02:12 AM
基础知识
MP4 简介
mp4 ⽂件格式⼜被称为 MPEG-4 Part 14,出自 MPEG-4 标准第 14 部分 。它是⼀种多媒体格式容器,⼴泛用于包装视频和⾳频数据流、海报、字幕和元数据等。(顺便⼀提,目前流行的视频编码格式 AVC/H264 定义在 MPEG-4 Part 10)。
mp4 ⽂件格式基于 Apple 公司的 QuickTime 格式,因此,QuickTime File Format Specification 也可以作为我们研究 mp4 的重要参考。
MP4 文件结构
- 在MP4文件中,数据被组织成一个个的"track",每个track包含一个或多个"chunk",每个chunk又包含多个"sample":
- Track:MP4文件可以包含一个或多个track,每个track代表不同的媒体类型,比如音频、视频或字幕等。每个track由一个唯一的ID来标识,并包含一些元数据,比如媒体类型、时长、码率等
- Chunk:每个track被分成一个个的chunk,每个chunk包含一定数量的sample。chunk是为了优化数据的读取而存在的,通过将大的数据块分成小的chunk,可以更快地从文件中读取所需数据。每个chunk包含一个唯一的ID和一些元数据,比如大小、偏移量等
- Sample:每个chunk又由多个sample组成,sample是最小的音频或视频单元。比如一个视频文件可以被分成多个chunk,每个chunk包含数秒的视频数据,每个chunk又被分成多个视频帧(sample),每个视频帧就是一个独立的图像帧。同样的,一个音频文件也可以被分成多个chunk,每个chunk包含数秒的音频数据,每个chunk又被分成多个音频采样(sample),每个采样就是一个独立的声音样本。每个sample包含一些元数据,比如采样率、位深度、编码格式等
- mp4 ⽂件由 box 组成,每个 box 分为 Header 和 Data。其中 Header 部分包含了 box 的类型和大小,Data 包含了子 box 或者数据,box 可以嵌套子 box。 下图是⼀个典型mp4文件的基本结构(可在 MP4Box 的 Box View 查看):
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fcd4d64d3-fed1-4432-bf12-6a5dfc65ce00%2FUntitled.png?table=block&id=d5bc9d8e-0a01-41f7-9a9b-db1994d5999e)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fb8bdf370-5ed0-4054-92a8-596803d95adb%2FUntitled.png?table=block&id=fca3b346-4395-49d9-a6df-eea965b41c81)
- MP4 ⽂件的基本组成单元是 box,也就是说 MP4 文件是由各种各样的 box 组成的,有 parent box,还有 children box。因此,这些 boxes 之间存在⼀定的层次关系,总结如下表所示,表中标记出了各个 box 必选或可选特性,√ 代表 box 必选:
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fb2ad8798-8c08-4845-8933-5ab2cb8149b9%2FUntitled.png?table=block&id=e095aff8-d126-4da5-9c72-50a061d0e686)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F5bba0f78-753b-4032-abed-eb0ed3005484%2FUntitled.png?table=block&id=e0ef93e8-63bd-4c41-8bcf-9eb1beffa0e2)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F20afa7df-f5df-4959-890e-6f85f82260df%2FUntitled.png?table=block&id=79ded40e-d218-4d35-88d2-0a8128f21eef)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fbe0c0b86-57d8-47f7-a5bf-9e113c4f0e36%2FUntitled.png?table=block&id=96e92701-0569-4ac0-adaf-8f0c11d833ae)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F512f30b3-34f7-4a33-b7ec-a56752935ef6%2FUntitled.png?table=block&id=a3c390d6-3013-4404-a7eb-47a63b611214)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fbe55a4f9-a82d-422a-b1fb-cc97b7d73be6%2FUntitled.png?table=block&id=1679efbb-5359-4c13-a940-fe3e557b9aee)
ftyp(File Type Box)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F13b438b2-7f3f-48c5-8935-676a58affb36%2FUntitled.png?table=block&id=e898e521-0cb7-4a50-ad69-296b9633ba09)
mdat(Media Data Box)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fbef2fac5-636e-4650-9b28-5d1b7a52233b%2FUntitled.png?table=block&id=276d5cc9-03e5-4e4b-9426-9df1da11b38d)
moov(Movie Box)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F2529a42d-f79f-469e-b518-cb9c43066a49%2FUntitled.png?table=block&id=0c817404-ab2c-4c21-8f34-a16ee74ef8ca)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F7ee79a9d-f9a4-4713-9612-300411872605%2FUntitled.png?table=block&id=4b4a195d-e005-4f27-b5f8-a31e96f26921)
Moov Insider
mp4 的媒体数据信息主要存放在 Moov Box 中,是我们需要分析的重点。moov 的主要组成部分如下:
- mvhd(Movie Header Box):
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F183b775a-088a-47e9-8e92-b40a0193d393%2FUntitled.png?table=block&id=238102aa-24e9-42cd-bcd1-0bc6bcca9628)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F323b4afe-9a2f-4355-a0d8-c9cdd37a34bb%2FUntitled.png?table=block&id=a4f6533e-f04f-4bee-bcac-810d45660436)
- udta(User Data Box):自定义数据
- track(Track Box):
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F34e75322-7366-4e3a-80d9-925ffe25aebb%2FUntitled.png?table=block&id=03969c15-0581-4d35-94b8-65635fd3c336)
- tkhd(Track Header Box):
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F21b59b56-4e47-438e-beea-fd3368a6404f%2FUntitled.png?table=block&id=127d00e5-47e3-4602-8eba-384a1a8a46f5)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Faec425af-0eb4-4345-9916-deb836412406%2FUntitled.png?table=block&id=91a8d60d-aa7c-41a8-b238-0237b1dd118b)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F0ae53571-a8ae-448d-be48-5a5ce406d4b3%2FUntitled.png?table=block&id=2798250b-a175-4279-a3cb-00fa5966a134)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F53814a63-1bc9-427d-94da-e945f1d7d5ea%2FUntitled.png?table=block&id=f715fbb2-50f3-48b7-9901-7203a123beb1)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fab789ddc-1bcb-4592-8e29-8c3ea79d870a%2FUntitled.png?table=block&id=d973f941-1aa2-404e-b858-15bd37a93712)
- mdia(Media Box):
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F120b3c55-17fa-4665-9b8f-8bfcd841122c%2FUntitled.png?table=block&id=1dafdd1f-22c9-45d4-a0c5-a556e1b23c2a)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fd24b53a0-b4b2-4eea-9487-67d92c271c4b%2FUntitled.png?table=block&id=102b14e1-6ba6-4e3a-889d-be70fcbed9bc)
- mdhd(Media Header Box):存放视频流创建时间,长度等信息
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F41b03111-c319-46ff-aa55-8cad454b8188%2FUntitled.png?table=block&id=fb35bf98-3576-4244-a6f3-332c33823877)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F6229f521-13ba-418b-b3c5-324ccaeff3e4%2FUntitled.png?table=block&id=b6ea2b39-4656-44d9-b3b0-589573f9e4e1)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Ffdc8b97a-ada7-4088-a0a6-1cdf0c47b3f7%2FUntitled.png?table=block&id=60a8b1e1-1caf-4c46-b9f8-9c77c0e099ee)
- hdlr(Handler Reference Box):媒体的播放过程信息
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fc2d947ad-fa7b-4e27-be27-8f308a39d015%2FUntitled.png?table=block&id=e76615f1-a259-4785-a4bb-f545ae8e59c8)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F9b60c6c6-6467-4237-8cda-405fe52e76ed%2FUntitled.png?table=block&id=f70188ee-4302-4abe-9cad-e07cfef2a989)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F35fa19e9-206a-4a7a-aab8-79c41a62cefe%2FUntitled.png?table=block&id=cf108d2b-71b0-4213-9417-4938aebbb357)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fe847cf43-61cc-4f84-ae35-6cd00123d2e9%2FUntitled.png?table=block&id=5dfe3f00-80e2-4d48-8886-1d7fb94fba28)
- minf(Media Information Box):
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F22a04500-ea0d-4d8e-a6ed-11ce2df741c9%2FUntitled.png?table=block&id=1e2c16ec-53b5-4250-bda2-aa0d475f7c89)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F5841cd48-6c1f-455f-a9b5-6902ef2eb874%2FUntitled.png?table=block&id=1337df63-b5a4-40a0-99ae-208ed0070480)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F1a611569-679a-441b-8609-fb03cc7e81d0%2FUntitled.png?table=block&id=53fcdcc2-5a9f-49e5-a862-8d98c2e26403)
Stbl Insider
- stbl(Sample Table Box):
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F26151d92-5850-421d-b109-364b7eb98fc7%2FUntitled.png?table=block&id=ff484d12-88f5-4e27-b9a2-9249e4767e40)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F31c2a758-1f76-4f92-bf9c-183931b8e280%2FUntitled.png?table=block&id=35ff20ac-d6a0-4202-9e75-3d5998914035)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F68272d50-03be-45ed-9ed2-664e08a64587%2FUntitled.png?table=block&id=04cb7af4-5811-42c0-825f-b98a05fa7493)
- stsd(Sample Description Box):存放解码必须的描述信息
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F38ec9707-0a96-4935-9f8d-4813b8215e0d%2FUntitled.png?table=block&id=b4480788-f812-47d6-8864-3c02501f3a3c)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fd319f1e9-a6c9-45e1-9bd5-621641fe918f%2FUntitled.png?table=block&id=b3dcbd43-e7b4-4197-b36f-e9ff69936b7c)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fd4dcbbd1-0680-4fc8-a164-b12d8638e5a6%2FUntitled.png?table=block&id=79870963-2c93-44bb-89ee-9e150545eebb)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fd9417931-ec87-4e57-b222-b4c47c65ebb5%2FUntitled.png?table=block&id=23c5d046-dd9b-44f3-9741-b1254ea605f0)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F4819510e-1dbb-4a9b-8d22-d7653f82d587%2FUntitled.png?table=block&id=862a6c93-74f2-4d88-a154-ed50f1119768)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F53670744-e8fb-436d-a085-0f2f895d3fc7%2FUntitled.png?table=block&id=100c94af-95b6-4754-94d5-3ac3f59a1244)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F745324fc-384a-4f5d-a266-871f9291400f%2FUntitled.png?table=block&id=33638fd9-0b68-4e9c-bfbb-06f2a7f2b0da)
- stts(Time-to-Sample Box):
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fe7cbd086-a34a-42de-b8aa-846a6346f453%2FUntitled.png?table=block&id=a869d977-1efb-4c8b-886c-7310a7e6befa)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fe9ecb49f-c5d8-4096-856d-168ad141a135%2FUntitled.png?table=block&id=fd0ef557-a93c-46f9-93ed-beffa4c34499)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F70250756-72ad-47f2-b4a7-a46505a6bdab%2FUntitled.png?table=block&id=321857ad-31e9-4fcb-bb03-13445d11fd87)
- stss(Sync Sample Box):同步sample表,存放关键帧列表,关键帧是为了⽀持随机访问
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fd3c5a799-b1cd-4227-81cb-7afbc59ad26d%2FUntitled.png?table=block&id=75cba0f9-cb4a-4eb8-bf9c-1c2ee73c2fa3)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fd9805859-6351-4727-a3f9-daf6b60342c7%2FUntitled.png?table=block&id=060ba32f-20d9-4d56-b3dc-baab9997fcb4)
- stsc(Sample-To-Chunk Box):
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F8582fec2-1c0c-40f7-a660-a0c5c5e1d7ca%2FUntitled.png?table=block&id=493c00c7-3c31-4915-8ae6-78452216b421)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F0af3a329-451f-4792-bb48-f78187177f99%2FUntitled.png?table=block&id=9e7d0978-b608-4f86-9cff-ef4ffa1d0264)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F00b5bcc9-bd0e-4b75-925c-2faee33135df%2FUntitled.png?table=block&id=0ab69a87-4e3a-4461-bccb-3cc9b43f4415)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fb8e4d5d3-2614-4c29-8be2-5dd287454079%2FUntitled.png?table=block&id=2d16972d-cf5d-4eaf-aac5-a8ede076ccec)
- stsz(Sample Size Box):指定了每个 sample 的 size。Sample Size Atom 包含两 sample 总数和⼀张包含了每个 sample size 的表
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F6066af2c-19a3-49af-9225-7e6be3528eae%2FUntitled.png?table=block&id=6e752edf-bba4-4798-841d-cd267e7f1d90)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F68c806fb-5490-4ab0-8a29-23e11cca5362%2FUntitled.png?table=block&id=ae926f8f-050b-4155-a0ef-9cc607a4cfa3)
- stco(Chunk Offset Box):
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F1544986e-022f-41ce-b318-9e35f6aa3276%2FUntitled.png?table=block&id=0ba179de-1724-4f8e-bc1f-86308977a438)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Ff7fd3392-e482-47b1-a4a5-46580671462b%2FUntitled.png?table=block&id=2a9db635-e53b-4ca4-96d1-8295fb6dc2c2)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fe47835cb-121e-4b72-af0c-b41d10e3ce3a%2FUntitled.png?table=block&id=2ade86fb-7bc1-4b19-b173-bf2553f7165d)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F55ddf2b4-d7a9-43a1-b998-5d3318ee61a0%2FUntitled.png?table=block&id=bd2c05f9-f981-4176-8990-d08999eb32d9)
如何计算 sample 偏移位置
上文提到通过 stco 并不能直接获取某个 sample 的偏移位置,下⾯举例说明如何获取某⼀个 pts 对应的 sample 在⽂件中的位置。
大体需要以下步骤:
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F063a0810-2756-4df3-9c49-9987672e98f3%2FUntitled.png?table=block&id=4870775a-c526-4d4f-b7d0-23798bf2794e)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F58f413db-a122-41ab-a96c-ab3870254ec2%2FUntitled.png?table=block&id=2dbc0429-2507-4932-b59e-42aaef529b55)
![notion image](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F759ccd95-9042-49d2-951a-ea9752b09cc5%2FUntitled.png?table=block&id=b7d616c2-e80b-4b48-8a81-b0a0e9c74286)
- 作者:OctalZero
- 链接:https://octalzero.com/article/f4698d4c-1b5f-40d0-9b17-fab4456395cc
- 声明:本文采用 CC BY-NC-SA 4.0 许可协议,转载请注明出处。
相关文章