GPAC MP4box 2.1-DEV-rev593-g007bf61a0 is vulnerable to Buffer Overflow in gf_hevc_read_sps_bs_internal function of media_tools/av_parsers.c:8261
Thanks for reporting your issue. Please make sure these boxes are checked before submitting your issue - thank you!

- [X] I looked for a similar issue and couldn't find any.
- [X] I tried with the latest version of GPAC. Installers available at http://gpac.io/downloads/gpac-nightly-builds/
- [X] I give enough information for contributors to reproduce my issue (meaningful title, github labels, platform and compiler, command-line ...). I can share files anonymously with this dropbox: https://www.mediafire.com/filedrop/filedrop_hosted.php?drop=eec9e058a9486fe4e99c33021481d9e1826ca9dbc242a6cfaab0fe95da5e5d95

Detailed guidelines: http://gpac.io/2013/07/16/how-to-file-a-bug-properly/

# Description

buffer overflow in gf_hevc_read_sps_bs_internal function of media_tools/av_parsers.c:8261

```c
	//sps_rep_format_idx = 0;
	if (multiLayerExtSpsFlag) {
		sps->update_rep_format_flag = gf_bs_read_int_log(bs, 1, "update_rep_format_flag");
		if (sps->update_rep_format_flag) {
			sps->rep_format_idx = gf_bs_read_int_log(bs, 8, "rep_format_idx");
			if (sps->rep_format_idx>15) {
				return -1;
			}
		} else {
			sps->rep_format_idx = vps->rep_format_idx[layer_id]; // overflow
		}
```


# Version info

latest version

```
MP4Box - GPAC version 2.1-DEV-rev593-g007bf61a0-master
(c) 2000-2022 Telecom Paris distributed under LGPL v2.1+ - http://gpac.io

Please cite our work in your research:
	GPAC Filters: https://doi.org/10.1145/3339825.3394929
	GPAC: https://doi.org/10.1145/1291233.1291452

GPAC Configuration: --enable-sanitizer
Features: GPAC_CONFIG_LINUX GPAC_64_BITS GPAC_HAS_IPV6 GPAC_HAS_SSL GPAC_HAS_SOCK_UN GPAC_MINIMAL_ODF GPAC_HAS_QJS GPAC_HAS_LINUX_DVB  GPAC_DISABLE_3D
```

# Reproduce

compile and run

```
./configure --enable-sanitizer
make
./MP4Box import -add poc_bof8.mov
```

Crash reported by sanitizer

```
[iso file] Unknown box type dvbs in parent stsd
[HEVC] Error parsing NAL unit type 16
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] Error parsing NAL unit type 32
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] Error parsing NAL unit type 33
[HEVC] Error parsing NAL unit type 16
Track Importing HEVC - Width -10 Height -20316159 FPS 25000/1000
[HEVC] Error parsing NAL Unit 8 (size 0 type 0 frame 0 last POC 0) - skipping
[HEVC] Error parsing NAL unit type 16
[HEVC] Error parsing NAL unit type 0
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] Error parsing NAL unit type 32
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] Error parsing NAL unit type 33
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] Error parsing NAL unit type 34
[Core] exp-golomb read failed, not enough bits in bitstream !
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] Wrong number of layer sets in VPS 5
[HEVC] Error parsing NAL unit type 32
[HEVC] Error parsing Video Param Set
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] Error parsing NAL unit type 32
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] Error parsing NAL unit type 33
[Core] exp-golomb read failed, not enough bits in bitstream !
[Core] exp-golomb read failed, not enough bits in bitstream !
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] Error parsing NAL unit type 34
[Core] exp-golomb read failed, not enough bits in bitstream !
[Core] exp-golomb read failed, not enough bits in bitstream !
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] Error parsing NAL unit type 34
[Core] exp-golomb read failed, not enough bits in bitstream !
[Core] exp-golomb read failed, not enough bits in bitstream !
[Core] exp-golomb read failed, not enough bits in bitstream !
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] sorry, 11 layers in VPS but only 4 supported
[HEVC] Error parsing NAL unit type 32
[HEVC] Error parsing Video Param Set
media_tools/av_parsers.c:8261:45: runtime error: index 45 out of bounds for type 'u32 [16]'
```

# POC

[poc_bof8.zip](https://github.com/gpac/gpac/files/10212064/poc_bof8.zip)

# Impact

Potentially causing DoS and RCE

# Credit 

Xdchase
buffer overflow in gf_hevc_read_sps_bs_internal function of media_tools/av_parsers.c:8261
This is a GitHub Issue
repo:gpac
owner:gpac
Title : buffer overflow in gf_hevc_read_sps_bs_internal function of media_tools/av_parsers.c:8261
Issue date:

--- start body ---
Thanks for reporting your issue. Please make sure these boxes are checked before submitting your issue - thank you!

- [X] I looked for a similar issue and couldn't find any.
- [X] I tried with the latest version of GPAC. Installers available at http://gpac.io/downloads/gpac-nightly-builds/
- [X] I give enough information for contributors to reproduce my issue (meaningful title, github labels, platform and compiler, command-line ...). I can share files anonymously with this dropbox: https://www.mediafire.com/filedrop/filedrop_hosted.php?drop=eec9e058a9486fe4e99c33021481d9e1826ca9dbc242a6cfaab0fe95da5e5d95

Detailed guidelines: http://gpac.io/2013/07/16/how-to-file-a-bug-properly/

# Description

buffer overflow in gf_hevc_read_sps_bs_internal function of media_tools/av_parsers.c:8261

```c
	//sps_rep_format_idx = 0;
	if (multiLayerExtSpsFlag) {
		sps->update_rep_format_flag = gf_bs_read_int_log(bs, 1, "update_rep_format_flag");
		if (sps->update_rep_format_flag) {
			sps->rep_format_idx = gf_bs_read_int_log(bs, 8, "rep_format_idx");
			if (sps->rep_format_idx>15) {
				return -1;
			}
		} else {
			sps->rep_format_idx = vps->rep_format_idx[layer_id]; // overflow
		}
```


# Version info

latest version

```
MP4Box - GPAC version 2.1-DEV-rev593-g007bf61a0-master
(c) 2000-2022 Telecom Paris distributed under LGPL v2.1+ - http://gpac.io

Please cite our work in your research:
	GPAC Filters: https://doi.org/10.1145/3339825.3394929
	GPAC: https://doi.org/10.1145/1291233.1291452

GPAC Configuration: --enable-sanitizer
Features: GPAC_CONFIG_LINUX GPAC_64_BITS GPAC_HAS_IPV6 GPAC_HAS_SSL GPAC_HAS_SOCK_UN GPAC_MINIMAL_ODF GPAC_HAS_QJS GPAC_HAS_LINUX_DVB  GPAC_DISABLE_3D
```

# Reproduce

compile and run

```
./configure --enable-sanitizer
make
./MP4Box import -add poc_bof8.mov
```

Crash reported by sanitizer

```
[iso file] Unknown box type dvbs in parent stsd
[HEVC] Error parsing NAL unit type 16
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] Error parsing NAL unit type 32
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] Error parsing NAL unit type 33
[HEVC] Error parsing NAL unit type 16
Track Importing HEVC - Width -10 Height -20316159 FPS 25000/1000
[HEVC] Error parsing NAL Unit 8 (size 0 type 0 frame 0 last POC 0) - skipping
[HEVC] Error parsing NAL unit type 16
[HEVC] Error parsing NAL unit type 0
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] Error parsing NAL unit type 32
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] Error parsing NAL unit type 33
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] Error parsing NAL unit type 34
[Core] exp-golomb read failed, not enough bits in bitstream !
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] Wrong number of layer sets in VPS 5
[HEVC] Error parsing NAL unit type 32
[HEVC] Error parsing Video Param Set
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] Error parsing NAL unit type 32
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] Error parsing NAL unit type 33
[Core] exp-golomb read failed, not enough bits in bitstream !
[Core] exp-golomb read failed, not enough bits in bitstream !
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] Error parsing NAL unit type 34
[Core] exp-golomb read failed, not enough bits in bitstream !
[Core] exp-golomb read failed, not enough bits in bitstream !
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] Error parsing NAL unit type 34
[Core] exp-golomb read failed, not enough bits in bitstream !
[Core] exp-golomb read failed, not enough bits in bitstream !
[Core] exp-golomb read failed, not enough bits in bitstream !
[Core] exp-golomb read failed, not enough bits in bitstream !
[HEVC] sorry, 11 layers in VPS but only 4 supported
[HEVC] Error parsing NAL unit type 32
[HEVC] Error parsing Video Param Set
media_tools/av_parsers.c:8261:45: runtime error: index 45 out of bounds for type 'u32 [16]'
```

# POC

[poc_bof8.zip](https://github.com/gpac/gpac/files/10212064/poc_bof8.zip)

# Impact

Potentially causing DoS and RCE

# Credit 

Xdchase
--- end body ---

A buffer overflow vulnerability in the gf_hevc_read_sps_bs_internal function of GPAC's media_tools/av_parsers.c can lead to potential Denial of Service (DoS) and Remote Code Execution (RCE). The vulnerability arises when the rep_format_idx is set without proper bounds checking, allowing an attacker to exploit this by providing a specially crafted input file, which can cause the application to crash or execute arbitrary code.