Files changed (1) hide show
  1. config.json +54 -0
config.json ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "openbmb/MiniCPM-V-2_6",
3
+ "version": 2.6,
4
+ "architectures": [
5
+ "MiniCPMV"
6
+ ],
7
+ "auto_map": {
8
+ "AutoConfig": "configuration_minicpm.MiniCPMVConfig",
9
+ "AutoModel": "modeling_minicpmv.MiniCPMV",
10
+ "AutoModelForCausalLM": "modeling_minicpmv.MiniCPMV"
11
+ },
12
+ "attention_dropout": 0.0,
13
+ "bos_token_id": 151643,
14
+ "eos_token_id": 151645,
15
+ "hidden_act": "silu",
16
+ "hidden_size": 3584,
17
+ "initializer_range": 0.02,
18
+ "intermediate_size": 18944,
19
+ "max_position_embeddings": 32768,
20
+ "max_window_layers": 28,
21
+ "num_attention_heads": 28,
22
+ "num_hidden_layers": 28,
23
+ "num_key_value_heads": 4,
24
+ "rms_norm_eps": 1e-06,
25
+ "rope_theta": 1000000.0,
26
+ "sliding_window": 131072,
27
+ "tie_word_embeddings": false,
28
+ "torch_dtype": "bfloat16",
29
+ "transformers_version": "4.40.0",
30
+ "use_cache": true,
31
+ "use_sliding_window": false,
32
+ "vocab_size": 151666,
33
+ "batch_vision_input": true,
34
+ "drop_vision_last_layer": false,
35
+ "image_size": 448,
36
+ "model_type": "minicpmv",
37
+ "patch_size": 14,
38
+ "query_num": 64,
39
+ "slice_config": {
40
+ "max_slice_nums": 9,
41
+ "patch_size": 14,
42
+ "model_type": "minicpmv"
43
+ },
44
+ "slice_mode": true,
45
+ "vision_config": {
46
+ "hidden_size": 1152,
47
+ "image_size": 980,
48
+ "intermediate_size": 4304,
49
+ "model_type": "siglip",
50
+ "num_attention_heads": 16,
51
+ "num_hidden_layers": 27,
52
+ "patch_size": 14
53
+ }
54
+ }