msluszniak commited on
Commit
3f4dd7d
·
verified ·
1 Parent(s): 2a9233a

Add the missing coreml/config.json; drop the image fp32 variant

Browse files

model.json points at {backend}/config.json, but this repo had no coreml/config.json at all, so the Core ML artifacts were published without a contract. Added, following the same shape as the published xnnpack config: a single variant with `components` mapping the image and text towers to their files.

The image fp32 variant is removed. Its text counterpart was already dropped, so the pair was mismatched, and fp16 is equivalent: relative L2 0.0026 and cosine 0.999997 against fp32 across 8 images. Core ML fp32 also cannot run on the ANE and measured slower everywhere.

Note the Core ML text tower takes two inputs (ids and mask) at [1,74] where the xnnpack text tower takes one at [1,77]; that difference is pre-existing and preserved here.

v0.9.0 and below are untouched.

coreml/clip_vit_base_patch32_image_coreml_fp32.pte DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:764f2ed8c91204d27f7c7679acdf074b6c118b7e51b1fd71fef995447f8a8594
3
- size 351813648
 
 
 
 
coreml/config.json ADDED
@@ -0,0 +1,75 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "$schema": "https://huggingface.co/software-mansion/react-native-executorch-spec/resolve/main/config.schema.json",
3
+ "model": "clip_vit_base_patch32",
4
+ "family": "clip",
5
+ "capabilities": [
6
+ "text-embedding",
7
+ "image-embedding"
8
+ ],
9
+ "backend": "coreml",
10
+ "license": "mit",
11
+ "tokenizer": "../tokenizer.json",
12
+ "variants": [
13
+ {
14
+ "file": null,
15
+ "components": {
16
+ "image": "clip_vit_base_patch32_image_coreml_fp16.pte",
17
+ "text": "clip_vit_base_patch32_text_coreml_fp16.pte"
18
+ },
19
+ "precision": "fp16",
20
+ "quantized": false,
21
+ "default": true,
22
+ "methods": {
23
+ "image": {
24
+ "inputs": [
25
+ {
26
+ "shape": [
27
+ 1,
28
+ 3,
29
+ 224,
30
+ 224
31
+ ],
32
+ "dtype": "float32"
33
+ }
34
+ ],
35
+ "outputs": [
36
+ {
37
+ "shape": [
38
+ 1,
39
+ 512
40
+ ],
41
+ "dtype": "float32"
42
+ }
43
+ ]
44
+ },
45
+ "text": {
46
+ "inputs": [
47
+ {
48
+ "shape": [
49
+ 1,
50
+ 74
51
+ ],
52
+ "dtype": "int64"
53
+ },
54
+ {
55
+ "shape": [
56
+ 1,
57
+ 74
58
+ ],
59
+ "dtype": "int64"
60
+ }
61
+ ],
62
+ "outputs": [
63
+ {
64
+ "shape": [
65
+ 1,
66
+ 512
67
+ ],
68
+ "dtype": "float32"
69
+ }
70
+ ]
71
+ }
72
+ }
73
+ }
74
+ ]
75
+ }