snapshot of https://github.com/ggml-org/whisper.cpp.git @ da54572229bcf64ba367d96c7ef15770376c4280 (2026-09-15)

2026-09-15 17:23:35 +05:00
commit 948ef47106
1955 changed files with 719115 additions and 0 deletions
--- a/models/.gitignore
+++ b/models/.gitignore
@@ -0,0 +1 @@
+*.bin
--- a/models/README.md
+++ b/models/README.md
@@ -0,0 +1,111 @@
+## Whisper model files in custom `ggml` format
+
+The [original Whisper PyTorch models provided by OpenAI](https://github.com/openai/whisper/blob/main/whisper/__init__.py#L17-L30)
+are converted to custom `ggml` format in order to be able to load them in C/C++.
+Conversion is performed using the [convert-pt-to-ggml.py](convert-pt-to-ggml.py) script.
+
+There are three ways to obtain `ggml` models:
+
+### 1. Use [download-ggml-model.sh](download-ggml-model.sh) to download pre-converted models
+
+Example download:
+
+```text
+$ ./download-ggml-model.sh base.en
+Downloading ggml model base.en ...
+models/ggml-base.en.bin          100%[=============================================>] 141.11M  5.41MB/s    in 22s
+Done! Model 'base.en' saved in 'models/ggml-base.en.bin'
+You can now use it like this:
+
+  $ ./build/bin/whisper-cli -m models/ggml-base.en.bin -f samples/jfk.wav
+```
+
+### 2. Manually download pre-converted models
+
+`ggml` models are available from the following locations:
+
+- https://huggingface.co/ggerganov/whisper.cpp/tree/main
+
+### 3. Convert with [convert-pt-to-ggml.py](convert-pt-to-ggml.py)
+
+Download one of the [models provided by OpenAI](https://github.com/openai/whisper/blob/main/whisper/__init__.py#L17-L30) and generate the `ggml` files using the [convert-pt-to-ggml.py](convert-pt-to-ggml.py) script.
+
+Example conversion, assuming the original PyTorch files have been downloaded into `~/.cache/whisper`. Change `~/path/to/repo/whisper/` to the location for your copy of the Whisper source:
+
+```bash
+mkdir models/whisper-medium
+python models/convert-pt-to-ggml.py ~/.cache/whisper/medium.pt ~/path/to/repo/whisper/ ./models/whisper-medium
+mv ./models/whisper-medium/ggml-model.bin models/ggml-medium.bin
+rmdir models/whisper-medium
+```
+
+## Available models
+
+| Model               | Disk    | SHA                                        |
+| ------------------- | ------- | ------------------------------------------ |
+| tiny                | 75 MiB  | `bd577a113a864445d4c299885e0cb97d4ba92b5f` |
+| tiny.en             | 75 MiB  | `c78c86eb1a8faa21b369bcd33207cc90d64ae9df` |
+| base                | 142 MiB | `465707469ff3a37a2b9b8d8f89f2f99de7299dac` |
+| base.en             | 142 MiB | `137c40403d78fd54d454da0f9bd998f78703390c` |
+| small               | 466 MiB | `55356645c2b361a969dfd0ef2c5a50d530afd8d5` |
+| small.en            | 466 MiB | `db8a495a91d927739e50b3fc1cc4c6b8f6c2d022` |
+| small.en-tdrz       | 465 MiB | `b6c6e7e89af1a35c08e6de56b66ca6a02a2fdfa1` |
+| medium              | 1.5 GiB | `fd9727b6e1217c2f614f9b698455c4ffd82463b4` |
+| medium.en           | 1.5 GiB | `8c30f0e44ce9560643ebd10bbe50cd20eafd3723` |
+| large-v1            | 2.9 GiB | `b1caaf735c4cc1429223d5a74f0f4d0b9b59a299` |
+| large-v2            | 2.9 GiB | `0f4c8e34f21cf1a914c59d8b3ce882345ad349d6` |
+| large-v2-q5_0       | 1.1 GiB | `00e39f2196344e901b3a2bd5814807a769bd1630` |
+| large-v3            | 2.9 GiB | `ad82bf6a9043ceed055076d0fd39f5f186ff8062` |
+| large-v3-q5_0       | 1.1 GiB | `e6e2ed78495d403bef4b7cff42ef4aaadcfea8de` |
+| large-v3-turbo      | 1.5 GiB | `4af2b29d7ec73d781377bfd1758ca957a807e941` |
+| large-v3-turbo-q5_0 | 547 MiB | `e050f7970618a659205450ad97eb95a18d69c9ee` |
+
+Models are multilingual unless the model name includes `.en`. Models ending in `-q5_0` are [quantized](../README.md#quantization). Models ending in `-tdrz` support local diarization (marking of speaker turns) using [tinydiarize](https://github.com/akashmjn/tinydiarize). More information about models is available [upstream (openai/whisper)](https://github.com/openai/whisper#available-models-and-languages). The list above is a subset of the models supported by the [download-ggml-model.sh](download-ggml-model.sh) script, but many more are available at https://huggingface.co/ggerganov/whisper.cpp/tree/main and elsewhere.
+
+## Model files for testing purposes
+
+The model files prefixed with `for-tests-` are empty (i.e. do not contain any weights) and are used by the CI for
+testing purposes. They are directly included in this repository for convenience and the Github Actions CI uses them to
+run various sanitizer tests.
+
+## Fine-tuned models
+
+There are community efforts for creating fine-tuned Whisper models using extra training data. For example, this
+[blog post](https://huggingface.co/blog/fine-tune-whisper) describes a method for fine-tuning using Hugging Face (HF)
+Transformer implementation of Whisper. The produced models are in slightly different format compared to the original
+OpenAI format. To read the HF models you can use the [convert-h5-to-ggml.py](convert-h5-to-ggml.py) script like this:
+
+```bash
+git clone https://github.com/openai/whisper
+git clone https://github.com/ggml-org/whisper.cpp
+
+# clone HF fine-tuned model (this is just an example)
+git clone https://huggingface.co/openai/whisper-medium
+
+# convert the model to ggml
+python3 ./whisper.cpp/models/convert-h5-to-ggml.py ./whisper-medium/ ./whisper .
+```
+
+## Distilled models
+
+Initial support for https://huggingface.co/distil-whisper is available.
+
+Currently, the chunk-based transcription strategy is not implemented, so there can be sub-optimal quality when using the distilled models with `whisper.cpp`.
+
+```bash
+# clone OpenAI whisper and whisper.cpp
+git clone https://github.com/openai/whisper
+git clone https://github.com/ggml-org/whisper.cpp
+
+# get the models
+cd whisper.cpp/models
+git clone https://huggingface.co/distil-whisper/distil-medium.en
+git clone https://huggingface.co/distil-whisper/distil-large-v2
+
+# convert to ggml
+python3 ./convert-h5-to-ggml.py ./distil-medium.en/ ../../whisper .
+mv ggml-model.bin ggml-medium.en-distil.bin
+
+python3 ./convert-h5-to-ggml.py ./distil-large-v2/ ../../whisper .
+mv ggml-model.bin ggml-large-v2-distil.bin
+```
--- a/models/convert-h5-to-coreml.py
+++ b/models/convert-h5-to-coreml.py
@@ -0,0 +1,117 @@
+import argparse
+import importlib.util
+
+spec = importlib.util.spec_from_file_location('whisper_to_coreml', 'models/convert-whisper-to-coreml.py')
+whisper_to_coreml = importlib.util.module_from_spec(spec)
+spec.loader.exec_module(whisper_to_coreml)
+
+from whisper import load_model
+
+from copy import deepcopy
+import torch
+from transformers import WhisperForConditionalGeneration
+from huggingface_hub import metadata_update
+
+# https://github.com/bayartsogt-ya/whisper-multiple-hf-datasets/blob/main/src/multiple_datasets/hub_default_utils.py
+WHISPER_MAPPING = {
+    "layers": "blocks",
+    "fc1": "mlp.0",
+    "fc2": "mlp.2",
+    "final_layer_norm": "mlp_ln",
+    "layers": "blocks",
+    ".self_attn.q_proj": ".attn.query",
+    ".self_attn.k_proj": ".attn.key",
+    ".self_attn.v_proj": ".attn.value",
+    ".self_attn_layer_norm": ".attn_ln",
+    ".self_attn.out_proj": ".attn.out",
+    ".encoder_attn.q_proj": ".cross_attn.query",
+    ".encoder_attn.k_proj": ".cross_attn.key",
+    ".encoder_attn.v_proj": ".cross_attn.value",
+    ".encoder_attn_layer_norm": ".cross_attn_ln",
+    ".encoder_attn.out_proj": ".cross_attn.out",
+    "decoder.layer_norm.": "decoder.ln.",
+    "encoder.layer_norm.": "encoder.ln_post.",
+    "embed_tokens": "token_embedding",
+    "encoder.embed_positions.weight": "encoder.positional_embedding",
+    "decoder.embed_positions.weight": "decoder.positional_embedding",
+    "layer_norm": "ln_post",
+}
+
+# https://github.com/bayartsogt-ya/whisper-multiple-hf-datasets/blob/main/src/multiple_datasets/hub_default_utils.py
+def rename_keys(s_dict):
+    keys = list(s_dict.keys())
+    for key in keys:
+        new_key = key
+        for k, v in WHISPER_MAPPING.items():
+            if k in key:
+                new_key = new_key.replace(k, v)
+
+        print(f"{key} -> {new_key}")
+
+        s_dict[new_key] = s_dict.pop(key)
+    return s_dict
+
+# https://github.com/bayartsogt-ya/whisper-multiple-hf-datasets/blob/main/src/multiple_datasets/hub_default_utils.py
+def convert_hf_whisper(hf_model_name_or_path: str, whisper_state_path: str):
+    transformer_model = WhisperForConditionalGeneration.from_pretrained(hf_model_name_or_path)
+    config = transformer_model.config
+
+    # first build dims
+    dims = {
+        'n_mels': config.num_mel_bins,
+        'n_vocab': config.vocab_size,
+        'n_audio_ctx': config.max_source_positions,
+        'n_audio_state': config.d_model,
+        'n_audio_head': config.encoder_attention_heads,
+        'n_audio_layer': config.encoder_layers,
+        'n_text_ctx': config.max_target_positions,
+        'n_text_state': config.d_model,
+        'n_text_head': config.decoder_attention_heads,
+        'n_text_layer': config.decoder_layers
+    }
+
+    state_dict = deepcopy(transformer_model.model.state_dict())
+    state_dict = rename_keys(state_dict)
+
+    torch.save({"dims": dims, "model_state_dict": state_dict}, whisper_state_path)
+
+# Ported from models/convert-whisper-to-coreml.py
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--model-name", type=str, help="name of model to convert (e.g. tiny, tiny.en, base, base.en, small, small.en, medium, medium.en, large-v1, large-v2, large-v3, large-v3-turbo)", required=True)
+    parser.add_argument("--model-path", type=str, help="path to the model (e.g. if published on HuggingFace: Oblivion208/whisper-tiny-cantonese)", required=True)
+    parser.add_argument("--encoder-only", type=bool, help="only convert encoder", default=False)
+    parser.add_argument("--quantize",     type=bool, help="quantize weights to F16", default=False)
+    parser.add_argument("--optimize-ane", type=bool, help="optimize for ANE execution (currently broken)", default=False)
+    args = parser.parse_args()
+
+    if args.model_name not in ["tiny", "tiny.en", "base", "base.en", "small", "small.en", "medium", "medium.en", "large-v1", "large-v2", "large-v3", "large-v3-turbo"]:
+        raise ValueError("Invalid model name")
+
+    pt_target_path = f"models/hf-{args.model_name}.pt"
+    convert_hf_whisper(args.model_path, pt_target_path)
+
+    whisper = load_model(pt_target_path).cpu()
+    hparams = whisper.dims
+    print(hparams)
+
+    if args.optimize_ane:
+        whisperANE = whisper_to_coreml.WhisperANE(hparams).eval()
+        whisperANE.load_state_dict(whisper.state_dict())
+
+        encoder = whisperANE.encoder
+        decoder = whisperANE.decoder
+    else:
+        encoder = whisper.encoder
+        decoder = whisper.decoder
+
+    # Convert encoder
+    encoder = whisper_to_coreml.convert_encoder(hparams, encoder, quantize=args.quantize)
+    encoder.save(f"models/coreml-encoder-{args.model_name}.mlpackage")
+
+    if args.encoder_only is False:
+        # Convert decoder
+        decoder = whisper_to_coreml.convert_decoder(hparams, decoder, quantize=args.quantize)
+        decoder.save(f"models/coreml-decoder-{args.model_name}.mlpackage")
+
+    print("done converting")
--- a/models/convert-h5-to-ggml.py
+++ b/models/convert-h5-to-ggml.py
@@ -0,0 +1,218 @@
+# Convert Hugging Face fine-tuned models to ggml format
+#
+# Usage:
+#
+#   git clone https://github.com/openai/whisper
+#   git clone https://github.com/ggml-org/whisper.cpp
+#   git clone https://huggingface.co/openai/whisper-medium
+#
+#   python3 ./whisper.cpp/models/convert-h5-to-ggml.py ./whisper-medium/ ./whisper .
+#
+# This script is similar to "convert-pt-to-ggml.py"
+#
+# For more info:
+#
+#   https://github.com/ggml-org/whisper.cpp/issues/157
+#
+
+import io
+import os
+import sys
+import struct
+import json
+import code
+import torch
+import numpy as np
+from pathlib import Path
+
+from transformers import WhisperForConditionalGeneration
+
+conv_map = {
+        'self_attn.k_proj'              : 'attn.key',
+        'self_attn.q_proj'              : 'attn.query',
+        'self_attn.v_proj'              : 'attn.value',
+        'self_attn.out_proj'            : 'attn.out',
+        'self_attn_layer_norm'          : 'attn_ln',
+        'encoder_attn.q_proj'           : 'cross_attn.query',
+        'encoder_attn.v_proj'           : 'cross_attn.value',
+        'encoder_attn.out_proj'         : 'cross_attn.out',
+        'encoder_attn_layer_norm'       : 'cross_attn_ln',
+        'fc1'                           : 'mlp.0',
+        'fc2'                           : 'mlp.2',
+        'final_layer_norm'              : 'mlp_ln',
+        'encoder.layer_norm.bias'       : 'encoder.ln_post.bias',
+        'encoder.layer_norm.weight'     : 'encoder.ln_post.weight',
+        'encoder.embed_positions.weight': 'encoder.positional_embedding',
+        'decoder.layer_norm.bias'       : 'decoder.ln.bias',
+        'decoder.layer_norm.weight'     : 'decoder.ln.weight',
+        'decoder.embed_positions.weight': 'decoder.positional_embedding',
+        'decoder.embed_tokens.weight'   : 'decoder.token_embedding.weight',
+        'proj_out.weight'               : 'decoder.proj.weight',
+        }
+
+# ref: https://github.com/openai/gpt-2/blob/master/src/encoder.py
+def bytes_to_unicode():
+    """
+    Returns list of utf-8 byte and a corresponding list of unicode strings.
+    The reversible bpe codes work on unicode strings.
+    This means you need a large # of unicode characters in your vocab if you want to avoid UNKs.
+    When you're at something like a 10B token dataset you end up needing around 5K for decent coverage.
+    This is a significant percentage of your normal, say, 32K bpe vocab.
+    To avoid that, we want lookup tables between utf-8 bytes and unicode strings.
+    And avoids mapping to whitespace/control characters the bpe code barfs on.
+    """
+    bs = list(range(ord("!"), ord("~")+1))+list(range(ord("¡"), ord("¬")+1))+list(range(ord("®"), ord("ÿ")+1))
+    cs = bs[:]
+    n = 0
+    for b in range(2**8):
+        if b not in bs:
+            bs.append(b)
+            cs.append(2**8+n)
+            n += 1
+    cs = [chr(n) for n in cs]
+    return dict(zip(bs, cs))
+
+if len(sys.argv) < 4:
+    print("Usage: convert-h5-to-ggml.py dir_model path-to-whisper-repo dir-output [use-f32]\n")
+    sys.exit(1)
+
+dir_model   = Path(sys.argv[1])
+dir_whisper = Path(sys.argv[2])
+dir_out     = Path(sys.argv[3])
+
+encoder = json.load((dir_model / "vocab.json").open("r", encoding="utf8"))
+encoder_added = json.load((dir_model / "added_tokens.json").open( "r", encoding="utf8"))
+hparams = json.load((dir_model / "config.json").open("r", encoding="utf8"))
+
+# Add this block to handle missing 'max_length'
+if "max_length" not in hparams or hparams["max_length"] is None:
+    hparams["max_length"] = hparams.get("max_target_positions", 448)  # Default to 448 if missing
+elif not isinstance(hparams["max_length"], int):
+    try:
+        hparams["max_length"] = int(hparams["max_length"])  # Convert if necessary
+    except ValueError:
+        print(f"Warning: Invalid max_length value '{hparams['max_length']}', using default 448.")
+        hparams["max_length"] = 448
+        
+model = WhisperForConditionalGeneration.from_pretrained(dir_model)
+
+#code.interact(local=locals())
+
+n_mels = hparams["num_mel_bins"]
+with np.load(os.path.join(dir_whisper, "whisper/assets", "mel_filters.npz")) as f:
+    filters = torch.from_numpy(f[f"mel_{n_mels}"])
+
+dir_tokenizer = dir_model
+
+fname_out = dir_out / "ggml-model.bin"
+
+tokens = json.load(open(dir_tokenizer / "vocab.json", "r", encoding="utf8"))
+
+# use 16-bit or 32-bit floats
+use_f16 = True
+if len(sys.argv) > 4:
+    use_f16 = False
+    fname_out = dir_out / "ggml-model-f32.bin"
+
+fout = open(fname_out, "wb")
+
+fout.write(struct.pack("i", 0x67676d6c)) # magic: ggml in hex
+fout.write(struct.pack("i", hparams["vocab_size"]))
+fout.write(struct.pack("i", hparams["max_source_positions"]))
+fout.write(struct.pack("i", hparams["d_model"]))
+fout.write(struct.pack("i", hparams["encoder_attention_heads"]))
+fout.write(struct.pack("i", hparams["encoder_layers"]))
+fout.write(struct.pack("i", hparams["max_length"]))
+fout.write(struct.pack("i", hparams["d_model"]))
+fout.write(struct.pack("i", hparams["decoder_attention_heads"]))
+fout.write(struct.pack("i", hparams["decoder_layers"]))
+fout.write(struct.pack("i", hparams["num_mel_bins"]))
+fout.write(struct.pack("i", use_f16))
+
+fout.write(struct.pack("i", filters.shape[0]))
+fout.write(struct.pack("i", filters.shape[1]))
+for i in range(filters.shape[0]):
+    for j in range(filters.shape[1]):
+        fout.write(struct.pack("f", filters[i][j]))
+
+byte_encoder = bytes_to_unicode()
+byte_decoder = {v:k for k, v in byte_encoder.items()}
+
+fout.write(struct.pack("i", len(tokens)))
+
+tokens = sorted(tokens.items(), key=lambda x: x[1])
+for key in tokens:
+    text = bytearray([byte_decoder[c] for c in key[0]])
+    fout.write(struct.pack("i", len(text)))
+    fout.write(text)
+
+list_vars = model.state_dict()
+for name in list_vars.keys():
+    # this seems to not be used
+    # ref: https://github.com/huggingface/transformers/blob/9a5b84a0076a04fe9596da72e8668069d4f09ea0/src/transformers/models/whisper/modeling_whisper.py#L1099-L1106
+    if name == "proj_out.weight":
+        print('Skipping', name)
+        continue
+
+    src = name
+
+    nn = name
+    if name != "proj_out.weight":
+        nn = nn.split(".")[1:]
+    else:
+        nn = nn.split(".")
+
+    if nn[1] == "layers":
+        nn[1] = "blocks"
+        if ".".join(nn[3:-1]) == "encoder_attn.k_proj":
+            mapped = "attn.key" if nn[0] == "encoder" else "cross_attn.key"
+        else:
+            mapped = conv_map[".".join(nn[3:-1])]
+        name = ".".join(nn[:3] + [mapped] + nn[-1:])
+    else:
+        name = ".".join(nn)
+        name = conv_map[name] if name in conv_map else name
+
+    print(src, ' -> ', name)
+    data = list_vars[src].squeeze().numpy()
+    data = data.astype(np.float16)
+
+    # reshape conv bias from [n] to [n, 1]
+    if name in ["encoder.conv1.bias", "encoder.conv2.bias"]:
+        data = data.reshape(data.shape[0], 1)
+        print("  Reshaped variable: " , name , " to shape: ", data.shape)
+
+    n_dims = len(data.shape)
+    print(name, n_dims, data.shape)
+
+    # looks like the whisper models are in f16 by default
+    # so we need to convert the small tensors to f32 until we fully support f16 in ggml
+    # ftype == 0 -> float32, ftype == 1 -> float16
+    ftype = 1
+    if use_f16:
+        if n_dims < 2 or \
+                name == "encoder.conv1.bias"   or \
+                name == "encoder.conv2.bias"   or \
+                name == "encoder.positional_embedding" or \
+                name == "decoder.positional_embedding":
+            print("  Converting to float32")
+            data = data.astype(np.float32)
+            ftype = 0
+    else:
+        data = data.astype(np.float32)
+        ftype = 0
+
+    # header
+    str_ = name.encode('utf-8')
+    fout.write(struct.pack("iii", n_dims, len(str_), ftype))
+    for i in range(n_dims):
+        fout.write(struct.pack("i", data.shape[n_dims - 1 - i]))
+    fout.write(str_)
+
+    # data
+    data.tofile(fout)
+
+fout.close()
+
+print("Done. Output file: " , fname_out)
+print("")
--- a/models/convert-parakeet-to-ggml.py
+++ b/models/convert-parakeet-to-ggml.py
@@ -0,0 +1,337 @@
+#!/usr/bin/env python3
+# Convert Parakeet TDT model from NeMo format to ggml format
+#
+# Usage: python convert-parakeet-to-ggml.py --model parakeet-model.nemo --output-dir output-dir [--use-f32]
+#
+# The NeMo file is a tar archive containing:
+#   - model_weights.ckpt (PyTorch checkpoint)
+#   - model_config.yaml (model configuration)
+#   - tokenizer files
+#
+# This script extracts the NeMo archive, loads the model weights and configuration,
+# and saves them in ggml format compatible with whisper.cpp.
+#
+
+import torch
+import argparse
+import io
+import os
+import sys
+import struct
+import tarfile
+import tempfile
+import shutil
+import yaml
+import numpy as np
+from pathlib import Path
+from typing import Optional
+
+def hz_to_mel(freq):
+    return 2595.0 * np.log10(1.0 + freq / 700.0)
+
+def mel_to_hz(mel):
+    return 700.0 * (10.0**(mel / 2595.0) - 1.0)
+
+def extract_nemo_archive(nemo_path, extract_dir):
+    print(f"Extracting {nemo_path} to {extract_dir}")
+    with tarfile.open(nemo_path, 'r') as tar:
+        tar.extractall(path=extract_dir)
+    print("Extraction complete")
+
+def load_model_config(config_path):
+    with open(config_path, 'r', encoding='utf-8') as f:
+        config = yaml.safe_load(f)
+    return config
+
+def load_tokenizer(extract_dir, config):
+    tokenizer_model_path = None
+    tokenizer_vocab_path = None
+
+    for file in os.listdir(extract_dir):
+        if file.endswith('_tokenizer.model'):
+            tokenizer_model_path = os.path.join(extract_dir, file)
+        elif file.endswith('tokenizer.vocab'):
+            tokenizer_vocab_path = os.path.join(extract_dir, file)
+
+    if not tokenizer_model_path:
+        raise FileNotFoundError("Tokenizer model file not found")
+
+    if not tokenizer_vocab_path:
+        raise FileNotFoundError("Tokenizer vocab file not found")
+
+    tokens = {}
+    with open(tokenizer_vocab_path, 'r', encoding='utf-8') as f:
+        for idx, line in enumerate(f):
+            parts = line.strip().split('\t')
+            if len(parts) >= 1:
+                token = parts[0]
+                tokens[token.encode('utf-8')] = idx
+
+    print(f"Loaded {len(tokens)} tokens from {os.path.basename(tokenizer_vocab_path)}")
+
+    if len(tokens) != 8192:
+        print(f"WARNING: Expected 8192 tokens, got {len(tokens)}")
+
+    return tokens
+
+def write_tensor(fout, name, data, use_f16=True, force_f32=False):
+    if 'pre_encode.conv' in name and 'bias' in name and len(data.shape) == 1:
+        data = data.reshape(1, -1, 1, 1)
+        print(f"  Reshaped conv bias {name} to {data.shape}")
+
+    n_dims = len(data.shape)
+
+    ftype = 1 if use_f16 and not force_f32 else 0
+    if force_f32:
+        data = data.astype(np.float32)
+    elif use_f16:
+        if n_dims < 2 or 'bias' in name or 'norm' in name or \
+                ('pre_encode.conv' in name and n_dims == 4) or \
+                'depthwise_conv.weight' in name:
+            data = data.astype(np.float32)
+            ftype = 0
+        else:
+            data = data.astype(np.float16)
+    else:
+        data = data.astype(np.float32)
+
+    dims_reversed = [data.shape[n_dims - 1 - i] for i in range(n_dims)]
+    print(f"Processing: {name} {list(data.shape)}, dtype: {data.dtype}, n_dims: {n_dims}, reversed: {dims_reversed}")
+    name_bytes = name.encode('utf-8')
+    fout.write(struct.pack("iii", n_dims, len(name_bytes), ftype))
+    for i in range(n_dims):
+        fout.write(struct.pack("i", data.shape[n_dims - 1 - i]))
+    fout.write(name_bytes)
+
+    data.tofile(fout)
+
+def convert_parakeet_to_ggml(nemo_path, output_dir, use_f16=True, out_name=None):
+    nemo_path = Path(nemo_path)
+    output_dir = Path(output_dir)
+    output_dir.mkdir(parents=True, exist_ok=True)
+
+    # Create temporary directory for extraction
+    with tempfile.TemporaryDirectory() as temp_dir:
+        extract_nemo_archive(nemo_path, temp_dir)
+
+        config_path = os.path.join(temp_dir, 'model_config.yaml')
+        config = load_model_config(config_path)
+
+        print("Model configuration:")
+        print(f"  Sample rate: {config['sample_rate']}")
+        print(f"  Encoder layers: {config['encoder']['n_layers']}")
+        print(f"  Encoder d_model: {config['encoder']['d_model']}")
+        print(f"  Mel features: {config['preprocessor']['features']}")
+
+        weights_path = os.path.join(temp_dir, 'model_weights.ckpt')
+        print(f"\nLoading model weights from {weights_path}")
+        checkpoint = torch.load(weights_path, map_location='cpu')
+
+        # Extract state dict
+        if 'state_dict' in checkpoint:
+            state_dict = checkpoint['state_dict']
+        else:
+            state_dict = checkpoint
+
+        print(f"Loaded {len(state_dict)} tensors")
+
+        # Load tokenizer
+        print("\nLoading tokenizer...")
+        tokens = load_tokenizer(temp_dir, config)
+        print(f"Loaded {len(tokens)} tokens")
+
+        # Prepare hyperparameters for the Parakeet ggml format.
+        hparams = {
+            'n_audio_ctx': 5000,
+            'n_audio_state': config['encoder']['d_model'],
+            'n_audio_head': config['encoder']['n_heads'],
+            'n_audio_layer': config['encoder']['n_layers'],
+            'n_mels': config['preprocessor']['features'],
+            'n_fft': config['preprocessor']['n_fft'],
+            'subsampling_factor': config['encoder']['subsampling_factor'],
+            'n_subsampling_channels': config['encoder']['subsampling_conv_channels'],
+            'n_conv_kernel': config['encoder']['conv_kernel_size'],
+
+            'n_pred_dim': config['decoder']['prednet']['pred_hidden'],
+            'n_pred_layers': config['decoder']['prednet']['pred_rnn_layers'],
+            'n_vocab': config['decoder']['vocab_size'],
+            'n_tdt_durations': config['model_defaults']['num_tdt_durations'],
+            'n_max_tokens': config['decoding']['greedy']['max_symbols'],
+        }
+
+        print("\nGGML hyperparameters:")
+        for key, value in hparams.items():
+            print(f"  {key}: {value}")
+
+        # Create output file
+        if out_name:
+            fname_out = output_dir / out_name
+        else:
+            fname_out = output_dir / ("ggml-model-f32.bin" if not use_f16 else "ggml-model.bin")
+        print(f"\nWriting to {fname_out}")
+
+        with open(fname_out, 'wb') as fout:
+            # Write magic number
+            fout.write(struct.pack("i", 0x67676d6c))  # 'ggml' in hex
+
+            # Write hyperparameters
+            fout.write(struct.pack("i", hparams['n_vocab']))
+            fout.write(struct.pack("i", hparams['n_audio_ctx']))
+            fout.write(struct.pack("i", hparams['n_audio_state']))
+            fout.write(struct.pack("i", hparams['n_audio_head']))
+            fout.write(struct.pack("i", hparams['n_audio_layer']))
+            fout.write(struct.pack("i", hparams['n_mels']))
+            fout.write(struct.pack("i", 1 if use_f16 else 0))
+            fout.write(struct.pack("i", hparams['n_fft']))
+            fout.write(struct.pack("i", hparams['subsampling_factor']))
+            fout.write(struct.pack("i", hparams['n_subsampling_channels']))
+            fout.write(struct.pack("i", hparams['n_conv_kernel']))
+            fout.write(struct.pack("i", hparams['n_pred_dim']))
+            fout.write(struct.pack("i", hparams['n_pred_layers']))
+            fout.write(struct.pack("i", hparams['n_tdt_durations']))
+            fout.write(struct.pack("i", hparams['n_max_tokens']))
+
+            # Extract mel filterbank from model
+            fb_key = None
+            for key in state_dict.keys():
+                if 'featurizer.fb' in key or 'filterbank' in key.lower():
+                    fb_key = key
+                    break
+
+            if not fb_key:
+                print("\nERROR: Mel filterbank not found in model!")
+                print("Expected tensor with 'featurizer.fb' or 'filterbank' in name")
+                print("\nAvailable preprocessor tensors:")
+                for key in sorted(state_dict.keys()):
+                    if 'preprocessor' in key or 'featurizer' in key:
+                        print(f"  {key}: {state_dict[key].shape}")
+                raise ValueError("Mel filterbank tensor not found in model")
+
+            print(f"\nUsing model's mel filterbank from: {fb_key}")
+            mel_filters = state_dict[fb_key].squeeze().numpy().astype(np.float32)
+            print(f"  Filterbank shape: {mel_filters.shape}")
+            print(f"  Filterbank min/max values: {mel_filters.min():.6f} / {mel_filters.max():.6f}")
+            print(f"  Filterbank non-zero elements: {np.count_nonzero(mel_filters)} / {mel_filters.size}")
+            print(f"  First row sum: {mel_filters[0].sum():.6f}")
+
+            if len(mel_filters.shape) != 2:
+                raise ValueError(f"Expected 2D filterbank, got shape {mel_filters.shape}")
+
+            n_mels, n_freqs = mel_filters.shape
+            fout.write(struct.pack("i", n_mels))      # n_mel
+            fout.write(struct.pack("i", n_freqs))     # n_fb (frequency bins)
+
+            # Write mel filterbank
+            for i in range(n_mels):
+                for j in range(n_freqs):
+                    fout.write(struct.pack("f", mel_filters[i, j]))
+
+            # Extract window function from model
+            window_key = None
+            for key in state_dict.keys():
+                if 'featurizer.window' in key or 'preproc' in key and 'window' in key:
+                    window_key = key
+                    break
+
+            if not window_key:
+                print("\nERROR: Window function not found in model!")
+                print("Expected tensor with 'featurizer.window' in name")
+                raise ValueError("Window function tensor not found in model")
+
+            print(f"\nUsing model's window function from: {window_key}")
+            window = state_dict[window_key].squeeze().numpy().astype(np.float32)
+            print(f"  Window shape: {window.shape}")
+            print(f"  Window min/max values: {window.min():.6f} / {window.max():.6f}")
+            print(f"  Window non-zero elements: {np.count_nonzero(window)} / {window.size}")
+            print(f"  Window sum: {window.sum():.6f}")
+
+            if len(window.shape) != 1:
+                raise ValueError(f"Expected 1D window, got shape {window.shape}")
+
+            n_window = window.shape[0]
+            fout.write(struct.pack("i", n_window))
+
+            # Write window function
+            for i in range(n_window):
+                fout.write(struct.pack("f", window[i]))
+
+            # Write TDT durations
+            tdt_durations = config['model_defaults']['tdt_durations']
+            if len(tdt_durations) != hparams['n_tdt_durations']:
+                raise ValueError(f"TDT durations count mismatch: {len(tdt_durations)} vs {hparams['n_tdt_durations']}")
+
+            for duration in tdt_durations:
+                fout.write(struct.pack("I", duration))
+
+            fout.write(struct.pack("i", len(tokens)))
+            for token_bytes, idx in sorted(tokens.items(), key=lambda x: x[1]):
+                fout.write(struct.pack("i", len(token_bytes)))
+                fout.write(token_bytes)
+
+            # Pre-collect prediction LSTM input-hidden biases so they can be
+            # folded into the hidden-hidden bias during the main write loop.
+            lstm_prefix = 'decoder.prediction.dec_rnn.lstm'
+            pred_bias_ih = {}
+            for key, t in state_dict.items():
+                if f'{lstm_prefix}.bias_ih_l' in key:
+                    layer_idx = int(key.rsplit('bias_ih_l', 1)[1])
+                    pred_bias_ih[layer_idx] = t.squeeze().numpy().astype(np.float32)
+
+            print("\nConverting model weights...")
+            for name, tensor in state_dict.items():
+                # Skip the filterbank and window - already written in preprocessing section
+                if name == fb_key:
+                    continue
+                if name == window_key:
+                    continue
+
+                # bias_ih is folded into bias_hh below; skip writing it separately
+                if f'{lstm_prefix}.bias_ih_l' in name:
+                    continue
+
+                # Don't squeeze Conv2d weights - they need to preserve all 4 dimensions
+                if 'conv' in name and 'weight' in name and len(tensor.shape) == 4:
+                    data = tensor.numpy()
+                else:
+                    data = tensor.squeeze().numpy()
+
+                # For prediction LSTM weights/biases:
+                # Fold bias_ih into bias_hh (bias_ih already skipped above).
+                # Reorder gates (input, forget, cell, output) from PyTorch layout
+                # [i, f, g, o] to [i, f, o, g] so the three sigmoid-gated outputs
+                # (i, f, o) are contiguous.
+                if name.startswith(f'{lstm_prefix}.'):
+                    if f'{lstm_prefix}.bias_hh_l' in name:
+                        layer_idx = int(name.rsplit('bias_hh_l', 1)[1])
+                        data = data.astype(np.float32) + pred_bias_ih[layer_idx]
+                        name = name.replace('bias_hh_l', 'bias_h_l')
+                    h = data.shape[0] // 4
+                    data = np.concatenate([data[:h], data[h:2*h], data[3*h:], data[2*h:3*h]], axis=0)
+
+                write_tensor(fout, name, data, use_f16=use_f16)
+
+        print(f"\nConversion complete!")
+        print(f"Output file: {fname_out}")
+        print(f"File size: {fname_out.stat().st_size / (1024**2):.2f} MB")
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(
+        description='Convert Parakeet TDT model from NeMo format to ggml format'
+    )
+    parser.add_argument('--model', type=str, required=True,
+                        help='Path to Parakeet .nemo model file')
+    parser.add_argument('--out-dir', type=str, required=True,
+                        help='Directory to write ggml model file')
+    parser.add_argument('--use-f32', action='store_true', default=False,
+                        help='Use f32 instead of f16 (default: f16)')
+    parser.add_argument('--out-name', type=str, default=None,
+                        help='Output file name (default: ggml-model.bin or ggml-model-f32.bin)')
+
+    args = parser.parse_args()
+
+    if not os.path.exists(args.model):
+        print(f"Error: {args.model} not found")
+        sys.exit(1)
+
+    use_f16 = not args.use_f32
+    convert_parakeet_to_ggml(args.model, args.out_dir, use_f16, args.out_name)
--- a/models/convert-pt-to-ggml.py
+++ b/models/convert-pt-to-ggml.py
@@ -0,0 +1,342 @@
+# Convert Whisper transformer model from PyTorch to ggml format
+#
+# Usage: python convert-pt-to-ggml.py ~/.cache/whisper/medium.pt ~/path/to/repo/whisper/ ./models/whisper-medium
+#
+# You need to clone the original repo in ~/path/to/repo/whisper/
+#
+#  git clone https://github.com/openai/whisper ~/path/to/repo/whisper/
+#
+# It is used to various assets needed by the algorithm:
+#
+#  - tokenizer
+#  - mel filters
+#
+# Also, you need to have the original models in ~/.cache/whisper/
+# See the original repo for more details.
+#
+# This script loads the specified model and whisper assets and saves them in ggml format.
+# The output is a single binary file containing the following information:
+#
+#  - hparams
+#  - mel filters
+#  - tokenizer vocab
+#  - model variables
+#
+# For each variable, write the following:
+#
+#  - Number of dimensions (int)
+#  - Name length (int)
+#  - Dimensions (int[n_dims])
+#  - Name (char[name_length])
+#  - Data (float[n_dims])
+#
+
+import io
+import os
+import sys
+import struct
+import json
+import code
+import torch
+import numpy as np
+import base64
+from pathlib import Path
+#from transformers import GPTJForCausalLM
+#from transformers import GPT2TokenizerFast
+
+# ref: https://github.com/openai/whisper/blob/8cf36f3508c9acd341a45eb2364239a3d81458b9/whisper/tokenizer.py#L10-L110
+#LANGUAGES = {
+#    "en": "english",
+#    "zh": "chinese",
+#    "de": "german",
+#    "es": "spanish",
+#    "ru": "russian",
+#    "ko": "korean",
+#    "fr": "french",
+#    "ja": "japanese",
+#    "pt": "portuguese",
+#    "tr": "turkish",
+#    "pl": "polish",
+#    "ca": "catalan",
+#    "nl": "dutch",
+#    "ar": "arabic",
+#    "sv": "swedish",
+#    "it": "italian",
+#    "id": "indonesian",
+#    "hi": "hindi",
+#    "fi": "finnish",
+#    "vi": "vietnamese",
+#    "iw": "hebrew",
+#    "uk": "ukrainian",
+#    "el": "greek",
+#    "ms": "malay",
+#    "cs": "czech",
+#    "ro": "romanian",
+#    "da": "danish",
+#    "hu": "hungarian",
+#    "ta": "tamil",
+#    "no": "norwegian",
+#    "th": "thai",
+#    "ur": "urdu",
+#    "hr": "croatian",
+#    "bg": "bulgarian",
+#    "lt": "lithuanian",
+#    "la": "latin",
+#    "mi": "maori",
+#    "ml": "malayalam",
+#    "cy": "welsh",
+#    "sk": "slovak",
+#    "te": "telugu",
+#    "fa": "persian",
+#    "lv": "latvian",
+#    "bn": "bengali",
+#    "sr": "serbian",
+#    "az": "azerbaijani",
+#    "sl": "slovenian",
+#    "kn": "kannada",
+#    "et": "estonian",
+#    "mk": "macedonian",
+#    "br": "breton",
+#    "eu": "basque",
+#    "is": "icelandic",
+#    "hy": "armenian",
+#    "ne": "nepali",
+#    "mn": "mongolian",
+#    "bs": "bosnian",
+#    "kk": "kazakh",
+#    "sq": "albanian",
+#    "sw": "swahili",
+#    "gl": "galician",
+#    "mr": "marathi",
+#    "pa": "punjabi",
+#    "si": "sinhala",
+#    "km": "khmer",
+#    "sn": "shona",
+#    "yo": "yoruba",
+#    "so": "somali",
+#    "af": "afrikaans",
+#    "oc": "occitan",
+#    "ka": "georgian",
+#    "be": "belarusian",
+#    "tg": "tajik",
+#    "sd": "sindhi",
+#    "gu": "gujarati",
+#    "am": "amharic",
+#    "yi": "yiddish",
+#    "lo": "lao",
+#    "uz": "uzbek",
+#    "fo": "faroese",
+#    "ht": "haitian creole",
+#    "ps": "pashto",
+#    "tk": "turkmen",
+#    "nn": "nynorsk",
+#    "mt": "maltese",
+#    "sa": "sanskrit",
+#    "lb": "luxembourgish",
+#    "my": "myanmar",
+#    "bo": "tibetan",
+#    "tl": "tagalog",
+#    "mg": "malagasy",
+#    "as": "assamese",
+#    "tt": "tatar",
+#    "haw": "hawaiian",
+#    "ln": "lingala",
+#    "ha": "hausa",
+#    "ba": "bashkir",
+#    "jw": "javanese",
+#    "su": "sundanese",
+#}
+
+## ref: https://github.com/openai/whisper/blob/8cf36f3508c9acd341a45eb2364239a3d81458b9/whisper/tokenizer.py#L273-L292
+#def build_tokenizer(path_to_whisper_repo: str, name: str = "gpt2"):
+#    os.environ["TOKENIZERS_PARALLELISM"] = "false"
+#    path = os.path.join(path_to_whisper_repo, "whisper/assets", name)
+#    tokenizer = GPT2TokenizerFast.from_pretrained(path)
+#
+#    specials = [
+#        "<|startoftranscript|>",
+#        *[f"<|{lang}|>" for lang in LANGUAGES.keys()],
+#        "<|translate|>",
+#        "<|transcribe|>",
+#        "<|startoflm|>",
+#        "<|startofprev|>",
+#        "<|nocaptions|>",
+#        "<|notimestamps|>",
+#    ]
+#
+#    tokenizer.add_special_tokens(dict(additional_special_tokens=specials))
+#    return tokenizer
+
+# ref: https://github.com/openai/gpt-2/blob/master/src/encoder.py
+def bytes_to_unicode():
+    """
+    Returns list of utf-8 byte and a corresponding list of unicode strings.
+    The reversible bpe codes work on unicode strings.
+    This means you need a large # of unicode characters in your vocab if you want to avoid UNKs.
+    When you're at something like a 10B token dataset you end up needing around 5K for decent coverage.
+    This is a signficant percentage of your normal, say, 32K bpe vocab.
+    To avoid that, we want lookup tables between utf-8 bytes and unicode strings.
+    And avoids mapping to whitespace/control characters the bpe code barfs on.
+    """
+    bs = list(range(ord("!"), ord("~")+1))+list(range(ord("¡"), ord("¬")+1))+list(range(ord("®"), ord("ÿ")+1))
+    cs = bs[:]
+    n = 0
+    for b in range(2**8):
+        if b not in bs:
+            bs.append(b)
+            cs.append(2**8+n)
+            n += 1
+    cs = [chr(n) for n in cs]
+    return dict(zip(bs, cs))
+
+
+if len(sys.argv) < 4:
+    print("Usage: convert-pt-to-ggml.py model.pt path-to-whisper-repo dir-output [use-f32]\n")
+    sys.exit(1)
+
+fname_inp   = Path(sys.argv[1])
+dir_whisper = Path(sys.argv[2])
+dir_out     = Path(sys.argv[3])
+
+# try to load PyTorch binary data
+try:
+    model_bytes = open(fname_inp, "rb").read()
+    with io.BytesIO(model_bytes) as fp:
+        checkpoint = torch.load(fp, map_location="cpu")
+except Exception:
+    print("Error: failed to load PyTorch model file:" , fname_inp)
+    sys.exit(1)
+
+hparams = checkpoint["dims"]
+print("hparams:", hparams)
+
+list_vars = checkpoint["model_state_dict"]
+
+#print(list_vars['encoder.positional_embedding'])
+#print(list_vars['encoder.conv1.weight'])
+#print(list_vars['encoder.conv1.weight'].shape)
+
+# load mel filters
+n_mels = hparams["n_mels"]
+with np.load(dir_whisper / "whisper" / "assets" / "mel_filters.npz") as f:
+    filters = torch.from_numpy(f[f"mel_{n_mels}"])
+    #print (filters)
+
+#code.interact(local=locals())
+
+# load tokenizer
+# for backwards compatibility, also check for older hf_transformers format tokenizer files
+# old format: dir_whisper/whisper/assets/[multilingual/gpt2]/vocab.json
+# new format: dir_whisper/whisper/assets/[multilingual/gpt2].tiktoken
+multilingual = hparams["n_vocab"] >= 51865
+tokenizer = dir_whisper / "whisper" / "assets" / (multilingual and "multilingual.tiktoken" or "gpt2.tiktoken")
+tokenizer_type = "tiktoken"
+if not tokenizer.is_file():
+    tokenizer = dir_whisper / "whisper" / "assets" / (multilingual and "multilingual" or "gpt2") / "vocab.json"
+    tokenizer_type = "hf_transformers"
+    if not tokenizer.is_file():
+        print("Error: failed to find either tiktoken or hf_transformers tokenizer file:", tokenizer)
+        sys.exit(1)
+
+byte_encoder = bytes_to_unicode()
+byte_decoder = {v:k for k, v in byte_encoder.items()}
+
+if tokenizer_type == "tiktoken":
+    with open(tokenizer, "rb") as f:
+        contents = f.read()
+        tokens = {base64.b64decode(token): int(rank) for token, rank in (line.split() for line in contents.splitlines() if line)}
+elif tokenizer_type == "hf_transformers":
+    with open(tokenizer, "r", encoding="utf8") as f:
+        _tokens_raw = json.load(f)
+        if '<|endoftext|>' in _tokens_raw:
+            # ensures exact same model as tokenizer_type == tiktoken
+            # details: https://github.com/ggerganov/whisper.cpp/pull/725
+            del _tokens_raw['<|endoftext|>']
+        tokens = {bytes([byte_decoder[c] for c in token]): int(idx) for token, idx in _tokens_raw.items()}
+
+# output in the same directory as the model
+fname_out = dir_out / "ggml-model.bin"
+
+# use 16-bit or 32-bit floats
+use_f16 = True
+if len(sys.argv) > 4:
+    use_f16 = False
+    fname_out = dir_out / "ggml-model-f32.bin"
+
+fout = fname_out.open("wb")
+
+fout.write(struct.pack("i", 0x67676d6c)) # magic: ggml in hex
+fout.write(struct.pack("i", hparams["n_vocab"]))
+fout.write(struct.pack("i", hparams["n_audio_ctx"]))
+fout.write(struct.pack("i", hparams["n_audio_state"]))
+fout.write(struct.pack("i", hparams["n_audio_head"]))
+fout.write(struct.pack("i", hparams["n_audio_layer"]))
+fout.write(struct.pack("i", hparams["n_text_ctx"]))
+fout.write(struct.pack("i", hparams["n_text_state"]))
+fout.write(struct.pack("i", hparams["n_text_head"]))
+fout.write(struct.pack("i", hparams["n_text_layer"]))
+fout.write(struct.pack("i", hparams["n_mels"]))
+fout.write(struct.pack("i", use_f16))
+
+# write mel filters
+fout.write(struct.pack("i", filters.shape[0]))
+fout.write(struct.pack("i", filters.shape[1]))
+for i in range(filters.shape[0]):
+    for j in range(filters.shape[1]):
+        fout.write(struct.pack("f", filters[i][j]))
+
+# write tokenizer
+fout.write(struct.pack("i", len(tokens)))
+
+for key in tokens:
+    fout.write(struct.pack("i", len(key)))
+    fout.write(key)
+
+for name in list_vars.keys():
+    data = list_vars[name].squeeze().numpy()
+    print("Processing variable: " , name ,  " with shape: ", data.shape)
+
+    # reshape conv bias from [n] to [n, 1]
+    if name in ["encoder.conv1.bias", "encoder.conv2.bias"]:
+        data = data.reshape(data.shape[0], 1)
+        print(f"  Reshaped variable: {name} to shape: ", data.shape)
+
+    n_dims = len(data.shape)
+
+    # looks like the whisper models are in f16 by default
+    # so we need to convert the small tensors to f32 until we fully support f16 in ggml
+    # ftype == 0 -> float32, ftype == 1 -> float16
+    ftype = 1
+    if use_f16:
+        if n_dims < 2 or \
+                name == "encoder.conv1.bias"   or \
+                name == "encoder.conv2.bias"   or \
+                name == "encoder.positional_embedding" or \
+                name == "decoder.positional_embedding":
+            print("  Converting to float32")
+            data = data.astype(np.float32)
+            ftype = 0
+    else:
+        data = data.astype(np.float32)
+        ftype = 0
+
+    #if name.startswith("encoder"):
+    #    if name.endswith("mlp.0.weight") or \
+    #       name.endswith("mlp.2.weight"):
+    #        print("  Transposing")
+    #        data = data.transpose()
+
+    # header
+    str_ = name.encode('utf-8')
+    fout.write(struct.pack("iii", n_dims, len(str_), ftype))
+    for i in range(n_dims):
+        fout.write(struct.pack("i", data.shape[n_dims - 1 - i]))
+    fout.write(str_)
+
+    # data
+    data.tofile(fout)
+
+fout.close()
+
+print("Done. Output file: " , fname_out)
+print("")
--- a/models/convert-silero-vad-to-ggml.py
+++ b/models/convert-silero-vad-to-ggml.py
@@ -0,0 +1,196 @@
+import os
+import struct
+import argparse
+import torch
+import numpy as np
+from silero_vad import load_silero_vad, __version__ as silero_version
+
+def convert_silero_vad(output_path, print_tensors=True):
+    model = load_silero_vad()
+    state_dict = model.state_dict()
+
+    # Clean up state dict keys - filter out 8k model
+    cleaned_dict = {}
+    for key, value in state_dict.items():
+        # Skip 8k model
+        if "_8k" not in key:
+            clean_key = key
+            if not key.startswith("_model."):
+                clean_key = "_model." + key
+            cleaned_dict[clean_key] = value
+
+    base, ext = os.path.splitext(output_path)
+    output_file = f"{base}-v{silero_version}-ggml{ext}"
+    print(f"Saving GGML Silero-VAD model to {output_file}")
+
+    print("\nTensor info for debugging:")
+    for key, tensor in cleaned_dict.items():
+        print(f"  - {key}: {tensor.shape} ({tensor.dtype})")
+    print()
+
+    with open(output_file, "wb") as fout:
+        # Write magic and version
+        fout.write(struct.pack("i", 0x67676d6c))
+
+        model_type = "silero-16k"
+        str_len = len(model_type)
+        fout.write(struct.pack("i", str_len))
+        fout.write(model_type.encode('utf-8'))
+
+        version_parts = silero_version.split('.')
+        major, minor, patch = map(int, version_parts)
+        print(f"Version: {major}.{minor}.{patch}")
+        fout.write(struct.pack("i", major))
+        fout.write(struct.pack("i", minor))
+        fout.write(struct.pack("i", patch))
+
+        # Write model architecture parameters
+        window_size = 512
+        fout.write(struct.pack("i", window_size))
+        context_size = 64
+        fout.write(struct.pack("i", context_size))
+
+        n_encoder_layers = 4
+        fout.write(struct.pack("i", n_encoder_layers))
+
+        # Write encoder dimensions
+        input_channels = 129
+        encoder_in_channels = [input_channels, 128, 64, 64]
+        encoder_out_channels = [128, 64, 64, 128]
+        kernel_size = 3
+
+        for i in range(n_encoder_layers):
+            fout.write(struct.pack("i", encoder_in_channels[i]))
+            fout.write(struct.pack("i", encoder_out_channels[i]))
+            fout.write(struct.pack("i", kernel_size))
+
+        # Write LSTM dimensions
+        lstm_input_size = 128
+        lstm_hidden_size = 128
+        fout.write(struct.pack("i", lstm_input_size))
+        fout.write(struct.pack("i", lstm_hidden_size))
+
+        # Write final conv dimensions
+        final_conv_in = 128
+        final_conv_out = 1
+        fout.write(struct.pack("i", final_conv_in))
+        fout.write(struct.pack("i", final_conv_out))
+
+        # Define tensor keys to write
+        tensor_keys = []
+
+        # Encoder weights
+        for i in range(n_encoder_layers):
+            weight_key = f"_model.encoder.{i}.reparam_conv.weight"
+            bias_key = f"_model.encoder.{i}.reparam_conv.bias"
+            if weight_key in cleaned_dict and bias_key in cleaned_dict:
+                tensor_keys.append(weight_key)
+                tensor_keys.append(bias_key)
+
+        # LSTM weights
+        lstm_keys = [
+            "_model.decoder.rnn.weight_ih",
+            "_model.decoder.rnn.weight_hh",
+            "_model.decoder.rnn.bias_ih",
+            "_model.decoder.rnn.bias_hh"
+        ]
+        tensor_keys.extend([k for k in lstm_keys if k in cleaned_dict])
+
+        # Final conv weights
+        final_keys = [
+            "_model.decoder.decoder.2.weight",
+            "_model.decoder.decoder.2.bias"
+        ]
+        tensor_keys.extend([k for k in final_keys if k in cleaned_dict])
+
+        # STFT basis - add this last
+        stft_tensor = "_model.stft.forward_basis_buffer"
+        tensor_keys.append(stft_tensor)
+
+        print(f"Writing {len(tensor_keys)} tensors:")
+        for key in tensor_keys:
+            if key in cleaned_dict:
+                print(f"  - {key}: {cleaned_dict[key].shape}")
+            else:
+                print(f"  - {key}: MISSING")
+
+        # Process each tensor
+        for key in tensor_keys:
+            if key not in cleaned_dict:
+                print(f"Warning: Missing tensor {key}, skipping")
+                continue
+
+            tensor = cleaned_dict[key]
+
+            # Special handling for STFT tensor
+            if key == "_model.stft.forward_basis_buffer":
+                # Get the original numpy array without squeezing
+                data = tensor.detach().cpu().numpy()
+                # Ensure it has the expected shape
+                print(f"STFT tensor original shape: {data.shape}")
+                n_dims = 3
+                tensor_shape = [data.shape[2], data.shape[1], data.shape[0]]
+                is_conv_weight = True
+            else:
+                # For other tensors, we can use standard processing
+                data = tensor.detach().cpu().squeeze().numpy()
+                tensor_shape = list(data.shape)
+
+                # Ensure we have at most 4 dimensions for GGML
+                n_dims = min(len(tensor_shape), 4)
+
+                # Reverse dimensions for GGML
+                tensor_shape = tensor_shape[:n_dims]
+                tensor_shape.reverse()
+
+                # Check if this is a convolution weight tensor
+                is_conv_weight = "weight" in key and ("encoder" in key or "_model.decoder.decoder.2" in key)
+
+            # Convert to float16 for convolution weights
+            if is_conv_weight:
+                data = data.astype(np.float16)
+                ftype = 1  # float16
+            else:
+                ftype = 0  # float32
+
+            # Debug printing of tensor info
+            print(f"\nWriting tensor: {key}")
+            print(f"  Original shape: {tensor.shape}")
+            print(f"  Processed shape: {data.shape}")
+            print(f"  GGML dimensions: {n_dims}")
+            print(f"  GGML shape: {tensor_shape}")
+            print(f"  Type: {'float16' if ftype == 1 else 'float32'}")
+
+            # Convert tensor name to bytes
+            name_bytes = key.encode('utf-8')
+            name_length = len(name_bytes)
+
+            # Write tensor header
+            fout.write(struct.pack("i", n_dims))
+            fout.write(struct.pack("i", name_length))
+            fout.write(struct.pack("i", ftype))
+
+            # Write tensor dimensions
+            for i in range(n_dims):
+                size = tensor_shape[i] if i < len(tensor_shape) else 1
+                fout.write(struct.pack("i", size))
+                print(f"  Writing dimension {i}: {size}")
+
+            # Write tensor name
+            fout.write(name_bytes)
+
+            # Write tensor data
+            data.tofile(fout)
+
+            print(f"  Wrote {data.size * (2 if ftype==1 else 4)} bytes")
+
+    print(f"\nDone! Model has been converted to GGML format: {output_file}")
+    print(f"File size: {os.path.getsize(output_file)} bytes")
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="Convert Silero-VAD PyTorch model to GGML format")
+    parser.add_argument("--output", type=str, required=True, help="Path to output GGML model file")
+    parser.add_argument("--print-tensors", action="store_true", help="Print tensor values", default=True)
+    args = parser.parse_args()
+
+    convert_silero_vad(args.output, args.print_tensors)
--- a/models/convert-whisper-to-coreml.py
+++ b/models/convert-whisper-to-coreml.py
@@ -0,0 +1,333 @@
+import argparse
+import torch
+import torch.nn.functional as F
+import coremltools as ct
+
+from torch import Tensor
+from torch import nn
+from typing import Dict
+from typing import Optional
+from ane_transformers.reference.layer_norm import LayerNormANE as LayerNormANEBase
+from whisper.model import Whisper, AudioEncoder, TextDecoder, ResidualAttentionBlock, MultiHeadAttention, ModelDimensions
+from whisper import load_model
+
+
+def _str_to_bool(v):
+    if isinstance(v, bool):
+        return v
+    if v.lower() in ("true", "1", "yes"):
+        return True
+    if v.lower() in ("false", "0", "no"):
+        return False
+    raise argparse.ArgumentTypeError(f"boolean value expected, got '{v}'")
+
+# Disable PyTorch Scaled Dot-Product Attention (SDPA) to avoid compatibility issues.
+# The Whisper implementation expects a specific behavior from
+# torch.nn.functional.scaled_dot_product_attention that differs between PyTorch
+# versions. Setting use_sdpa=False forces Whisper to use its manual attention
+# implementation instead, which is more stable across different PyTorch versions
+# (2.5.0 required by coremltools vs newer versions).
+import whisper.model
+whisper.model.MultiHeadAttention.use_sdpa = False
+
+# Use for changing dim of input in encoder and decoder embeddings
+def linear_to_conv2d_map(state_dict, prefix, local_metadata, strict,
+                         missing_keys, unexpected_keys, error_msgs):
+    """
+    Unsqueeze twice to map nn.Linear weights to nn.Conv2d weights
+    """
+    for k in state_dict:
+        is_attention = all(substr in k for substr in ['attn', '.weight'])
+        is_mlp = any(k.endswith(s) for s in ['mlp.0.weight', 'mlp.2.weight'])
+
+        if (is_attention or is_mlp) and len(state_dict[k].shape) == 2:
+            state_dict[k] = state_dict[k][:, :, None, None]
+
+
+def correct_for_bias_scale_order_inversion(state_dict, prefix, local_metadata,
+                                           strict, missing_keys,
+                                           unexpected_keys, error_msgs):
+    state_dict[prefix + 'bias'] = state_dict[prefix + 'bias'] / state_dict[prefix + 'weight']
+    return state_dict
+
+class LayerNormANE(LayerNormANEBase):
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self._register_load_state_dict_pre_hook(
+            correct_for_bias_scale_order_inversion)
+
+class MultiHeadAttentionANE(MultiHeadAttention):
+    def __init__(self, n_state: int, n_head: int):
+        super().__init__(n_state, n_head)
+        self.query =  nn.Conv2d(n_state, n_state, kernel_size=1)
+        self.key = nn.Conv2d(n_state, n_state, kernel_size=1, bias=False)
+        self.value = nn.Conv2d(n_state, n_state, kernel_size=1)
+        self.out = nn.Conv2d(n_state, n_state, kernel_size=1)
+
+    def forward(self,
+                x: Tensor,
+                xa: Optional[Tensor] = None,
+                mask: Optional[Tensor] = None,
+                kv_cache: Optional[dict] = None):
+
+        q = self.query(x)
+
+        if kv_cache is None or xa is None or self.key not in kv_cache:
+            # hooks, if installed (i.e. kv_cache is not None), will prepend the cached kv tensors;
+            # otherwise, perform key/value projections for self- or cross-attention as usual.
+            k = self.key(x if xa is None else xa)
+            v = self.value(x if xa is None else xa)
+
+        else:
+            # for cross-attention, calculate keys and values once and reuse in subsequent calls.
+            k = kv_cache[self.key]
+            v = kv_cache[self.value]
+
+        wv, qk = self.qkv_attention_ane(q, k, v, mask)
+
+        return self.out(wv), qk
+
+    def qkv_attention_ane(self, q: Tensor, k: Tensor, v: Tensor, mask: Optional[Tensor] = None):
+
+        _, dim, _, seqlen = q.size()
+
+        dim_per_head = dim // self.n_head
+
+        scale = float(dim_per_head)**-0.5
+
+        q = q * scale
+
+        mh_q = q.split(dim_per_head, dim=1)
+        mh_k = k.transpose(1,3).split(dim_per_head, dim=3)
+        mh_v = v.split(dim_per_head, dim=1)
+
+        mh_qk = [
+            torch.einsum('bchq,bkhc->bkhq', [qi, ki])
+            for qi, ki in zip(mh_q, mh_k)
+        ]  # (batch_size, max_seq_length, 1, max_seq_length) * n_heads
+
+        if mask is not None:
+            for head_idx in range(self.n_head):
+                mh_qk[head_idx] = mh_qk[head_idx] + mask[:, :seqlen, :, :seqlen]
+
+        attn_weights = [aw.softmax(dim=1) for aw in mh_qk]  # (batch_size, max_seq_length, 1, max_seq_length) * n_heads
+        attn = [torch.einsum('bkhq,bchk->bchq', wi, vi) for wi, vi in zip(attn_weights, mh_v)]  # (batch_size, dim_per_head, 1, max_seq_length) * n_heads
+        attn = torch.cat(attn, dim=1)  # (batch_size, dim, 1, max_seq_length)
+
+        return attn, torch.cat(mh_qk, dim=1).float().detach()
+
+
+class ResidualAttentionBlockANE(ResidualAttentionBlock):
+    def __init__(self, n_state: int, n_head: int, cross_attention: bool = False):
+        super().__init__(n_state, n_head, cross_attention)
+        self.attn =  MultiHeadAttentionANE(n_state, n_head)
+        self.attn_ln = LayerNormANE(n_state)
+        self.cross_attn =  MultiHeadAttentionANE(n_state, n_head) if cross_attention else None
+        self.cross_attn_ln =  LayerNormANE(n_state) if cross_attention else None
+
+        n_mlp = n_state * 4
+        self.mlp =  nn.Sequential(
+            nn.Conv2d(n_state, n_mlp, kernel_size=1),
+            nn.GELU(),
+            nn.Conv2d(n_mlp, n_state, kernel_size=1)
+        )
+        self.mlp_ln = LayerNormANE(n_state)
+
+
+class AudioEncoderANE(AudioEncoder):
+    def __init__(self, n_mels: int, n_ctx: int, n_state: int, n_head: int, n_layer: int):
+        super().__init__(n_mels, n_ctx, n_state, n_head, n_layer)
+
+        self.blocks = nn.ModuleList(
+            [ResidualAttentionBlockANE(n_state, n_head) for _ in range(n_layer)]
+        )
+        self.ln_post = LayerNormANE(n_state)
+
+    def forward(self, x: Tensor):
+        """
+        x : torch.Tensor, shape = (batch_size, n_mels, n_ctx)
+            the mel spectrogram of the audio
+        """
+        x = F.gelu(self.conv1(x))
+        x = F.gelu(self.conv2(x))
+
+        assert x.shape[1:] == self.positional_embedding.shape[::-1], "incorrect audio shape"
+
+        # Add positional embedding and add dummy dim for ANE
+        x = (x + self.positional_embedding.transpose(0,1)).to(x.dtype).unsqueeze(2)
+
+        for block in self.blocks:
+            x = block(x)
+
+        x = self.ln_post(x)
+        x = x.squeeze(2).transpose(1, 2)
+
+        return x
+
+class TextDecoderANE(TextDecoder):
+
+    def __init__(self, n_vocab: int, n_ctx: int, n_state: int, n_head: int, n_layer: int):
+        super().__init__(n_vocab, n_ctx, n_state, n_head, n_layer)
+
+        self.blocks= nn.ModuleList(
+            [ResidualAttentionBlockANE(n_state, n_head, cross_attention=True) for _ in range(n_layer)]
+        )
+        self.ln= LayerNormANE(n_state)
+
+    def forward(self, x: Tensor, xa: Tensor, kv_cache: Optional[dict] = None):
+        """
+        x : torch.LongTensor, shape = (batch_size, <= n_ctx)
+            the text tokens
+        xa : torch.Tensor, shape = (batch_size, n_mels, n_audio_ctx)
+            the encoded audio features to be attended on
+        """
+        offset = next(iter(kv_cache.values())).shape[3] if kv_cache else 0
+        x = self.token_embedding(x) + self.positional_embedding[offset : offset + x.shape[-1]]
+        x = x.to(xa.dtype)
+
+        # Reformat for ANE
+        mask = self.mask[None, None, :, :].permute(0,3,1,2)
+        x = x.transpose(1,2).unsqueeze(2)
+
+        for block in self.blocks:
+            x = block(x, xa, mask=mask, kv_cache=kv_cache)
+
+        x = self.ln(x)
+
+        # Reformat back from ANE
+        x = x.permute(0,2,3,1).squeeze(0)
+
+        # ANE can only load tensors with dim size of at most 16,384 - whisper uses 51,864 (en) or 51,865 (multi-lang) tokens so we need to compute in chunks
+        if self.token_embedding.weight.shape[0] >= 51865:
+            # split in 11 chunks - 4715 each
+            splits = self.token_embedding.weight.split(self.token_embedding.weight.shape[0]//11, dim=0)
+            logits = torch.cat([torch.einsum('bid,jd->bij', x, split) for split in splits]).view(*x.shape[:2], -1)
+        else:
+            # split in 12 chunks - 4322 each
+            assert(self.token_embedding.weight.shape[0] == 51864)
+            splits = self.token_embedding.weight.split(self.token_embedding.weight.shape[0]//12, dim=0)
+            logits = torch.cat([torch.einsum('bid,jd->bij', x, split) for split in splits]).view(*x.shape[:2], -1)
+
+        return logits
+
+class WhisperANE(Whisper):
+    def __init__(self, dims: ModelDimensions):
+        super().__init__(dims)
+
+        self.encoder = AudioEncoderANE(
+            self.dims.n_mels,
+            self.dims.n_audio_ctx,
+            self.dims.n_audio_state,
+            self.dims.n_audio_head,
+            self.dims.n_audio_layer,
+        )
+        self.decoder = TextDecoderANE(
+            self.dims.n_vocab,
+            self.dims.n_text_ctx,
+            self.dims.n_text_state,
+            self.dims.n_text_head,
+            self.dims.n_text_layer,
+        )
+
+        self._register_load_state_dict_pre_hook(linear_to_conv2d_map)
+
+    def forward(self, mel: torch.Tensor, tokens: torch.Tensor) -> Dict[str, torch.Tensor]:
+        return self.decoder(tokens, self.encoder(mel))
+
+    def install_kv_cache_hooks(self, cache: Optional[dict] = None):
+        cache = {**cache} if cache is not None else {}
+        hooks = []
+
+        def save_to_cache(module, _, output):
+            if module not in cache or output.shape[3] > self.decoder.positional_embedding.shape[0]:
+                cache[module] = output  # save as-is, for the first token or cross attention
+            else:
+                cache[module] = torch.cat([cache[module], output], dim=3).detach()
+            return cache[module]
+
+        def install_hooks(layer: nn.Module):
+            if isinstance(layer, MultiHeadAttentionANE):
+                hooks.append(layer.key.register_forward_hook(save_to_cache))
+                hooks.append(layer.value.register_forward_hook(save_to_cache))
+
+        self.decoder.apply(install_hooks)
+        return cache, hooks
+
+def convert_encoder(hparams, model, quantize=False):
+    model.eval()
+
+    input_shape = (1, hparams.n_mels, 3000)
+    input_data = torch.randn(input_shape)
+    traced_model = torch.jit.trace(model, input_data)
+
+    model = ct.convert(
+        traced_model,
+        convert_to="mlprogram",
+        inputs=[ct.TensorType(name="logmel_data", shape=input_shape)],
+        outputs=[ct.TensorType(name="output")],
+        compute_units=ct.ComputeUnit.ALL,
+        compute_precision=ct.precision.FLOAT16 if quantize else ct.precision.FLOAT32,
+    )
+
+    return model
+
+def convert_decoder(hparams, model, quantize=False):
+    model.eval()
+
+    tokens_shape = (1, 1)
+    audio_shape = (1, hparams.n_audio_ctx, hparams.n_audio_state)
+
+    audio_data = torch.randn(audio_shape)
+    token_data = torch.randint(hparams.n_vocab, tokens_shape).long()
+
+    traced_model = torch.jit.trace(model, (token_data, audio_data))
+
+    model = ct.convert(
+        traced_model,
+        convert_to="mlprogram",
+        inputs=[
+            ct.TensorType(name="token_data", shape=tokens_shape, dtype=int),
+            ct.TensorType(name="audio_data", shape=audio_shape)
+        ],
+        compute_precision=ct.precision.FLOAT16 if quantize else ct.precision.FLOAT32,
+    )
+
+    return model
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--model", type=str, help="model to convert (e.g. tiny, tiny.en, base, base.en, small, small.en, medium, medium.en, large-v1, large-v2, large-v3, large-v3-turbo)", required=True)
+    parser.add_argument("--encoder-only", type=_str_to_bool, help="only convert encoder", default=False)
+    parser.add_argument("--quantize",     type=_str_to_bool, help="quantize weights to F16", default=False)
+    parser.add_argument("--optimize-ane", type=_str_to_bool, help="optimize for ANE execution", default=False)
+    args = parser.parse_args()
+
+    if args.model not in ["tiny", "tiny.en", "base", "base.en", "small", "small.en", "small.en-tdrz", "medium", "medium.en", "large-v1", "large-v2", "large-v3", "large-v3-turbo"]:
+        raise ValueError("Invalid model name")
+
+    whisper = load_model(args.model).cpu()
+    hparams = whisper.dims
+    print(hparams)
+
+    if args.optimize_ane:
+        whisperANE = WhisperANE(hparams).eval()
+        whisperANE.load_state_dict(whisper.state_dict())
+
+        encoder = whisperANE.encoder
+        decoder = whisperANE.decoder
+    else:
+        encoder = whisper.encoder
+        decoder = whisper.decoder
+
+    # Convert encoder
+    encoder = convert_encoder(hparams, encoder, quantize=args.quantize)
+    encoder.save(f"models/coreml-encoder-{args.model}.mlpackage")
+
+    if args.encoder_only is False:
+        # Convert decoder
+        decoder = convert_decoder(hparams, decoder, quantize=args.quantize)
+        decoder.save(f"models/coreml-decoder-{args.model}.mlpackage")
+
+    print("done converting")
--- a/models/convert-whisper-to-openvino.py
+++ b/models/convert-whisper-to-openvino.py
@@ -0,0 +1,59 @@
+import argparse
+import torch
+from whisper import load_model
+import os
+from openvino.frontend import FrontEndManager
+from openvino import serialize
+import shutil
+
+def convert_encoder(hparams, encoder, mname):
+    encoder.eval()
+
+    mel = torch.zeros((1, hparams.n_mels, 3000))
+
+    onnx_folder = os.path.join(os.path.dirname(__file__), "onnx_encoder")
+
+    #create a directory to store the onnx model, and other collateral that is saved during onnx export procedure
+    if not os.path.isdir(onnx_folder):
+        os.makedirs(onnx_folder)
+
+    onnx_path = os.path.join(onnx_folder, "whisper_encoder.onnx")
+
+    # Export the PyTorch model to ONNX
+    torch.onnx.export(
+        encoder,
+        mel,
+        onnx_path,
+        input_names=["mel"],
+        output_names=["output_features"]
+    )
+
+    # Convert ONNX to OpenVINO IR format using the frontend
+    fem = FrontEndManager()
+    onnx_fe = fem.load_by_framework("onnx")
+    onnx_model = onnx_fe.load(onnx_path)
+    ov_model = onnx_fe.convert(onnx_model)
+
+    # Serialize the OpenVINO model to XML and BIN files
+    serialize(ov_model, xml_path=os.path.join(os.path.dirname(__file__), "ggml-" + mname + "-encoder-openvino.xml"))
+
+    # Cleanup
+    if os.path.isdir(onnx_folder):
+        shutil.rmtree(onnx_folder)
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--model", type=str, help="model to convert (e.g. tiny, tiny.en, base, base.en, small, small.en, medium, medium.en, large-v1, large-v2, large-v3, large-v3-turbo)", required=True)
+    args = parser.parse_args()
+
+    if args.model not in ["tiny", "tiny.en", "base", "base.en", "small", "small.en", "medium", "medium.en", "large-v1", "large-v2", "large-v3", "large-v3-turbo"]:
+        raise ValueError("Invalid model name")
+
+    whisper = load_model(args.model).cpu()
+    hparams = whisper.dims
+
+    encoder = whisper.encoder
+
+    # Convert encoder to onnx
+    convert_encoder(hparams, encoder, args.model)
--- a/models/download-coreml-model.sh
+++ b/models/download-coreml-model.sh
@@ -0,0 +1,102 @@
+#!/bin/sh
+
+printf "whisper.cpp: this script hasn't been maintained and is not functional atm\n"
+exit 1
+
+# This script downloads Whisper model files that have already been converted to Core ML format.
+# This way you don't have to convert them yourself.
+
+src="https://huggingface.co/datasets/ggerganov/whisper.cpp-coreml"
+pfx="resolve/main/ggml"
+
+# get the path of this script
+get_script_path() {
+    if [ -x "$(command -v realpath)" ]; then
+       dirname "$(realpath "$0")"
+    else
+        _ret="$(cd -- "$(dirname "$0")" >/dev/null 2>&1 || exit ; pwd -P)"
+        echo "$_ret"
+    fi
+}
+
+script_path="$(get_script_path)"
+
+# Check if the script is inside a /bin/ directory
+case "$script_path" in
+    */bin) default_download_path="$PWD" ;;  # Use current directory as default download path if in /bin/
+    *) default_download_path="$script_path" ;;  # Otherwise, use script directory
+esac
+
+models_path="${2:-$default_download_path}"
+
+# Whisper models
+models="tiny.en tiny base.en base small.en small medium.en medium large-v1 large-v2 large-v3 large-v3-turbo"
+
+# list available models
+list_models() {
+        printf "\n"
+        printf "  Available models:"
+        for model in $models; do
+                printf " %s" "$models"
+        done
+        printf "\n\n"
+}
+
+if [ "$#" -lt 1 ] || [ "$#" -gt 2 ]; then
+    printf "Usage: %s <model> [models_path]\n" "$0"
+    list_models
+
+    exit 1
+fi
+
+model=$1
+
+if ! echo "$models" | grep -q -w "$model"; then
+    printf "Invalid model: %s\n" "$model"
+    list_models
+
+    exit 1
+fi
+
+# download Core ML model
+
+printf "Downloading Core ML model %s from '%s' ...\n" "$model" "$src"
+
+cd "$models_path" || exit
+
+if [ -f "ggml-$model.mlmodel" ]; then
+    printf "Model %s already exists. Skipping download.\n" "$model"
+    exit 0
+fi
+
+if [ -x "$(command -v wget)" ]; then
+    wget --quiet --show-progress -O ggml-"$model".mlmodel $src/$pfx-"$model".mlmodel
+elif [ -x "$(command -v curl)" ]; then
+    curl -L --output ggml-"$model".mlmodel $src/$pfx-"$model".mlmodel
+else
+    printf "Either wget or curl is required to download models.\n"
+    exit 1
+fi
+
+
+if [ $? -ne 0 ]; then
+    printf "Failed to download Core ML model %s \n" "$model"
+    printf "Please try again later or download the original Whisper model files and convert them yourself.\n"
+    exit 1
+fi
+
+# Check if 'whisper-cli' is available in the system PATH
+if command -v whisper-cli >/dev/null 2>&1; then
+    # If found, use 'whisper-cli' (relying on PATH resolution)
+    whisper_cmd="whisper-cli"
+else
+    # If not found, use the local build version
+    whisper_cmd="./build/bin/whisper-cli"
+fi
+
+printf "Done! Model '%s' saved in '%s/ggml-%s.bin'\n" "$model" "$models_path" "$model"
+printf "Run the following command to compile it:\n\n"
+printf "  $ xcrun coremlc compile %s/ggml-%s.mlmodel %s\n\n" "$models_path" "$model" "$models_path"
+printf "You can now use it like this:\n\n"
+printf "  $ %s -m %s/ggml-%s.bin -f samples/jfk.wav\n" "$whisper_cmd" "$models_path" "$model"
+printf "\n"
--- a/models/download-ggml-model.cmd
+++ b/models/download-ggml-model.cmd
@@ -0,0 +1,114 @@
+@echo off
+
+rem Save the original working directory
+set "orig_dir=%CD%"
+
+rem Get the script directory
+set "script_dir=%~dp0"
+
+rem Check if the script directory contains "\bin\" (case-insensitive)
+echo %script_dir% | findstr /i "\\bin\\" >nul
+if %ERRORLEVEL%==0 (
+  rem If script is in a \bin\ directory, use the original working directory as default download path
+  set "default_download_path=%orig_dir%"
+) else (
+  rem Otherwise, use script directory
+  pushd %~dp0
+  set "default_download_path=%CD%"
+  popd
+)
+
+rem Set the root path to be the parent directory of the script
+for %%d in (%~dp0..) do set "root_path=%%~fd"
+
+rem Count number of arguments passed to script
+set argc=0
+for %%x in (%*) do set /A argc+=1
+
+set models=tiny tiny-q5_1 tiny-q8_0 ^
+tiny.en tiny.en-q5_1 tiny.en-q8_0 ^
+base base-q5_1 base-q8_0 ^
+base.en base.en-q5_1 base.en-q8_0 ^
+small small-q5_1 small-q8_0 ^
+small.en small.en-q5_1 small.en-q8_0 small.en-tdrz ^
+medium medium-q5_0 medium-q8_0 ^
+medium.en medium.en-q5_0 medium.en-q8_0 ^
+large-v1 ^
+large-v2 large-v2-q5_0 large-v2-q8_0 ^
+large-v3 large-v3-q5_0 ^
+large-v3-turbo large-v3-turbo-q5_0 large-v3-turbo-q8_0
+
+rem If argc is not equal to 1 or 2, print usage information and exit
+if %argc% NEQ 1 (
+  if %argc% NEQ 2 (
+    echo.
+    echo Usage: download-ggml-model.cmd model [models_path]
+    CALL :list_models
+    goto :eof
+  )
+)
+
+if %argc% EQU 2 (
+  set models_path=%2
+) else (
+  set models_path=%default_download_path%
+)
+
+set model=%1
+
+for %%b in (%models%) do (
+  if "%%b"=="%model%" (
+    CALL :download_model
+    goto :eof
+  )
+)
+
+echo Invalid model: %model%
+CALL :list_models
+goto :eof
+
+:download_model
+echo Downloading ggml model %model%...
+
+if exist "%models_path%\\ggml-%model%.bin" (
+  echo Model %model% already exists. Skipping download.
+  goto :eof
+)
+echo %model% | findstr tdrz
+if %ERRORLEVEL% neq 0 (
+ PowerShell -NoProfile -ExecutionPolicy Bypass -Command "Start-BitsTransfer -Source https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-%model%.bin -Destination \"%models_path%\\ggml-%model%.bin\""
+) else (
+  PowerShell -NoProfile -ExecutionPolicy Bypass -Command "Start-BitsTransfer -Source https://huggingface.co/akashmjn/tinydiarize-whisper.cpp/resolve/main/ggml-%model%.bin -Destination \"%models_path%\\ggml-%model%.bin\""
+
+)
+
+if %ERRORLEVEL% neq 0 (
+  echo Failed to download ggml model %model%
+  echo Please try again later or download the original Whisper model files and convert them yourself.
+  goto :eof
+)
+
+rem Check if 'whisper-cli' is available in the system PATH
+where whisper-cli >nul 2>&1
+if %ERRORLEVEL%==0 (
+  rem If found, suggest 'whisper-cli' (relying on PATH resolution)
+  set "whisper_cmd=whisper-cli"
+) else (
+  rem If not found, suggest the local build version
+  set "whisper_cmd=%root_path%\build\bin\Release\whisper-cli.exe"
+)
+
+echo Done! Model %model% saved in %models_path%\ggml-%model%.bin
+echo You can now use it like this:
+echo %whisper_cmd% -m %models_path%\ggml-%model%.bin -f samples\jfk.wav
+
+goto :eof
+
+:list_models
+  echo.
+  echo Available models:
+  (for %%a in (%models%) do (
+    echo %%a
+  ))
+  echo.
+  exit /b
--- a/models/download-ggml-model.sh
+++ b/models/download-ggml-model.sh
@@ -0,0 +1,155 @@
+#!/bin/sh
+
+# This script downloads Whisper model files that have already been converted to ggml format.
+# This way you don't have to convert them yourself.
+
+#src="https://ggml.ggerganov.com"
+#pfx="ggml-model-whisper"
+
+src="https://huggingface.co/ggerganov/whisper.cpp"
+pfx="resolve/main/ggml"
+
+BOLD="\033[1m"
+RESET='\033[0m'
+
+# get the path of this script
+get_script_path() {
+    if [ -x "$(command -v realpath)" ]; then
+        dirname "$(realpath "$0")"
+    else
+        _ret="$(cd -- "$(dirname "$0")" >/dev/null 2>&1 || exit ; pwd -P)"
+        echo "$_ret"
+    fi
+}
+
+script_path="$(get_script_path)"
+
+# Check if the script is inside a /bin/ directory
+case "$script_path" in
+    */bin) default_download_path="$PWD" ;;  # Use current directory as default download path if in /bin/
+    *) default_download_path="$script_path" ;;  # Otherwise, use script directory
+esac
+
+models_path="${2:-$default_download_path}"
+
+# Whisper models
+models="tiny
+tiny.en
+tiny-q5_1
+tiny.en-q5_1
+tiny-q8_0
+base
+base.en
+base-q5_1
+base.en-q5_1
+base-q8_0
+small
+small.en
+small.en-tdrz
+small-q5_1
+small.en-q5_1
+small-q8_0
+medium
+medium.en
+medium-q5_0
+medium.en-q5_0
+medium-q8_0
+large-v1
+large-v2
+large-v2-q5_0
+large-v2-q8_0
+large-v3
+large-v3-q5_0
+large-v3-turbo
+large-v3-turbo-q5_0
+large-v3-turbo-q8_0"
+
+# list available models
+list_models() {
+    printf "\n"
+    printf "Available models:"
+    model_class=""
+    for model in $models; do
+        this_model_class="${model%%[.-]*}"
+        if [ "$this_model_class" != "$model_class" ]; then
+            printf "\n "
+            model_class=$this_model_class
+        fi
+        printf " %s" "$model"
+    done
+    printf "\n\n"
+}
+
+if [ "$#" -lt 1 ] || [ "$#" -gt 2 ]; then
+    printf "Usage: %s <model> [models_path]\n" "$0"
+    list_models
+    printf "___________________________________________________________\n"
+    printf "${BOLD}.en${RESET} = english-only ${BOLD}-q5_[01]${RESET} = quantized ${BOLD}-tdrz${RESET} = tinydiarize\n"
+
+    exit 1
+fi
+
+model=$1
+
+if ! echo "$models" | grep -q -w "$model"; then
+    printf "Invalid model: %s\n" "$model"
+    list_models
+
+    exit 1
+fi
+
+# check if model contains `tdrz` and update the src and pfx accordingly
+if echo "$model" | grep -q "tdrz"; then
+    src="https://huggingface.co/akashmjn/tinydiarize-whisper.cpp"
+    pfx="resolve/main/ggml"
+fi
+
+echo "$model" | grep -q '^"tdrz"*$'
+
+# download ggml model
+
+printf "Downloading ggml model %s from '%s' ...\n" "$model" "$src"
+
+cd "$models_path" || exit
+
+if [ -f "ggml-$model.bin" ]; then
+    printf "Model %s already exists. Skipping download.\n" "$model"
+    exit 0
+fi
+
+if [ -x "$(command -v wget2)" ]; then
+    wget2 --no-config --progress bar -O ggml-"$model".bin $src/$pfx-"$model".bin
+elif [ -x "$(command -v curl)" ]; then
+    curl -L --fail \
+         --retry 5 \
+         --retry-delay 5 \
+         --retry-all-errors \
+         --retry-connrefused \
+         ${HF_TOKEN:+--header "Authorization: Bearer $HF_TOKEN"} \
+         --output ggml-"$model".bin $src/$pfx-"$model".bin
+elif [ -x "$(command -v wget)" ]; then
+    wget --no-config --quiet --show-progress -O ggml-"$model".bin $src/$pfx-"$model".bin
+else
+    printf "Either wget2, curl, or wget is required to download models.\n"
+    exit 1
+fi
+
+if [ $? -ne 0 ]; then
+    printf "Failed to download ggml model %s \n" "$model"
+    printf "Please try again later or download the original Whisper model files and convert them yourself.\n"
+    exit 1
+fi
+
+# Check if 'whisper-cli' is available in the system PATH
+if command -v whisper-cli >/dev/null 2>&1; then
+    # If found, use 'whisper-cli' (relying on PATH resolution)
+    whisper_cmd="whisper-cli"
+else
+    # If not found, use the local build version
+    whisper_cmd="./build/bin/whisper-cli"
+fi
+
+printf "Done! Model '%s' saved in '%s/ggml-%s.bin'\n" "$model" "$models_path" "$model"
+printf "You can now use it like this:\n\n"
+printf "  $ %s -m %s/ggml-%s.bin -f samples/jfk.wav\n" "$whisper_cmd" "$models_path" "$model"
+printf "\n"
--- a/models/download-vad-model.cmd
+++ b/models/download-vad-model.cmd
@@ -0,0 +1,99 @@
+@echo off
+
+rem Save the original working directory
+set "orig_dir=%CD%"
+
+rem Get the script directory
+set "script_dir=%~dp0"
+
+rem Check if the script directory contains "\bin\" (case-insensitive)
+echo %script_dir% | findstr /i "\\bin\\" >nul
+if %ERRORLEVEL%==0 (
+  rem If script is in a \bin\ directory, use the original working directory as default download path
+  set "default_download_path=%orig_dir%"
+) else (
+  rem Otherwise, use script directory
+  pushd %~dp0
+  set "default_download_path=%CD%"
+  popd
+)
+
+rem Set the root path to be the parent directory of the script
+for %%d in (%~dp0..) do set "root_path=%%~fd"
+
+rem Count number of arguments passed to script
+set argc=0
+for %%x in (%*) do set /A argc+=1
+
+set models=silero-v5.1.2 silero-v6.2.0
+
+rem If argc is not equal to 1 or 2, print usage information and exit
+if %argc% NEQ 1 (
+  if %argc% NEQ 2 (
+    echo.
+    echo Usage: download-vad-model.cmd model [models_path]
+    CALL :list_models
+    goto :eof
+  )
+)
+
+if %argc% EQU 2 (
+  set models_path=%2
+) else (
+  set models_path=%default_download_path%
+)
+
+set model=%1
+
+for %%b in (%models%) do (
+  if "%%b"=="%model%" (
+    CALL :download_model
+    goto :eof
+  )
+)
+
+echo Invalid model: %model%
+CALL :list_models
+goto :eof
+
+:download_model
+echo Downloading vad model %model%...
+
+if exist "%models_path%\\ggml-%model%.bin" (
+  echo Model %model% already exists. Skipping download.
+  goto :eof
+)
+
+
+PowerShell -NoProfile -ExecutionPolicy Bypass -Command "Start-BitsTransfer -Source https://huggingface.co/ggml-org/whisper-vad/resolve/main/ggml-%model%.bin -Destination \"%models_path%\\ggml-%model%.bin\""
+
+if %ERRORLEVEL% neq 0 (
+  echo Failed to download ggml model %model%
+  echo Please try again later or download the original Whisper model files and convert them yourself.
+  goto :eof
+)
+
+rem Check if 'whisper-cli' is available in the system PATH
+where whisper-cli >nul 2>&1
+if %ERRORLEVEL%==0 (
+  rem If found, suggest 'whisper-cli' (relying on PATH resolution)
+  set "whisper_cmd=whisper-cli"
+) else (
+  rem If not found, suggest the local build version
+  set "whisper_cmd=%root_path%\build\bin\Release\whisper-cli.exe"
+)
+
+echo Done! Model %model% saved in %models_path%\ggml-%model%.bin
+echo You can now use it like this:
+echo %whisper_cmd% -vm %models_path%\ggml-%model%.bin --vad -m models/ggml-base.en.bin -f samples\jfk.wav
+
+goto :eof
+
+:list_models
+  echo.
+  echo Available models:
+  (for %%a in (%models%) do (
+    echo %%a
+  ))
+  echo.
+  exit /b
--- a/models/download-vad-model.sh
+++ b/models/download-vad-model.sh
@@ -0,0 +1,105 @@
+#!/bin/sh
+
+# This script downloads Whisper VAD model files that have already been converted
+# to ggml format. This way you don't have to convert them yourself.
+
+src="https://huggingface.co/ggml-org/whisper-vad"
+pfx="resolve/main/ggml"
+
+BOLD="\033[1m"
+RESET='\033[0m'
+
+# get the path of this script
+get_script_path() {
+    if [ -x "$(command -v realpath)" ]; then
+        dirname "$(realpath "$0")"
+    else
+        _ret="$(cd -- "$(dirname "$0")" >/dev/null 2>&1 || exit ; pwd -P)"
+        echo "$_ret"
+    fi
+}
+
+script_path="$(get_script_path)"
+
+# Check if the script is inside a /bin/ directory
+case "$script_path" in
+    */bin) default_download_path="$PWD" ;;  # Use current directory as default download path if in /bin/
+    *) default_download_path="$script_path" ;;  # Otherwise, use script directory
+esac
+
+models_path="${2:-$default_download_path}"
+
+# Whisper VAD models
+models="silero-v5.1.2 silero-v6.2.0"
+
+# list available models
+list_models() {
+    printf "\n"
+    printf "Available models:"
+    model_class=""
+    for model in $models; do
+        this_model_class="${model%%[.-]*}"
+        if [ "$this_model_class" != "$model_class" ]; then
+            printf "\n "
+            model_class=$this_model_class
+        fi
+        printf " %s" "$model"
+    done
+    printf "\n\n"
+}
+
+if [ "$#" -lt 1 ] || [ "$#" -gt 2 ]; then
+    printf "Usage: %s <model> [models_path]\n" "$0"
+    list_models
+    exit 1
+fi
+
+model=$1
+
+if ! echo "$models" | grep -q -w "$model"; then
+    printf "Invalid model: %s\n" "$model"
+    list_models
+
+    exit 1
+fi
+
+# download ggml model
+printf "Downloading ggml model %s from '%s' ...\n" "$model" "$src"
+
+cd "$models_path" || exit
+
+if [ -f "ggml-$model.bin" ]; then
+    printf "Model %s already exists. Skipping download.\n" "$model"
+    exit 0
+fi
+
+if [ -x "$(command -v wget2)" ]; then
+    wget2 --no-config --progress bar -O ggml-"$model".bin $src/$pfx-"$model".bin
+elif [ -x "$(command -v wget)" ]; then
+    wget --no-config --quiet --show-progress -O ggml-"$model".bin $src/$pfx-"$model".bin
+elif [ -x "$(command -v curl)" ]; then
+    curl -L --output ggml-"$model".bin $src/$pfx-"$model".bin
+else
+    printf "Either wget or curl is required to download models.\n"
+    exit 1
+fi
+
+if [ $? -ne 0 ]; then
+    printf "Failed to download ggml model %s \n" "$model"
+    printf "Please try again later or download the original Whisper model files and convert them yourself.\n"
+    exit 1
+fi
+
+# Check if 'whisper-cli' is available in the system PATH
+if command -v whisper-cli >/dev/null 2>&1; then
+    # If found, use 'whisper-cli' (relying on PATH resolution)
+    whisper_cmd="whisper-cli"
+else
+    # If not found, use the local build version
+    whisper_cmd="./build/bin/whisper-cli"
+fi
+
+printf "Done! Model '%s' saved in '%s/ggml-%s.bin'\n" "$model" "$models_path" "$model"
+printf "You can now use it like this:\n\n"
+printf "  $ %s -vm %s/ggml-%s.bin --vad -f samples/jfk.wav -m models/ggml-base.en.bin\n" "$whisper_cmd" "$models_path" "$model"
+printf "\n"
--- a/models/download-vitisai-model.cmd
+++ b/models/download-vitisai-model.cmd
@@ -0,0 +1,32 @@
+@echo off
+setlocal
+
+set "script=%~dp0download-vitisai-model.ps1"
+
+if "%~1"=="" (
+  PowerShell -NoProfile -ExecutionPolicy Bypass -File "%script%"
+  exit /b %ERRORLEVEL%
+)
+
+if /I "%~1"=="--list" (
+  PowerShell -NoProfile -ExecutionPolicy Bypass -File "%script%" -List
+  exit /b %ERRORLEVEL%
+)
+
+if /I "%~1"=="-l" (
+  PowerShell -NoProfile -ExecutionPolicy Bypass -File "%script%" -List
+  exit /b %ERRORLEVEL%
+)
+
+if /I "%~1"=="list" (
+  PowerShell -NoProfile -ExecutionPolicy Bypass -File "%script%" -List
+  exit /b %ERRORLEVEL%
+)
+
+if "%~2"=="" (
+  PowerShell -NoProfile -ExecutionPolicy Bypass -File "%script%" -Model "%~1"
+) else (
+  PowerShell -NoProfile -ExecutionPolicy Bypass -File "%script%" -Model "%~1" -ModelsPath "%~2"
+)
+
+exit /b %ERRORLEVEL%
--- a/models/download-vitisai-model.ps1
+++ b/models/download-vitisai-model.ps1
@@ -0,0 +1,218 @@
+param(
+    [Parameter(Position = 0)]
+    [string] $Model,
+
+    [Parameter(Position = 1)]
+    [string] $ModelsPath,
+
+    [switch] $List
+)
+
+$ErrorActionPreference = "Stop"
+
+$Source = "https://huggingface.co"
+$CollectionApi = "https://huggingface.co/api/collections/amd/ryzen-ai-whisper-npu-optimized-onnx-models"
+$CollectionUrl = "https://huggingface.co/collections/amd/ryzen-ai-whisper-npu-optimized-onnx-models"
+
+$ScriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
+if ($ScriptDir -match "\\bin$") {
+    $DefaultDownloadPath = (Get-Location).Path
+} else {
+    $DefaultDownloadPath = $ScriptDir
+}
+
+if (-not $ModelsPath) {
+    $ModelsPath = $DefaultDownloadPath
+}
+
+function Get-HfHeaders {
+    $headers = @{}
+    if ($env:HF_TOKEN) {
+        $headers["Authorization"] = "Bearer $env:HF_TOKEN"
+    }
+    return $headers
+}
+
+function Invoke-HfJson {
+    param([string] $Uri)
+
+    $headers = Get-HfHeaders
+    if ($headers.Count -gt 0) {
+        return Invoke-RestMethod -Uri $Uri -Headers $headers
+    }
+
+    return Invoke-RestMethod -Uri $Uri
+}
+
+function Normalize-ModelName {
+    param([string] $Name)
+
+    # HF currently publishes ggml-small-en-encoder-vitisai.rai, while the
+    # matching ggml model is ggml-small.en.bin.
+    if ($Name.EndsWith("-en")) {
+        return $Name.Substring(0, $Name.Length - 3) + ".en"
+    }
+
+    return $Name
+}
+
+function Get-VitisAiModels {
+    $collection = Invoke-HfJson -Uri $CollectionApi
+    $seen = @{}
+    $rows = New-Object System.Collections.Generic.List[object]
+
+    foreach ($item in $collection.items) {
+        if ($item.type -ne "model") {
+            continue
+        }
+
+        $repo = $item.id
+        if (-not $repo) {
+            continue
+        }
+
+        $modelInfo = Invoke-HfJson -Uri "$Source/api/models/$repo"
+        foreach ($sibling in $modelInfo.siblings) {
+            $filename = [string] $sibling.rfilename
+            $match = [regex]::Match($filename, "^ggml-(.+)-encoder-vitisai\.rai$")
+            if (-not $match.Success) {
+                continue
+            }
+
+            $rawName = $match.Groups[1].Value
+            $modelName = Normalize-ModelName -Name $rawName
+            if ($seen.ContainsKey($modelName)) {
+                continue
+            }
+
+            $seen[$modelName] = $true
+            $destination = "ggml-$modelName-encoder-vitisai.rai"
+            $url = "$Source/$repo/resolve/main/$([uri]::EscapeDataString($filename))"
+
+            $rows.Add([pscustomobject]@{
+                Model = $modelName
+                RawName = $rawName
+                Repo = $repo
+                SourceFile = $filename
+                DestinationFile = $destination
+                DownloadUrl = $url
+            })
+        }
+    }
+
+    $order = @{
+        "tiny" = 10
+        "tiny.en" = 11
+        "base" = 20
+        "base.en" = 21
+        "small" = 30
+        "small.en" = 31
+        "medium" = 40
+        "medium.en" = 41
+        "large-v1" = 50
+        "large-v2" = 60
+        "large-v3" = 70
+        "large-v3-turbo" = 80
+    }
+
+    return $rows | Sort-Object `
+        @{ Expression = { if ($order.ContainsKey($_.Model)) { $order[$_.Model] } else { 1000 } } }, `
+        @{ Expression = { $_.Model } }
+}
+
+function Show-Models {
+    $models = Get-VitisAiModels
+
+    Write-Host ""
+    Write-Host "Available VitisAI encoder caches from ${CollectionUrl}:"
+    foreach ($entry in $models) {
+        if ($entry.Model -eq $entry.RawName) {
+            Write-Host ("  {0,-18} {1}" -f $entry.Model, $entry.Repo)
+        } else {
+            Write-Host ("  {0,-18} {1} (source name: {2})" -f $entry.Model, $entry.Repo, $entry.RawName)
+        }
+    }
+    Write-Host ""
+}
+
+function Show-Usage {
+    Write-Host "Usage: download-vitisai-model.cmd --list"
+    Write-Host "       download-vitisai-model.cmd <model> [models_path]"
+    Write-Host ""
+    Write-Host "Downloads ggml-<model>-encoder-vitisai.rai next to ggml-<model>.bin."
+    Write-Host "Use the same model name as download-ggml-model.cmd."
+    Write-Host ""
+}
+
+if ($List -or $Model -eq "--list" -or $Model -eq "-l" -or $Model -eq "list") {
+    Show-Models
+    exit 0
+}
+
+if (-not $Model) {
+    Show-Usage
+    Show-Models
+    exit 1
+}
+
+$models = Get-VitisAiModels
+$entry = $models | Where-Object { $_.Model -eq $Model -or $_.RawName -eq $Model } | Select-Object -First 1
+if (-not $entry) {
+    Write-Host "Invalid model: $Model"
+    foreach ($available in $models) {
+        Write-Host "  $($available.Model)"
+    }
+    exit 1
+}
+
+New-Item -ItemType Directory -Force -Path $ModelsPath | Out-Null
+$destinationPath = Join-Path $ModelsPath $entry.DestinationFile
+
+Write-Host "Downloading VitisAI encoder cache $($entry.Model) from '$($entry.Repo)' ..."
+if (Test-Path $destinationPath) {
+    Write-Host "VitisAI encoder cache $($entry.DestinationFile) already exists. Skipping download."
+    exit 0
+}
+
+$headers = Get-HfHeaders
+$downloaded = $false
+for ($attempt = 1; $attempt -le 5; ++$attempt) {
+    try {
+        if ($headers.Count -gt 0) {
+            Invoke-WebRequest -Uri $entry.DownloadUrl -Headers $headers -OutFile $destinationPath
+        } else {
+            Invoke-WebRequest -Uri $entry.DownloadUrl -OutFile $destinationPath
+        }
+        $downloaded = $true
+        break
+    } catch {
+        if ($attempt -eq 5) {
+            if (Test-Path $destinationPath) {
+                Remove-Item -Force $destinationPath
+            }
+            Write-Host "Failed to download VitisAI encoder cache $($entry.Model) from $($entry.DownloadUrl)"
+            throw
+        }
+        Start-Sleep -Seconds 5
+    }
+}
+
+if (-not $downloaded) {
+    exit 1
+}
+
+$whisperCmd = "whisper-cli"
+if (-not (Get-Command whisper-cli -ErrorAction SilentlyContinue)) {
+    $rootPath = Split-Path -Parent $ScriptDir
+    $whisperCmd = Join-Path $rootPath "build\bin\Release\whisper-cli.exe"
+}
+
+Write-Host "Done! VitisAI encoder cache '$($entry.Model)' saved in '$destinationPath'"
+if ($entry.RawName -ne $entry.Model) {
+    Write-Host "Source cache '$($entry.SourceFile)' was renamed to match ggml model name '$($entry.Model)'."
+}
+Write-Host "Use it with the matching ggml model:"
+Write-Host ""
+Write-Host "  $ScriptDir\download-ggml-model.cmd $($entry.Model) $ModelsPath"
+Write-Host "  $whisperCmd -m $ModelsPath\ggml-$($entry.Model).bin -f samples\jfk.wav"
+Write-Host ""
--- a/models/download-vitisai-model.sh
+++ b/models/download-vitisai-model.sh
@@ -0,0 +1,226 @@
+#!/bin/sh
+
+# This script downloads prebuilt VitisAI encoder cache files for Whisper models.
+# The cache file is saved next to the ggml model file and follows the loader
+# convention: ggml-<model>-encoder-vitisai.rai
+
+src="https://huggingface.co"
+collection_api="https://huggingface.co/api/collections/amd/ryzen-ai-whisper-npu-optimized-onnx-models"
+collection_url="https://huggingface.co/collections/amd/ryzen-ai-whisper-npu-optimized-onnx-models"
+
+BOLD="\033[1m"
+RESET='\033[0m'
+
+# get the path of this script
+get_script_path() {
+    if [ -x "$(command -v realpath)" ]; then
+        dirname "$(realpath "$0")"
+    else
+        _ret="$(cd -- "$(dirname "$0")" >/dev/null 2>&1 || exit ; pwd -P)"
+        echo "$_ret"
+    fi
+}
+
+find_python() {
+    if command -v python3 >/dev/null 2>&1; then
+        printf "%s\n" "python3"
+    elif command -v python >/dev/null 2>&1; then
+        printf "%s\n" "python"
+    else
+        return 1
+    fi
+}
+
+script_path="$(get_script_path)"
+
+# Check if the script is inside a /bin/ directory
+case "$script_path" in
+    */bin) default_download_path="$PWD" ;;  # Use current directory as default download path if in /bin/
+    *) default_download_path="$script_path" ;;  # Otherwise, use script directory
+esac
+
+models_path="${2:-$default_download_path}"
+
+discover_models() {
+    python_cmd="$(find_python)" || {
+        printf "Python is required to query available VitisAI caches from Hugging Face.\n" >&2
+        return 1
+    }
+
+    "$python_cmd" - "$src" "$collection_api" <<'PY'
+import json
+import os
+import re
+import sys
+import urllib.parse
+import urllib.request
+
+src = sys.argv[1].rstrip("/")
+collection_api = sys.argv[2]
+headers = {}
+token = os.environ.get("HF_TOKEN")
+if token:
+    headers["Authorization"] = "Bearer " + token
+
+def load_json(url):
+    req = urllib.request.Request(url, headers=headers)
+    with urllib.request.urlopen(req) as response:
+        return json.load(response)
+
+def normalize_model_name(name):
+    # HF currently publishes ggml-small-en-encoder-vitisai.rai, while the
+    # matching ggml model is ggml-small.en.bin.
+    if name.endswith("-en"):
+        return name[:-3] + ".en"
+    return name
+
+collection = load_json(collection_api)
+rows = []
+seen = set()
+
+for item in collection.get("items", []):
+    if item.get("type") != "model":
+        continue
+
+    repo = item.get("id")
+    if not repo:
+        continue
+
+    model_info = load_json(src + "/api/models/" + repo)
+    for sibling in model_info.get("siblings", []):
+        filename = sibling.get("rfilename", "")
+        match = re.match(r"^ggml-(.+)-encoder-vitisai\.rai$", filename)
+        if not match:
+            continue
+
+        raw_name = match.group(1)
+        model_name = normalize_model_name(raw_name)
+        if model_name in seen:
+            continue
+        seen.add(model_name)
+
+        destination = "ggml-%s-encoder-vitisai.rai" % model_name
+        url = "%s/%s/resolve/main/%s" % (src, repo, urllib.parse.quote(filename))
+        rows.append((model_name, raw_name, repo, filename, destination, url))
+
+order = {
+    "tiny": 10,
+    "tiny.en": 11,
+    "base": 20,
+    "base.en": 21,
+    "small": 30,
+    "small.en": 31,
+    "medium": 40,
+    "medium.en": 41,
+    "large-v1": 50,
+    "large-v2": 60,
+    "large-v3": 70,
+    "large-v3-turbo": 80,
+}
+
+for row in sorted(rows, key=lambda item: (order.get(item[0], 1000), item[0])):
+    print("|".join(row))
+PY
+}
+
+list_models() {
+    models="$(discover_models)" || exit 1
+
+    printf "\n"
+    printf "Available VitisAI encoder caches from %s:\n" "$collection_url"
+    printf "%s\n" "$models" | while IFS='|' read -r model raw repo _source _destination _url; do
+        if [ "$model" = "$raw" ]; then
+            printf "  %-18s %s\n" "$model" "$repo"
+        else
+            printf "  %-18s %s (source name: %s)\n" "$model" "$repo" "$raw"
+        fi
+    done
+    printf "\n"
+}
+
+usage() {
+    printf "Usage: %s --list\n" "$0"
+    printf "       %s <model> [models_path]\n" "$0"
+    printf "\n"
+    printf "Downloads ggml-<model>-encoder-vitisai.rai next to ggml-<model>.bin.\n"
+    printf "Use the same model name as %s/download-ggml-model.sh.\n" "$script_path"
+    printf "\n"
+}
+
+if [ "$#" -eq 1 ] && { [ "$1" = "--list" ] || [ "$1" = "-l" ] || [ "$1" = "list" ]; }; then
+    list_models
+    exit 0
+fi
+
+if [ "$#" -lt 1 ] || [ "$#" -gt 2 ]; then
+    usage
+    list_models
+    printf "___________________________________________________________\n"
+    printf "Example: %s ${BOLD}small${RESET} %s\n" "$0" "$default_download_path"
+    exit 1
+fi
+
+model=$1
+models="$(discover_models)" || exit 1
+
+match="$(printf "%s\n" "$models" | awk -F '|' -v model="$model" '$1 == model || $2 == model { print; exit }')"
+if [ -z "$match" ]; then
+    printf "Invalid model: %s\n" "$model"
+    printf "%s\n" "$models" | while IFS='|' read -r available _raw _repo _source _destination _url; do
+        printf "  %s\n" "$available"
+    done
+    exit 1
+fi
+
+IFS='|' read -r model raw_name repo source_file destination_file download_url <<EOF
+$match
+EOF
+
+printf "Downloading VitisAI encoder cache %s from '%s' ...\n" "$model" "$repo"
+
+mkdir -p "$models_path" || exit
+cd "$models_path" || exit
+
+if [ -f "$destination_file" ]; then
+    printf "VitisAI encoder cache %s already exists. Skipping download.\n" "$destination_file"
+    exit 0
+fi
+
+if [ -x "$(command -v wget2)" ]; then
+    wget2 --no-config --progress bar -O "$destination_file" ${HF_TOKEN:+--header "Authorization: Bearer $HF_TOKEN"} "$download_url"
+elif [ -x "$(command -v curl)" ]; then
+    curl -L --fail \
+         --retry 5 \
+         --retry-delay 5 \
+         --retry-all-errors \
+         --retry-connrefused \
+         ${HF_TOKEN:+--header "Authorization: Bearer $HF_TOKEN"} \
+         --output "$destination_file" "$download_url"
+elif [ -x "$(command -v wget)" ]; then
+    wget --no-config --quiet --show-progress ${HF_TOKEN:+--header "Authorization: Bearer $HF_TOKEN"} -O "$destination_file" "$download_url"
+else
+    printf "Either wget2, curl, or wget is required to download VitisAI encoder caches.\n"
+    exit 1
+fi
+
+if [ $? -ne 0 ]; then
+    printf "Failed to download VitisAI encoder cache %s from %s\n" "$model" "$download_url"
+    rm -f "$destination_file"
+    exit 1
+fi
+
+# Check if 'whisper-cli' is available in the system PATH
+if command -v whisper-cli >/dev/null 2>&1; then
+    whisper_cmd="whisper-cli"
+else
+    whisper_cmd="./build/bin/whisper-cli"
+fi
+
+printf "Done! VitisAI encoder cache '%s' saved in '%s/%s'\n" "$model" "$models_path" "$destination_file"
+if [ "$raw_name" != "$model" ]; then
+    printf "Source cache '%s' was renamed to match ggml model name '%s'.\n" "$source_file" "$model"
+fi
+printf "Use it with the matching ggml model:\n\n"
+printf "  $ %s/download-ggml-model.sh %s %s\n" "$script_path" "$model" "$models_path"
+printf "  $ %s -m %s/ggml-%s.bin -f samples/jfk.wav\n" "$whisper_cmd" "$models_path" "$model"
+printf "\n"
--- a/models/generate-coreml-interface.sh
+++ b/models/generate-coreml-interface.sh
@@ -0,0 +1,31 @@
+#!/bin/sh
+#
+# This generates:
+#   - src/coreml/whisper-encoder-impl.h and src/coreml/whisper-encoder-impl.m
+#   - src/coreml/whisper-decoder-impl.h and src/coreml/whisper-decoder-impl.m
+#
+
+set -e
+
+wd=$(dirname "$0")
+cd "$wd/../" || exit
+
+python3 models/convert-whisper-to-coreml.py --model tiny.en
+
+mv -v models/coreml-encoder-tiny.en.mlpackage models/whisper-encoder-impl.mlpackage
+xcrun coremlc generate models/whisper-encoder-impl.mlpackage src/coreml/
+mv src/coreml/whisper_encoder_impl.h src/coreml/whisper-encoder-impl.h
+mv src/coreml/whisper_encoder_impl.m src/coreml/whisper-encoder-impl.m
+sed -i '' 's/whisper_encoder_impl\.h/whisper-encoder-impl.h/g' src/coreml/whisper-encoder-impl.m
+sed -i '' 's/whisper_encoder_impl\.m/whisper-encoder-impl.m/g' src/coreml/whisper-encoder-impl.m
+sed -i '' 's/whisper_encoder_impl\.h/whisper-encoder-impl.h/g' src/coreml/whisper-encoder-impl.h
+
+mv -v models/coreml-decoder-tiny.en.mlpackage models/whisper-decoder-impl.mlpackage
+xcrun coremlc generate models/whisper-decoder-impl.mlpackage src/coreml/
+mv src/coreml/whisper_decoder_impl.h src/coreml/whisper-decoder-impl.h
+mv src/coreml/whisper_decoder_impl.m src/coreml/whisper-decoder-impl.m
+sed -i '' 's/whisper_decoder_impl\.h/whisper-decoder-impl.h/g' src/coreml/whisper-decoder-impl.m
+sed -i '' 's/whisper_decoder_impl\.m/whisper-decoder-impl.m/g' src/coreml/whisper-decoder-impl.m
+sed -i '' 's/whisper_decoder_impl\.h/whisper-decoder-impl.h/g' src/coreml/whisper-decoder-impl.h
+
+rm -rfv models/whisper-encoder-impl.mlpackage models/whisper-decoder-impl.mlpackage
--- a/models/generate-coreml-model.sh
+++ b/models/generate-coreml-model.sh
@@ -0,0 +1,38 @@
+#!/bin/sh
+
+set -e
+
+# Usage: ./generate-coreml-model.sh <model-name>
+if [ $# -eq 0 ]; then
+  echo "No model name supplied"
+  echo "Usage for Whisper models: ./generate-coreml-model.sh <model-name>"
+  echo "Usage for HuggingFace models: ./generate-coreml-model.sh -h5 <model-name> <model-path>"
+  exit 1
+elif [ "$1" = "-h5" ] && [ $# != 3 ]; then
+  echo "No model name and model path supplied for a HuggingFace model"
+  echo "Usage for HuggingFace models: ./generate-coreml-model.sh -h5 <model-name> <model-path>"
+  exit 1
+fi
+
+mname="$1"
+
+wd=$(dirname "$0")
+cd "$wd/../" || exit
+
+if [ "$mname" = "-h5" ]; then
+  mname="$2"
+  mpath="$3"
+  echo "$mpath"
+  python3 models/convert-h5-to-coreml.py --model-name "$mname" --model-path "$mpath" --encoder-only True
+else
+  python3 models/convert-whisper-to-coreml.py --model "$mname" --encoder-only True --optimize-ane True
+fi
+
+xcrun coremlc compile models/coreml-encoder-"${mname}".mlpackage models/
+rm -rf models/ggml-"${mname}"-encoder.mlmodelc
+mv -v models/coreml-encoder-"${mname}".mlmodelc models/ggml-"${mname}"-encoder.mlmodelc
+
+# TODO: decoder (sometime in the future maybe)
+#xcrun coremlc compile models/whisper-decoder-${mname}.mlpackage models/
+#rm -rf models/ggml-${mname}-decoder.mlmodelc
+#mv -v models/coreml_decoder_${mname}.mlmodelc models/ggml-${mname}-decoder.mlmodelc
--- a/models/generate-parakeet-test-model.py
+++ b/models/generate-parakeet-test-model.py
@@ -0,0 +1,190 @@
+#!/usr/bin/env python3
+import struct
+import sys
+import numpy as np
+from pathlib import Path
+import argparse
+
+def write_tensor(fout, name, data):
+    n_dims = len(data.shape)
+    data = data.astype(np.float32)
+    ftype = 0  # GGML_TYPE_F32
+
+    name_bytes = name.encode('utf-8')
+    fout.write(struct.pack("iii", n_dims, len(name_bytes), ftype))
+    for i in range(n_dims):
+        fout.write(struct.pack("i", data.shape[n_dims - 1 - i]))
+    fout.write(name_bytes)
+    data.tofile(fout)
+
+def generate(output_path, n_fft_override=None):
+    rng = np.random.default_rng(42)
+
+    hparams = {
+        'n_vocab':                10,
+        'n_audio_ctx':            3200,
+        'n_audio_state':          8,
+        'n_audio_head':           2,
+        'n_audio_layer':          1,
+        'n_mels':                 16,
+        'ftype':                  0,
+        'n_fft':                  64,
+        'subsampling_factor':     8,
+        'n_subsampling_channels': 4,
+        'n_conv_kernel':          3,
+        'n_pred_dim':             8,
+        'n_pred_layers':          1,
+        'n_tdt_durations':        2,
+        'n_max_tokens':           5,
+    }
+
+    if n_fft_override is not None:
+        hparams['n_fft'] = n_fft_override
+
+    n_vocab    = hparams['n_vocab']
+    n_state    = hparams['n_audio_state']
+    n_head     = hparams['n_audio_head']
+    n_layer    = hparams['n_audio_layer']
+    n_mels     = hparams['n_mels']
+    n_fft      = hparams['n_fft']
+    n_sub_fac  = hparams['subsampling_factor']
+    n_sub_ch   = hparams['n_subsampling_channels']
+    n_conv_ker = hparams['n_conv_kernel']
+    dec_dim    = hparams['n_pred_dim']
+    n_pred_l   = hparams['n_pred_layers']
+    n_tdt      = hparams['n_tdt_durations']
+
+    n_pre_enc     = (n_mels // n_sub_fac) * n_sub_ch
+    n_head_dim    = n_state // n_head
+    n_pred_embed  = n_vocab + 1
+    n_lstm_gates  = 4 * dec_dim
+    n_joint_out   = n_vocab + n_tdt + 1
+    n_freqs       = n_fft // 2 + 1
+
+    def f32(*shape):
+        return rng.standard_normal(shape).astype(np.float32)
+
+    with open(output_path, 'wb') as fout:
+        fout.write(struct.pack("I", 0x67676d6c))
+
+        for key in ['n_vocab',
+                    'n_audio_ctx',
+                    'n_audio_state',
+                    'n_audio_head',
+                    'n_audio_layer',
+                    'n_mels',
+                    'ftype',
+                    'n_fft',
+                    'subsampling_factor',
+                    'n_subsampling_channels',
+                    'n_conv_kernel',
+                    'n_pred_dim',
+                    'n_pred_layers',
+                    'n_tdt_durations',
+                    'n_max_tokens']:
+            fout.write(struct.pack("i", hparams[key]))
+
+        fout.write(struct.pack("i", n_mels))
+        fout.write(struct.pack("i", n_freqs))
+        # Mel filter must be non-negative.
+        np.abs(f32(n_mels, n_freqs)).tofile(fout)
+
+        fout.write(struct.pack("i", n_fft))
+        f32(n_fft).tofile(fout)
+
+        for d in range(n_tdt):
+            fout.write(struct.pack("I", d))
+
+        tokens = ['<unk>', '<s>', '</s>'] + [chr(ord('a') + i) for i in range(n_vocab - 3)]
+        assert len(tokens) == n_vocab
+        fout.write(struct.pack("i", n_vocab))
+        for tok in tokens:
+            tok_bytes = tok.encode('utf-8')
+            fout.write(struct.pack("i", len(tok_bytes)))
+            fout.write(tok_bytes)
+
+        write_tensor(fout, "encoder.pre_encode.out.weight",    f32(n_state, n_pre_enc))
+        write_tensor(fout, "encoder.pre_encode.out.bias",      f32(n_state))
+
+        write_tensor(fout, "encoder.pre_encode.conv.0.weight", f32(n_sub_ch, 1, 3, 3))
+        write_tensor(fout, "encoder.pre_encode.conv.0.bias",   f32(1, n_sub_ch, 1, 1))
+
+        write_tensor(fout, "encoder.pre_encode.conv.2.weight", f32(n_sub_ch, 1, 3, 3))
+        write_tensor(fout, "encoder.pre_encode.conv.2.bias",   f32(1, n_sub_ch, 1, 1))
+
+        write_tensor(fout, "encoder.pre_encode.conv.3.weight", f32(n_sub_ch, n_sub_ch, 1, 1))
+        write_tensor(fout, "encoder.pre_encode.conv.3.bias",   f32(1, n_sub_ch, 1, 1))
+
+        write_tensor(fout, "encoder.pre_encode.conv.5.weight", f32(n_sub_ch, 1, 3, 3))
+        write_tensor(fout, "encoder.pre_encode.conv.5.bias",   f32(1, n_sub_ch, 1, 1))
+
+        write_tensor(fout, "encoder.pre_encode.conv.6.weight", f32(n_sub_ch, n_sub_ch, 1, 1))
+        write_tensor(fout, "encoder.pre_encode.conv.6.bias",   f32(1, n_sub_ch, 1, 1))
+
+        for i in range(n_layer):
+            p = f"encoder.layers.{i}"
+
+            write_tensor(fout, f"{p}.norm_feed_forward1.weight",     f32(n_state))
+            write_tensor(fout, f"{p}.norm_feed_forward1.bias",       f32(n_state))
+            write_tensor(fout, f"{p}.feed_forward1.linear1.weight",  f32(4*n_state, n_state))
+            write_tensor(fout, f"{p}.feed_forward1.linear2.weight",  f32(n_state, 4*n_state))
+
+            write_tensor(fout, f"{p}.norm_conv.weight",              f32(n_state))
+            write_tensor(fout, f"{p}.norm_conv.bias",                f32(n_state))
+            write_tensor(fout, f"{p}.conv.pointwise_conv1.weight",   f32(2*n_state, n_state))
+            write_tensor(fout, f"{p}.conv.depthwise_conv.weight",    f32(n_state, n_conv_ker))
+            write_tensor(fout, f"{p}.conv.batch_norm.weight",        f32(n_state))
+            write_tensor(fout, f"{p}.conv.batch_norm.bias",          f32(n_state))
+            write_tensor(fout, f"{p}.conv.batch_norm.running_mean",  f32(n_state))
+            write_tensor(fout, f"{p}.conv.batch_norm.running_var",   np.abs(f32(n_state)))
+            num_batches = np.zeros(1, dtype=np.int32)
+            write_tensor(fout, f"{p}.conv.batch_norm.num_batches_tracked", num_batches)
+            write_tensor(fout, f"{p}.conv.pointwise_conv2.weight",   f32(n_state, n_state))
+
+            write_tensor(fout, f"{p}.norm_self_att.weight",          f32(n_state))
+            write_tensor(fout, f"{p}.norm_self_att.bias",            f32(n_state))
+
+            write_tensor(fout, f"{p}.self_attn.pos_bias_u",          f32(n_head, n_head_dim))
+            write_tensor(fout, f"{p}.self_attn.pos_bias_v",          f32(n_head, n_head_dim))
+            write_tensor(fout, f"{p}.self_attn.linear_q.weight",     f32(n_state, n_state))
+            write_tensor(fout, f"{p}.self_attn.linear_k.weight",     f32(n_state, n_state))
+            write_tensor(fout, f"{p}.self_attn.linear_v.weight",     f32(n_state, n_state))
+            write_tensor(fout, f"{p}.self_attn.linear_out.weight",   f32(n_state, n_state))
+            write_tensor(fout, f"{p}.self_attn.linear_pos.weight",   f32(n_state, n_state))
+
+            write_tensor(fout, f"{p}.norm_feed_forward2.weight",     f32(n_state))
+            write_tensor(fout, f"{p}.norm_feed_forward2.bias",       f32(n_state))
+            write_tensor(fout, f"{p}.feed_forward2.linear1.weight",  f32(4*n_state, n_state))
+            write_tensor(fout, f"{p}.feed_forward2.linear2.weight",  f32(n_state, 4*n_state))
+
+            write_tensor(fout, f"{p}.norm_out.weight",               f32(n_state))
+            write_tensor(fout, f"{p}.norm_out.bias",                 f32(n_state))
+
+        write_tensor(fout, "decoder.prediction.embed.weight", f32(n_pred_embed, dec_dim))
+
+        def reorder_gates(data):
+            h = data.shape[0] // 4
+            return np.concatenate([data[:h], data[h:2*h], data[3*h:], data[2*h:3*h]], axis=0)
+
+        for i in range(n_pred_l):
+            base = f"decoder.prediction.dec_rnn.lstm"
+            write_tensor(fout, f"{base}.weight_ih_l{i}", reorder_gates(f32(n_lstm_gates, dec_dim)))
+            write_tensor(fout, f"{base}.weight_hh_l{i}", reorder_gates(f32(n_lstm_gates, dec_dim)))
+            write_tensor(fout, f"{base}.bias_h_l{i}",    reorder_gates(f32(n_lstm_gates) + f32(n_lstm_gates)))
+
+        write_tensor(fout, "joint.pred.weight",        f32(dec_dim, dec_dim))
+        write_tensor(fout, "joint.pred.bias",          f32(dec_dim))
+        write_tensor(fout, "joint.enc.weight",         f32(dec_dim, n_state))
+        write_tensor(fout, "joint.enc.bias",           f32(dec_dim))
+        write_tensor(fout, "joint.joint_net.2.weight", f32(n_joint_out, dec_dim))
+        write_tensor(fout, "joint.joint_net.2.bias",   f32(n_joint_out))
+
+    size = Path(output_path).stat().st_size
+    print(f"Generated {output_path} ({size / 1024:.1f} KB)")
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser()
+    parser.add_argument('output', nargs='?', default='models/for-tests-ggml-parakeet-tdt.bin')
+    parser.add_argument('--n-fft',type=int, default=None)
+    args = parser.parse_args()
+    generate(args.output, args.n_fft)
--- a/models/ggml_to_pt.py
+++ b/models/ggml_to_pt.py
@@ -0,0 +1,109 @@
+import struct
+import torch
+import numpy as np
+from collections import OrderedDict
+from pathlib import Path
+import sys
+
+if len(sys.argv) < 3:
+    print(
+        "Usage: convert-ggml-to-pt.py model.bin dir-output\n")
+    sys.exit(1)
+
+fname_inp = Path(sys.argv[1])
+dir_out = Path(sys.argv[2])
+fname_out = dir_out / "torch-model.pt"
+
+
+
+# Open the ggml file
+with open(fname_inp, "rb") as f:
+    # Read magic number and hyperparameters
+    magic_number, n_vocab, n_audio_ctx, n_audio_state, n_audio_head, n_audio_layer, n_text_ctx, n_text_state, n_text_head, n_text_layer, n_mels, use_f16 = struct.unpack("12i", f.read(48))
+    print(f"Magic number: {magic_number}")
+    print(f"Vocab size: {n_vocab}")
+    print(f"Audio context size: {n_audio_ctx}")
+    print(f"Audio state size: {n_audio_state}")
+    print(f"Audio head size: {n_audio_head}")
+    print(f"Audio layer size: {n_audio_layer}")
+    print(f"Text context size: {n_text_ctx}")
+    print(f"Text head size: {n_text_head}")
+    print(f"Mel size: {n_mels}")
+    # Read mel filters
+    # mel_filters = np.fromfile(f, dtype=np.float32, count=n_mels * 2).reshape(n_mels, 2)
+    # print(f"Mel filters: {mel_filters}")
+    filters_shape_0 = struct.unpack("i", f.read(4))[0]
+    print(f"Filters shape 0: {filters_shape_0}")
+    filters_shape_1 = struct.unpack("i", f.read(4))[0]
+    print(f"Filters shape 1: {filters_shape_1}")
+
+    # Read tokenizer tokens
+    # bytes = f.read(4)
+    # print(bytes)
+    
+
+    # for i in range(filters.shape[0]):
+    # for j in range(filters.shape[1]):
+    #     fout.write(struct.pack("f", filters[i][j]))
+    mel_filters = np.zeros((filters_shape_0, filters_shape_1))
+
+    for i in range(filters_shape_0):
+        for j in range(filters_shape_1):
+            mel_filters[i][j] = struct.unpack("f", f.read(4))[0]
+            
+    bytes_data = f.read(4) 
+    num_tokens = struct.unpack("i", bytes_data)[0]
+    tokens = {}
+
+
+    for _ in range(num_tokens):
+        token_len = struct.unpack("i", f.read(4))[0]
+        token = f.read(token_len)
+        tokens[token] = {}
+    
+    # Read model variables
+    model_state_dict = OrderedDict()
+    while True:
+        try:
+            n_dims, name_length, ftype = struct.unpack("iii", f.read(12))
+        except struct.error:
+            break  # End of file
+        dims = [struct.unpack("i", f.read(4))[0] for _ in range(n_dims)]
+        dims = dims[::-1]
+        name = f.read(name_length).decode("utf-8")
+        if ftype == 1:  # f16
+            data = np.fromfile(f, dtype=np.float16, count=np.prod(dims)).reshape(dims)
+        else:  # f32
+            data = np.fromfile(f, dtype=np.float32, count=np.prod(dims)).reshape(dims)
+
+            
+        if name in  ["encoder.conv1.bias", "encoder.conv2.bias"]:
+            
+            data = data[:, 0]
+        
+            
+        model_state_dict[name] = torch.from_numpy(data)
+    
+# Now you have the model's state_dict stored in model_state_dict
+# You can load this state_dict into a model with the same architecture
+
+# dims = ModelDimensions(**checkpoint["dims"])
+# model = Whisper(dims)
+from whisper import Whisper, ModelDimensions
+dims = ModelDimensions(
+    n_mels=n_mels,
+    n_audio_ctx=n_audio_ctx,
+    n_audio_state=n_audio_state,
+    n_audio_head=n_audio_head,
+    n_audio_layer=n_audio_layer,
+    n_text_ctx=n_text_ctx,
+    n_text_state=n_text_state,
+    n_text_head=n_text_head,
+    n_text_layer=n_text_layer,
+    n_vocab=n_vocab,
+)
+model = Whisper(dims)  # Replace with your model's class
+model.load_state_dict(model_state_dict)
+
+# Save the model in PyTorch format
+torch.save(model.state_dict(), fname_out)
--- a/models/requirements-coreml.txt
+++ b/models/requirements-coreml.txt
@@ -0,0 +1,4 @@
+torch
+coremltools
+openai-whisper
+ane_transformers
--- a/models/requirements-openvino.txt
+++ b/models/requirements-openvino.txt
@@ -0,0 +1,3 @@
+openvino>=2023.3.0
+openai-whisper
+onnxscript
--- a/models/requirements-parakeet.txt
+++ b/models/requirements-parakeet.txt
@@ -0,0 +1,3 @@
+torch
+numpy
+pyyaml