948ef47106
Some checks failed
Bindings Tests (Go) / ubuntu-22 (push) Failing after 31s
CI (sycl) / ubuntu-22-cmake-sycl (linux/arm/v7, icx, icpx, ON) (push) Failing after 3s
CI (sycl) / ubuntu-22-cmake-sycl (linux/ppc64le, icx, icpx, ON) (push) Failing after 2s
CI (sycl) / ubuntu-22-cmake-sycl-fp16 (linux/amd64, icx, icpx, ON) (push) Failing after 3s
CI (sycl) / ubuntu-22-cmake-sycl-fp16 (linux/arm/v7, icx, icpx, ON) (push) Failing after 3s
CI (sycl) / ubuntu-22-cmake-sycl-fp16 (linux/arm64, icx, icpx, ON) (push) Failing after 3s
CI (sycl) / ubuntu-22-cmake-sycl-fp16 (linux/ppc64le, icx, icpx, ON) (push) Failing after 3s
CI (vad) / vad (push) Failing after 3s
CI (wasm) / emscripten (Release) (push) Failing after 4s
Bindings Tests (Ruby) / ubuntu-22 (push) Failing after 37s
CI (android) / android (push) Failing after 16s
CI (android) / android_java (push) Failing after 3s
CI (clang) / ubuntu-22-clang (linux/amd64, Debug) (push) Failing after 3s
CI (clang) / ubuntu-22-clang (linux/amd64, Release) (push) Failing after 2s
CI (clang) / ubuntu-22-clang (linux/ppc64le, Debug) (push) Failing after 3s
CI (clang) / ubuntu-22-clang (linux/ppc64le, Release) (push) Failing after 3s
CI (cpu) / ggml-ci-x64-cpu-low-perf (push) Failing after 3s
CI (cpu) / ggml-ci-x64-cpu-high-perf (push) Failing after 2s
CI (gcc) / ubuntu-22-gcc (linux/amd64, Debug) (push) Failing after 3s
CI (gcc) / ubuntu-22-gcc (linux/amd64, Release) (push) Failing after 3s
CI (gcc) / ubuntu-22-gcc (linux/ppc64le, Debug) (push) Failing after 3s
CI (gcc) / ubuntu-22-gcc (linux/ppc64le, Release) (push) Failing after 4s
CI (gcc) / ubuntu-22-gcc-arm-v7 (linux/arm/v7, Debug) (push) Failing after 3s
CI (gcc) / ubuntu-22-gcc-arm-v7 (linux/arm/v7, Release) (push) Failing after 3s
CI (quantize) / quantize (push) Failing after 3s
CI (sanitize) / ubuntu-22-gcc-sanitized (ADDRESS) (push) Failing after 3s
CI (sanitize) / ubuntu-22-gcc-sanitized (THREAD) (push) Failing after 3s
CI (sanitize) / ubuntu-22-gcc-sanitized (UNDEFINED) (push) Failing after 2s
CI (sycl) / ubuntu-22-cmake-sycl (linux/amd64, icx, icpx, ON) (push) Failing after 3s
CI (sycl) / ubuntu-22-cmake-sycl (linux/arm64, icx, icpx, ON) (push) Failing after 3s
Examples WASM / deploy-wasm-github-pages (push) Failing after 3s
Examples Tests / addon_node-ubuntu-22 (20.x) (push) Failing after 3s
Examples Tests / addon_node-ubuntu-22 (22.x) (push) Failing after 2s
CI (coreml) / coreml-base-en (push) Has been cancelled
CI (self-hosted) / gpu-vulkan (push) Has been cancelled
CI (self-hosted) / npu-amd-windows (push) Has been cancelled
CI (self-hosted) / gpu-vulkan-nvidia-cm2 (push) Has been cancelled
CI (clang) / ubuntu-22-clang-arm64 (Debug) (push) Has been cancelled
CI (clang) / ubuntu-22-clang-arm64 (Release) (push) Has been cancelled
CI (self-hosted) / gpu-cuda (push) Has been cancelled
CI (self-hosted) / gpu-vulkan-nvidia-cm (push) Has been cancelled
CI (cpu) / ggml-ci-arm64-cpu-low-perf (push) Has been cancelled
CI (cpu) / ggml-ci-arm64-cpu-high-perf (push) Has been cancelled
CI (cpu) / ggml-ci-arm64-cpu-high-perf-sve (push) Has been cancelled
CI (freebsd) / freeBSD-latest (push) Has been cancelled
CI (self-hosted) / gpu-metal (push) Has been cancelled
CI (gcc) / ubuntu-22-gcc-arm64 (Debug) (push) Has been cancelled
CI (gcc) / ubuntu-22-gcc-arm64 (Release) (push) Has been cancelled
CI (macOS) / macOS-latest (generic/platform=iOS) (push) Has been cancelled
CI (macOS) / macOS-latest (generic/platform=macOS) (push) Has been cancelled
CI (macOS) / macOS-latest (generic/platform=tvOS) (push) Has been cancelled
CI (self-hosted) / npu-amd-linux (push) Has been cancelled
CI (windows) / windows-msys2 (Release, clang, clang-x86_64, CLANG64) (push) Has been cancelled
CI (windows) / windows-msys2 (Release, gcc, ucrt-x86_64, UCRT64) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/main-cuda.Dockerfile platform:linux/amd64 runs_on:ubuntu-24.04 tag:main-cuda]) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/main-intel.Dockerfile platform:linux/amd64 runs_on:ubuntu-24.04 tag:main-intel]) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/main-musa.Dockerfile platform:linux/amd64 runs_on:ubuntu-24.04 tag:main-musa]) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/main-vulkan.Dockerfile platform:linux/amd64 runs_on:ubuntu-24.04 tag:main-vulkan]) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/main-vulkan.Dockerfile platform:linux/arm64 runs_on:ubuntu-24.04-arm tag:main-vulkan-arm64]) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/main.Dockerfile platform:linux/amd64 runs_on:ubuntu-24.04 tag:main]) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/main.Dockerfile platform:linux/arm64 runs_on:ubuntu-24.04-arm tag:main-arm64]) (push) Has been cancelled
Mark inactive issues stale / mark-issues (push) Failing after 1s
82 lines
2.6 KiB
Markdown
82 lines
2.6 KiB
Markdown
# whisper.cpp
|
|
|
|
Node.js package for Whisper speech recognition
|
|
|
|
Package: https://www.npmjs.com/package/whisper.cpp
|
|
|
|
## Details
|
|
|
|
The performance is comparable to when running `whisper.cpp` in the browser via WASM.
|
|
|
|
The API is currently very rudimentary: [bindings/javascript/emscripten.cpp](/bindings/javascript/emscripten.cpp)
|
|
|
|
For sample usage check [tests/test-whisper.js](/tests/test-whisper.js)
|
|
|
|
## Package building + test
|
|
|
|
```bash
|
|
# load emscripten
|
|
source /path/to/emsdk/emsdk_env.sh
|
|
|
|
# clone repo
|
|
git clone https://github.com/ggerganov/whisper.cpp
|
|
cd whisper.cpp
|
|
|
|
# grab base.en model
|
|
./models/download-ggml-model.sh base.en
|
|
|
|
# prepare PCM sample for testing
|
|
ffmpeg -i samples/jfk.wav -f f32le -acodec pcm_f32le samples/jfk.pcmf32
|
|
|
|
# build
|
|
mkdir build-em && cd build-em
|
|
emcmake cmake .. && make -j
|
|
|
|
# run test
|
|
node ../tests/test-whisper.js
|
|
|
|
# For Node.js versions prior to v16.4.0, experimental features need to be enabled:
|
|
node --experimental-wasm-threads --experimental-wasm-simd ../tests/test-whisper.js
|
|
|
|
# publish npm package
|
|
make publish-npm
|
|
```
|
|
|
|
## Sample run
|
|
|
|
```text
|
|
$ node --experimental-wasm-threads --experimental-wasm-simd ../tests/test-whisper.js
|
|
|
|
whisper_model_load: loading model from 'whisper.bin'
|
|
whisper_model_load: n_vocab = 51864
|
|
whisper_model_load: n_audio_ctx = 1500
|
|
whisper_model_load: n_audio_state = 512
|
|
whisper_model_load: n_audio_head = 8
|
|
whisper_model_load: n_audio_layer = 6
|
|
whisper_model_load: n_text_ctx = 448
|
|
whisper_model_load: n_text_state = 512
|
|
whisper_model_load: n_text_head = 8
|
|
whisper_model_load: n_text_layer = 6
|
|
whisper_model_load: n_mels = 80
|
|
whisper_model_load: f16 = 1
|
|
whisper_model_load: type = 2
|
|
whisper_model_load: adding 1607 extra tokens
|
|
whisper_model_load: mem_required = 506.00 MB
|
|
whisper_model_load: ggml ctx size = 140.60 MB
|
|
whisper_model_load: memory size = 22.83 MB
|
|
whisper_model_load: model size = 140.54 MB
|
|
|
|
system_info: n_threads = 8 / 10 | AVX = 0 | AVX2 = 0 | AVX512 = 0 | NEON = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 1 | BLAS = 0 |
|
|
|
|
operator(): processing 176000 samples, 11.0 sec, 8 threads, 1 processors, lang = en, task = transcribe ...
|
|
|
|
[00:00:00.000 --> 00:00:11.000] And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country.
|
|
|
|
whisper_print_timings: load time = 162.37 ms
|
|
whisper_print_timings: mel time = 183.70 ms
|
|
whisper_print_timings: sample time = 4.27 ms
|
|
whisper_print_timings: encode time = 8582.63 ms / 1430.44 ms per layer
|
|
whisper_print_timings: decode time = 436.16 ms / 72.69 ms per layer
|
|
whisper_print_timings: total time = 9370.90 ms
|
|
```
|