Packages

erllama

0.1.2
0.8.0 retired 0.7.0 retired 0.6.2 retired 0.6.1 retired 0.6.0 retired 0.5.1 retired 0.5.0 retired 0.4.0 retired 0.3.0 retired 0.2.0 retired 0.1.2 retired 0.1.1 retired 0.1.0 retired

Native Erlang/OTP wrapper around llama.cpp with token-exact tiered KV cache; cache more warm state than fits in RAM

Retired package: Renamed - Renamed to barrel_inference: https://github.com/barrel-platform/barrel_inference
HexDocs

Current section

Files

Jump to
Readme
13 Versions
0 Dependencies
Dependants
Files
Activity
erllama c_src llama.cpp ggml src ggml-cuda mmid.cuh
Raw

c_src/llama.cpp/ggml/src/ggml-cuda/mmid.cuh 

#pragma once



void ggml_cuda_launch_mm_ids_helper(

        const int32_t * ids, int32_t * ids_src1, int32_t * ids_dst, int32_t * expert_bounds,

        int n_experts, int n_tokens, int n_expert_used, int nchannels_y, int si1, int sis1, cudaStream_t stream);