Search results

ggml-cuda : add TQ2_0 kernels, for ternary inference on GPU

Apple Metalhttps://en.wikipedia.org/wiki/Metal_(API)https://en.wikipedia.org/wiki/Metal_(API)

enhancementNew feature or requestNew feature or request

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

Nvidia GPUIssues specific to Nvidia GPUsIssues specific to Nvidia GPUs

performanceSpeed related topicsSpeed related topics

pythonpython script changespython script changes

Review Complexity : HighGenerally require indepth knowledge of LLMs or GPUsGenerally require indepth knowledge of LLMs or GPUs

testingEverything test relatedEverything test related

Status: Open (in progress).

ggml-org/llama.cppnumber 11183

#11183 In ggml-org/llama.cpp;

· compilade opened on Jan 10, 2025

ggml: skip excess iteration for pair whose vars same element when i2 == i1

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

Review Complexity : HighGenerally require indepth knowledge of LLMs or GPUsGenerally require indepth knowledge of LLMs or GPUs

Status: Open (in progress).

ggml-org/llama.cppnumber 9177

#9177 In ggml-org/llama.cpp;

· GermanAizek opened on Aug 25, 2024

ggml : make GeLU faster and more accurate on CPU

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

Review Complexity : HighGenerally require indepth knowledge of LLMs or GPUsGenerally require indepth knowledge of LLMs or GPUs

Status: Open (in progress).

ggml-org/llama.cppnumber 8878

#8878 In ggml-org/llama.cpp;

· jart opened on Aug 5, 2024

Add support for loongarch backend in sgemm.cpp

Review Complexity : HighGenerally require indepth knowledge of LLMs or GPUsGenerally require indepth knowledge of LLMs or GPUs

Status: Open (in progress).

ggml-org/llama.cppnumber 8726

#8726 In ggml-org/llama.cpp;

· Tianzhengshuyuan opened on Jul 27, 2024

llama : support Jamba hybrid Transformer-Mamba models

androidIssues specific to AndroidIssues specific to Android

embeddingsembedding related topicsembedding related topics

enhancementNew feature or requestNew feature or request

examples

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

modelModel specificModel specific

need feedbackTesting and feedback with results are neededTesting and feedback with results are needed

pythonpython script changespython script changes

refactoringRefactoringRefactoring

Review Complexity : HighGenerally require indepth knowledge of LLMs or GPUsGenerally require indepth knowledge of LLMs or GPUs

server

Status: Draft (not ready).

ggml-org/llama.cppnumber 7531

#7531 In ggml-org/llama.cpp;

· compilade opened on May 25, 2024

Introduce Q8_0 and Q4_0 with Bf16 delta values

examples

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

pythonpython script changespython script changes

Review Complexity : HighGenerally require indepth knowledge of LLMs or GPUsGenerally require indepth knowledge of LLMs or GPUs

Tensor Encoding Schemehttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemeshttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemes

Status: Open (in progress).

ggml-org/llama.cppnumber 7497

#7497 In ggml-org/llama.cpp;

· Srihari-mcw opened on May 23, 2024

Add token healing to <code>main</code> and <code>server</code>

enhancementNew feature or requestNew feature or request

examples

help wantedExtra attention is neededExtra attention is needed

need feedbackTesting and feedback with results are neededTesting and feedback with results are needed

Review Complexity : HighGenerally require indepth knowledge of LLMs or GPUsGenerally require indepth knowledge of LLMs or GPUs

server

Status: Open (in progress).

ggml-org/llama.cppnumber 7187

#7187 In ggml-org/llama.cpp;

· mare5x opened on May 9, 2024

support MiniCPM-V-2

demoDemonstrate some concept or idea, not intended to be mergedDemonstrate some concept or idea, not intended to be merged

enhancementNew feature or requestNew feature or request

examples

pythonpython script changespython script changes

Review Complexity : HighGenerally require indepth knowledge of LLMs or GPUsGenerally require indepth knowledge of LLMs or GPUs

Status: Open (in progress).

ggml-org/llama.cppnumber 6919

#6919 In ggml-org/llama.cpp;

· Achazwl opened on Apr 26, 2024

cuda : use amd wave sharing intrinsics for warp_reduce functions

performanceSpeed related topicsSpeed related topics

Review Complexity : HighGenerally require indepth knowledge of LLMs or GPUsGenerally require indepth knowledge of LLMs or GPUs

Status: Open (in progress).

ggml-org/llama.cppnumber 6522

#6522 In ggml-org/llama.cpp;

· Engininja2 opened on Apr 7, 2024

Adding Support for Custom Qwen2moe Architectures with mergekit-qwen2

modelModel specificModel specific

Review Complexity : HighGenerally require indepth knowledge of LLMs or GPUsGenerally require indepth knowledge of LLMs or GPUs

Status: Draft (not ready).

ggml-org/llama.cppnumber 6453

#6453 In ggml-org/llama.cpp;

· DisOOM opened on Apr 3, 2024

Smooth Sampling / Quadratic Sampling support

generation qualityQuality of model outputQuality of model output

performanceSpeed related topicsSpeed related topics

Review Complexity : HighGenerally require indepth knowledge of LLMs or GPUsGenerally require indepth knowledge of LLMs or GPUs

Status: Open (in progress).

ggml-org/llama.cppnumber 6445

#6445 In ggml-org/llama.cpp;

· kalomaze opened on Apr 2, 2024

Xeon Phi (Knights Corner) Support.

enhancementNew feature or requestNew feature or request

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

performanceSpeed related topicsSpeed related topics

Review Complexity : HighGenerally require indepth knowledge of LLMs or GPUsGenerally require indepth knowledge of LLMs or GPUs

Status: Open (in progress).

ggml-org/llama.cppnumber 6440

#6440 In ggml-org/llama.cpp;

· julialongtin opened on Apr 2, 2024