首页下载资源后端C/C++ 中的 LLM 推理

ZIPC/C++ 中的 LLM 推理

2301_8048821418.45MB需要积分:1

资源文件列表:

llama.cpp-master.zip 大约有1096个文件
  1. llama.cpp-master/
  2. llama.cpp-master/.clang-tidy 791B
  3. llama.cpp-master/.devops/
  4. llama.cpp-master/.devops/cloud-v-pipeline 1.05KB
  5. llama.cpp-master/.devops/full-cuda.Dockerfile 848B
  6. llama.cpp-master/.devops/full-rocm.Dockerfile 1.1KB
  7. llama.cpp-master/.devops/full.Dockerfile 470B
  8. llama.cpp-master/.devops/llama-cli-cuda.Dockerfile 864B
  9. llama.cpp-master/.devops/llama-cli-intel.Dockerfile 740B
  10. llama.cpp-master/.devops/llama-cli-rocm.Dockerfile 1.01KB
  11. llama.cpp-master/.devops/llama-cli-vulkan.Dockerfile 724B
  12. llama.cpp-master/.devops/llama-cli.Dockerfile 376B
  13. llama.cpp-master/.devops/llama-cpp-cuda.srpm.spec 2.56KB
  14. llama.cpp-master/.devops/llama-cpp.srpm.spec 2.63KB
  15. llama.cpp-master/.devops/llama-server-cuda.Dockerfile 1020B
  16. llama.cpp-master/.devops/llama-server-intel.Dockerfile 900B
  17. llama.cpp-master/.devops/llama-server-rocm.Dockerfile 1.18KB
  18. llama.cpp-master/.devops/llama-server-vulkan.Dockerfile 843B
  19. llama.cpp-master/.devops/llama-server.Dockerfile 519B
  20. llama.cpp-master/.devops/nix/
  21. llama.cpp-master/.devops/nix/apps.nix 434B
  22. llama.cpp-master/.devops/nix/devshells.nix 279B
  23. llama.cpp-master/.devops/nix/docker.nix 850B
  24. llama.cpp-master/.devops/nix/jetson-support.nix 1.05KB
  25. llama.cpp-master/.devops/nix/nixpkgs-instances.nix 1.67KB
  26. llama.cpp-master/.devops/nix/package.nix 9.74KB
  27. llama.cpp-master/.devops/nix/scope.nix 514B
  28. llama.cpp-master/.devops/nix/sif.nix 729B
  29. llama.cpp-master/.devops/tools.sh 1.67KB
  30. llama.cpp-master/.dockerignore 158B
  31. llama.cpp-master/.ecrc 80B
  32. llama.cpp-master/.editorconfig 599B
  33. llama.cpp-master/.flake8 544B
  34. llama.cpp-master/.github/
  35. llama.cpp-master/.github/ISSUE_TEMPLATE/
  36. llama.cpp-master/.github/ISSUE_TEMPLATE/01-bug-low.yml 1.7KB
  37. llama.cpp-master/.github/ISSUE_TEMPLATE/02-bug-medium.yml 1.71KB
  38. llama.cpp-master/.github/ISSUE_TEMPLATE/03-bug-high.yml 1.72KB
  39. llama.cpp-master/.github/ISSUE_TEMPLATE/04-bug-critical.yml 1.7KB
  40. llama.cpp-master/.github/ISSUE_TEMPLATE/05-enhancement.yml 2.35KB
  41. llama.cpp-master/.github/ISSUE_TEMPLATE/06-research.yml 1.69KB
  42. llama.cpp-master/.github/ISSUE_TEMPLATE/07-refactor.yml 1.2KB
  43. llama.cpp-master/.github/ISSUE_TEMPLATE/config.yml 524B
  44. llama.cpp-master/.github/labeler.yml 2.32KB
  45. llama.cpp-master/.github/pull_request_template.md 193B
  46. llama.cpp-master/.github/workflows/
  47. llama.cpp-master/.github/workflows/bench.yml 10.42KB
  48. llama.cpp-master/.github/workflows/build.yml 42.25KB
  49. llama.cpp-master/.github/workflows/close-issue.yml 717B
  50. llama.cpp-master/.github/workflows/docker.yml 5.02KB
  51. llama.cpp-master/.github/workflows/editorconfig.yml 607B
  52. llama.cpp-master/.github/workflows/gguf-publish.yml 1.2KB
  53. llama.cpp-master/.github/workflows/labeler.yml 355B
  54. llama.cpp-master/.github/workflows/nix-ci-aarch64.yml 2.23KB
  55. llama.cpp-master/.github/workflows/nix-ci.yml 2.51KB
  56. llama.cpp-master/.github/workflows/nix-flake-update.yml 607B
  57. llama.cpp-master/.github/workflows/nix-publish-flake.yml 1.11KB
  58. llama.cpp-master/.github/workflows/python-check-requirements.yml 966B
  59. llama.cpp-master/.github/workflows/python-lint.yml 561B
  60. llama.cpp-master/.github/workflows/python-type-check.yml 995B
  61. llama.cpp-master/.github/workflows/server.yml 6KB
  62. llama.cpp-master/.gitignore 1.45KB
  63. llama.cpp-master/.gitmodules 94B
  64. llama.cpp-master/.pre-commit-config.yaml 447B
  65. llama.cpp-master/AUTHORS 32.93KB
  66. llama.cpp-master/CMakeLists.txt 6.29KB
  67. llama.cpp-master/CMakePresets.json 2.82KB
  68. llama.cpp-master/CONTRIBUTING.md 2.2KB
  69. llama.cpp-master/LICENSE 1.05KB
  70. llama.cpp-master/Makefile 48.57KB
  71. llama.cpp-master/Package.swift 2KB
  72. llama.cpp-master/README.md 28.74KB
  73. llama.cpp-master/SECURITY.md 4.97KB
  74. llama.cpp-master/ci/
  75. llama.cpp-master/ci/README.md 1.06KB
  76. llama.cpp-master/ci/run.sh 37.51KB
  77. llama.cpp-master/cmake/
  78. llama.cpp-master/cmake/arm64-windows-llvm.cmake 592B
  79. llama.cpp-master/cmake/arm64-windows-msvc.cmake 192B
  80. llama.cpp-master/cmake/build-info.cmake 1.57KB
  81. llama.cpp-master/cmake/git-vars.cmake 717B
  82. llama.cpp-master/cmake/llama-config.cmake.in 2.39KB
  83. llama.cpp-master/cmake/llama.pc.in 250B
  84. llama.cpp-master/common/
  85. llama.cpp-master/common/CMakeLists.txt 2.74KB
  86. llama.cpp-master/common/base64.hpp 12.58KB
  87. llama.cpp-master/common/build-info.cpp.in 186B
  88. llama.cpp-master/common/cmake/
  89. llama.cpp-master/common/cmake/build-info-gen-cpp.cmake 943B
  90. llama.cpp-master/common/common.cpp 135.3KB
  91. llama.cpp-master/common/common.h 19.7KB
  92. llama.cpp-master/common/console.cpp 15.86KB
  93. llama.cpp-master/common/console.h 359B
  94. llama.cpp-master/common/grammar-parser.cpp 21.75KB
  95. llama.cpp-master/common/grammar-parser.h 874B
  96. llama.cpp-master/common/json-schema-to-grammar.cpp 42.49KB
  97. llama.cpp-master/common/json-schema-to-grammar.h 211B
  98. llama.cpp-master/common/json.hpp 898.69KB
  99. llama.cpp-master/common/log.h 24.09KB
  100. llama.cpp-master/common/ngram-cache.cpp 11.08KB
  101. llama.cpp-master/common/ngram-cache.h 3.98KB
  102. llama.cpp-master/common/sampling.cpp 17.68KB
  103. llama.cpp-master/common/sampling.h 6.33KB
  104. llama.cpp-master/common/stb_image.h 313.42KB
  105. llama.cpp-master/common/train.cpp 64.78KB
  106. llama.cpp-master/common/train.h 7.7KB
  107. llama.cpp-master/convert_hf_to_gguf.py 169.04KB
  108. llama.cpp-master/convert_hf_to_gguf_update.py 14.43KB
  109. llama.cpp-master/convert_llama_ggml_to_gguf.py 18.63KB
  110. llama.cpp-master/convert_lora_to_gguf.py 14.04KB
  111. llama.cpp-master/docs/
  112. llama.cpp-master/docs/android.md 2.42KB
  113. llama.cpp-master/docs/backend/
  114. llama.cpp-master/docs/backend/BLIS.md 1.7KB
  115. llama.cpp-master/docs/backend/SYCL.md 23.63KB
  116. llama.cpp-master/docs/build.md 19.97KB
  117. llama.cpp-master/docs/development/
  118. llama.cpp-master/docs/development/HOWTO-add-model.md 4.8KB
  119. llama.cpp-master/docs/development/debugging-tests.md 3.1KB
  120. llama.cpp-master/docs/development/llama-star/
  121. llama.cpp-master/docs/development/llama-star/idea-arch.key 477.14KB
  122. llama.cpp-master/docs/development/llama-star/idea-arch.pdf 41.34KB
  123. llama.cpp-master/docs/development/token_generation_performance_tips.md 2.25KB
  124. llama.cpp-master/docs/docker.md 4.72KB
  125. llama.cpp-master/docs/install.md 872B
  126. llama.cpp-master/examples/
  127. llama.cpp-master/examples/CMakeLists.txt 1.33KB
  128. llama.cpp-master/examples/Miku.sh 2.57KB
  129. llama.cpp-master/examples/baby-llama/
  130. llama.cpp-master/examples/baby-llama/CMakeLists.txt 239B
  131. llama.cpp-master/examples/baby-llama/baby-llama.cpp 61.05KB
  132. llama.cpp-master/examples/base-translate.sh 1001B
  133. llama.cpp-master/examples/batched-bench/
  134. llama.cpp-master/examples/batched-bench/CMakeLists.txt 245B
  135. llama.cpp-master/examples/batched-bench/README.md 2.74KB
  136. llama.cpp-master/examples/batched-bench/batched-bench.cpp 6.63KB
  137. llama.cpp-master/examples/batched.swift/
  138. llama.cpp-master/examples/batched.swift/.gitignore 173B
  139. llama.cpp-master/examples/batched.swift/Makefile 230B
  140. llama.cpp-master/examples/batched.swift/Package.swift 766B
  141. llama.cpp-master/examples/batched.swift/README.md 112B
  142. llama.cpp-master/examples/batched.swift/Sources/
  143. llama.cpp-master/examples/batched.swift/Sources/main.swift 7.57KB
  144. llama.cpp-master/examples/batched/
  145. llama.cpp-master/examples/batched/CMakeLists.txt 233B
  146. llama.cpp-master/examples/batched/README.md 1.39KB
  147. llama.cpp-master/examples/batched/batched.cpp 7.81KB
  148. llama.cpp-master/examples/benchmark/
  149. llama.cpp-master/examples/benchmark/CMakeLists.txt 312B
  150. llama.cpp-master/examples/benchmark/benchmark-matmult.cpp 9.61KB
  151. llama.cpp-master/examples/chat-13B.bat 2.39KB
  152. llama.cpp-master/examples/chat-13B.sh 1.31KB
  153. llama.cpp-master/examples/chat-persistent.sh 4.93KB
  154. llama.cpp-master/examples/chat-vicuna.sh 1.3KB
  155. llama.cpp-master/examples/chat.sh 349B
  156. llama.cpp-master/examples/convert-llama2c-to-ggml/
  157. llama.cpp-master/examples/convert-llama2c-to-ggml/CMakeLists.txt 265B
  158. llama.cpp-master/examples/convert-llama2c-to-ggml/README.md 1.52KB
  159. llama.cpp-master/examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp 34.26KB
  160. llama.cpp-master/examples/convert_legacy_llama.py 58.15KB
  161. llama.cpp-master/examples/cvector-generator/
  162. llama.cpp-master/examples/cvector-generator/CMakeLists.txt 261B
  163. llama.cpp-master/examples/cvector-generator/README.md 1.54KB
  164. llama.cpp-master/examples/cvector-generator/completions.txt 6.75KB
  165. llama.cpp-master/examples/cvector-generator/cvector-generator.cpp 17.91KB
  166. llama.cpp-master/examples/cvector-generator/mean.hpp 1.49KB
  167. llama.cpp-master/examples/cvector-generator/negative.txt 989B
  168. llama.cpp-master/examples/cvector-generator/pca.hpp 11.43KB
  169. llama.cpp-master/examples/cvector-generator/positive.txt 955B
  170. llama.cpp-master/examples/deprecation-warning/
  171. llama.cpp-master/examples/deprecation-warning/README.md 1.64KB
  172. llama.cpp-master/examples/deprecation-warning/deprecation-warning.cpp 1.15KB
  173. llama.cpp-master/examples/embedding/
  174. llama.cpp-master/examples/embedding/CMakeLists.txt 237B
  175. llama.cpp-master/examples/embedding/README.md 2.14KB
  176. llama.cpp-master/examples/embedding/embedding.cpp 9.36KB
  177. llama.cpp-master/examples/eval-callback/
  178. llama.cpp-master/examples/eval-callback/CMakeLists.txt 530B
  179. llama.cpp-master/examples/eval-callback/README.md 4.61KB
  180. llama.cpp-master/examples/eval-callback/eval-callback.cpp 6.09KB
  181. llama.cpp-master/examples/export-lora/
  182. llama.cpp-master/examples/export-lora/CMakeLists.txt 241B
  183. llama.cpp-master/examples/export-lora/README.md 1.12KB
  184. llama.cpp-master/examples/export-lora/export-lora.cpp 15.98KB
  185. llama.cpp-master/examples/gbnf-validator/
  186. llama.cpp-master/examples/gbnf-validator/CMakeLists.txt 247B
  187. llama.cpp-master/examples/gbnf-validator/gbnf-validator.cpp 4.36KB
  188. llama.cpp-master/examples/gguf-hash/
  189. llama.cpp-master/examples/gguf-hash/CMakeLists.txt 618B
  190. llama.cpp-master/examples/gguf-hash/README.md 10.41KB
  191. llama.cpp-master/examples/gguf-hash/deps/
  192. llama.cpp-master/examples/gguf-hash/deps/rotate-bits/
  193. llama.cpp-master/examples/gguf-hash/deps/rotate-bits/package.json 255B
  194. llama.cpp-master/examples/gguf-hash/deps/rotate-bits/rotate-bits.h 1017B
  195. llama.cpp-master/examples/gguf-hash/deps/sha1/
  196. llama.cpp-master/examples/gguf-hash/deps/sha1/package.json 200B
  197. llama.cpp-master/examples/gguf-hash/deps/sha1/sha1.c 7.44KB
  198. llama.cpp-master/examples/gguf-hash/deps/sha1/sha1.h 717B
  199. llama.cpp-master/examples/gguf-hash/deps/sha256/
  200. llama.cpp-master/examples/gguf-hash/deps/sha256/package.json 283B
  201. llama.cpp-master/examples/gguf-hash/deps/sha256/sha256.c 5.16KB
  202. llama.cpp-master/examples/gguf-hash/deps/sha256/sha256.h 549B
  203. llama.cpp-master/examples/gguf-hash/deps/xxhash/
  204. llama.cpp-master/examples/gguf-hash/deps/xxhash/clib.json 255B
  205. llama.cpp-master/examples/gguf-hash/deps/xxhash/xxhash.c 1.81KB
  206. llama.cpp-master/examples/gguf-hash/deps/xxhash/xxhash.h 258.54KB
  207. llama.cpp-master/examples/gguf-hash/gguf-hash.cpp 23.38KB
  208. llama.cpp-master/examples/gguf-split/
  209. llama.cpp-master/examples/gguf-split/CMakeLists.txt 239B
  210. llama.cpp-master/examples/gguf-split/README.md 343B
  211. llama.cpp-master/examples/gguf-split/gguf-split.cpp 19.4KB
  212. llama.cpp-master/examples/gguf-split/tests.sh 2.12KB
  213. llama.cpp-master/examples/gguf/
  214. llama.cpp-master/examples/gguf/CMakeLists.txt 219B
  215. llama.cpp-master/examples/gguf/gguf.cpp 7.92KB
  216. llama.cpp-master/examples/gritlm/
  217. llama.cpp-master/examples/gritlm/CMakeLists.txt 231B
  218. llama.cpp-master/examples/gritlm/README.md 2.73KB
  219. llama.cpp-master/examples/gritlm/gritlm.cpp 9.74KB
  220. llama.cpp-master/examples/imatrix/
  221. llama.cpp-master/examples/imatrix/CMakeLists.txt 233B
  222. llama.cpp-master/examples/imatrix/README.md 2KB
  223. llama.cpp-master/examples/imatrix/imatrix.cpp 22.23KB
  224. llama.cpp-master/examples/infill/
  225. llama.cpp-master/examples/infill/CMakeLists.txt 231B
  226. llama.cpp-master/examples/infill/README.md 2.61KB
  227. llama.cpp-master/examples/infill/infill.cpp 23.62KB
  228. llama.cpp-master/examples/jeopardy/
  229. llama.cpp-master/examples/jeopardy/README.md 1KB
  230. llama.cpp-master/examples/jeopardy/graph.py 1.61KB
  231. llama.cpp-master/examples/jeopardy/jeopardy.sh 851B
  232. llama.cpp-master/examples/jeopardy/qasheet.csv 16.28KB
  233. llama.cpp-master/examples/jeopardy/questions.txt 12.02KB
  234. llama.cpp-master/examples/json_schema_pydantic_example.py 3.14KB
  235. llama.cpp-master/examples/json_schema_to_grammar.py 32.91KB
  236. llama.cpp-master/examples/llama-bench/
  237. llama.cpp-master/examples/llama-bench/CMakeLists.txt 235B
  238. llama.cpp-master/examples/llama-bench/README.md 13.99KB
  239. llama.cpp-master/examples/llama-bench/llama-bench.cpp 51.46KB
  240. llama.cpp-master/examples/llama.android/
  241. llama.cpp-master/examples/llama.android/.gitignore 431B
  242. llama.cpp-master/examples/llama.android/README.md
  243. llama.cpp-master/examples/llama.android/app/
  244. llama.cpp-master/examples/llama.android/app/.gitignore 7B
  245. llama.cpp-master/examples/llama.android/app/build.gradle.kts 1.96KB
  246. llama.cpp-master/examples/llama.android/app/proguard-rules.pro 751B
  247. llama.cpp-master/examples/llama.android/app/src/
  248. llama.cpp-master/examples/llama.android/app/src/main/
  249. llama.cpp-master/examples/llama.android/app/src/main/AndroidManifest.xml 1.02KB
  250. llama.cpp-master/examples/llama.android/app/src/main/java/
  251. llama.cpp-master/examples/llama.android/app/src/main/java/com/
  252. llama.cpp-master/examples/llama.android/app/src/main/java/com/example/
  253. llama.cpp-master/examples/llama.android/app/src/main/java/com/example/llama/
  254. llama.cpp-master/examples/llama.android/app/src/main/java/com/example/llama/Downloadable.kt 4.42KB
  255. llama.cpp-master/examples/llama.android/app/src/main/java/com/example/llama/MainActivity.kt 5.5KB
  256. llama.cpp-master/examples/llama.android/app/src/main/java/com/example/llama/MainViewModel.kt 2.84KB
  257. llama.cpp-master/examples/llama.android/app/src/main/java/com/example/llama/ui/
  258. llama.cpp-master/examples/llama.android/app/src/main/java/com/example/llama/ui/theme/
  259. llama.cpp-master/examples/llama.android/app/src/main/java/com/example/llama/ui/theme/Color.kt 282B
  260. llama.cpp-master/examples/llama.android/app/src/main/java/com/example/llama/ui/theme/Theme.kt 2.14KB
  261. llama.cpp-master/examples/llama.android/app/src/main/java/com/example/llama/ui/theme/Type.kt 987B
  262. llama.cpp-master/examples/llama.android/app/src/main/res/
  263. llama.cpp-master/examples/llama.android/app/src/main/res/drawable/
  264. llama.cpp-master/examples/llama.android/app/src/main/res/drawable/ic_launcher_background.xml 5.47KB
  265. llama.cpp-master/examples/llama.android/app/src/main/res/drawable/ic_launcher_foreground.xml 1.66KB
  266. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-anydpi/
  267. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-anydpi/ic_launcher.xml 344B
  268. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-anydpi/ic_launcher_round.xml 344B
  269. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-hdpi/
  270. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-hdpi/ic_launcher.webp 1.37KB
  271. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-hdpi/ic_launcher_round.webp 2.83KB
  272. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-mdpi/
  273. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-mdpi/ic_launcher.webp 982B
  274. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-mdpi/ic_launcher_round.webp 1.73KB
  275. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-xhdpi/
  276. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-xhdpi/ic_launcher.webp 1.86KB
  277. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-xhdpi/ic_launcher_round.webp 3.83KB
  278. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-xxhdpi/
  279. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-xxhdpi/ic_launcher.webp 2.82KB
  280. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.webp 5.78KB
  281. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-xxxhdpi/
  282. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-xxxhdpi/ic_launcher.webp 3.75KB
  283. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.webp 7.6KB
  284. llama.cpp-master/examples/llama.android/app/src/main/res/values/
  285. llama.cpp-master/examples/llama.android/app/src/main/res/values/colors.xml 379B
  286. llama.cpp-master/examples/llama.android/app/src/main/res/values/strings.xml 75B
  287. llama.cpp-master/examples/llama.android/app/src/main/res/values/themes.xml 155B
  288. llama.cpp-master/examples/llama.android/app/src/main/res/xml/
  289. llama.cpp-master/examples/llama.android/app/src/main/res/xml/backup_rules.xml 479B
  290. llama.cpp-master/examples/llama.android/app/src/main/res/xml/data_extraction_rules.xml 552B
  291. llama.cpp-master/examples/llama.android/build.gradle.kts 299B
  292. llama.cpp-master/examples/llama.android/gradle.properties 1.33KB
  293. llama.cpp-master/examples/llama.android/gradle/
  294. llama.cpp-master/examples/llama.android/gradle/wrapper/
  295. llama.cpp-master/examples/llama.android/gradle/wrapper/gradle-wrapper.jar 57.82KB
  296. llama.cpp-master/examples/llama.android/gradle/wrapper/gradle-wrapper.properties 231B
  297. llama.cpp-master/examples/llama.android/gradlew 5.63KB
  298. llama.cpp-master/examples/llama.android/llama/
  299. llama.cpp-master/examples/llama.android/llama/.gitignore 7B
  300. llama.cpp-master/examples/llama.android/llama/build.gradle.kts 1.69KB
  301. llama.cpp-master/examples/llama.android/llama/consumer-rules.pro
  302. llama.cpp-master/examples/llama.android/llama/proguard-rules.pro 751B
  303. llama.cpp-master/examples/llama.android/llama/src/
  304. llama.cpp-master/examples/llama.android/llama/src/androidTest/
  305. llama.cpp-master/examples/llama.android/llama/src/androidTest/java/
  306. llama.cpp-master/examples/llama.android/llama/src/androidTest/java/android/
  307. llama.cpp-master/examples/llama.android/llama/src/androidTest/java/android/llama/
  308. llama.cpp-master/examples/llama.android/llama/src/androidTest/java/android/llama/cpp/
  309. llama.cpp-master/examples/llama.android/llama/src/androidTest/java/android/llama/cpp/ExampleInstrumentedTest.kt 667B
  310. llama.cpp-master/examples/llama.android/llama/src/main/
  311. llama.cpp-master/examples/llama.android/llama/src/main/AndroidManifest.xml 122B
  312. llama.cpp-master/examples/llama.android/llama/src/main/cpp/
  313. llama.cpp-master/examples/llama.android/llama/src/main/cpp/CMakeLists.txt 2.12KB
  314. llama.cpp-master/examples/llama.android/llama/src/main/cpp/llama-android.cpp 13.28KB
  315. llama.cpp-master/examples/llama.android/llama/src/main/java/
  316. llama.cpp-master/examples/llama.android/llama/src/main/java/android/
  317. llama.cpp-master/examples/llama.android/llama/src/main/java/android/llama/
  318. llama.cpp-master/examples/llama.android/llama/src/main/java/android/llama/cpp/
  319. llama.cpp-master/examples/llama.android/llama/src/main/java/android/llama/cpp/LLamaAndroid.kt 5.32KB
  320. llama.cpp-master/examples/llama.android/llama/src/test/
  321. llama.cpp-master/examples/llama.android/llama/src/test/java/
  322. llama.cpp-master/examples/llama.android/llama/src/test/java/android/
  323. llama.cpp-master/examples/llama.android/llama/src/test/java/android/llama/
  324. llama.cpp-master/examples/llama.android/llama/src/test/java/android/llama/cpp/
  325. llama.cpp-master/examples/llama.android/llama/src/test/java/android/llama/cpp/ExampleUnitTest.kt 342B
  326. llama.cpp-master/examples/llama.android/settings.gradle.kts 349B
  327. llama.cpp-master/examples/llama.swiftui/
  328. llama.cpp-master/examples/llama.swiftui/.gitignore 24B
  329. llama.cpp-master/examples/llama.swiftui/README.md 517B
  330. llama.cpp-master/examples/llama.swiftui/llama.cpp.swift/
  331. llama.cpp-master/examples/llama.swiftui/llama.cpp.swift/LibLlama.swift 11.24KB
  332. llama.cpp-master/examples/llama.swiftui/llama.swiftui.xcodeproj/
  333. llama.cpp-master/examples/llama.swiftui/llama.swiftui.xcodeproj/project.pbxproj 18KB
  334. llama.cpp-master/examples/llama.swiftui/llama.swiftui.xcodeproj/project.xcworkspace/
  335. llama.cpp-master/examples/llama.swiftui/llama.swiftui.xcodeproj/project.xcworkspace/contents.xcworkspacedata 135B
  336. llama.cpp-master/examples/llama.swiftui/llama.swiftui.xcodeproj/project.xcworkspace/xcshareddata/
  337. llama.cpp-master/examples/llama.swiftui/llama.swiftui.xcodeproj/project.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist 244B
  338. llama.cpp-master/examples/llama.swiftui/llama.swiftui/
  339. llama.cpp-master/examples/llama.swiftui/llama.swiftui/Assets.xcassets/
  340. llama.cpp-master/examples/llama.swiftui/llama.swiftui/Assets.xcassets/AppIcon.appiconset/
  341. llama.cpp-master/examples/llama.swiftui/llama.swiftui/Assets.xcassets/AppIcon.appiconset/Contents.json 177B
  342. llama.cpp-master/examples/llama.swiftui/llama.swiftui/Assets.xcassets/Contents.json 63B
  343. llama.cpp-master/examples/llama.swiftui/llama.swiftui/Models/
  344. llama.cpp-master/examples/llama.swiftui/llama.swiftui/Models/LlamaState.swift 6.99KB
  345. llama.cpp-master/examples/llama.swiftui/llama.swiftui/Resources/
  346. llama.cpp-master/examples/llama.swiftui/llama.swiftui/Resources/models/
  347. llama.cpp-master/examples/llama.swiftui/llama.swiftui/Resources/models/.gitignore
  348. llama.cpp-master/examples/llama.swiftui/llama.swiftui/UI/
  349. llama.cpp-master/examples/llama.swiftui/llama.swiftui/UI/ContentView.swift 4.73KB
  350. llama.cpp-master/examples/llama.swiftui/llama.swiftui/UI/DownloadButton.swift 4.41KB
  351. llama.cpp-master/examples/llama.swiftui/llama.swiftui/UI/InputButton.swift 4.74KB
  352. llama.cpp-master/examples/llama.swiftui/llama.swiftui/UI/LoadCustomButton.swift 1.29KB
  353. llama.cpp-master/examples/llama.swiftui/llama.swiftui/llama_swiftuiApp.swift 146B
  354. llama.cpp-master/examples/llama.vim 5.05KB
  355. llama.cpp-master/examples/llava/
  356. llama.cpp-master/examples/llava/CMakeLists.txt 1.26KB
  357. llama.cpp-master/examples/llava/MobileVLM-README.md 18.42KB
  358. llama.cpp-master/examples/llava/README.md 5.18KB
  359. llama.cpp-master/examples/llava/android/
  360. llama.cpp-master/examples/llava/android/adb_run.sh 2.55KB
  361. llama.cpp-master/examples/llava/android/build_64.sh 207B
  362. llama.cpp-master/examples/llava/clip.cpp 85.03KB
  363. llama.cpp-master/examples/llava/clip.h 2.83KB
  364. llama.cpp-master/examples/llava/convert_image_encoder_to_gguf.py 13.58KB
  365. llama.cpp-master/examples/llava/llava-cli.cpp 12.57KB
  366. llama.cpp-master/examples/llava/llava.cpp 17.95KB
  367. llama.cpp-master/examples/llava/llava.h 1.79KB
  368. llama.cpp-master/examples/llava/llava_surgery.py 1.34KB
  369. llama.cpp-master/examples/llava/llava_surgery_v2.py 6.89KB
  370. llama.cpp-master/examples/llava/requirements.txt 143B
  371. llama.cpp-master/examples/llm.vim 921B
  372. llama.cpp-master/examples/lookahead/
  373. llama.cpp-master/examples/lookahead/CMakeLists.txt 237B
  374. llama.cpp-master/examples/lookahead/README.md 195B
  375. llama.cpp-master/examples/lookahead/lookahead.cpp 16.12KB
  376. llama.cpp-master/examples/lookup/
  377. llama.cpp-master/examples/lookup/CMakeLists.txt 965B
  378. llama.cpp-master/examples/lookup/README.md 487B
  379. llama.cpp-master/examples/lookup/lookup-create.cpp 1.17KB
  380. llama.cpp-master/examples/lookup/lookup-merge.cpp 1.34KB
  381. llama.cpp-master/examples/lookup/lookup-stats.cpp 5.6KB
  382. llama.cpp-master/examples/lookup/lookup.cpp 8.37KB
  383. llama.cpp-master/examples/main-cmake-pkg/
  384. llama.cpp-master/examples/main-cmake-pkg/.gitignore 387B
  385. llama.cpp-master/examples/main-cmake-pkg/CMakeLists.txt 1.25KB
  386. llama.cpp-master/examples/main-cmake-pkg/README.md 1.25KB
  387. llama.cpp-master/examples/main/
  388. llama.cpp-master/examples/main/CMakeLists.txt 226B
  389. llama.cpp-master/examples/main/README.md 27.16KB
  390. llama.cpp-master/examples/main/main.cpp 39.6KB
  391. llama.cpp-master/examples/parallel/
  392. llama.cpp-master/examples/parallel/CMakeLists.txt 235B
  393. llama.cpp-master/examples/parallel/README.md 93B
  394. llama.cpp-master/examples/parallel/parallel.cpp 15.49KB
  395. llama.cpp-master/examples/passkey/
  396. llama.cpp-master/examples/passkey/CMakeLists.txt 233B
  397. llama.cpp-master/examples/passkey/README.md 409B
  398. llama.cpp-master/examples/passkey/passkey.cpp 8.7KB
  399. llama.cpp-master/examples/perplexity/
  400. llama.cpp-master/examples/perplexity/CMakeLists.txt 239B
  401. llama.cpp-master/examples/perplexity/README.md 19.51KB
  402. llama.cpp-master/examples/perplexity/perplexity.cpp 79.32KB
  403. llama.cpp-master/examples/pydantic_models_to_grammar.py 54.87KB
  404. llama.cpp-master/examples/pydantic_models_to_grammar_examples.py 13.4KB
  405. llama.cpp-master/examples/quantize-stats/
  406. llama.cpp-master/examples/quantize-stats/CMakeLists.txt 310B
  407. llama.cpp-master/examples/quantize-stats/quantize-stats.cpp 15.66KB
  408. llama.cpp-master/examples/quantize/
  409. llama.cpp-master/examples/quantize/CMakeLists.txt 294B
  410. llama.cpp-master/examples/quantize/README.md 5.05KB
  411. llama.cpp-master/examples/quantize/quantize.cpp 18.61KB
  412. llama.cpp-master/examples/quantize/tests.sh 1.5KB
  413. llama.cpp-master/examples/reason-act.sh 355B
  414. llama.cpp-master/examples/regex_to_grammar.py 431B
  415. llama.cpp-master/examples/retrieval/
  416. llama.cpp-master/examples/retrieval/CMakeLists.txt 237B
  417. llama.cpp-master/examples/retrieval/README.md 2.09KB
  418. llama.cpp-master/examples/retrieval/retrieval.cpp 10.11KB
  419. llama.cpp-master/examples/rpc/
  420. llama.cpp-master/examples/rpc/CMakeLists.txt 95B
  421. llama.cpp-master/examples/rpc/README.md 2.28KB
  422. llama.cpp-master/examples/rpc/rpc-server.cpp 4.15KB
  423. llama.cpp-master/examples/save-load-state/
  424. llama.cpp-master/examples/save-load-state/CMakeLists.txt 249B
  425. llama.cpp-master/examples/save-load-state/save-load-state.cpp 8.36KB
  426. llama.cpp-master/examples/server-llama2-13B.sh 790B
  427. llama.cpp-master/examples/server/
  428. llama.cpp-master/examples/server/CMakeLists.txt 1.79KB
  429. llama.cpp-master/examples/server/README.md 42.75KB
  430. llama.cpp-master/examples/server/bench/
  431. llama.cpp-master/examples/server/bench/README.md 4.2KB
  432. llama.cpp-master/examples/server/bench/bench.py 12.99KB
  433. llama.cpp-master/examples/server/bench/prometheus.yml 183B
  434. llama.cpp-master/examples/server/bench/requirements.txt 20B
  435. llama.cpp-master/examples/server/bench/script.js 5.76KB
  436. llama.cpp-master/examples/server/chat-llama2.sh 2.46KB
  437. llama.cpp-master/examples/server/chat.mjs 3.79KB
  438. llama.cpp-master/examples/server/chat.sh 1.93KB
  439. llama.cpp-master/examples/server/deps.sh 374B
  440. llama.cpp-master/examples/server/httplib.h 303.63KB
  441. llama.cpp-master/examples/server/public/
  442. llama.cpp-master/examples/server/public/colorthemes.css 11.12KB
  443. llama.cpp-master/examples/server/public/completion.js 5.81KB
  444. llama.cpp-master/examples/server/public/favicon.ico 4.03KB
  445. llama.cpp-master/examples/server/public/index-new.html 47.66KB
  446. llama.cpp-master/examples/server/public/index.html 41.58KB
  447. llama.cpp-master/examples/server/public/index.js 22.53KB
  448. llama.cpp-master/examples/server/public/json-schema-to-grammar.mjs 28.5KB
  449. llama.cpp-master/examples/server/public/prompt-formats.js 6.02KB
  450. llama.cpp-master/examples/server/public/style.css 19.6KB
  451. llama.cpp-master/examples/server/public/system-prompts.js 10.5KB
  452. llama.cpp-master/examples/server/public/theme-beeninorder.css 6.95KB
  453. llama.cpp-master/examples/server/public/theme-ketivah.css 7.14KB
  454. llama.cpp-master/examples/server/public/theme-mangotango.css 6.58KB
  455. llama.cpp-master/examples/server/public/theme-playground.css 6.83KB
  456. llama.cpp-master/examples/server/public/theme-polarnight.css 8.01KB
  457. llama.cpp-master/examples/server/public/theme-snowstorm.css 8KB
  458. llama.cpp-master/examples/server/public_simplechat/
  459. llama.cpp-master/examples/server/public_simplechat/datautils.mjs 8.94KB
  460. llama.cpp-master/examples/server/public_simplechat/index.html 1.91KB
  461. llama.cpp-master/examples/server/public_simplechat/readme.md 14.42KB
  462. llama.cpp-master/examples/server/public_simplechat/simplechat.css 1KB
  463. llama.cpp-master/examples/server/public_simplechat/simplechat.js 30.53KB
  464. llama.cpp-master/examples/server/public_simplechat/simplechat_screens.webp 20.88KB
  465. llama.cpp-master/examples/server/public_simplechat/ui.mjs 5.94KB
  466. llama.cpp-master/examples/server/server.cpp 139.03KB
  467. llama.cpp-master/examples/server/tests/
  468. llama.cpp-master/examples/server/tests/README.md 2.79KB
  469. llama.cpp-master/examples/server/tests/features/
  470. llama.cpp-master/examples/server/tests/features/embeddings.feature 2.42KB
  471. llama.cpp-master/examples/server/tests/features/environment.py 2.53KB
  472. llama.cpp-master/examples/server/tests/features/issues.feature 139B
  473. llama.cpp-master/examples/server/tests/features/lora.feature 1.14KB
  474. llama.cpp-master/examples/server/tests/features/parallel.feature 2.7KB
  475. llama.cpp-master/examples/server/tests/features/passkey.feature 2.66KB
  476. llama.cpp-master/examples/server/tests/features/results.feature 4.24KB
  477. llama.cpp-master/examples/server/tests/features/security.feature 2.48KB
  478. llama.cpp-master/examples/server/tests/features/server.feature 4.94KB
  479. llama.cpp-master/examples/server/tests/features/slotsave.feature 2.41KB
  480. llama.cpp-master/examples/server/tests/features/steps/
  481. llama.cpp-master/examples/server/tests/features/steps/steps.py 54.7KB
  482. llama.cpp-master/examples/server/tests/features/wrong_usages.feature 794B
  483. llama.cpp-master/examples/server/tests/requirements.txt 125B
  484. llama.cpp-master/examples/server/tests/tests.sh 197B
  485. llama.cpp-master/examples/server/themes/
  486. llama.cpp-master/examples/server/themes/README.md 182B
  487. llama.cpp-master/examples/server/themes/buttons-top/
  488. llama.cpp-master/examples/server/themes/buttons-top/README.md 260B
  489. llama.cpp-master/examples/server/themes/buttons-top/buttons_top.png 116.94KB
  490. llama.cpp-master/examples/server/themes/buttons-top/favicon.ico 4.03KB
  491. llama.cpp-master/examples/server/themes/buttons-top/index.html 33.74KB
  492. llama.cpp-master/examples/server/themes/wild/
  493. llama.cpp-master/examples/server/themes/wild/README.md 127B
  494. llama.cpp-master/examples/server/themes/wild/favicon.ico 4.03KB
  495. llama.cpp-master/examples/server/themes/wild/index.html 33.86KB
  496. llama.cpp-master/examples/server/themes/wild/llama_cpp.png 74.69KB
  497. llama.cpp-master/examples/server/themes/wild/llamapattern.png 253.5KB
  498. llama.cpp-master/examples/server/themes/wild/wild.png 484.83KB
  499. llama.cpp-master/examples/server/utils.hpp 21.02KB
  500. llama.cpp-master/examples/server_embd.py 971B
  501. llama.cpp-master/examples/simple/
  502. llama.cpp-master/examples/simple/CMakeLists.txt 231B
  503. llama.cpp-master/examples/simple/README.md 915B
  504. llama.cpp-master/examples/simple/simple.cpp 4.87KB
  505. llama.cpp-master/examples/speculative/
  506. llama.cpp-master/examples/speculative/CMakeLists.txt 241B
  507. llama.cpp-master/examples/speculative/README.md 285B
  508. llama.cpp-master/examples/speculative/speculative.cpp 23.86KB
  509. llama.cpp-master/examples/sycl/
  510. llama.cpp-master/examples/sycl/CMakeLists.txt 335B
  511. llama.cpp-master/examples/sycl/README.md 1.43KB
  512. llama.cpp-master/examples/sycl/build.sh 582B
  513. llama.cpp-master/examples/sycl/ls-sycl-device.cpp 195B
  514. llama.cpp-master/examples/sycl/run-llama2.sh 1.23KB
  515. llama.cpp-master/examples/sycl/win-build-sycl.bat 845B
  516. llama.cpp-master/examples/sycl/win-run-llama2.bat 330B
  517. llama.cpp-master/examples/tokenize/
  518. llama.cpp-master/examples/tokenize/CMakeLists.txt 235B
  519. llama.cpp-master/examples/tokenize/tokenize.cpp 13.4KB
  520. llama.cpp-master/examples/ts-type-to-grammar.sh 920B
  521. llama.cpp-master/flake.lock 1.52KB
  522. llama.cpp-master/flake.nix 7.18KB
  523. llama.cpp-master/ggml/
  524. llama.cpp-master/ggml/.gitignore 56B
  525. llama.cpp-master/ggml/CMakeLists.txt 9.36KB
  526. llama.cpp-master/ggml/cmake/
  527. llama.cpp-master/ggml/cmake/FindSIMD.cmake 2.59KB
  528. llama.cpp-master/ggml/include/
  529. llama.cpp-master/ggml/include/ggml-alloc.h 2.92KB
  530. llama.cpp-master/ggml/include/ggml-backend.h 13.38KB
  531. llama.cpp-master/ggml/include/ggml-blas.h 526B
  532. llama.cpp-master/ggml/include/ggml-cann.h 4.55KB
  533. llama.cpp-master/ggml/include/ggml-cuda.h 1.59KB
  534. llama.cpp-master/ggml/include/ggml-kompute.h 1KB
  535. llama.cpp-master/ggml/include/ggml-metal.h 2.25KB
  536. llama.cpp-master/ggml/include/ggml-rpc.h 673B
  537. llama.cpp-master/ggml/include/ggml-sycl.h 1.46KB
  538. llama.cpp-master/ggml/include/ggml-vulkan.h 946B
  539. llama.cpp-master/ggml/include/ggml.h 89.94KB
  540. llama.cpp-master/ggml/src/
  541. llama.cpp-master/ggml/src/CMakeLists.txt 50.03KB
  542. llama.cpp-master/ggml/src/ggml-aarch64.c 91.2KB
  543. llama.cpp-master/ggml/src/ggml-aarch64.h 1.96KB
  544. llama.cpp-master/ggml/src/ggml-alloc.c 37.49KB
  545. llama.cpp-master/ggml/src/ggml-backend-impl.h 7.43KB
  546. llama.cpp-master/ggml/src/ggml-backend.c 83.33KB
  547. llama.cpp-master/ggml/src/ggml-blas.cpp 12.15KB
  548. llama.cpp-master/ggml/src/ggml-cann.cpp 69.96KB
  549. llama.cpp-master/ggml/src/ggml-cann/
  550. llama.cpp-master/ggml/src/ggml-cann/.clang-format 4.42KB
  551. llama.cpp-master/ggml/src/ggml-cann/Doxyfile 109.97KB
  552. llama.cpp-master/ggml/src/ggml-cann/acl_tensor.cpp 6.92KB
  553. llama.cpp-master/ggml/src/ggml-cann/acl_tensor.h 12.17KB
  554. llama.cpp-master/ggml/src/ggml-cann/aclnn_ops.cpp 122.73KB
  555. llama.cpp-master/ggml/src/ggml-cann/aclnn_ops.h 25.25KB
  556. llama.cpp-master/ggml/src/ggml-cann/common.h 9.25KB
  557. llama.cpp-master/ggml/src/ggml-cann/kernels/
  558. llama.cpp-master/ggml/src/ggml-cann/kernels/CMakeLists.txt 1.03KB
  559. llama.cpp-master/ggml/src/ggml-cann/kernels/ascendc_kernels.h 693B
  560. llama.cpp-master/ggml/src/ggml-cann/kernels/dup.cpp 7.99KB
  561. llama.cpp-master/ggml/src/ggml-cann/kernels/get_row_f16.cpp 6.79KB
  562. llama.cpp-master/ggml/src/ggml-cann/kernels/get_row_f32.cpp 6.53KB
  563. llama.cpp-master/ggml/src/ggml-cann/kernels/get_row_q4_0.cpp 7.03KB
  564. llama.cpp-master/ggml/src/ggml-cann/kernels/get_row_q8_0.cpp 6.96KB
  565. llama.cpp-master/ggml/src/ggml-cann/kernels/quantize_f16_q8_0.cpp 7.37KB
  566. llama.cpp-master/ggml/src/ggml-cann/kernels/quantize_f32_q8_0.cpp 7.31KB
  567. llama.cpp-master/ggml/src/ggml-cann/kernels/quantize_float_to_q4_0.cpp 10.67KB
  568. llama.cpp-master/ggml/src/ggml-common.h 129.8KB
  569. llama.cpp-master/ggml/src/ggml-cuda.cu 120.79KB
  570. llama.cpp-master/ggml/src/ggml-cuda/
  571. llama.cpp-master/ggml/src/ggml-cuda/acc.cu 1.93KB
  572. llama.cpp-master/ggml/src/ggml-cuda/acc.cuh 131B
  573. llama.cpp-master/ggml/src/ggml-cuda/arange.cu 1.19KB
  574. llama.cpp-master/ggml/src/ggml-cuda/arange.cuh 137B
  575. llama.cpp-master/ggml/src/ggml-cuda/argsort.cu 3.35KB
  576. llama.cpp-master/ggml/src/ggml-cuda/argsort.cuh 102B
  577. llama.cpp-master/ggml/src/ggml-cuda/binbcast.cu 10.29KB
  578. llama.cpp-master/ggml/src/ggml-cuda/binbcast.cuh 326B
  579. llama.cpp-master/ggml/src/ggml-cuda/clamp.cu 1.14KB
  580. llama.cpp-master/ggml/src/ggml-cuda/clamp.cuh 135B
  581. llama.cpp-master/ggml/src/ggml-cuda/common.cuh 20.76KB
  582. llama.cpp-master/ggml/src/ggml-cuda/concat.cu 6.35KB
  583. llama.cpp-master/ggml/src/ggml-cuda/concat.cuh 137B
  584. llama.cpp-master/ggml/src/ggml-cuda/conv-transpose-1d.cu 3.25KB
  585. llama.cpp-master/ggml/src/ggml-cuda/conv-transpose-1d.cuh 158B
  586. llama.cpp-master/ggml/src/ggml-cuda/convert.cu 25.03KB
  587. llama.cpp-master/ggml/src/ggml-cuda/convert.cuh 391B
  588. llama.cpp-master/ggml/src/ggml-cuda/cpy.cu 19.95KB
  589. llama.cpp-master/ggml/src/ggml-cuda/cpy.cuh 298B
  590. llama.cpp-master/ggml/src/ggml-cuda/dequantize.cuh 2.59KB
  591. llama.cpp-master/ggml/src/ggml-cuda/diagmask.cu 1.72KB
  592. llama.cpp-master/ggml/src/ggml-cuda/diagmask.cuh 150B
  593. llama.cpp-master/ggml/src/ggml-cuda/dmmv.cu 27.48KB
  594. llama.cpp-master/ggml/src/ggml-cuda/dmmv.cuh 642B
  595. llama.cpp-master/ggml/src/ggml-cuda/fattn-common.cuh 23.67KB
  596. llama.cpp-master/ggml/src/ggml-cuda/fattn-tile-f16.cu 11.13KB
  597. llama.cpp-master/ggml/src/ggml-cuda/fattn-tile-f16.cuh 115B
  598. llama.cpp-master/ggml/src/ggml-cuda/fattn-tile-f32.cu 11.05KB
  599. llama.cpp-master/ggml/src/ggml-cuda/fattn-tile-f32.cuh 115B
  600. llama.cpp-master/ggml/src/ggml-cuda/fattn-vec-f16.cuh 14.64KB
  601. llama.cpp-master/ggml/src/ggml-cuda/fattn-vec-f32.cuh 13.71KB
  602. llama.cpp-master/ggml/src/ggml-cuda/fattn-wmma-f16.cuh 20.11KB
  603. llama.cpp-master/ggml/src/ggml-cuda/fattn.cu 13.84KB
  604. llama.cpp-master/ggml/src/ggml-cuda/fattn.cuh 106B
  605. llama.cpp-master/ggml/src/ggml-cuda/getrows.cu 6.83KB
  606. llama.cpp-master/ggml/src/ggml-cuda/getrows.cuh 141B
  607. llama.cpp-master/ggml/src/ggml-cuda/im2col.cu 4.45KB
  608. llama.cpp-master/ggml/src/ggml-cuda/im2col.cuh 137B
  609. llama.cpp-master/ggml/src/ggml-cuda/mma.cuh 7.41KB
  610. llama.cpp-master/ggml/src/ggml-cuda/mmq.cu 4.58KB
  611. llama.cpp-master/ggml/src/ggml-cuda/mmq.cuh 110.48KB
  612. llama.cpp-master/ggml/src/ggml-cuda/mmvq.cu 18.78KB
  613. llama.cpp-master/ggml/src/ggml-cuda/mmvq.cuh 481B
  614. llama.cpp-master/ggml/src/ggml-cuda/norm.cu 7.03KB
  615. llama.cpp-master/ggml/src/ggml-cuda/norm.cuh 263B
  616. llama.cpp-master/ggml/src/ggml-cuda/pad.cu 1.75KB
  617. llama.cpp-master/ggml/src/ggml-cuda/pad.cuh 131B
  618. llama.cpp-master/ggml/src/ggml-cuda/pool2d.cu 3.23KB
  619. llama.cpp-master/ggml/src/ggml-cuda/pool2d.cuh 137B
  620. llama.cpp-master/ggml/src/ggml-cuda/quantize.cu 5.34KB
  621. llama.cpp-master/ggml/src/ggml-cuda/quantize.cuh 979B
  622. llama.cpp-master/ggml/src/ggml-cuda/rope.cu 10.46KB
  623. llama.cpp-master/ggml/src/ggml-cuda/rope.cuh 133B
  624. llama.cpp-master/ggml/src/ggml-cuda/scale.cu 1021B
  625. llama.cpp-master/ggml/src/ggml-cuda/scale.cuh 135B
  626. llama.cpp-master/ggml/src/ggml-cuda/softmax.cu 7.54KB
  627. llama.cpp-master/ggml/src/ggml-cuda/softmax.cuh 142B
  628. llama.cpp-master/ggml/src/ggml-cuda/sumrows.cu 1.17KB
  629. llama.cpp-master/ggml/src/ggml-cuda/sumrows.cuh 103B
  630. llama.cpp-master/ggml/src/ggml-cuda/template-instances/
  631. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-f16-f16.cu 177B
  632. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-f16-q4_0.cu 178B
  633. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-f16-q4_1.cu 178B
  634. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-f16-q5_0.cu 178B
  635. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-f16-q5_1.cu 178B
  636. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-f16-q8_0.cu 178B
  637. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_0-f16.cu 178B
  638. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_0-q4_0.cu 179B
  639. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_0-q4_1.cu 179B
  640. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_0-q5_0.cu 179B
  641. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_0-q5_1.cu 179B
  642. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_0-q8_0.cu 179B
  643. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_1-f16.cu 178B
  644. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_1-q4_0.cu 179B
  645. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_1-q4_1.cu 179B
  646. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_1-q5_0.cu 179B
  647. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_1-q5_1.cu 179B
  648. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_1-q8_0.cu 179B
  649. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_0-f16.cu 178B
  650. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_0-q4_0.cu 179B
  651. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_0-q4_1.cu 179B
  652. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_0-q5_0.cu 179B
  653. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_0-q5_1.cu 179B
  654. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_0-q8_0.cu 179B
  655. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_1-f16.cu 178B
  656. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_1-q4_0.cu 179B
  657. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_1-q4_1.cu 179B
  658. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_1-q5_0.cu 179B
  659. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_1-q5_1.cu 179B
  660. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_1-q8_0.cu 179B
  661. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q8_0-f16.cu 178B
  662. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q8_0-q4_0.cu 179B
  663. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q8_0-q4_1.cu 179B
  664. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q8_0-q5_0.cu 179B
  665. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q8_0-q5_1.cu 179B
  666. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q8_0-q8_0.cu 179B
  667. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs256-f16-f16.cu 177B
  668. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs64-f16-f16.cu 176B
  669. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs64-f16-q4_0.cu 177B
  670. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs64-f16-q4_1.cu 177B
  671. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs64-f16-q5_0.cu 177B
  672. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs64-f16-q5_1.cu 177B
  673. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs64-f16-q8_0.cu 177B
  674. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-f16-f16.cu 177B
  675. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-f16-q4_0.cu 178B
  676. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-f16-q4_1.cu 178B
  677. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-f16-q5_0.cu 178B
  678. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-f16-q5_1.cu 178B
  679. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-f16-q8_0.cu 178B
  680. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_0-f16.cu 178B
  681. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_0-q4_0.cu 179B
  682. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_0-q4_1.cu 179B
  683. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_0-q5_0.cu 179B
  684. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_0-q5_1.cu 179B
  685. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_0-q8_0.cu 179B
  686. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_1-f16.cu 178B
  687. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_1-q4_0.cu 179B
  688. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_1-q4_1.cu 179B
  689. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_1-q5_0.cu 179B
  690. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_1-q5_1.cu 179B
  691. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_1-q8_0.cu 179B
  692. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_0-f16.cu 178B
  693. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_0-q4_0.cu 179B
  694. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_0-q4_1.cu 179B
  695. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_0-q5_0.cu 179B
  696. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_0-q5_1.cu 179B
  697. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_0-q8_0.cu 179B
  698. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_1-f16.cu 178B
  699. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_1-q4_0.cu 179B
  700. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_1-q4_1.cu 179B
  701. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_1-q5_0.cu 179B
  702. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_1-q5_1.cu 179B
  703. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_1-q8_0.cu 179B
  704. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q8_0-f16.cu 178B
  705. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q8_0-q4_0.cu 179B
  706. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q8_0-q4_1.cu 179B
  707. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q8_0-q5_0.cu 179B
  708. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q8_0-q5_1.cu 179B
  709. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q8_0-q8_0.cu 179B
  710. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs256-f16-f16.cu 177B
  711. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs64-f16-f16.cu 176B
  712. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs64-f16-q4_0.cu 177B
  713. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs64-f16-q4_1.cu 177B
  714. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs64-f16-q5_0.cu 177B
  715. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs64-f16-q5_1.cu 177B
  716. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs64-f16-q8_0.cu 177B
  717. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-wmma-f16-instance-kqfloat-cpb16.cu 367B
  718. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-wmma-f16-instance-kqfloat-cpb32.cu 325B
  719. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-wmma-f16-instance-kqhalf-cpb16.cu 361B
  720. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-wmma-f16-instance-kqhalf-cpb32.cu 361B
  721. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-wmma-f16-instance-kqhalf-cpb8.cu 276B
  722. llama.cpp-master/ggml/src/ggml-cuda/template-instances/generate_cu_files.py 2.76KB
  723. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-iq1_s.cu 139B
  724. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-iq2_s.cu 139B
  725. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-iq2_xs.cu 140B
  726. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-iq2_xxs.cu 141B
  727. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-iq3_s.cu 139B
  728. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-iq3_xxs.cu 141B
  729. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-iq4_nl.cu 140B
  730. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-iq4_xs.cu 140B
  731. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-q2_k.cu 138B
  732. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-q3_k.cu 138B
  733. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-q4_0.cu 138B
  734. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-q4_1.cu 138B
  735. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-q4_k.cu 138B
  736. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-q5_0.cu 138B
  737. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-q5_1.cu 138B
  738. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-q5_k.cu 138B
  739. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-q6_k.cu 138B
  740. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-q8_0.cu 138B
  741. llama.cpp-master/ggml/src/ggml-cuda/tsembd.cu 1.76KB
  742. llama.cpp-master/ggml/src/ggml-cuda/tsembd.cuh 161B
  743. llama.cpp-master/ggml/src/ggml-cuda/unary.cu 10.58KB
  744. llama.cpp-master/ggml/src/ggml-cuda/unary.cuh 1.18KB
  745. llama.cpp-master/ggml/src/ggml-cuda/upscale.cu 2.07KB
  746. llama.cpp-master/ggml/src/ggml-cuda/upscale.cuh 139B
  747. llama.cpp-master/ggml/src/ggml-cuda/vecdotq.cuh 38.2KB
  748. llama.cpp-master/ggml/src/ggml-cuda/vendors/
  749. llama.cpp-master/ggml/src/ggml-cuda/vendors/cuda.h 462B
  750. llama.cpp-master/ggml/src/ggml-cuda/vendors/hip.h 7.1KB
  751. llama.cpp-master/ggml/src/ggml-cuda/vendors/musa.h 7.04KB
  752. llama.cpp-master/ggml/src/ggml-impl.h 20.06KB
  753. llama.cpp-master/ggml/src/ggml-kompute.cpp 79.01KB
  754. llama.cpp-master/ggml/src/ggml-metal.m 187.1KB
  755. llama.cpp-master/ggml/src/ggml-metal.metal 221.39KB
  756. llama.cpp-master/ggml/src/ggml-quants.c 637.5KB
  757. llama.cpp-master/ggml/src/ggml-quants.h 11.48KB
  758. llama.cpp-master/ggml/src/ggml-rpc.cpp 43.35KB
  759. llama.cpp-master/ggml/src/ggml-sycl.cpp 209.44KB
  760. llama.cpp-master/ggml/src/ggml-sycl/
  761. llama.cpp-master/ggml/src/ggml-sycl/backend.hpp 653B
  762. llama.cpp-master/ggml/src/ggml-sycl/common.cpp 1.49KB
  763. llama.cpp-master/ggml/src/ggml-sycl/common.hpp 10.68KB
  764. llama.cpp-master/ggml/src/ggml-sycl/concat.cpp 7.39KB
  765. llama.cpp-master/ggml/src/ggml-sycl/concat.hpp 575B
  766. llama.cpp-master/ggml/src/ggml-sycl/conv.cpp 3.16KB
  767. llama.cpp-master/ggml/src/ggml-sycl/conv.hpp 550B
  768. llama.cpp-master/ggml/src/ggml-sycl/convert.cpp 21.31KB
  769. llama.cpp-master/ggml/src/ggml-sycl/convert.hpp 778B
  770. llama.cpp-master/ggml/src/ggml-sycl/dequantize.hpp 22.99KB
  771. llama.cpp-master/ggml/src/ggml-sycl/dmmv.cpp 40.63KB
  772. llama.cpp-master/ggml/src/ggml-sycl/dmmv.hpp 808B
  773. llama.cpp-master/ggml/src/ggml-sycl/dpct/
  774. llama.cpp-master/ggml/src/ggml-sycl/dpct/helper.hpp 120.39KB
  775. llama.cpp-master/ggml/src/ggml-sycl/mmq.cpp 116.7KB
  776. llama.cpp-master/ggml/src/ggml-sycl/mmq.hpp 819B
  777. llama.cpp-master/ggml/src/ggml-sycl/mmvq.cpp 38.85KB
  778. llama.cpp-master/ggml/src/ggml-sycl/mmvq.hpp 799B
  779. llama.cpp-master/ggml/src/ggml-sycl/norm.cpp 13.23KB
  780. llama.cpp-master/ggml/src/ggml-sycl/norm.hpp 1.08KB
  781. llama.cpp-master/ggml/src/ggml-sycl/presets.hpp 1.97KB
  782. llama.cpp-master/ggml/src/ggml-sycl/rope.cpp 10.38KB
  783. llama.cpp-master/ggml/src/ggml-sycl/rope.hpp 633B
  784. llama.cpp-master/ggml/src/ggml-sycl/softmax.cpp 10.88KB
  785. llama.cpp-master/ggml/src/ggml-sycl/softmax.hpp 652B
  786. llama.cpp-master/ggml/src/ggml-sycl/tsembd.cpp 2.53KB
  787. llama.cpp-master/ggml/src/ggml-sycl/tsembd.hpp 560B
  788. llama.cpp-master/ggml/src/ggml-sycl/vecdotq.hpp 38.82KB
  789. llama.cpp-master/ggml/src/ggml-vulkan.cpp 405.37KB
  790. llama.cpp-master/ggml/src/ggml.c 713.38KB
  791. llama.cpp-master/ggml/src/kompute/
  792. llama.cpp-master/ggml/src/kompute-shaders/
  793. llama.cpp-master/ggml/src/kompute-shaders/common.comp 3.53KB
  794. llama.cpp-master/ggml/src/kompute-shaders/op_add.comp 1.61KB
  795. llama.cpp-master/ggml/src/kompute-shaders/op_addrow.comp 640B
  796. llama.cpp-master/ggml/src/kompute-shaders/op_cpy_f16_f16.comp 1.5KB
  797. llama.cpp-master/ggml/src/kompute-shaders/op_cpy_f16_f32.comp 1.49KB
  798. llama.cpp-master/ggml/src/kompute-shaders/op_cpy_f32_f16.comp 1.49KB
  799. llama.cpp-master/ggml/src/kompute-shaders/op_cpy_f32_f32.comp 1.49KB
  800. llama.cpp-master/ggml/src/kompute-shaders/op_diagmask.comp 726B
  801. llama.cpp-master/ggml/src/kompute-shaders/op_gelu.comp 604B
  802. llama.cpp-master/ggml/src/kompute-shaders/op_getrows.comp 609B
  803. llama.cpp-master/ggml/src/kompute-shaders/op_getrows_f16.comp 787B
  804. llama.cpp-master/ggml/src/kompute-shaders/op_getrows_f32.comp 762B
  805. llama.cpp-master/ggml/src/kompute-shaders/op_getrows_q4_0.comp 919B
  806. llama.cpp-master/ggml/src/kompute-shaders/op_getrows_q4_1.comp 962B
  807. llama.cpp-master/ggml/src/kompute-shaders/op_getrows_q6_k.comp 1.16KB
  808. llama.cpp-master/ggml/src/kompute-shaders/op_mul.comp 1.33KB
  809. llama.cpp-master/ggml/src/kompute-shaders/op_mul_mat_f16.comp 1.59KB
  810. llama.cpp-master/ggml/src/kompute-shaders/op_mul_mat_mat_f32.comp 1.27KB
  811. llama.cpp-master/ggml/src/kompute-shaders/op_mul_mat_q4_0.comp 1018B
  812. llama.cpp-master/ggml/src/kompute-shaders/op_mul_mat_q4_1.comp 1.04KB
  813. llama.cpp-master/ggml/src/kompute-shaders/op_mul_mat_q6_k.comp 3.54KB
  814. llama.cpp-master/ggml/src/kompute-shaders/op_mul_mat_q8_0.comp 2.19KB
  815. llama.cpp-master/ggml/src/kompute-shaders/op_mul_mv_q_n.comp 1.76KB
  816. llama.cpp-master/ggml/src/kompute-shaders/op_mul_mv_q_n_pre.comp 521B
  817. llama.cpp-master/ggml/src/kompute-shaders/op_norm.comp 2.25KB
  818. llama.cpp-master/ggml/src/kompute-shaders/op_relu.comp 508B
  819. llama.cpp-master/ggml/src/kompute-shaders/op_rmsnorm.comp 1.38KB
  820. llama.cpp-master/ggml/src/kompute-shaders/op_rope_f16.comp 2.84KB
  821. llama.cpp-master/ggml/src/kompute-shaders/op_rope_f32.comp 2.74KB
  822. llama.cpp-master/ggml/src/kompute-shaders/op_scale.comp 432B
  823. llama.cpp-master/ggml/src/kompute-shaders/op_scale_8.comp 528B
  824. llama.cpp-master/ggml/src/kompute-shaders/op_silu.comp 543B
  825. llama.cpp-master/ggml/src/kompute-shaders/op_softmax.comp 1.75KB
  826. llama.cpp-master/ggml/src/kompute-shaders/rope_common.comp 2.25KB
  827. llama.cpp-master/ggml/src/llamafile/
  828. llama.cpp-master/ggml/src/llamafile/sgemm.cpp 30.9KB
  829. llama.cpp-master/ggml/src/llamafile/sgemm.h 302B
  830. llama.cpp-master/ggml/src/vulkan-shaders/
  831. llama.cpp-master/ggml/src/vulkan-shaders/CMakeLists.txt 268B
  832. llama.cpp-master/ggml/src/vulkan-shaders/add.comp 287B
  833. llama.cpp-master/ggml/src/vulkan-shaders/argsort.comp 1.96KB
  834. llama.cpp-master/ggml/src/vulkan-shaders/clamp.comp 340B
  835. llama.cpp-master/ggml/src/vulkan-shaders/concat.comp 1.25KB
  836. llama.cpp-master/ggml/src/vulkan-shaders/copy.comp 352B
  837. llama.cpp-master/ggml/src/vulkan-shaders/dequant_f32.comp 442B
  838. llama.cpp-master/ggml/src/vulkan-shaders/dequant_funcs.comp 2.33KB
  839. llama.cpp-master/ggml/src/vulkan-shaders/dequant_head.comp 249B
  840. llama.cpp-master/ggml/src/vulkan-shaders/dequant_iq4_nl.comp 871B
  841. llama.cpp-master/ggml/src/vulkan-shaders/dequant_q2_k.comp 1.44KB
  842. llama.cpp-master/ggml/src/vulkan-shaders/dequant_q3_k.comp 1.68KB
  843. llama.cpp-master/ggml/src/vulkan-shaders/dequant_q4_0.comp 861B
  844. llama.cpp-master/ggml/src/vulkan-shaders/dequant_q4_1.comp 892B
  845. llama.cpp-master/ggml/src/vulkan-shaders/dequant_q4_k.comp 1.97KB
  846. llama.cpp-master/ggml/src/vulkan-shaders/dequant_q5_0.comp 1.02KB
  847. llama.cpp-master/ggml/src/vulkan-shaders/dequant_q5_1.comp 1.02KB
  848. llama.cpp-master/ggml/src/vulkan-shaders/dequant_q5_k.comp 2.41KB
  849. llama.cpp-master/ggml/src/vulkan-shaders/dequant_q6_k.comp 1.39KB
  850. llama.cpp-master/ggml/src/vulkan-shaders/dequant_q8_0.comp 839B
  851. llama.cpp-master/ggml/src/vulkan-shaders/diag_mask_inf.comp 799B
  852. llama.cpp-master/ggml/src/vulkan-shaders/div.comp 287B
  853. llama.cpp-master/ggml/src/vulkan-shaders/gelu.comp 767B
  854. llama.cpp-master/ggml/src/vulkan-shaders/gelu_quick.comp 631B
  855. llama.cpp-master/ggml/src/vulkan-shaders/generic_binary_head.comp 2.11KB
  856. llama.cpp-master/ggml/src/vulkan-shaders/generic_head.comp 160B
  857. llama.cpp-master/ggml/src/vulkan-shaders/generic_unary_head.comp 1.48KB
  858. llama.cpp-master/ggml/src/vulkan-shaders/get_rows.comp 702B
  859. llama.cpp-master/ggml/src/vulkan-shaders/get_rows_quant.comp 970B
  860. llama.cpp-master/ggml/src/vulkan-shaders/group_norm.comp 1.68KB
  861. llama.cpp-master/ggml/src/vulkan-shaders/im2col.comp 1.49KB
  862. llama.cpp-master/ggml/src/vulkan-shaders/leaky_relu.comp 586B
  863. llama.cpp-master/ggml/src/vulkan-shaders/mul.comp 287B
  864. llama.cpp-master/ggml/src/vulkan-shaders/mul_mat_split_k_reduce.comp 592B
  865. llama.cpp-master/ggml/src/vulkan-shaders/mul_mat_vec.comp 1.62KB
  866. llama.cpp-master/ggml/src/vulkan-shaders/mul_mat_vec_base.comp 1.75KB
  867. llama.cpp-master/ggml/src/vulkan-shaders/mul_mat_vec_nc.comp 1.76KB
  868. llama.cpp-master/ggml/src/vulkan-shaders/mul_mat_vec_p021.comp 1.86KB
  869. llama.cpp-master/ggml/src/vulkan-shaders/mul_mat_vec_q2_k.comp 4.33KB
  870. llama.cpp-master/ggml/src/vulkan-shaders/mul_mat_vec_q3_k.comp 4.36KB
  871. llama.cpp-master/ggml/src/vulkan-shaders/mul_mat_vec_q4_k.comp 8.02KB
  872. llama.cpp-master/ggml/src/vulkan-shaders/mul_mat_vec_q5_k.comp 6.92KB
  873. llama.cpp-master/ggml/src/vulkan-shaders/mul_mat_vec_q6_k.comp 4.91KB
  874. llama.cpp-master/ggml/src/vulkan-shaders/mul_mm.comp 21.02KB
  875. llama.cpp-master/ggml/src/vulkan-shaders/norm.comp 1.25KB
  876. llama.cpp-master/ggml/src/vulkan-shaders/pad.comp 884B
  877. llama.cpp-master/ggml/src/vulkan-shaders/relu.comp 520B
  878. llama.cpp-master/ggml/src/vulkan-shaders/rms_norm.comp 1.25KB
  879. llama.cpp-master/ggml/src/vulkan-shaders/rope_head.comp 1.47KB
  880. llama.cpp-master/ggml/src/vulkan-shaders/rope_neox.comp 929B
  881. llama.cpp-master/ggml/src/vulkan-shaders/rope_norm.comp 901B
  882. llama.cpp-master/ggml/src/vulkan-shaders/scale.comp 273B
  883. llama.cpp-master/ggml/src/vulkan-shaders/silu.comp 565B
  884. llama.cpp-master/ggml/src/vulkan-shaders/soft_max.comp 2.61KB
  885. llama.cpp-master/ggml/src/vulkan-shaders/square.comp 288B
  886. llama.cpp-master/ggml/src/vulkan-shaders/sum_rows.comp 940B
  887. llama.cpp-master/ggml/src/vulkan-shaders/tanh.comp 519B
  888. llama.cpp-master/ggml/src/vulkan-shaders/timestep_embedding.comp 1KB
  889. llama.cpp-master/ggml/src/vulkan-shaders/types.comp 3.44KB
  890. llama.cpp-master/ggml/src/vulkan-shaders/upscale.comp 1.07KB
  891. llama.cpp-master/ggml/src/vulkan-shaders/vulkan-shaders-gen.cpp 22.18KB
  892. llama.cpp-master/gguf-py/
  893. llama.cpp-master/gguf-py/LICENSE 1.05KB
  894. llama.cpp-master/gguf-py/README.md 2.66KB
  895. llama.cpp-master/gguf-py/examples/
  896. llama.cpp-master/gguf-py/examples/reader.py 1.54KB
  897. llama.cpp-master/gguf-py/examples/writer.py 1.09KB
  898. llama.cpp-master/gguf-py/gguf/
  899. llama.cpp-master/gguf-py/gguf/__init__.py 219B
  900. llama.cpp-master/gguf-py/gguf/constants.py 45.91KB
  901. llama.cpp-master/gguf-py/gguf/gguf.py 478B
  902. llama.cpp-master/gguf-py/gguf/gguf_reader.py 12.08KB
  903. llama.cpp-master/gguf-py/gguf/gguf_writer.py 34.14KB
  904. llama.cpp-master/gguf-py/gguf/lazy.py 8.33KB
  905. llama.cpp-master/gguf-py/gguf/metadata.py 25.12KB
  906. llama.cpp-master/gguf-py/gguf/py.typed
  907. llama.cpp-master/gguf-py/gguf/quants.py 4.21KB
  908. llama.cpp-master/gguf-py/gguf/tensor_mapping.py 29.44KB
  909. llama.cpp-master/gguf-py/gguf/utility.py 2.87KB
  910. llama.cpp-master/gguf-py/gguf/vocab.py 18.6KB
  911. llama.cpp-master/gguf-py/pyproject.toml 1013B
  912. llama.cpp-master/gguf-py/scripts/
  913. llama.cpp-master/gguf-py/scripts/__init__.py 297B
  914. llama.cpp-master/gguf-py/scripts/gguf_convert_endian.py 5.16KB
  915. llama.cpp-master/gguf-py/scripts/gguf_dump.py 21.42KB
  916. llama.cpp-master/gguf-py/scripts/gguf_hash.py 3.62KB
  917. llama.cpp-master/gguf-py/scripts/gguf_new_metadata.py 10.46KB
  918. llama.cpp-master/gguf-py/scripts/gguf_set_metadata.py 4.03KB
  919. llama.cpp-master/gguf-py/tests/
  920. llama.cpp-master/gguf-py/tests/__init__.py 29B
  921. llama.cpp-master/gguf-py/tests/test_metadata.py 12.46KB
  922. llama.cpp-master/grammars/
  923. llama.cpp-master/grammars/README.md 16.44KB
  924. llama.cpp-master/grammars/arithmetic.gbnf 177B
  925. llama.cpp-master/grammars/c.gbnf 1.35KB
  926. llama.cpp-master/grammars/chess.gbnf 565B
  927. llama.cpp-master/grammars/japanese.gbnf 249B
  928. llama.cpp-master/grammars/json.gbnf 601B
  929. llama.cpp-master/grammars/json_arr.gbnf 796B
  930. llama.cpp-master/grammars/list.gbnf 109B
  931. llama.cpp-master/include/
  932. llama.cpp-master/include/llama.h 56.21KB
  933. llama.cpp-master/media/
  934. llama.cpp-master/media/llama-leader.jpeg 195.26KB
  935. llama.cpp-master/media/llama0-banner.png 141.23KB
  936. llama.cpp-master/media/llama0-logo.png 175.72KB
  937. llama.cpp-master/media/llama1-banner.png 32.55KB
  938. llama.cpp-master/media/llama1-logo.png 31.73KB
  939. llama.cpp-master/media/matmul.png 259.48KB
  940. llama.cpp-master/media/matmul.svg 51.38KB
  941. llama.cpp-master/models/
  942. llama.cpp-master/models/.editorconfig 12B
  943. llama.cpp-master/models/ggml-vocab-aquila.gguf 4.6MB
  944. llama.cpp-master/models/ggml-vocab-baichuan.gguf 1.28MB
  945. llama.cpp-master/models/ggml-vocab-bert-bge.gguf 612.84KB
  946. llama.cpp-master/models/ggml-vocab-bert-bge.gguf.inp 1.9KB
  947. llama.cpp-master/models/ggml-vocab-bert-bge.gguf.out 1.59KB
  948. llama.cpp-master/models/ggml-vocab-command-r.gguf 10.37MB
  949. llama.cpp-master/models/ggml-vocab-command-r.gguf.inp 1.9KB
  950. llama.cpp-master/models/ggml-vocab-command-r.gguf.out 1.86KB
  951. llama.cpp-master/models/ggml-vocab-deepseek-coder.gguf 1.1MB
  952. llama.cpp-master/models/ggml-vocab-deepseek-coder.gguf.inp 1.9KB
  953. llama.cpp-master/models/ggml-vocab-deepseek-coder.gguf.out 2.05KB
  954. llama.cpp-master/models/ggml-vocab-deepseek-llm.gguf 3.79MB
  955. llama.cpp-master/models/ggml-vocab-deepseek-llm.gguf.inp 1.9KB
  956. llama.cpp-master/models/ggml-vocab-deepseek-llm.gguf.out 1.88KB
  957. llama.cpp-master/models/ggml-vocab-falcon.gguf 2.18MB
  958. llama.cpp-master/models/ggml-vocab-falcon.gguf.inp 1.9KB
  959. llama.cpp-master/models/ggml-vocab-falcon.gguf.out 1.96KB
  960. llama.cpp-master/models/ggml-vocab-gpt-2.gguf 1.68MB
  961. llama.cpp-master/models/ggml-vocab-gpt-2.gguf.inp 1.9KB
  962. llama.cpp-master/models/ggml-vocab-gpt-2.gguf.out 2.1KB
  963. llama.cpp-master/models/ggml-vocab-gpt-neox.gguf 1.69MB
  964. llama.cpp-master/models/ggml-vocab-llama-bpe.gguf 7.46MB
  965. llama.cpp-master/models/ggml-vocab-llama-bpe.gguf.inp 1.9KB
  966. llama.cpp-master/models/ggml-vocab-llama-bpe.gguf.out 1.7KB
  967. llama.cpp-master/models/ggml-vocab-llama-spm.gguf 706.9KB
  968. llama.cpp-master/models/ggml-vocab-llama-spm.gguf.inp 1.9KB
  969. llama.cpp-master/models/ggml-vocab-llama-spm.gguf.out 2.62KB
  970. llama.cpp-master/models/ggml-vocab-mpt.gguf 1.69MB
  971. llama.cpp-master/models/ggml-vocab-mpt.gguf.inp 1.9KB
  972. llama.cpp-master/models/ggml-vocab-mpt.gguf.out 1.85KB
  973. llama.cpp-master/models/ggml-vocab-phi-3.gguf 709KB
  974. llama.cpp-master/models/ggml-vocab-phi-3.gguf.inp 1.9KB
  975. llama.cpp-master/models/ggml-vocab-phi-3.gguf.out 2.62KB
  976. llama.cpp-master/models/ggml-vocab-qwen2.gguf 5.65MB
  977. llama.cpp-master/models/ggml-vocab-qwen2.gguf.inp 1.9KB
  978. llama.cpp-master/models/ggml-vocab-qwen2.gguf.out 1.72KB
  979. llama.cpp-master/models/ggml-vocab-refact.gguf 1.64MB
  980. llama.cpp-master/models/ggml-vocab-refact.gguf.inp 1.9KB
  981. llama.cpp-master/models/ggml-vocab-refact.gguf.out 1.87KB
  982. llama.cpp-master/models/ggml-vocab-starcoder.gguf 1.64MB
  983. llama.cpp-master/models/ggml-vocab-starcoder.gguf.inp 1.9KB
  984. llama.cpp-master/models/ggml-vocab-starcoder.gguf.out 1.88KB
  985. llama.cpp-master/mypy.ini 163B
  986. llama.cpp-master/pocs/
  987. llama.cpp-master/pocs/CMakeLists.txt 171B
  988. llama.cpp-master/pocs/vdot/
  989. llama.cpp-master/pocs/vdot/CMakeLists.txt 387B
  990. llama.cpp-master/pocs/vdot/q8dot.cpp 5.23KB
  991. llama.cpp-master/pocs/vdot/vdot.cpp 13.18KB
  992. llama.cpp-master/poetry.lock 121.86KB
  993. llama.cpp-master/prompts/
  994. llama.cpp-master/prompts/LLM-questions.txt 2.54KB
  995. llama.cpp-master/prompts/alpaca.txt 106B
  996. llama.cpp-master/prompts/assistant.txt 2.29KB
  997. llama.cpp-master/prompts/chat-with-baichuan.txt 90B
  998. llama.cpp-master/prompts/chat-with-bob.txt 386B
  999. llama.cpp-master/prompts/chat-with-qwen.txt 28B
  1000. llama.cpp-master/prompts/chat-with-vicuna-v0.txt 446B
  1001. llama.cpp-master/prompts/chat-with-vicuna-v1.txt 426B
  1002. llama.cpp-master/prompts/chat.txt 1.79KB
  1003. llama.cpp-master/prompts/dan-modified.txt 1.5KB
  1004. llama.cpp-master/prompts/dan.txt 1.62KB
  1005. llama.cpp-master/prompts/mnemonics.txt 4.97KB
  1006. llama.cpp-master/prompts/parallel-questions.txt 1.68KB
  1007. llama.cpp-master/prompts/reason-act.txt 758B
  1008. llama.cpp-master/pyproject.toml 1.25KB
  1009. llama.cpp-master/pyrightconfig.json 528B
  1010. llama.cpp-master/requirements.txt 505B
  1011. llama.cpp-master/requirements/
  1012. llama.cpp-master/requirements/requirements-all.txt 428B
  1013. llama.cpp-master/requirements/requirements-compare-llama-bench.txt 34B
  1014. llama.cpp-master/requirements/requirements-convert_hf_to_gguf.txt 111B
  1015. llama.cpp-master/requirements/requirements-convert_hf_to_gguf_update.txt 111B
  1016. llama.cpp-master/requirements/requirements-convert_legacy_llama.txt 99B
  1017. llama.cpp-master/requirements/requirements-convert_llama_ggml_to_gguf.txt 43B
  1018. llama.cpp-master/requirements/requirements-convert_lora_to_gguf.txt 96B
  1019. llama.cpp-master/requirements/requirements-pydantic.txt 48B
  1020. llama.cpp-master/requirements/requirements-test-tokenizer-random.txt 13B
  1021. llama.cpp-master/scripts/
  1022. llama.cpp-master/scripts/build-info.sh 717B
  1023. llama.cpp-master/scripts/check-requirements.sh 4.34KB
  1024. llama.cpp-master/scripts/ci-run.sh 1.28KB
  1025. llama.cpp-master/scripts/compare-commits.sh 749B
  1026. llama.cpp-master/scripts/compare-llama-bench.py 14.3KB
  1027. llama.cpp-master/scripts/debug-test.sh 5.01KB
  1028. llama.cpp-master/scripts/gen-authors.sh 337B
  1029. llama.cpp-master/scripts/gen-unicode-data.py 6.28KB
  1030. llama.cpp-master/scripts/get-flags.mk 1.27KB
  1031. llama.cpp-master/scripts/get-hellaswag.sh 263B
  1032. llama.cpp-master/scripts/get-pg.sh 1.36KB
  1033. llama.cpp-master/scripts/get-wikitext-103.sh 210B
  1034. llama.cpp-master/scripts/get-wikitext-2.sh 253B
  1035. llama.cpp-master/scripts/get-winogrande.sh 292B
  1036. llama.cpp-master/scripts/hf.sh 2.26KB
  1037. llama.cpp-master/scripts/install-oneapi.bat 802B
  1038. llama.cpp-master/scripts/pod-llama.sh 8.17KB
  1039. llama.cpp-master/scripts/qnt-all.sh 558B
  1040. llama.cpp-master/scripts/run-all-perf.sh 549B
  1041. llama.cpp-master/scripts/run-all-ppl.sh 554B
  1042. llama.cpp-master/scripts/run-with-preset.py 5.47KB
  1043. llama.cpp-master/scripts/server-llm.sh 11.22KB
  1044. llama.cpp-master/scripts/sync-ggml-am.sh 7.89KB
  1045. llama.cpp-master/scripts/sync-ggml.last 41B
  1046. llama.cpp-master/scripts/sync-ggml.sh 2.58KB
  1047. llama.cpp-master/scripts/verify-checksum-models.py 2.42KB
  1048. llama.cpp-master/scripts/xxd.cmake 647B
  1049. llama.cpp-master/spm-headers/
  1050. llama.cpp-master/spm-headers/ggml-alloc.h 28B
  1051. llama.cpp-master/spm-headers/ggml-backend.h 30B
  1052. llama.cpp-master/spm-headers/ggml-metal.h 28B
  1053. llama.cpp-master/spm-headers/ggml.h 22B
  1054. llama.cpp-master/spm-headers/llama.h 18B
  1055. llama.cpp-master/src/
  1056. llama.cpp-master/src/CMakeLists.txt 749B
  1057. llama.cpp-master/src/llama-grammar.cpp 19.4KB
  1058. llama.cpp-master/src/llama-grammar.h 1.09KB
  1059. llama.cpp-master/src/llama-impl.h 795B
  1060. llama.cpp-master/src/llama-sampling.cpp 22.09KB
  1061. llama.cpp-master/src/llama-sampling.h 2.63KB
  1062. llama.cpp-master/src/llama-vocab.cpp 66.57KB
  1063. llama.cpp-master/src/llama-vocab.h 4.64KB
  1064. llama.cpp-master/src/llama.cpp 789.47KB
  1065. llama.cpp-master/src/unicode-data.cpp 164.26KB
  1066. llama.cpp-master/src/unicode-data.h 582B
  1067. llama.cpp-master/src/unicode.cpp 29.9KB
  1068. llama.cpp-master/src/unicode.h 2.14KB
  1069. llama.cpp-master/tests/
  1070. llama.cpp-master/tests/.gitignore 25B
  1071. llama.cpp-master/tests/CMakeLists.txt 7.07KB
  1072. llama.cpp-master/tests/get-model.cpp 594B
  1073. llama.cpp-master/tests/get-model.h 53B
  1074. llama.cpp-master/tests/run-json-schema-to-grammar.mjs 395B
  1075. llama.cpp-master/tests/test-autorelease.cpp 719B
  1076. llama.cpp-master/tests/test-backend-ops.cpp 91.17KB
  1077. llama.cpp-master/tests/test-c.c 96B
  1078. llama.cpp-master/tests/test-chat-template.cpp 19.85KB
  1079. llama.cpp-master/tests/test-double-float.cpp 1.79KB
  1080. llama.cpp-master/tests/test-grad0.cpp 52.65KB
  1081. llama.cpp-master/tests/test-grammar-integration.cpp 35.63KB
  1082. llama.cpp-master/tests/test-grammar-parser.cpp 16.37KB
  1083. llama.cpp-master/tests/test-json-schema-to-grammar.cpp 38.9KB
  1084. llama.cpp-master/tests/test-llama-grammar.cpp 11.04KB
  1085. llama.cpp-master/tests/test-model-load-cancel.cpp 763B
  1086. llama.cpp-master/tests/test-opt.cpp 5.04KB
  1087. llama.cpp-master/tests/test-quantize-fns.cpp 6.54KB
  1088. llama.cpp-master/tests/test-quantize-perf.cpp 13.69KB
  1089. llama.cpp-master/tests/test-rope.cpp 6.12KB
  1090. llama.cpp-master/tests/test-sampling.cpp 13.34KB
  1091. llama.cpp-master/tests/test-tokenizer-0.cpp 10.46KB
  1092. llama.cpp-master/tests/test-tokenizer-0.py 1.92KB
  1093. llama.cpp-master/tests/test-tokenizer-0.sh 921B
  1094. llama.cpp-master/tests/test-tokenizer-1-bpe.cpp 4.68KB
  1095. llama.cpp-master/tests/test-tokenizer-1-spm.cpp 3.48KB
  1096. llama.cpp-master/tests/test-tokenizer-random.py 21.46KB

资源介绍:

描述 主要目标llama.cpp是在各种硬件(本地和云端)上以最少的设置和最先进的性能实现 LLM 推理。 纯 C/C++ 实现,无任何依赖项 Apple 芯片是一流的——通过 ARM NEON、Accelerate 和 Metal 框架进行了优化 AVX、AVX2 和 AVX512 支持 x86 架构 1.5 位、2 位、3 位、4 位、5 位、6 位和 8 位整数量化,可加快推理速度并减少内存使用 用于在 NVIDIA GPU 上运行 LLM 的自定义 CUDA 内核(通过 HIP 支持 AMD GPU) Vulkan 和 SYCL 后端支持 CPU+GPU 混合推理,部分加速大于 VRAM 总容量的模型
# LLaMA.cpp HTTP Server Fast, lightweight, pure C/C++ HTTP server based on [httplib](https://github.com/yhirose/cpp-httplib), [nlohmann::json](https://github.com/nlohmann/json) and **llama.cpp**. Set of LLM REST APIs and a simple web front end to interact with llama.cpp. **Features:** * LLM inference of F16 and quantized models on GPU and CPU * [OpenAI API](https://github.com/openai/openai-openapi) compatible chat completions and embeddings routes * Parallel decoding with multi-user support * Continuous batching * Multimodal (wip) * Monitoring endpoints * Schema-constrained JSON response format The project is under active development, and we are [looking for feedback and contributors](https://github.com/ggerganov/llama.cpp/issues/4216). ## Usage ``` usage: ./llama-server [options] general: -h, --help, --usage print usage and exit --version show version and build info -v, --verbose print verbose information --verbosity N set specific verbosity level (default: 0) --verbose-prompt print a verbose prompt before generation (default: false) --no-display-prompt don't print prompt at generation (default: false) -co, --color colorise output to distinguish prompt and user input from generations (default: false) -s, --seed SEED RNG seed (default: -1, use random seed for < 0) -t, --threads N number of threads to use during generation (default: 8) -tb, --threads-batch N number of threads to use during batch and prompt processing (default: same as --threads) -td, --threads-draft N number of threads to use during generation (default: same as --threads) -tbd, --threads-batch-draft N number of threads to use during batch and prompt processing (default: same as --threads-draft) --draft N number of tokens to draft for speculative decoding (default: 5) -ps, --p-split N speculative decoding split probability (default: 0.1) -lcs, --lookup-cache-static FNAME path to static lookup cache to use for lookup decoding (not updated by generation) -lcd, --lookup-cache-dynamic FNAME path to dynamic lookup cache to use for lookup decoding (updated by generation) -c, --ctx-size N size of the prompt context (default: 0, 0 = loaded from model) -n, --predict N number of tokens to predict (default: -1, -1 = infinity, -2 = until context filled) -b, --batch-size N logical maximum batch size (default: 2048) -ub, --ubatch-size N physical maximum batch size (default: 512) --keep N number of tokens to keep from the initial prompt (default: 0, -1 = all) --chunks N max number of chunks to process (default: -1, -1 = all) -fa, --flash-attn enable Flash Attention (default: disabled) -p, --prompt PROMPT prompt to start generation with in conversation mode, this will be used as system prompt (default: '') -f, --file FNAME a file containing the prompt (default: none) --in-file FNAME an input file (repeat to specify multiple files) -bf, --binary-file FNAME binary file containing the prompt (default: none) -e, --escape process escapes sequences (\n, \r, \t, \', \", \\) (default: true) --no-escape do not process escape sequences -ptc, --print-token-count N print token count every N tokens (default: -1) --prompt-cache FNAME file to cache prompt state for faster startup (default: none) --prompt-cache-all if specified, saves user input and generations to cache as well not supported with --interactive or other interactive options --prompt-cache-ro if specified, uses the prompt cache but does not update it -r, --reverse-prompt PROMPT halt generation at PROMPT, return control in interactive mode can be specified more than once for multiple prompts -sp, --special special tokens output enabled (default: false) -cnv, --conversation run in conversation mode, does not print special tokens and suffix/prefix if suffix/prefix are not specified, default chat template will be used (default: false) -i, --interactive run in interactive mode (default: false) -if, --interactive-first run in interactive mode and wait for input right away (default: false) -mli, --multiline-input allows you to write or paste multiple lines without ending each in '\' --in-prefix-bos prefix BOS to user inputs, preceding the `--in-prefix` string --in-prefix STRING string to prefix user inputs with (default: empty) --in-suffix STRING string to suffix after user inputs with (default: empty) --spm-infill use Suffix/Prefix/Middle pattern for infill (instead of Prefix/Suffix/Middle) as some models prefer this. (default: disabled) sampling: --samplers SAMPLERS samplers that will be used for generation in the order, separated by ';' (default: top_k;tfs_z;typical_p;top_p;min_p;temperature) --sampling-seq SEQUENCE simplified sequence for samplers that will be used (default: kfypmt) --ignore-eos ignore end of stream token and continue generating (implies --logit-bias EOS-inf) --penalize-nl penalize newline tokens (default: false) --temp N temperature (default: 0.8) --top-k N top-k sampling (default: 40, 0 = disabled) --top-p N top-p sampling (default: 0.9, 1.0 = disabled) --min-p N min-p sampling (default: 0.1, 0.0 = disabled) --tfs N tail free sampling, parameter z (default: 1.0, 1.0 = disabled) --typical N locally typical sampling, parameter p (default: 1.0, 1.0 = disabled) --repeat-last-n N last n tokens to consider for penalize (default: 64, 0 = disabled, -1 = ctx_size) --repeat-penalty N penalize repeat sequence of tokens (default: 1.0, 1.0 = disabled) --presence-penalty N repeat alpha presence penalty (default: 0.0, 0.0 = disabled) --frequency-penalty N repeat alpha frequency penalty (default: 0.0, 0.0 = disabled) --dynatemp-range N dynamic temperature range (default: 0.0, 0.0 = disabled) --dynatemp-exp N dynamic temperature exponent (default: 1.0) --mirostat N use Mirostat sampling. Top K, Nucleus, Tail Free and Locally Typical samplers are ignored if used. (default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0) --mirostat-lr N Mirostat learning rate, parameter eta (default: 0.1) --mirostat-ent N Mirostat target entropy, parameter tau (default: 5.0) -l TOKEN_ID(+/-)BIAS modifies the likelihood of token appearing in the completion, i.e. `--logit-bias 15043+1` to increase likelihood of token ' Hello', or `--logit-bias 15043-1` to decrease likelihood of token ' Hello' --cfg-negative-prompt PROMPT negative prompt to use for guidance (default: '') --cfg-negative-prompt-file FNAME negative prompt file to use for guidance --cfg-scale N strength of guidance (default: 1.0, 1.0 = disable)
100+评论
captcha