首页下载资源后端C/C++ 中的 LLM 推理

ZIPC/C++ 中的 LLM 推理

2301_8048821418.45MB需要积分:1

资源文件列表:

llama.cpp-master.zip 大约有1096个文件
  1. llama.cpp-master/
  2. llama.cpp-master/.clang-tidy 791B
  3. llama.cpp-master/.devops/
  4. llama.cpp-master/.devops/cloud-v-pipeline 1.05KB
  5. llama.cpp-master/.devops/full-cuda.Dockerfile 848B
  6. llama.cpp-master/.devops/full-rocm.Dockerfile 1.1KB
  7. llama.cpp-master/.devops/full.Dockerfile 470B
  8. llama.cpp-master/.devops/llama-cli-cuda.Dockerfile 864B
  9. llama.cpp-master/.devops/llama-cli-intel.Dockerfile 740B
  10. llama.cpp-master/.devops/llama-cli-rocm.Dockerfile 1.01KB
  11. llama.cpp-master/.devops/llama-cli-vulkan.Dockerfile 724B
  12. llama.cpp-master/.devops/llama-cli.Dockerfile 376B
  13. llama.cpp-master/.devops/llama-cpp-cuda.srpm.spec 2.56KB
  14. llama.cpp-master/.devops/llama-cpp.srpm.spec 2.63KB
  15. llama.cpp-master/.devops/llama-server-cuda.Dockerfile 1020B
  16. llama.cpp-master/.devops/llama-server-intel.Dockerfile 900B
  17. llama.cpp-master/.devops/llama-server-rocm.Dockerfile 1.18KB
  18. llama.cpp-master/.devops/llama-server-vulkan.Dockerfile 843B
  19. llama.cpp-master/.devops/llama-server.Dockerfile 519B
  20. llama.cpp-master/.devops/nix/
  21. llama.cpp-master/.devops/nix/apps.nix 434B
  22. llama.cpp-master/.devops/nix/devshells.nix 279B
  23. llama.cpp-master/.devops/nix/docker.nix 850B
  24. llama.cpp-master/.devops/nix/jetson-support.nix 1.05KB
  25. llama.cpp-master/.devops/nix/nixpkgs-instances.nix 1.67KB
  26. llama.cpp-master/.devops/nix/package.nix 9.74KB
  27. llama.cpp-master/.devops/nix/scope.nix 514B
  28. llama.cpp-master/.devops/nix/sif.nix 729B
  29. llama.cpp-master/.devops/tools.sh 1.67KB
  30. llama.cpp-master/.dockerignore 158B
  31. llama.cpp-master/.ecrc 80B
  32. llama.cpp-master/.editorconfig 599B
  33. llama.cpp-master/.flake8 544B
  34. llama.cpp-master/.github/
  35. llama.cpp-master/.github/ISSUE_TEMPLATE/
  36. llama.cpp-master/.github/ISSUE_TEMPLATE/01-bug-low.yml 1.7KB
  37. llama.cpp-master/.github/ISSUE_TEMPLATE/02-bug-medium.yml 1.71KB
  38. llama.cpp-master/.github/ISSUE_TEMPLATE/03-bug-high.yml 1.72KB
  39. llama.cpp-master/.github/ISSUE_TEMPLATE/04-bug-critical.yml 1.7KB
  40. llama.cpp-master/.github/ISSUE_TEMPLATE/05-enhancement.yml 2.35KB
  41. llama.cpp-master/.github/ISSUE_TEMPLATE/06-research.yml 1.69KB
  42. llama.cpp-master/.github/ISSUE_TEMPLATE/07-refactor.yml 1.2KB
  43. llama.cpp-master/.github/ISSUE_TEMPLATE/config.yml 524B
  44. llama.cpp-master/.github/labeler.yml 2.32KB
  45. llama.cpp-master/.github/pull_request_template.md 193B
  46. llama.cpp-master/.github/workflows/
  47. llama.cpp-master/.github/workflows/bench.yml 10.42KB
  48. llama.cpp-master/.github/workflows/build.yml 42.25KB
  49. llama.cpp-master/.github/workflows/close-issue.yml 717B
  50. llama.cpp-master/.github/workflows/docker.yml 5.02KB
  51. llama.cpp-master/.github/workflows/editorconfig.yml 607B
  52. llama.cpp-master/.github/workflows/gguf-publish.yml 1.2KB
  53. llama.cpp-master/.github/workflows/labeler.yml 355B
  54. llama.cpp-master/.github/workflows/nix-ci-aarch64.yml 2.23KB
  55. llama.cpp-master/.github/workflows/nix-ci.yml 2.51KB
  56. llama.cpp-master/.github/workflows/nix-flake-update.yml 607B
  57. llama.cpp-master/.github/workflows/nix-publish-flake.yml 1.11KB
  58. llama.cpp-master/.github/workflows/python-check-requirements.yml 966B
  59. llama.cpp-master/.github/workflows/python-lint.yml 561B
  60. llama.cpp-master/.github/workflows/python-type-check.yml 995B
  61. llama.cpp-master/.github/workflows/server.yml 6KB
  62. llama.cpp-master/.gitignore 1.45KB
  63. llama.cpp-master/.gitmodules 94B
  64. llama.cpp-master/.pre-commit-config.yaml 447B
  65. llama.cpp-master/AUTHORS 32.93KB
  66. llama.cpp-master/CMakeLists.txt 6.29KB
  67. llama.cpp-master/CMakePresets.json 2.82KB
  68. llama.cpp-master/CONTRIBUTING.md 2.2KB
  69. llama.cpp-master/LICENSE 1.05KB
  70. llama.cpp-master/Makefile 48.57KB
  71. llama.cpp-master/Package.swift 2KB
  72. llama.cpp-master/README.md 28.74KB
  73. llama.cpp-master/SECURITY.md 4.97KB
  74. llama.cpp-master/ci/
  75. llama.cpp-master/ci/README.md 1.06KB
  76. llama.cpp-master/ci/run.sh 37.51KB
  77. llama.cpp-master/cmake/
  78. llama.cpp-master/cmake/arm64-windows-llvm.cmake 592B
  79. llama.cpp-master/cmake/arm64-windows-msvc.cmake 192B
  80. llama.cpp-master/cmake/build-info.cmake 1.57KB
  81. llama.cpp-master/cmake/git-vars.cmake 717B
  82. llama.cpp-master/cmake/llama-config.cmake.in 2.39KB
  83. llama.cpp-master/cmake/llama.pc.in 250B
  84. llama.cpp-master/common/
  85. llama.cpp-master/common/CMakeLists.txt 2.74KB
  86. llama.cpp-master/common/base64.hpp 12.58KB
  87. llama.cpp-master/common/build-info.cpp.in 186B
  88. llama.cpp-master/common/cmake/
  89. llama.cpp-master/common/cmake/build-info-gen-cpp.cmake 943B
  90. llama.cpp-master/common/common.cpp 135.3KB
  91. llama.cpp-master/common/common.h 19.7KB
  92. llama.cpp-master/common/console.cpp 15.86KB
  93. llama.cpp-master/common/console.h 359B
  94. llama.cpp-master/common/grammar-parser.cpp 21.75KB
  95. llama.cpp-master/common/grammar-parser.h 874B
  96. llama.cpp-master/common/json-schema-to-grammar.cpp 42.49KB
  97. llama.cpp-master/common/json-schema-to-grammar.h 211B
  98. llama.cpp-master/common/json.hpp 898.69KB
  99. llama.cpp-master/common/log.h 24.09KB
  100. llama.cpp-master/common/ngram-cache.cpp 11.08KB
  101. llama.cpp-master/common/ngram-cache.h 3.98KB
  102. llama.cpp-master/common/sampling.cpp 17.68KB
  103. llama.cpp-master/common/sampling.h 6.33KB
  104. llama.cpp-master/common/stb_image.h 313.42KB
  105. llama.cpp-master/common/train.cpp 64.78KB
  106. llama.cpp-master/common/train.h 7.7KB
  107. llama.cpp-master/convert_hf_to_gguf.py 169.04KB
  108. llama.cpp-master/convert_hf_to_gguf_update.py 14.43KB
  109. llama.cpp-master/convert_llama_ggml_to_gguf.py 18.63KB
  110. llama.cpp-master/convert_lora_to_gguf.py 14.04KB
  111. llama.cpp-master/docs/
  112. llama.cpp-master/docs/android.md 2.42KB
  113. llama.cpp-master/docs/backend/
  114. llama.cpp-master/docs/backend/BLIS.md 1.7KB
  115. llama.cpp-master/docs/backend/SYCL.md 23.63KB
  116. llama.cpp-master/docs/build.md 19.97KB
  117. llama.cpp-master/docs/development/
  118. llama.cpp-master/docs/development/HOWTO-add-model.md 4.8KB
  119. llama.cpp-master/docs/development/debugging-tests.md 3.1KB
  120. llama.cpp-master/docs/development/llama-star/
  121. llama.cpp-master/docs/development/llama-star/idea-arch.key 477.14KB
  122. llama.cpp-master/docs/development/llama-star/idea-arch.pdf 41.34KB
  123. llama.cpp-master/docs/development/token_generation_performance_tips.md 2.25KB
  124. llama.cpp-master/docs/docker.md 4.72KB
  125. llama.cpp-master/docs/install.md 872B
  126. llama.cpp-master/examples/
  127. llama.cpp-master/examples/CMakeLists.txt 1.33KB
  128. llama.cpp-master/examples/Miku.sh 2.57KB
  129. llama.cpp-master/examples/baby-llama/
  130. llama.cpp-master/examples/baby-llama/CMakeLists.txt 239B
  131. llama.cpp-master/examples/baby-llama/baby-llama.cpp 61.05KB
  132. llama.cpp-master/examples/base-translate.sh 1001B
  133. llama.cpp-master/examples/batched-bench/
  134. llama.cpp-master/examples/batched-bench/CMakeLists.txt 245B
  135. llama.cpp-master/examples/batched-bench/README.md 2.74KB
  136. llama.cpp-master/examples/batched-bench/batched-bench.cpp 6.63KB
  137. llama.cpp-master/examples/batched.swift/
  138. llama.cpp-master/examples/batched.swift/.gitignore 173B
  139. llama.cpp-master/examples/batched.swift/Makefile 230B
  140. llama.cpp-master/examples/batched.swift/Package.swift 766B
  141. llama.cpp-master/examples/batched.swift/README.md 112B
  142. llama.cpp-master/examples/batched.swift/Sources/
  143. llama.cpp-master/examples/batched.swift/Sources/main.swift 7.57KB
  144. llama.cpp-master/examples/batched/
  145. llama.cpp-master/examples/batched/CMakeLists.txt 233B
  146. llama.cpp-master/examples/batched/README.md 1.39KB
  147. llama.cpp-master/examples/batched/batched.cpp 7.81KB
  148. llama.cpp-master/examples/benchmark/
  149. llama.cpp-master/examples/benchmark/CMakeLists.txt 312B
  150. llama.cpp-master/examples/benchmark/benchmark-matmult.cpp 9.61KB
  151. llama.cpp-master/examples/chat-13B.bat 2.39KB
  152. llama.cpp-master/examples/chat-13B.sh 1.31KB
  153. llama.cpp-master/examples/chat-persistent.sh 4.93KB
  154. llama.cpp-master/examples/chat-vicuna.sh 1.3KB
  155. llama.cpp-master/examples/chat.sh 349B
  156. llama.cpp-master/examples/convert-llama2c-to-ggml/
  157. llama.cpp-master/examples/convert-llama2c-to-ggml/CMakeLists.txt 265B
  158. llama.cpp-master/examples/convert-llama2c-to-ggml/README.md 1.52KB
  159. llama.cpp-master/examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp 34.26KB
  160. llama.cpp-master/examples/convert_legacy_llama.py 58.15KB
  161. llama.cpp-master/examples/cvector-generator/
  162. llama.cpp-master/examples/cvector-generator/CMakeLists.txt 261B
  163. llama.cpp-master/examples/cvector-generator/README.md 1.54KB
  164. llama.cpp-master/examples/cvector-generator/completions.txt 6.75KB
  165. llama.cpp-master/examples/cvector-generator/cvector-generator.cpp 17.91KB
  166. llama.cpp-master/examples/cvector-generator/mean.hpp 1.49KB
  167. llama.cpp-master/examples/cvector-generator/negative.txt 989B
  168. llama.cpp-master/examples/cvector-generator/pca.hpp 11.43KB
  169. llama.cpp-master/examples/cvector-generator/positive.txt 955B
  170. llama.cpp-master/examples/deprecation-warning/
  171. llama.cpp-master/examples/deprecation-warning/README.md 1.64KB
  172. llama.cpp-master/examples/deprecation-warning/deprecation-warning.cpp 1.15KB
  173. llama.cpp-master/examples/embedding/
  174. llama.cpp-master/examples/embedding/CMakeLists.txt 237B
  175. llama.cpp-master/examples/embedding/README.md 2.14KB
  176. llama.cpp-master/examples/embedding/embedding.cpp 9.36KB
  177. llama.cpp-master/examples/eval-callback/
  178. llama.cpp-master/examples/eval-callback/CMakeLists.txt 530B
  179. llama.cpp-master/examples/eval-callback/README.md 4.61KB
  180. llama.cpp-master/examples/eval-callback/eval-callback.cpp 6.09KB
  181. llama.cpp-master/examples/export-lora/
  182. llama.cpp-master/examples/export-lora/CMakeLists.txt 241B
  183. llama.cpp-master/examples/export-lora/README.md 1.12KB
  184. llama.cpp-master/examples/export-lora/export-lora.cpp 15.98KB
  185. llama.cpp-master/examples/gbnf-validator/
  186. llama.cpp-master/examples/gbnf-validator/CMakeLists.txt 247B
  187. llama.cpp-master/examples/gbnf-validator/gbnf-validator.cpp 4.36KB
  188. llama.cpp-master/examples/gguf-hash/
  189. llama.cpp-master/examples/gguf-hash/CMakeLists.txt 618B
  190. llama.cpp-master/examples/gguf-hash/README.md 10.41KB
  191. llama.cpp-master/examples/gguf-hash/deps/
  192. llama.cpp-master/examples/gguf-hash/deps/rotate-bits/
  193. llama.cpp-master/examples/gguf-hash/deps/rotate-bits/package.json 255B
  194. llama.cpp-master/examples/gguf-hash/deps/rotate-bits/rotate-bits.h 1017B
  195. llama.cpp-master/examples/gguf-hash/deps/sha1/
  196. llama.cpp-master/examples/gguf-hash/deps/sha1/package.json 200B
  197. llama.cpp-master/examples/gguf-hash/deps/sha1/sha1.c 7.44KB
  198. llama.cpp-master/examples/gguf-hash/deps/sha1/sha1.h 717B
  199. llama.cpp-master/examples/gguf-hash/deps/sha256/
  200. llama.cpp-master/examples/gguf-hash/deps/sha256/package.json 283B
  201. llama.cpp-master/examples/gguf-hash/deps/sha256/sha256.c 5.16KB
  202. llama.cpp-master/examples/gguf-hash/deps/sha256/sha256.h 549B
  203. llama.cpp-master/examples/gguf-hash/deps/xxhash/
  204. llama.cpp-master/examples/gguf-hash/deps/xxhash/clib.json 255B
  205. llama.cpp-master/examples/gguf-hash/deps/xxhash/xxhash.c 1.81KB
  206. llama.cpp-master/examples/gguf-hash/deps/xxhash/xxhash.h 258.54KB
  207. llama.cpp-master/examples/gguf-hash/gguf-hash.cpp 23.38KB
  208. llama.cpp-master/examples/gguf-split/
  209. llama.cpp-master/examples/gguf-split/CMakeLists.txt 239B
  210. llama.cpp-master/examples/gguf-split/README.md 343B
  211. llama.cpp-master/examples/gguf-split/gguf-split.cpp 19.4KB
  212. llama.cpp-master/examples/gguf-split/tests.sh 2.12KB
  213. llama.cpp-master/examples/gguf/
  214. llama.cpp-master/examples/gguf/CMakeLists.txt 219B
  215. llama.cpp-master/examples/gguf/gguf.cpp 7.92KB
  216. llama.cpp-master/examples/gritlm/
  217. llama.cpp-master/examples/gritlm/CMakeLists.txt 231B
  218. llama.cpp-master/examples/gritlm/README.md 2.73KB
  219. llama.cpp-master/examples/gritlm/gritlm.cpp 9.74KB
  220. llama.cpp-master/examples/imatrix/
  221. llama.cpp-master/examples/imatrix/CMakeLists.txt 233B
  222. llama.cpp-master/examples/imatrix/README.md 2KB
  223. llama.cpp-master/examples/imatrix/imatrix.cpp 22.23KB
  224. llama.cpp-master/examples/infill/
  225. llama.cpp-master/examples/infill/CMakeLists.txt 231B
  226. llama.cpp-master/examples/infill/README.md 2.61KB
  227. llama.cpp-master/examples/infill/infill.cpp 23.62KB
  228. llama.cpp-master/examples/jeopardy/
  229. llama.cpp-master/examples/jeopardy/README.md 1KB
  230. llama.cpp-master/examples/jeopardy/graph.py 1.61KB
  231. llama.cpp-master/examples/jeopardy/jeopardy.sh 851B
  232. llama.cpp-master/examples/jeopardy/qasheet.csv 16.28KB
  233. llama.cpp-master/examples/jeopardy/questions.txt 12.02KB
  234. llama.cpp-master/examples/json_schema_pydantic_example.py 3.14KB
  235. llama.cpp-master/examples/json_schema_to_grammar.py 32.91KB
  236. llama.cpp-master/examples/llama-bench/
  237. llama.cpp-master/examples/llama-bench/CMakeLists.txt 235B
  238. llama.cpp-master/examples/llama-bench/README.md 13.99KB
  239. llama.cpp-master/examples/llama-bench/llama-bench.cpp 51.46KB
  240. llama.cpp-master/examples/llama.android/
  241. llama.cpp-master/examples/llama.android/.gitignore 431B
  242. llama.cpp-master/examples/llama.android/README.md
  243. llama.cpp-master/examples/llama.android/app/
  244. llama.cpp-master/examples/llama.android/app/.gitignore 7B
  245. llama.cpp-master/examples/llama.android/app/build.gradle.kts 1.96KB
  246. llama.cpp-master/examples/llama.android/app/proguard-rules.pro 751B
  247. llama.cpp-master/examples/llama.android/app/src/
  248. llama.cpp-master/examples/llama.android/app/src/main/
  249. llama.cpp-master/examples/llama.android/app/src/main/AndroidManifest.xml 1.02KB
  250. llama.cpp-master/examples/llama.android/app/src/main/java/
  251. llama.cpp-master/examples/llama.android/app/src/main/java/com/
  252. llama.cpp-master/examples/llama.android/app/src/main/java/com/example/
  253. llama.cpp-master/examples/llama.android/app/src/main/java/com/example/llama/
  254. llama.cpp-master/examples/llama.android/app/src/main/java/com/example/llama/Downloadable.kt 4.42KB
  255. llama.cpp-master/examples/llama.android/app/src/main/java/com/example/llama/MainActivity.kt 5.5KB
  256. llama.cpp-master/examples/llama.android/app/src/main/java/com/example/llama/MainViewModel.kt 2.84KB
  257. llama.cpp-master/examples/llama.android/app/src/main/java/com/example/llama/ui/
  258. llama.cpp-master/examples/llama.android/app/src/main/java/com/example/llama/ui/theme/
  259. llama.cpp-master/examples/llama.android/app/src/main/java/com/example/llama/ui/theme/Color.kt 282B
  260. llama.cpp-master/examples/llama.android/app/src/main/java/com/example/llama/ui/theme/Theme.kt 2.14KB
  261. llama.cpp-master/examples/llama.android/app/src/main/java/com/example/llama/ui/theme/Type.kt 987B
  262. llama.cpp-master/examples/llama.android/app/src/main/res/
  263. llama.cpp-master/examples/llama.android/app/src/main/res/drawable/
  264. llama.cpp-master/examples/llama.android/app/src/main/res/drawable/ic_launcher_background.xml 5.47KB
  265. llama.cpp-master/examples/llama.android/app/src/main/res/drawable/ic_launcher_foreground.xml 1.66KB
  266. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-anydpi/
  267. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-anydpi/ic_launcher.xml 344B
  268. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-anydpi/ic_launcher_round.xml 344B
  269. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-hdpi/
  270. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-hdpi/ic_launcher.webp 1.37KB
  271. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-hdpi/ic_launcher_round.webp 2.83KB
  272. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-mdpi/
  273. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-mdpi/ic_launcher.webp 982B
  274. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-mdpi/ic_launcher_round.webp 1.73KB
  275. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-xhdpi/
  276. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-xhdpi/ic_launcher.webp 1.86KB
  277. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-xhdpi/ic_launcher_round.webp 3.83KB
  278. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-xxhdpi/
  279. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-xxhdpi/ic_launcher.webp 2.82KB
  280. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.webp 5.78KB
  281. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-xxxhdpi/
  282. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-xxxhdpi/ic_launcher.webp 3.75KB
  283. llama.cpp-master/examples/llama.android/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.webp 7.6KB
  284. llama.cpp-master/examples/llama.android/app/src/main/res/values/
  285. llama.cpp-master/examples/llama.android/app/src/main/res/values/colors.xml 379B
  286. llama.cpp-master/examples/llama.android/app/src/main/res/values/strings.xml 75B
  287. llama.cpp-master/examples/llama.android/app/src/main/res/values/themes.xml 155B
  288. llama.cpp-master/examples/llama.android/app/src/main/res/xml/
  289. llama.cpp-master/examples/llama.android/app/src/main/res/xml/backup_rules.xml 479B
  290. llama.cpp-master/examples/llama.android/app/src/main/res/xml/data_extraction_rules.xml 552B
  291. llama.cpp-master/examples/llama.android/build.gradle.kts 299B
  292. llama.cpp-master/examples/llama.android/gradle.properties 1.33KB
  293. llama.cpp-master/examples/llama.android/gradle/
  294. llama.cpp-master/examples/llama.android/gradle/wrapper/
  295. llama.cpp-master/examples/llama.android/gradle/wrapper/gradle-wrapper.jar 57.82KB
  296. llama.cpp-master/examples/llama.android/gradle/wrapper/gradle-wrapper.properties 231B
  297. llama.cpp-master/examples/llama.android/gradlew 5.63KB
  298. llama.cpp-master/examples/llama.android/llama/
  299. llama.cpp-master/examples/llama.android/llama/.gitignore 7B
  300. llama.cpp-master/examples/llama.android/llama/build.gradle.kts 1.69KB
  301. llama.cpp-master/examples/llama.android/llama/consumer-rules.pro
  302. llama.cpp-master/examples/llama.android/llama/proguard-rules.pro 751B
  303. llama.cpp-master/examples/llama.android/llama/src/
  304. llama.cpp-master/examples/llama.android/llama/src/androidTest/
  305. llama.cpp-master/examples/llama.android/llama/src/androidTest/java/
  306. llama.cpp-master/examples/llama.android/llama/src/androidTest/java/android/
  307. llama.cpp-master/examples/llama.android/llama/src/androidTest/java/android/llama/
  308. llama.cpp-master/examples/llama.android/llama/src/androidTest/java/android/llama/cpp/
  309. llama.cpp-master/examples/llama.android/llama/src/androidTest/java/android/llama/cpp/ExampleInstrumentedTest.kt 667B
  310. llama.cpp-master/examples/llama.android/llama/src/main/
  311. llama.cpp-master/examples/llama.android/llama/src/main/AndroidManifest.xml 122B
  312. llama.cpp-master/examples/llama.android/llama/src/main/cpp/
  313. llama.cpp-master/examples/llama.android/llama/src/main/cpp/CMakeLists.txt 2.12KB
  314. llama.cpp-master/examples/llama.android/llama/src/main/cpp/llama-android.cpp 13.28KB
  315. llama.cpp-master/examples/llama.android/llama/src/main/java/
  316. llama.cpp-master/examples/llama.android/llama/src/main/java/android/
  317. llama.cpp-master/examples/llama.android/llama/src/main/java/android/llama/
  318. llama.cpp-master/examples/llama.android/llama/src/main/java/android/llama/cpp/
  319. llama.cpp-master/examples/llama.android/llama/src/main/java/android/llama/cpp/LLamaAndroid.kt 5.32KB
  320. llama.cpp-master/examples/llama.android/llama/src/test/
  321. llama.cpp-master/examples/llama.android/llama/src/test/java/
  322. llama.cpp-master/examples/llama.android/llama/src/test/java/android/
  323. llama.cpp-master/examples/llama.android/llama/src/test/java/android/llama/
  324. llama.cpp-master/examples/llama.android/llama/src/test/java/android/llama/cpp/
  325. llama.cpp-master/examples/llama.android/llama/src/test/java/android/llama/cpp/ExampleUnitTest.kt 342B
  326. llama.cpp-master/examples/llama.android/settings.gradle.kts 349B
  327. llama.cpp-master/examples/llama.swiftui/
  328. llama.cpp-master/examples/llama.swiftui/.gitignore 24B
  329. llama.cpp-master/examples/llama.swiftui/README.md 517B
  330. llama.cpp-master/examples/llama.swiftui/llama.cpp.swift/
  331. llama.cpp-master/examples/llama.swiftui/llama.cpp.swift/LibLlama.swift 11.24KB
  332. llama.cpp-master/examples/llama.swiftui/llama.swiftui.xcodeproj/
  333. llama.cpp-master/examples/llama.swiftui/llama.swiftui.xcodeproj/project.pbxproj 18KB
  334. llama.cpp-master/examples/llama.swiftui/llama.swiftui.xcodeproj/project.xcworkspace/
  335. llama.cpp-master/examples/llama.swiftui/llama.swiftui.xcodeproj/project.xcworkspace/contents.xcworkspacedata 135B
  336. llama.cpp-master/examples/llama.swiftui/llama.swiftui.xcodeproj/project.xcworkspace/xcshareddata/
  337. llama.cpp-master/examples/llama.swiftui/llama.swiftui.xcodeproj/project.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist 244B
  338. llama.cpp-master/examples/llama.swiftui/llama.swiftui/
  339. llama.cpp-master/examples/llama.swiftui/llama.swiftui/Assets.xcassets/
  340. llama.cpp-master/examples/llama.swiftui/llama.swiftui/Assets.xcassets/AppIcon.appiconset/
  341. llama.cpp-master/examples/llama.swiftui/llama.swiftui/Assets.xcassets/AppIcon.appiconset/Contents.json 177B
  342. llama.cpp-master/examples/llama.swiftui/llama.swiftui/Assets.xcassets/Contents.json 63B
  343. llama.cpp-master/examples/llama.swiftui/llama.swiftui/Models/
  344. llama.cpp-master/examples/llama.swiftui/llama.swiftui/Models/LlamaState.swift 6.99KB
  345. llama.cpp-master/examples/llama.swiftui/llama.swiftui/Resources/
  346. llama.cpp-master/examples/llama.swiftui/llama.swiftui/Resources/models/
  347. llama.cpp-master/examples/llama.swiftui/llama.swiftui/Resources/models/.gitignore
  348. llama.cpp-master/examples/llama.swiftui/llama.swiftui/UI/
  349. llama.cpp-master/examples/llama.swiftui/llama.swiftui/UI/ContentView.swift 4.73KB
  350. llama.cpp-master/examples/llama.swiftui/llama.swiftui/UI/DownloadButton.swift 4.41KB
  351. llama.cpp-master/examples/llama.swiftui/llama.swiftui/UI/InputButton.swift 4.74KB
  352. llama.cpp-master/examples/llama.swiftui/llama.swiftui/UI/LoadCustomButton.swift 1.29KB
  353. llama.cpp-master/examples/llama.swiftui/llama.swiftui/llama_swiftuiApp.swift 146B
  354. llama.cpp-master/examples/llama.vim 5.05KB
  355. llama.cpp-master/examples/llava/
  356. llama.cpp-master/examples/llava/CMakeLists.txt 1.26KB
  357. llama.cpp-master/examples/llava/MobileVLM-README.md 18.42KB
  358. llama.cpp-master/examples/llava/README.md 5.18KB
  359. llama.cpp-master/examples/llava/android/
  360. llama.cpp-master/examples/llava/android/adb_run.sh 2.55KB
  361. llama.cpp-master/examples/llava/android/build_64.sh 207B
  362. llama.cpp-master/examples/llava/clip.cpp 85.03KB
  363. llama.cpp-master/examples/llava/clip.h 2.83KB
  364. llama.cpp-master/examples/llava/convert_image_encoder_to_gguf.py 13.58KB
  365. llama.cpp-master/examples/llava/llava-cli.cpp 12.57KB
  366. llama.cpp-master/examples/llava/llava.cpp 17.95KB
  367. llama.cpp-master/examples/llava/llava.h 1.79KB
  368. llama.cpp-master/examples/llava/llava_surgery.py 1.34KB
  369. llama.cpp-master/examples/llava/llava_surgery_v2.py 6.89KB
  370. llama.cpp-master/examples/llava/requirements.txt 143B
  371. llama.cpp-master/examples/llm.vim 921B
  372. llama.cpp-master/examples/lookahead/
  373. llama.cpp-master/examples/lookahead/CMakeLists.txt 237B
  374. llama.cpp-master/examples/lookahead/README.md 195B
  375. llama.cpp-master/examples/lookahead/lookahead.cpp 16.12KB
  376. llama.cpp-master/examples/lookup/
  377. llama.cpp-master/examples/lookup/CMakeLists.txt 965B
  378. llama.cpp-master/examples/lookup/README.md 487B
  379. llama.cpp-master/examples/lookup/lookup-create.cpp 1.17KB
  380. llama.cpp-master/examples/lookup/lookup-merge.cpp 1.34KB
  381. llama.cpp-master/examples/lookup/lookup-stats.cpp 5.6KB
  382. llama.cpp-master/examples/lookup/lookup.cpp 8.37KB
  383. llama.cpp-master/examples/main-cmake-pkg/
  384. llama.cpp-master/examples/main-cmake-pkg/.gitignore 387B
  385. llama.cpp-master/examples/main-cmake-pkg/CMakeLists.txt 1.25KB
  386. llama.cpp-master/examples/main-cmake-pkg/README.md 1.25KB
  387. llama.cpp-master/examples/main/
  388. llama.cpp-master/examples/main/CMakeLists.txt 226B
  389. llama.cpp-master/examples/main/README.md 27.16KB
  390. llama.cpp-master/examples/main/main.cpp 39.6KB
  391. llama.cpp-master/examples/parallel/
  392. llama.cpp-master/examples/parallel/CMakeLists.txt 235B
  393. llama.cpp-master/examples/parallel/README.md 93B
  394. llama.cpp-master/examples/parallel/parallel.cpp 15.49KB
  395. llama.cpp-master/examples/passkey/
  396. llama.cpp-master/examples/passkey/CMakeLists.txt 233B
  397. llama.cpp-master/examples/passkey/README.md 409B
  398. llama.cpp-master/examples/passkey/passkey.cpp 8.7KB
  399. llama.cpp-master/examples/perplexity/
  400. llama.cpp-master/examples/perplexity/CMakeLists.txt 239B
  401. llama.cpp-master/examples/perplexity/README.md 19.51KB
  402. llama.cpp-master/examples/perplexity/perplexity.cpp 79.32KB
  403. llama.cpp-master/examples/pydantic_models_to_grammar.py 54.87KB
  404. llama.cpp-master/examples/pydantic_models_to_grammar_examples.py 13.4KB
  405. llama.cpp-master/examples/quantize-stats/
  406. llama.cpp-master/examples/quantize-stats/CMakeLists.txt 310B
  407. llama.cpp-master/examples/quantize-stats/quantize-stats.cpp 15.66KB
  408. llama.cpp-master/examples/quantize/
  409. llama.cpp-master/examples/quantize/CMakeLists.txt 294B
  410. llama.cpp-master/examples/quantize/README.md 5.05KB
  411. llama.cpp-master/examples/quantize/quantize.cpp 18.61KB
  412. llama.cpp-master/examples/quantize/tests.sh 1.5KB
  413. llama.cpp-master/examples/reason-act.sh 355B
  414. llama.cpp-master/examples/regex_to_grammar.py 431B
  415. llama.cpp-master/examples/retrieval/
  416. llama.cpp-master/examples/retrieval/CMakeLists.txt 237B
  417. llama.cpp-master/examples/retrieval/README.md 2.09KB
  418. llama.cpp-master/examples/retrieval/retrieval.cpp 10.11KB
  419. llama.cpp-master/examples/rpc/
  420. llama.cpp-master/examples/rpc/CMakeLists.txt 95B
  421. llama.cpp-master/examples/rpc/README.md 2.28KB
  422. llama.cpp-master/examples/rpc/rpc-server.cpp 4.15KB
  423. llama.cpp-master/examples/save-load-state/
  424. llama.cpp-master/examples/save-load-state/CMakeLists.txt 249B
  425. llama.cpp-master/examples/save-load-state/save-load-state.cpp 8.36KB
  426. llama.cpp-master/examples/server-llama2-13B.sh 790B
  427. llama.cpp-master/examples/server/
  428. llama.cpp-master/examples/server/CMakeLists.txt 1.79KB
  429. llama.cpp-master/examples/server/README.md 42.75KB
  430. llama.cpp-master/examples/server/bench/
  431. llama.cpp-master/examples/server/bench/README.md 4.2KB
  432. llama.cpp-master/examples/server/bench/bench.py 12.99KB
  433. llama.cpp-master/examples/server/bench/prometheus.yml 183B
  434. llama.cpp-master/examples/server/bench/requirements.txt 20B
  435. llama.cpp-master/examples/server/bench/script.js 5.76KB
  436. llama.cpp-master/examples/server/chat-llama2.sh 2.46KB
  437. llama.cpp-master/examples/server/chat.mjs 3.79KB
  438. llama.cpp-master/examples/server/chat.sh 1.93KB
  439. llama.cpp-master/examples/server/deps.sh 374B
  440. llama.cpp-master/examples/server/httplib.h 303.63KB
  441. llama.cpp-master/examples/server/public/
  442. llama.cpp-master/examples/server/public/colorthemes.css 11.12KB
  443. llama.cpp-master/examples/server/public/completion.js 5.81KB
  444. llama.cpp-master/examples/server/public/favicon.ico 4.03KB
  445. llama.cpp-master/examples/server/public/index-new.html 47.66KB
  446. llama.cpp-master/examples/server/public/index.html 41.58KB
  447. llama.cpp-master/examples/server/public/index.js 22.53KB
  448. llama.cpp-master/examples/server/public/json-schema-to-grammar.mjs 28.5KB
  449. llama.cpp-master/examples/server/public/prompt-formats.js 6.02KB
  450. llama.cpp-master/examples/server/public/style.css 19.6KB
  451. llama.cpp-master/examples/server/public/system-prompts.js 10.5KB
  452. llama.cpp-master/examples/server/public/theme-beeninorder.css 6.95KB
  453. llama.cpp-master/examples/server/public/theme-ketivah.css 7.14KB
  454. llama.cpp-master/examples/server/public/theme-mangotango.css 6.58KB
  455. llama.cpp-master/examples/server/public/theme-playground.css 6.83KB
  456. llama.cpp-master/examples/server/public/theme-polarnight.css 8.01KB
  457. llama.cpp-master/examples/server/public/theme-snowstorm.css 8KB
  458. llama.cpp-master/examples/server/public_simplechat/
  459. llama.cpp-master/examples/server/public_simplechat/datautils.mjs 8.94KB
  460. llama.cpp-master/examples/server/public_simplechat/index.html 1.91KB
  461. llama.cpp-master/examples/server/public_simplechat/readme.md 14.42KB
  462. llama.cpp-master/examples/server/public_simplechat/simplechat.css 1KB
  463. llama.cpp-master/examples/server/public_simplechat/simplechat.js 30.53KB
  464. llama.cpp-master/examples/server/public_simplechat/simplechat_screens.webp 20.88KB
  465. llama.cpp-master/examples/server/public_simplechat/ui.mjs 5.94KB
  466. llama.cpp-master/examples/server/server.cpp 139.03KB
  467. llama.cpp-master/examples/server/tests/
  468. llama.cpp-master/examples/server/tests/README.md 2.79KB
  469. llama.cpp-master/examples/server/tests/features/
  470. llama.cpp-master/examples/server/tests/features/embeddings.feature 2.42KB
  471. llama.cpp-master/examples/server/tests/features/environment.py 2.53KB
  472. llama.cpp-master/examples/server/tests/features/issues.feature 139B
  473. llama.cpp-master/examples/server/tests/features/lora.feature 1.14KB
  474. llama.cpp-master/examples/server/tests/features/parallel.feature 2.7KB
  475. llama.cpp-master/examples/server/tests/features/passkey.feature 2.66KB
  476. llama.cpp-master/examples/server/tests/features/results.feature 4.24KB
  477. llama.cpp-master/examples/server/tests/features/security.feature 2.48KB
  478. llama.cpp-master/examples/server/tests/features/server.feature 4.94KB
  479. llama.cpp-master/examples/server/tests/features/slotsave.feature 2.41KB
  480. llama.cpp-master/examples/server/tests/features/steps/
  481. llama.cpp-master/examples/server/tests/features/steps/steps.py 54.7KB
  482. llama.cpp-master/examples/server/tests/features/wrong_usages.feature 794B
  483. llama.cpp-master/examples/server/tests/requirements.txt 125B
  484. llama.cpp-master/examples/server/tests/tests.sh 197B
  485. llama.cpp-master/examples/server/themes/
  486. llama.cpp-master/examples/server/themes/README.md 182B
  487. llama.cpp-master/examples/server/themes/buttons-top/
  488. llama.cpp-master/examples/server/themes/buttons-top/README.md 260B
  489. llama.cpp-master/examples/server/themes/buttons-top/buttons_top.png 116.94KB
  490. llama.cpp-master/examples/server/themes/buttons-top/favicon.ico 4.03KB
  491. llama.cpp-master/examples/server/themes/buttons-top/index.html 33.74KB
  492. llama.cpp-master/examples/server/themes/wild/
  493. llama.cpp-master/examples/server/themes/wild/README.md 127B
  494. llama.cpp-master/examples/server/themes/wild/favicon.ico 4.03KB
  495. llama.cpp-master/examples/server/themes/wild/index.html 33.86KB
  496. llama.cpp-master/examples/server/themes/wild/llama_cpp.png 74.69KB
  497. llama.cpp-master/examples/server/themes/wild/llamapattern.png 253.5KB
  498. llama.cpp-master/examples/server/themes/wild/wild.png 484.83KB
  499. llama.cpp-master/examples/server/utils.hpp 21.02KB
  500. llama.cpp-master/examples/server_embd.py 971B
  501. llama.cpp-master/examples/simple/
  502. llama.cpp-master/examples/simple/CMakeLists.txt 231B
  503. llama.cpp-master/examples/simple/README.md 915B
  504. llama.cpp-master/examples/simple/simple.cpp 4.87KB
  505. llama.cpp-master/examples/speculative/
  506. llama.cpp-master/examples/speculative/CMakeLists.txt 241B
  507. llama.cpp-master/examples/speculative/README.md 285B
  508. llama.cpp-master/examples/speculative/speculative.cpp 23.86KB
  509. llama.cpp-master/examples/sycl/
  510. llama.cpp-master/examples/sycl/CMakeLists.txt 335B
  511. llama.cpp-master/examples/sycl/README.md 1.43KB
  512. llama.cpp-master/examples/sycl/build.sh 582B
  513. llama.cpp-master/examples/sycl/ls-sycl-device.cpp 195B
  514. llama.cpp-master/examples/sycl/run-llama2.sh 1.23KB
  515. llama.cpp-master/examples/sycl/win-build-sycl.bat 845B
  516. llama.cpp-master/examples/sycl/win-run-llama2.bat 330B
  517. llama.cpp-master/examples/tokenize/
  518. llama.cpp-master/examples/tokenize/CMakeLists.txt 235B
  519. llama.cpp-master/examples/tokenize/tokenize.cpp 13.4KB
  520. llama.cpp-master/examples/ts-type-to-grammar.sh 920B
  521. llama.cpp-master/flake.lock 1.52KB
  522. llama.cpp-master/flake.nix 7.18KB
  523. llama.cpp-master/ggml/
  524. llama.cpp-master/ggml/.gitignore 56B
  525. llama.cpp-master/ggml/CMakeLists.txt 9.36KB
  526. llama.cpp-master/ggml/cmake/
  527. llama.cpp-master/ggml/cmake/FindSIMD.cmake 2.59KB
  528. llama.cpp-master/ggml/include/
  529. llama.cpp-master/ggml/include/ggml-alloc.h 2.92KB
  530. llama.cpp-master/ggml/include/ggml-backend.h 13.38KB
  531. llama.cpp-master/ggml/include/ggml-blas.h 526B
  532. llama.cpp-master/ggml/include/ggml-cann.h 4.55KB
  533. llama.cpp-master/ggml/include/ggml-cuda.h 1.59KB
  534. llama.cpp-master/ggml/include/ggml-kompute.h 1KB
  535. llama.cpp-master/ggml/include/ggml-metal.h 2.25KB
  536. llama.cpp-master/ggml/include/ggml-rpc.h 673B
  537. llama.cpp-master/ggml/include/ggml-sycl.h 1.46KB
  538. llama.cpp-master/ggml/include/ggml-vulkan.h 946B
  539. llama.cpp-master/ggml/include/ggml.h 89.94KB
  540. llama.cpp-master/ggml/src/
  541. llama.cpp-master/ggml/src/CMakeLists.txt 50.03KB
  542. llama.cpp-master/ggml/src/ggml-aarch64.c 91.2KB
  543. llama.cpp-master/ggml/src/ggml-aarch64.h 1.96KB
  544. llama.cpp-master/ggml/src/ggml-alloc.c 37.49KB
  545. llama.cpp-master/ggml/src/ggml-backend-impl.h 7.43KB
  546. llama.cpp-master/ggml/src/ggml-backend.c 83.33KB
  547. llama.cpp-master/ggml/src/ggml-blas.cpp 12.15KB
  548. llama.cpp-master/ggml/src/ggml-cann.cpp 69.96KB
  549. llama.cpp-master/ggml/src/ggml-cann/
  550. llama.cpp-master/ggml/src/ggml-cann/.clang-format 4.42KB
  551. llama.cpp-master/ggml/src/ggml-cann/Doxyfile 109.97KB
  552. llama.cpp-master/ggml/src/ggml-cann/acl_tensor.cpp 6.92KB
  553. llama.cpp-master/ggml/src/ggml-cann/acl_tensor.h 12.17KB
  554. llama.cpp-master/ggml/src/ggml-cann/aclnn_ops.cpp 122.73KB
  555. llama.cpp-master/ggml/src/ggml-cann/aclnn_ops.h 25.25KB
  556. llama.cpp-master/ggml/src/ggml-cann/common.h 9.25KB
  557. llama.cpp-master/ggml/src/ggml-cann/kernels/
  558. llama.cpp-master/ggml/src/ggml-cann/kernels/CMakeLists.txt 1.03KB
  559. llama.cpp-master/ggml/src/ggml-cann/kernels/ascendc_kernels.h 693B
  560. llama.cpp-master/ggml/src/ggml-cann/kernels/dup.cpp 7.99KB
  561. llama.cpp-master/ggml/src/ggml-cann/kernels/get_row_f16.cpp 6.79KB
  562. llama.cpp-master/ggml/src/ggml-cann/kernels/get_row_f32.cpp 6.53KB
  563. llama.cpp-master/ggml/src/ggml-cann/kernels/get_row_q4_0.cpp 7.03KB
  564. llama.cpp-master/ggml/src/ggml-cann/kernels/get_row_q8_0.cpp 6.96KB
  565. llama.cpp-master/ggml/src/ggml-cann/kernels/quantize_f16_q8_0.cpp 7.37KB
  566. llama.cpp-master/ggml/src/ggml-cann/kernels/quantize_f32_q8_0.cpp 7.31KB
  567. llama.cpp-master/ggml/src/ggml-cann/kernels/quantize_float_to_q4_0.cpp 10.67KB
  568. llama.cpp-master/ggml/src/ggml-common.h 129.8KB
  569. llama.cpp-master/ggml/src/ggml-cuda.cu 120.79KB
  570. llama.cpp-master/ggml/src/ggml-cuda/
  571. llama.cpp-master/ggml/src/ggml-cuda/acc.cu 1.93KB
  572. llama.cpp-master/ggml/src/ggml-cuda/acc.cuh 131B
  573. llama.cpp-master/ggml/src/ggml-cuda/arange.cu 1.19KB
  574. llama.cpp-master/ggml/src/ggml-cuda/arange.cuh 137B
  575. llama.cpp-master/ggml/src/ggml-cuda/argsort.cu 3.35KB
  576. llama.cpp-master/ggml/src/ggml-cuda/argsort.cuh 102B
  577. llama.cpp-master/ggml/src/ggml-cuda/binbcast.cu 10.29KB
  578. llama.cpp-master/ggml/src/ggml-cuda/binbcast.cuh 326B
  579. llama.cpp-master/ggml/src/ggml-cuda/clamp.cu 1.14KB
  580. llama.cpp-master/ggml/src/ggml-cuda/clamp.cuh 135B
  581. llama.cpp-master/ggml/src/ggml-cuda/common.cuh 20.76KB
  582. llama.cpp-master/ggml/src/ggml-cuda/concat.cu 6.35KB
  583. llama.cpp-master/ggml/src/ggml-cuda/concat.cuh 137B
  584. llama.cpp-master/ggml/src/ggml-cuda/conv-transpose-1d.cu 3.25KB
  585. llama.cpp-master/ggml/src/ggml-cuda/conv-transpose-1d.cuh 158B
  586. llama.cpp-master/ggml/src/ggml-cuda/convert.cu 25.03KB
  587. llama.cpp-master/ggml/src/ggml-cuda/convert.cuh 391B
  588. llama.cpp-master/ggml/src/ggml-cuda/cpy.cu 19.95KB
  589. llama.cpp-master/ggml/src/ggml-cuda/cpy.cuh 298B
  590. llama.cpp-master/ggml/src/ggml-cuda/dequantize.cuh 2.59KB
  591. llama.cpp-master/ggml/src/ggml-cuda/diagmask.cu 1.72KB
  592. llama.cpp-master/ggml/src/ggml-cuda/diagmask.cuh 150B
  593. llama.cpp-master/ggml/src/ggml-cuda/dmmv.cu 27.48KB
  594. llama.cpp-master/ggml/src/ggml-cuda/dmmv.cuh 642B
  595. llama.cpp-master/ggml/src/ggml-cuda/fattn-common.cuh 23.67KB
  596. llama.cpp-master/ggml/src/ggml-cuda/fattn-tile-f16.cu 11.13KB
  597. llama.cpp-master/ggml/src/ggml-cuda/fattn-tile-f16.cuh 115B
  598. llama.cpp-master/ggml/src/ggml-cuda/fattn-tile-f32.cu 11.05KB
  599. llama.cpp-master/ggml/src/ggml-cuda/fattn-tile-f32.cuh 115B
  600. llama.cpp-master/ggml/src/ggml-cuda/fattn-vec-f16.cuh 14.64KB
  601. llama.cpp-master/ggml/src/ggml-cuda/fattn-vec-f32.cuh 13.71KB
  602. llama.cpp-master/ggml/src/ggml-cuda/fattn-wmma-f16.cuh 20.11KB
  603. llama.cpp-master/ggml/src/ggml-cuda/fattn.cu 13.84KB
  604. llama.cpp-master/ggml/src/ggml-cuda/fattn.cuh 106B
  605. llama.cpp-master/ggml/src/ggml-cuda/getrows.cu 6.83KB
  606. llama.cpp-master/ggml/src/ggml-cuda/getrows.cuh 141B
  607. llama.cpp-master/ggml/src/ggml-cuda/im2col.cu 4.45KB
  608. llama.cpp-master/ggml/src/ggml-cuda/im2col.cuh 137B
  609. llama.cpp-master/ggml/src/ggml-cuda/mma.cuh 7.41KB
  610. llama.cpp-master/ggml/src/ggml-cuda/mmq.cu 4.58KB
  611. llama.cpp-master/ggml/src/ggml-cuda/mmq.cuh 110.48KB
  612. llama.cpp-master/ggml/src/ggml-cuda/mmvq.cu 18.78KB
  613. llama.cpp-master/ggml/src/ggml-cuda/mmvq.cuh 481B
  614. llama.cpp-master/ggml/src/ggml-cuda/norm.cu 7.03KB
  615. llama.cpp-master/ggml/src/ggml-cuda/norm.cuh 263B
  616. llama.cpp-master/ggml/src/ggml-cuda/pad.cu 1.75KB
  617. llama.cpp-master/ggml/src/ggml-cuda/pad.cuh 131B
  618. llama.cpp-master/ggml/src/ggml-cuda/pool2d.cu 3.23KB
  619. llama.cpp-master/ggml/src/ggml-cuda/pool2d.cuh 137B
  620. llama.cpp-master/ggml/src/ggml-cuda/quantize.cu 5.34KB
  621. llama.cpp-master/ggml/src/ggml-cuda/quantize.cuh 979B
  622. llama.cpp-master/ggml/src/ggml-cuda/rope.cu 10.46KB
  623. llama.cpp-master/ggml/src/ggml-cuda/rope.cuh 133B
  624. llama.cpp-master/ggml/src/ggml-cuda/scale.cu 1021B
  625. llama.cpp-master/ggml/src/ggml-cuda/scale.cuh 135B
  626. llama.cpp-master/ggml/src/ggml-cuda/softmax.cu 7.54KB
  627. llama.cpp-master/ggml/src/ggml-cuda/softmax.cuh 142B
  628. llama.cpp-master/ggml/src/ggml-cuda/sumrows.cu 1.17KB
  629. llama.cpp-master/ggml/src/ggml-cuda/sumrows.cuh 103B
  630. llama.cpp-master/ggml/src/ggml-cuda/template-instances/
  631. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-f16-f16.cu 177B
  632. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-f16-q4_0.cu 178B
  633. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-f16-q4_1.cu 178B
  634. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-f16-q5_0.cu 178B
  635. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-f16-q5_1.cu 178B
  636. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-f16-q8_0.cu 178B
  637. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_0-f16.cu 178B
  638. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_0-q4_0.cu 179B
  639. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_0-q4_1.cu 179B
  640. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_0-q5_0.cu 179B
  641. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_0-q5_1.cu 179B
  642. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_0-q8_0.cu 179B
  643. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_1-f16.cu 178B
  644. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_1-q4_0.cu 179B
  645. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_1-q4_1.cu 179B
  646. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_1-q5_0.cu 179B
  647. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_1-q5_1.cu 179B
  648. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q4_1-q8_0.cu 179B
  649. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_0-f16.cu 178B
  650. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_0-q4_0.cu 179B
  651. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_0-q4_1.cu 179B
  652. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_0-q5_0.cu 179B
  653. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_0-q5_1.cu 179B
  654. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_0-q8_0.cu 179B
  655. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_1-f16.cu 178B
  656. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_1-q4_0.cu 179B
  657. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_1-q4_1.cu 179B
  658. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_1-q5_0.cu 179B
  659. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_1-q5_1.cu 179B
  660. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_1-q8_0.cu 179B
  661. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q8_0-f16.cu 178B
  662. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q8_0-q4_0.cu 179B
  663. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q8_0-q4_1.cu 179B
  664. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q8_0-q5_0.cu 179B
  665. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q8_0-q5_1.cu 179B
  666. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q8_0-q8_0.cu 179B
  667. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs256-f16-f16.cu 177B
  668. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs64-f16-f16.cu 176B
  669. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs64-f16-q4_0.cu 177B
  670. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs64-f16-q4_1.cu 177B
  671. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs64-f16-q5_0.cu 177B
  672. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs64-f16-q5_1.cu 177B
  673. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f16-instance-hs64-f16-q8_0.cu 177B
  674. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-f16-f16.cu 177B
  675. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-f16-q4_0.cu 178B
  676. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-f16-q4_1.cu 178B
  677. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-f16-q5_0.cu 178B
  678. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-f16-q5_1.cu 178B
  679. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-f16-q8_0.cu 178B
  680. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_0-f16.cu 178B
  681. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_0-q4_0.cu 179B
  682. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_0-q4_1.cu 179B
  683. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_0-q5_0.cu 179B
  684. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_0-q5_1.cu 179B
  685. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_0-q8_0.cu 179B
  686. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_1-f16.cu 178B
  687. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_1-q4_0.cu 179B
  688. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_1-q4_1.cu 179B
  689. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_1-q5_0.cu 179B
  690. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_1-q5_1.cu 179B
  691. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q4_1-q8_0.cu 179B
  692. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_0-f16.cu 178B
  693. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_0-q4_0.cu 179B
  694. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_0-q4_1.cu 179B
  695. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_0-q5_0.cu 179B
  696. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_0-q5_1.cu 179B
  697. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_0-q8_0.cu 179B
  698. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_1-f16.cu 178B
  699. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_1-q4_0.cu 179B
  700. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_1-q4_1.cu 179B
  701. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_1-q5_0.cu 179B
  702. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_1-q5_1.cu 179B
  703. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q5_1-q8_0.cu 179B
  704. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q8_0-f16.cu 178B
  705. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q8_0-q4_0.cu 179B
  706. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q8_0-q4_1.cu 179B
  707. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q8_0-q5_0.cu 179B
  708. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q8_0-q5_1.cu 179B
  709. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs128-q8_0-q8_0.cu 179B
  710. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs256-f16-f16.cu 177B
  711. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs64-f16-f16.cu 176B
  712. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs64-f16-q4_0.cu 177B
  713. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs64-f16-q4_1.cu 177B
  714. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs64-f16-q5_0.cu 177B
  715. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs64-f16-q5_1.cu 177B
  716. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-vec-f32-instance-hs64-f16-q8_0.cu 177B
  717. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-wmma-f16-instance-kqfloat-cpb16.cu 367B
  718. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-wmma-f16-instance-kqfloat-cpb32.cu 325B
  719. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-wmma-f16-instance-kqhalf-cpb16.cu 361B
  720. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-wmma-f16-instance-kqhalf-cpb32.cu 361B
  721. llama.cpp-master/ggml/src/ggml-cuda/template-instances/fattn-wmma-f16-instance-kqhalf-cpb8.cu 276B
  722. llama.cpp-master/ggml/src/ggml-cuda/template-instances/generate_cu_files.py 2.76KB
  723. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-iq1_s.cu 139B
  724. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-iq2_s.cu 139B
  725. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-iq2_xs.cu 140B
  726. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-iq2_xxs.cu 141B
  727. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-iq3_s.cu 139B
  728. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-iq3_xxs.cu 141B
  729. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-iq4_nl.cu 140B
  730. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-iq4_xs.cu 140B
  731. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-q2_k.cu 138B
  732. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-q3_k.cu 138B
  733. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-q4_0.cu 138B
  734. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-q4_1.cu 138B
  735. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-q4_k.cu 138B
  736. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-q5_0.cu 138B
  737. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-q5_1.cu 138B
  738. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-q5_k.cu 138B
  739. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-q6_k.cu 138B
  740. llama.cpp-master/ggml/src/ggml-cuda/template-instances/mmq-instance-q8_0.cu 138B
  741. llama.cpp-master/ggml/src/ggml-cuda/tsembd.cu 1.76KB
  742. llama.cpp-master/ggml/src/ggml-cuda/tsembd.cuh 161B
  743. llama.cpp-master/ggml/src/ggml-cuda/unary.cu 10.58KB
  744. llama.cpp-master/ggml/src/ggml-cuda/unary.cuh 1.18KB
  745. llama.cpp-master/ggml/src/ggml-cuda/upscale.cu 2.07KB
  746. llama.cpp-master/ggml/src/ggml-cuda/upscale.cuh 139B
  747. llama.cpp-master/ggml/src/ggml-cuda/vecdotq.cuh 38.2KB
  748. llama.cpp-master/ggml/src/ggml-cuda/vendors/
  749. llama.cpp-master/ggml/src/ggml-cuda/vendors/cuda.h 462B
  750. llama.cpp-master/ggml/src/ggml-cuda/vendors/hip.h 7.1KB
  751. llama.cpp-master/ggml/src/ggml-cuda/vendors/musa.h 7.04KB
  752. llama.cpp-master/ggml/src/ggml-impl.h 20.06KB
  753. llama.cpp-master/ggml/src/ggml-kompute.cpp 79.01KB
  754. llama.cpp-master/ggml/src/ggml-metal.m 187.1KB
  755. llama.cpp-master/ggml/src/ggml-metal.metal 221.39KB
  756. llama.cpp-master/ggml/src/ggml-quants.c 637.5KB
  757. llama.cpp-master/ggml/src/ggml-quants.h 11.48KB
  758. llama.cpp-master/ggml/src/ggml-rpc.cpp 43.35KB
  759. llama.cpp-master/ggml/src/ggml-sycl.cpp 209.44KB
  760. llama.cpp-master/ggml/src/ggml-sycl/
  761. llama.cpp-master/ggml/src/ggml-sycl/backend.hpp 653B
  762. llama.cpp-master/ggml/src/ggml-sycl/common.cpp 1.49KB
  763. llama.cpp-master/ggml/src/ggml-sycl/common.hpp 10.68KB
  764. llama.cpp-master/ggml/src/ggml-sycl/concat.cpp 7.39KB
  765. llama.cpp-master/ggml/src/ggml-sycl/concat.hpp 575B
  766. llama.cpp-master/ggml/src/ggml-sycl/conv.cpp 3.16KB
  767. llama.cpp-master/ggml/src/ggml-sycl/conv.hpp 550B
  768. llama.cpp-master/ggml/src/ggml-sycl/convert.cpp 21.31KB
  769. llama.cpp-master/ggml/src/ggml-sycl/convert.hpp 778B
  770. llama.cpp-master/ggml/src/ggml-sycl/dequantize.hpp 22.99KB
  771. llama.cpp-master/ggml/src/ggml-sycl/dmmv.cpp 40.63KB
  772. llama.cpp-master/ggml/src/ggml-sycl/dmmv.hpp 808B
  773. llama.cpp-master/ggml/src/ggml-sycl/dpct/
  774. llama.cpp-master/ggml/src/ggml-sycl/dpct/helper.hpp 120.39KB
  775. llama.cpp-master/ggml/src/ggml-sycl/mmq.cpp 116.7KB
  776. llama.cpp-master/ggml/src/ggml-sycl/mmq.hpp 819B
  777. llama.cpp-master/ggml/src/ggml-sycl/mmvq.cpp 38.85KB
  778. llama.cpp-master/ggml/src/ggml-sycl/mmvq.hpp 799B
  779. llama.cpp-master/ggml/src/ggml-sycl/norm.cpp 13.23KB
  780. llama.cpp-master/ggml/src/ggml-sycl/norm.hpp 1.08KB
  781. llama.cpp-master/ggml/src/ggml-sycl/presets.hpp 1.97KB
  782. llama.cpp-master/ggml/src/ggml-sycl/rope.cpp 10.38KB
  783. llama.cpp-master/ggml/src/ggml-sycl/rope.hpp 633B
  784. llama.cpp-master/ggml/src/ggml-sycl/softmax.cpp 10.88KB
  785. llama.cpp-master/ggml/src/ggml-sycl/softmax.hpp 652B
  786. llama.cpp-master/ggml/src/ggml-sycl/tsembd.cpp 2.53KB
  787. llama.cpp-master/ggml/src/ggml-sycl/tsembd.hpp 560B
  788. llama.cpp-master/ggml/src/ggml-sycl/vecdotq.hpp 38.82KB
  789. llama.cpp-master/ggml/src/ggml-vulkan.cpp 405.37KB
  790. llama.cpp-master/ggml/src/ggml.c 713.38KB
  791. llama.cpp-master/ggml/src/kompute/
  792. llama.cpp-master/ggml/src/kompute-shaders/
  793. llama.cpp-master/ggml/src/kompute-shaders/common.comp 3.53KB
  794. llama.cpp-master/ggml/src/kompute-shaders/op_add.comp 1.61KB
  795. llama.cpp-master/ggml/src/kompute-shaders/op_addrow.comp 640B
  796. llama.cpp-master/ggml/src/kompute-shaders/op_cpy_f16_f16.comp 1.5KB
  797. llama.cpp-master/ggml/src/kompute-shaders/op_cpy_f16_f32.comp 1.49KB
  798. llama.cpp-master/ggml/src/kompute-shaders/op_cpy_f32_f16.comp 1.49KB
  799. llama.cpp-master/ggml/src/kompute-shaders/op_cpy_f32_f32.comp 1.49KB
  800. llama.cpp-master/ggml/src/kompute-shaders/op_diagmask.comp 726B
  801. llama.cpp-master/ggml/src/kompute-shaders/op_gelu.comp 604B
  802. llama.cpp-master/ggml/src/kompute-shaders/op_getrows.comp 609B
  803. llama.cpp-master/ggml/src/kompute-shaders/op_getrows_f16.comp 787B
  804. llama.cpp-master/ggml/src/kompute-shaders/op_getrows_f32.comp 762B
  805. llama.cpp-master/ggml/src/kompute-shaders/op_getrows_q4_0.comp 919B
  806. llama.cpp-master/ggml/src/kompute-shaders/op_getrows_q4_1.comp 962B
  807. llama.cpp-master/ggml/src/kompute-shaders/op_getrows_q6_k.comp 1.16KB
  808. llama.cpp-master/ggml/src/kompute-shaders/op_mul.comp 1.33KB
  809. llama.cpp-master/ggml/src/kompute-shaders/op_mul_mat_f16.comp 1.59KB
  810. llama.cpp-master/ggml/src/kompute-shaders/op_mul_mat_mat_f32.comp 1.27KB
  811. llama.cpp-master/ggml/src/kompute-shaders/op_mul_mat_q4_0.comp 1018B
  812. llama.cpp-master/ggml/src/kompute-shaders/op_mul_mat_q4_1.comp 1.04KB
  813. llama.cpp-master/ggml/src/kompute-shaders/op_mul_mat_q6_k.comp 3.54KB
  814. llama.cpp-master/ggml/src/kompute-shaders/op_mul_mat_q8_0.comp 2.19KB
  815. llama.cpp-master/ggml/src/kompute-shaders/op_mul_mv_q_n.comp 1.76KB
  816. llama.cpp-master/ggml/src/kompute-shaders/op_mul_mv_q_n_pre.comp 521B
  817. llama.cpp-master/ggml/src/kompute-shaders/op_norm.comp 2.25KB
  818. llama.cpp-master/ggml/src/kompute-shaders/op_relu.comp 508B
  819. llama.cpp-master/ggml/src/kompute-shaders/op_rmsnorm.comp 1.38KB
  820. llama.cpp-master/ggml/src/kompute-shaders/op_rope_f16.comp 2.84KB
  821. llama.cpp-master/ggml/src/kompute-shaders/op_rope_f32.comp 2.74KB
  822. llama.cpp-master/ggml/src/kompute-shaders/op_scale.comp 432B
  823. llama.cpp-master/ggml/src/kompute-shaders/op_scale_8.comp 528B
  824. llama.cpp-master/ggml/src/kompute-shaders/op_silu.comp 543B
  825. llama.cpp-master/ggml/src/kompute-shaders/op_softmax.comp 1.75KB
  826. llama.cpp-master/ggml/src/kompute-shaders/rope_common.comp 2.25KB
  827. llama.cpp-master/ggml/src/llamafile/
  828. llama.cpp-master/ggml/src/llamafile/sgemm.cpp 30.9KB
  829. llama.cpp-master/ggml/src/llamafile/sgemm.h 302B
  830. llama.cpp-master/ggml/src/vulkan-shaders/
  831. llama.cpp-master/ggml/src/vulkan-shaders/CMakeLists.txt 268B
  832. llama.cpp-master/ggml/src/vulkan-shaders/add.comp 287B
  833. llama.cpp-master/ggml/src/vulkan-shaders/argsort.comp 1.96KB
  834. llama.cpp-master/ggml/src/vulkan-shaders/clamp.comp 340B
  835. llama.cpp-master/ggml/src/vulkan-shaders/concat.comp 1.25KB
  836. llama.cpp-master/ggml/src/vulkan-shaders/copy.comp 352B
  837. llama.cpp-master/ggml/src/vulkan-shaders/dequant_f32.comp 442B
  838. llama.cpp-master/ggml/src/vulkan-shaders/dequant_funcs.comp 2.33KB
  839. llama.cpp-master/ggml/src/vulkan-shaders/dequant_head.comp 249B
  840. llama.cpp-master/ggml/src/vulkan-shaders/dequant_iq4_nl.comp 871B
  841. llama.cpp-master/ggml/src/vulkan-shaders/dequant_q2_k.comp 1.44KB
  842. llama.cpp-master/ggml/src/vulkan-shaders/dequant_q3_k.comp 1.68KB
  843. llama.cpp-master/ggml/src/vulkan-shaders/dequant_q4_0.comp 861B
  844. llama.cpp-master/ggml/src/vulkan-shaders/dequant_q4_1.comp 892B
  845. llama.cpp-master/ggml/src/vulkan-shaders/dequant_q4_k.comp 1.97KB
  846. llama.cpp-master/ggml/src/vulkan-shaders/dequant_q5_0.comp 1.02KB
  847. llama.cpp-master/ggml/src/vulkan-shaders/dequant_q5_1.comp 1.02KB
  848. llama.cpp-master/ggml/src/vulkan-shaders/dequant_q5_k.comp 2.41KB
  849. llama.cpp-master/ggml/src/vulkan-shaders/dequant_q6_k.comp 1.39KB
  850. llama.cpp-master/ggml/src/vulkan-shaders/dequant_q8_0.comp 839B
  851. llama.cpp-master/ggml/src/vulkan-shaders/diag_mask_inf.comp 799B
  852. llama.cpp-master/ggml/src/vulkan-shaders/div.comp 287B
  853. llama.cpp-master/ggml/src/vulkan-shaders/gelu.comp 767B
  854. llama.cpp-master/ggml/src/vulkan-shaders/gelu_quick.comp 631B
  855. llama.cpp-master/ggml/src/vulkan-shaders/generic_binary_head.comp 2.11KB
  856. llama.cpp-master/ggml/src/vulkan-shaders/generic_head.comp 160B
  857. llama.cpp-master/ggml/src/vulkan-shaders/generic_unary_head.comp 1.48KB
  858. llama.cpp-master/ggml/src/vulkan-shaders/get_rows.comp 702B
  859. llama.cpp-master/ggml/src/vulkan-shaders/get_rows_quant.comp 970B
  860. llama.cpp-master/ggml/src/vulkan-shaders/group_norm.comp 1.68KB
  861. llama.cpp-master/ggml/src/vulkan-shaders/im2col.comp 1.49KB
  862. llama.cpp-master/ggml/src/vulkan-shaders/leaky_relu.comp 586B
  863. llama.cpp-master/ggml/src/vulkan-shaders/mul.comp 287B
  864. llama.cpp-master/ggml/src/vulkan-shaders/mul_mat_split_k_reduce.comp 592B
  865. llama.cpp-master/ggml/src/vulkan-shaders/mul_mat_vec.comp 1.62KB
  866. llama.cpp-master/ggml/src/vulkan-shaders/mul_mat_vec_base.comp 1.75KB
  867. llama.cpp-master/ggml/src/vulkan-shaders/mul_mat_vec_nc.comp 1.76KB
  868. llama.cpp-master/ggml/src/vulkan-shaders/mul_mat_vec_p021.comp 1.86KB
  869. llama.cpp-master/ggml/src/vulkan-shaders/mul_mat_vec_q2_k.comp 4.33KB
  870. llama.cpp-master/ggml/src/vulkan-shaders/mul_mat_vec_q3_k.comp 4.36KB
  871. llama.cpp-master/ggml/src/vulkan-shaders/mul_mat_vec_q4_k.comp 8.02KB
  872. llama.cpp-master/ggml/src/vulkan-shaders/mul_mat_vec_q5_k.comp 6.92KB
  873. llama.cpp-master/ggml/src/vulkan-shaders/mul_mat_vec_q6_k.comp 4.91KB
  874. llama.cpp-master/ggml/src/vulkan-shaders/mul_mm.comp 21.02KB
  875. llama.cpp-master/ggml/src/vulkan-shaders/norm.comp 1.25KB
  876. llama.cpp-master/ggml/src/vulkan-shaders/pad.comp 884B
  877. llama.cpp-master/ggml/src/vulkan-shaders/relu.comp 520B
  878. llama.cpp-master/ggml/src/vulkan-shaders/rms_norm.comp 1.25KB
  879. llama.cpp-master/ggml/src/vulkan-shaders/rope_head.comp 1.47KB
  880. llama.cpp-master/ggml/src/vulkan-shaders/rope_neox.comp 929B
  881. llama.cpp-master/ggml/src/vulkan-shaders/rope_norm.comp 901B
  882. llama.cpp-master/ggml/src/vulkan-shaders/scale.comp 273B
  883. llama.cpp-master/ggml/src/vulkan-shaders/silu.comp 565B
  884. llama.cpp-master/ggml/src/vulkan-shaders/soft_max.comp 2.61KB
  885. llama.cpp-master/ggml/src/vulkan-shaders/square.comp 288B
  886. llama.cpp-master/ggml/src/vulkan-shaders/sum_rows.comp 940B
  887. llama.cpp-master/ggml/src/vulkan-shaders/tanh.comp 519B
  888. llama.cpp-master/ggml/src/vulkan-shaders/timestep_embedding.comp 1KB
  889. llama.cpp-master/ggml/src/vulkan-shaders/types.comp 3.44KB
  890. llama.cpp-master/ggml/src/vulkan-shaders/upscale.comp 1.07KB
  891. llama.cpp-master/ggml/src/vulkan-shaders/vulkan-shaders-gen.cpp 22.18KB
  892. llama.cpp-master/gguf-py/
  893. llama.cpp-master/gguf-py/LICENSE 1.05KB
  894. llama.cpp-master/gguf-py/README.md 2.66KB
  895. llama.cpp-master/gguf-py/examples/
  896. llama.cpp-master/gguf-py/examples/reader.py 1.54KB
  897. llama.cpp-master/gguf-py/examples/writer.py 1.09KB
  898. llama.cpp-master/gguf-py/gguf/
  899. llama.cpp-master/gguf-py/gguf/__init__.py 219B
  900. llama.cpp-master/gguf-py/gguf/constants.py 45.91KB
  901. llama.cpp-master/gguf-py/gguf/gguf.py 478B
  902. llama.cpp-master/gguf-py/gguf/gguf_reader.py 12.08KB
  903. llama.cpp-master/gguf-py/gguf/gguf_writer.py 34.14KB
  904. llama.cpp-master/gguf-py/gguf/lazy.py 8.33KB
  905. llama.cpp-master/gguf-py/gguf/metadata.py 25.12KB
  906. llama.cpp-master/gguf-py/gguf/py.typed
  907. llama.cpp-master/gguf-py/gguf/quants.py 4.21KB
  908. llama.cpp-master/gguf-py/gguf/tensor_mapping.py 29.44KB
  909. llama.cpp-master/gguf-py/gguf/utility.py 2.87KB
  910. llama.cpp-master/gguf-py/gguf/vocab.py 18.6KB
  911. llama.cpp-master/gguf-py/pyproject.toml 1013B
  912. llama.cpp-master/gguf-py/scripts/
  913. llama.cpp-master/gguf-py/scripts/__init__.py 297B
  914. llama.cpp-master/gguf-py/scripts/gguf_convert_endian.py 5.16KB
  915. llama.cpp-master/gguf-py/scripts/gguf_dump.py 21.42KB
  916. llama.cpp-master/gguf-py/scripts/gguf_hash.py 3.62KB
  917. llama.cpp-master/gguf-py/scripts/gguf_new_metadata.py 10.46KB
  918. llama.cpp-master/gguf-py/scripts/gguf_set_metadata.py 4.03KB
  919. llama.cpp-master/gguf-py/tests/
  920. llama.cpp-master/gguf-py/tests/__init__.py 29B
  921. llama.cpp-master/gguf-py/tests/test_metadata.py 12.46KB
  922. llama.cpp-master/grammars/
  923. llama.cpp-master/grammars/README.md 16.44KB
  924. llama.cpp-master/grammars/arithmetic.gbnf 177B
  925. llama.cpp-master/grammars/c.gbnf 1.35KB
  926. llama.cpp-master/grammars/chess.gbnf 565B
  927. llama.cpp-master/grammars/japanese.gbnf 249B
  928. llama.cpp-master/grammars/json.gbnf 601B
  929. llama.cpp-master/grammars/json_arr.gbnf 796B
  930. llama.cpp-master/grammars/list.gbnf 109B
  931. llama.cpp-master/include/
  932. llama.cpp-master/include/llama.h 56.21KB
  933. llama.cpp-master/media/
  934. llama.cpp-master/media/llama-leader.jpeg 195.26KB
  935. llama.cpp-master/media/llama0-banner.png 141.23KB
  936. llama.cpp-master/media/llama0-logo.png 175.72KB
  937. llama.cpp-master/media/llama1-banner.png 32.55KB
  938. llama.cpp-master/media/llama1-logo.png 31.73KB
  939. llama.cpp-master/media/matmul.png 259.48KB
  940. llama.cpp-master/media/matmul.svg 51.38KB
  941. llama.cpp-master/models/
  942. llama.cpp-master/models/.editorconfig 12B
  943. llama.cpp-master/models/ggml-vocab-aquila.gguf 4.6MB
  944. llama.cpp-master/models/ggml-vocab-baichuan.gguf 1.28MB
  945. llama.cpp-master/models/ggml-vocab-bert-bge.gguf 612.84KB
  946. llama.cpp-master/models/ggml-vocab-bert-bge.gguf.inp 1.9KB
  947. llama.cpp-master/models/ggml-vocab-bert-bge.gguf.out 1.59KB
  948. llama.cpp-master/models/ggml-vocab-command-r.gguf 10.37MB
  949. llama.cpp-master/models/ggml-vocab-command-r.gguf.inp 1.9KB
  950. llama.cpp-master/models/ggml-vocab-command-r.gguf.out 1.86KB
  951. llama.cpp-master/models/ggml-vocab-deepseek-coder.gguf 1.1MB
  952. llama.cpp-master/models/ggml-vocab-deepseek-coder.gguf.inp 1.9KB
  953. llama.cpp-master/models/ggml-vocab-deepseek-coder.gguf.out 2.05KB
  954. llama.cpp-master/models/ggml-vocab-deepseek-llm.gguf 3.79MB
  955. llama.cpp-master/models/ggml-vocab-deepseek-llm.gguf.inp 1.9KB
  956. llama.cpp-master/models/ggml-vocab-deepseek-llm.gguf.out 1.88KB
  957. llama.cpp-master/models/ggml-vocab-falcon.gguf 2.18MB
  958. llama.cpp-master/models/ggml-vocab-falcon.gguf.inp 1.9KB
  959. llama.cpp-master/models/ggml-vocab-falcon.gguf.out 1.96KB
  960. llama.cpp-master/models/ggml-vocab-gpt-2.gguf 1.68MB
  961. llama.cpp-master/models/ggml-vocab-gpt-2.gguf.inp 1.9KB
  962. llama.cpp-master/models/ggml-vocab-gpt-2.gguf.out 2.1KB
  963. llama.cpp-master/models/ggml-vocab-gpt-neox.gguf 1.69MB
  964. llama.cpp-master/models/ggml-vocab-llama-bpe.gguf 7.46MB
  965. llama.cpp-master/models/ggml-vocab-llama-bpe.gguf.inp 1.9KB
  966. llama.cpp-master/models/ggml-vocab-llama-bpe.gguf.out 1.7KB
  967. llama.cpp-master/models/ggml-vocab-llama-spm.gguf 706.9KB
  968. llama.cpp-master/models/ggml-vocab-llama-spm.gguf.inp 1.9KB
  969. llama.cpp-master/models/ggml-vocab-llama-spm.gguf.out 2.62KB
  970. llama.cpp-master/models/ggml-vocab-mpt.gguf 1.69MB
  971. llama.cpp-master/models/ggml-vocab-mpt.gguf.inp 1.9KB
  972. llama.cpp-master/models/ggml-vocab-mpt.gguf.out 1.85KB
  973. llama.cpp-master/models/ggml-vocab-phi-3.gguf 709KB
  974. llama.cpp-master/models/ggml-vocab-phi-3.gguf.inp 1.9KB
  975. llama.cpp-master/models/ggml-vocab-phi-3.gguf.out 2.62KB
  976. llama.cpp-master/models/ggml-vocab-qwen2.gguf 5.65MB
  977. llama.cpp-master/models/ggml-vocab-qwen2.gguf.inp 1.9KB
  978. llama.cpp-master/models/ggml-vocab-qwen2.gguf.out 1.72KB
  979. llama.cpp-master/models/ggml-vocab-refact.gguf 1.64MB
  980. llama.cpp-master/models/ggml-vocab-refact.gguf.inp 1.9KB
  981. llama.cpp-master/models/ggml-vocab-refact.gguf.out 1.87KB
  982. llama.cpp-master/models/ggml-vocab-starcoder.gguf 1.64MB
  983. llama.cpp-master/models/ggml-vocab-starcoder.gguf.inp 1.9KB
  984. llama.cpp-master/models/ggml-vocab-starcoder.gguf.out 1.88KB
  985. llama.cpp-master/mypy.ini 163B
  986. llama.cpp-master/pocs/
  987. llama.cpp-master/pocs/CMakeLists.txt 171B
  988. llama.cpp-master/pocs/vdot/
  989. llama.cpp-master/pocs/vdot/CMakeLists.txt 387B
  990. llama.cpp-master/pocs/vdot/q8dot.cpp 5.23KB
  991. llama.cpp-master/pocs/vdot/vdot.cpp 13.18KB
  992. llama.cpp-master/poetry.lock 121.86KB
  993. llama.cpp-master/prompts/
  994. llama.cpp-master/prompts/LLM-questions.txt 2.54KB
  995. llama.cpp-master/prompts/alpaca.txt 106B
  996. llama.cpp-master/prompts/assistant.txt 2.29KB
  997. llama.cpp-master/prompts/chat-with-baichuan.txt 90B
  998. llama.cpp-master/prompts/chat-with-bob.txt 386B
  999. llama.cpp-master/prompts/chat-with-qwen.txt 28B
  1000. llama.cpp-master/prompts/chat-with-vicuna-v0.txt 446B
  1001. llama.cpp-master/prompts/chat-with-vicuna-v1.txt 426B
  1002. llama.cpp-master/prompts/chat.txt 1.79KB
  1003. llama.cpp-master/prompts/dan-modified.txt 1.5KB
  1004. llama.cpp-master/prompts/dan.txt 1.62KB
  1005. llama.cpp-master/prompts/mnemonics.txt 4.97KB
  1006. llama.cpp-master/prompts/parallel-questions.txt 1.68KB
  1007. llama.cpp-master/prompts/reason-act.txt 758B
  1008. llama.cpp-master/pyproject.toml 1.25KB
  1009. llama.cpp-master/pyrightconfig.json 528B
  1010. llama.cpp-master/requirements.txt 505B
  1011. llama.cpp-master/requirements/
  1012. llama.cpp-master/requirements/requirements-all.txt 428B
  1013. llama.cpp-master/requirements/requirements-compare-llama-bench.txt 34B
  1014. llama.cpp-master/requirements/requirements-convert_hf_to_gguf.txt 111B
  1015. llama.cpp-master/requirements/requirements-convert_hf_to_gguf_update.txt 111B
  1016. llama.cpp-master/requirements/requirements-convert_legacy_llama.txt 99B
  1017. llama.cpp-master/requirements/requirements-convert_llama_ggml_to_gguf.txt 43B
  1018. llama.cpp-master/requirements/requirements-convert_lora_to_gguf.txt 96B
  1019. llama.cpp-master/requirements/requirements-pydantic.txt 48B
  1020. llama.cpp-master/requirements/requirements-test-tokenizer-random.txt 13B
  1021. llama.cpp-master/scripts/
  1022. llama.cpp-master/scripts/build-info.sh 717B
  1023. llama.cpp-master/scripts/check-requirements.sh 4.34KB
  1024. llama.cpp-master/scripts/ci-run.sh 1.28KB
  1025. llama.cpp-master/scripts/compare-commits.sh 749B
  1026. llama.cpp-master/scripts/compare-llama-bench.py 14.3KB
  1027. llama.cpp-master/scripts/debug-test.sh 5.01KB
  1028. llama.cpp-master/scripts/gen-authors.sh 337B
  1029. llama.cpp-master/scripts/gen-unicode-data.py 6.28KB
  1030. llama.cpp-master/scripts/get-flags.mk 1.27KB
  1031. llama.cpp-master/scripts/get-hellaswag.sh 263B
  1032. llama.cpp-master/scripts/get-pg.sh 1.36KB
  1033. llama.cpp-master/scripts/get-wikitext-103.sh 210B
  1034. llama.cpp-master/scripts/get-wikitext-2.sh 253B
  1035. llama.cpp-master/scripts/get-winogrande.sh 292B
  1036. llama.cpp-master/scripts/hf.sh 2.26KB
  1037. llama.cpp-master/scripts/install-oneapi.bat 802B
  1038. llama.cpp-master/scripts/pod-llama.sh 8.17KB
  1039. llama.cpp-master/scripts/qnt-all.sh 558B
  1040. llama.cpp-master/scripts/run-all-perf.sh 549B
  1041. llama.cpp-master/scripts/run-all-ppl.sh 554B
  1042. llama.cpp-master/scripts/run-with-preset.py 5.47KB
  1043. llama.cpp-master/scripts/server-llm.sh 11.22KB
  1044. llama.cpp-master/scripts/sync-ggml-am.sh 7.89KB
  1045. llama.cpp-master/scripts/sync-ggml.last 41B
  1046. llama.cpp-master/scripts/sync-ggml.sh 2.58KB
  1047. llama.cpp-master/scripts/verify-checksum-models.py 2.42KB
  1048. llama.cpp-master/scripts/xxd.cmake 647B
  1049. llama.cpp-master/spm-headers/
  1050. llama.cpp-master/spm-headers/ggml-alloc.h 28B
  1051. llama.cpp-master/spm-headers/ggml-backend.h 30B
  1052. llama.cpp-master/spm-headers/ggml-metal.h 28B
  1053. llama.cpp-master/spm-headers/ggml.h 22B
  1054. llama.cpp-master/spm-headers/llama.h 18B
  1055. llama.cpp-master/src/
  1056. llama.cpp-master/src/CMakeLists.txt 749B
  1057. llama.cpp-master/src/llama-grammar.cpp 19.4KB
  1058. llama.cpp-master/src/llama-grammar.h 1.09KB
  1059. llama.cpp-master/src/llama-impl.h 795B
  1060. llama.cpp-master/src/llama-sampling.cpp 22.09KB
  1061. llama.cpp-master/src/llama-sampling.h 2.63KB
  1062. llama.cpp-master/src/llama-vocab.cpp 66.57KB
  1063. llama.cpp-master/src/llama-vocab.h 4.64KB
  1064. llama.cpp-master/src/llama.cpp 789.47KB
  1065. llama.cpp-master/src/unicode-data.cpp 164.26KB
  1066. llama.cpp-master/src/unicode-data.h 582B
  1067. llama.cpp-master/src/unicode.cpp 29.9KB
  1068. llama.cpp-master/src/unicode.h 2.14KB
  1069. llama.cpp-master/tests/
  1070. llama.cpp-master/tests/.gitignore 25B
  1071. llama.cpp-master/tests/CMakeLists.txt 7.07KB
  1072. llama.cpp-master/tests/get-model.cpp 594B
  1073. llama.cpp-master/tests/get-model.h 53B
  1074. llama.cpp-master/tests/run-json-schema-to-grammar.mjs 395B
  1075. llama.cpp-master/tests/test-autorelease.cpp 719B
  1076. llama.cpp-master/tests/test-backend-ops.cpp 91.17KB
  1077. llama.cpp-master/tests/test-c.c 96B
  1078. llama.cpp-master/tests/test-chat-template.cpp 19.85KB
  1079. llama.cpp-master/tests/test-double-float.cpp 1.79KB
  1080. llama.cpp-master/tests/test-grad0.cpp 52.65KB
  1081. llama.cpp-master/tests/test-grammar-integration.cpp 35.63KB
  1082. llama.cpp-master/tests/test-grammar-parser.cpp 16.37KB
  1083. llama.cpp-master/tests/test-json-schema-to-grammar.cpp 38.9KB
  1084. llama.cpp-master/tests/test-llama-grammar.cpp 11.04KB
  1085. llama.cpp-master/tests/test-model-load-cancel.cpp 763B
  1086. llama.cpp-master/tests/test-opt.cpp 5.04KB
  1087. llama.cpp-master/tests/test-quantize-fns.cpp 6.54KB
  1088. llama.cpp-master/tests/test-quantize-perf.cpp 13.69KB
  1089. llama.cpp-master/tests/test-rope.cpp 6.12KB
  1090. llama.cpp-master/tests/test-sampling.cpp 13.34KB
  1091. llama.cpp-master/tests/test-tokenizer-0.cpp 10.46KB
  1092. llama.cpp-master/tests/test-tokenizer-0.py 1.92KB
  1093. llama.cpp-master/tests/test-tokenizer-0.sh 921B
  1094. llama.cpp-master/tests/test-tokenizer-1-bpe.cpp 4.68KB
  1095. llama.cpp-master/tests/test-tokenizer-1-spm.cpp 3.48KB
  1096. llama.cpp-master/tests/test-tokenizer-random.py 21.46KB

资源介绍:

描述 主要目标llama.cpp是在各种硬件（本地和云端）上以最少的设置和最先进的性能实现 LLM 推理。 纯 C/C++ 实现，无任何依赖项 Apple 芯片是一流的——通过 ARM NEON、Accelerate 和 Metal 框架进行了优化 AVX、AVX2 和 AVX512 支持 x86 架构 1.5 位、2 位、3 位、4 位、5 位、6 位和 8 位整数量化，可加快推理速度并减少内存使用 用于在 NVIDIA GPU 上运行 LLM 的自定义 CUDA 内核（通过 HIP 支持 AMD GPU） Vulkan 和 SYCL 后端支持 CPU+GPU 混合推理，部分加速大于 VRAM 总容量的模型
# LLaMA.cpp HTTP Server Fast, lightweight, pure C/C++ HTTP server based on [httplib](https://github.com/yhirose/cpp-httplib), [nlohmann::json](https://github.com/nlohmann/json) and **llama.cpp**. Set of LLM REST APIs and a simple web front end to interact with llama.cpp. **Features:** * LLM inference of F16 and quantized models on GPU and CPU * [OpenAI API](https://github.com/openai/openai-openapi) compatible chat completions and embeddings routes * Parallel decoding with multi-user support * Continuous batching * Multimodal (wip) * Monitoring endpoints * Schema-constrained JSON response format The project is under active development, and we are [looking for feedback and contributors](https://github.com/ggerganov/llama.cpp/issues/4216). ## Usage ``` usage: ./llama-server [options] general: -h, --help, --usage print usage and exit --version show version and build info -v, --verbose print verbose information --verbosity N set specific verbosity level (default: 0) --verbose-prompt print a verbose prompt before generation (default: false) --no-display-prompt don't print prompt at generation (default: false) -co, --color colorise output to distinguish prompt and user input from generations (default: false) -s, --seed SEED RNG seed (default: -1, use random seed for < 0) -t, --threads N number of threads to use during generation (default: 8) -tb, --threads-batch N number of threads to use during batch and prompt processing (default: same as --threads) -td, --threads-draft N number of threads to use during generation (default: same as --threads) -tbd, --threads-batch-draft N number of threads to use during batch and prompt processing (default: same as --threads-draft) --draft N number of tokens to draft for speculative decoding (default: 5) -ps, --p-split N speculative decoding split probability (default: 0.1) -lcs, --lookup-cache-static FNAME path to static lookup cache to use for lookup decoding (not updated by generation) -lcd, --lookup-cache-dynamic FNAME path to dynamic lookup cache to use for lookup decoding (updated by generation) -c, --ctx-size N size of the prompt context (default: 0, 0 = loaded from model) -n, --predict N number of tokens to predict (default: -1, -1 = infinity, -2 = until context filled) -b, --batch-size N logical maximum batch size (default: 2048) -ub, --ubatch-size N physical maximum batch size (default: 512) --keep N number of tokens to keep from the initial prompt (default: 0, -1 = all) --chunks N max number of chunks to process (default: -1, -1 = all) -fa, --flash-attn enable Flash Attention (default: disabled) -p, --prompt PROMPT prompt to start generation with in conversation mode, this will be used as system prompt (default: '') -f, --file FNAME a file containing the prompt (default: none) --in-file FNAME an input file (repeat to specify multiple files) -bf, --binary-file FNAME binary file containing the prompt (default: none) -e, --escape process escapes sequences (\n, \r, \t, \', \", \\) (default: true) --no-escape do not process escape sequences -ptc, --print-token-count N print token count every N tokens (default: -1) --prompt-cache FNAME file to cache prompt state for faster startup (default: none) --prompt-cache-all if specified, saves user input and generations to cache as well not supported with --interactive or other interactive options --prompt-cache-ro if specified, uses the prompt cache but does not update it -r, --reverse-prompt PROMPT halt generation at PROMPT, return control in interactive mode can be specified more than once for multiple prompts -sp, --special special tokens output enabled (default: false) -cnv, --conversation run in conversation mode, does not print special tokens and suffix/prefix if suffix/prefix are not specified, default chat template will be used (default: false) -i, --interactive run in interactive mode (default: false) -if, --interactive-first run in interactive mode and wait for input right away (default: false) -mli, --multiline-input allows you to write or paste multiple lines without ending each in '\' --in-prefix-bos prefix BOS to user inputs, preceding the `--in-prefix` string --in-prefix STRING string to prefix user inputs with (default: empty) --in-suffix STRING string to suffix after user inputs with (default: empty) --spm-infill use Suffix/Prefix/Middle pattern for infill (instead of Prefix/Suffix/Middle) as some models prefer this. (default: disabled) sampling: --samplers SAMPLERS samplers that will be used for generation in the order, separated by ';' (default: top_k;tfs_z;typical_p;top_p;min_p;temperature) --sampling-seq SEQUENCE simplified sequence for samplers that will be used (default: kfypmt) --ignore-eos ignore end of stream token and continue generating (implies --logit-bias EOS-inf) --penalize-nl penalize newline tokens (default: false) --temp N temperature (default: 0.8) --top-k N top-k sampling (default: 40, 0 = disabled) --top-p N top-p sampling (default: 0.9, 1.0 = disabled) --min-p N min-p sampling (default: 0.1, 0.0 = disabled) --tfs N tail free sampling, parameter z (default: 1.0, 1.0 = disabled) --typical N locally typical sampling, parameter p (default: 1.0, 1.0 = disabled) --repeat-last-n N last n tokens to consider for penalize (default: 64, 0 = disabled, -1 = ctx_size) --repeat-penalty N penalize repeat sequence of tokens (default: 1.0, 1.0 = disabled) --presence-penalty N repeat alpha presence penalty (default: 0.0, 0.0 = disabled) --frequency-penalty N repeat alpha frequency penalty (default: 0.0, 0.0 = disabled) --dynatemp-range N dynamic temperature range (default: 0.0, 0.0 = disabled) --dynatemp-exp N dynamic temperature exponent (default: 1.0) --mirostat N use Mirostat sampling. Top K, Nucleus, Tail Free and Locally Typical samplers are ignored if used. (default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0) --mirostat-lr N Mirostat learning rate, parameter eta (default: 0.1) --mirostat-ent N Mirostat target entropy, parameter tau (default: 5.0) -l TOKEN_ID(+/-)BIAS modifies the likelihood of token appearing in the completion, i.e. `--logit-bias 15043+1` to increase likelihood of token ' Hello', or `--logit-bias 15043-1` to decrease likelihood of token ' Hello' --cfg-negative-prompt PROMPT negative prompt to use for guidance (default: '') --cfg-negative-prompt-file FNAME negative prompt file to use for guidance --cfg-scale N strength of guidance (default: 1.0, 1.0 = disable)
100+评论
captcha

    相关资源

    1. rxtx项目运行资源 x64 jar 包

      ling_zhi_xin#物联网ZIP71.23MB7月前
      资源摘要:rxtx项目运行资源 x64 jar 包

    2. 这是一个简易的PS软件，使用VC6.0编写 数字图像处理实验编写的软件

      2301_80488214#网络技术ZIP2.89MB7月前
      资源摘要:这是一个简易的PS软件，使用VC6.0编写。数字图像处理实验编写的软件

    3. SecGPT网络安全大模型

      2301_80488214#安全技术ZIP4.41MB7月前
      资源摘要:模型介绍SecGPT的愿景是将人工智能技术引入网络安全领域，以提高网络防御的效率和效果。其使命是推动网络安全智能化，为社会提供更安全的数字生活环境。SecGPT可以作为基座安全模型，用于探索各种网络安全任务。以下是对SecGPT在网络安全任务中可能应用的进一步探讨：1. 漏洞分析：SecGPT可以与安全研究人员或开发团队进行多轮对话，共同分析和审查应用程序或系统中的潜在漏洞。它可以帮助识别和理解漏洞的性质、潜在影响以及可能的修复方法。2. 溯源分析：在网络入侵事件调查中，SecGPT可以与调查员合作，协助分析网络流量、日志和事件记录，以追踪攻击者的活动路径，从而支持安全溯源分析。3. 流量分析：SecGPT可以分析网络流量数据，识别异常流量模式，帮助检测潜在的网络攻击或入侵行为，并提供应对这些行为的建议。4. 攻击研判：在面对未知攻击时，SecGPT可以与安全团队一起进行多轮对话，共同分析攻击的特征、模式和可能的来源，以便更好地理解和应对威胁。

    4. Element Audio 插件主机

      2301_80488214#网络技术ZIP2.94MB7月前
      资源摘要:特征独立运行或作为 DAW 中的插件运行**将音频和 MIDI 从任意位置传输到任意位置现场演奏虚拟乐器和效果创建可重复使用的仪器和效果图与 MIDI 时钟进行外部同步子图 – 相互嵌套的图自定义键盘快捷键占位节点内置虚拟键盘多次撤消/重做脚本 - 自定义 DSP 和 DSP UI直接在图表中嵌入插件 UI

    5. 聊天室、websocket、socket.io、毕业设计

      2301_80488214#网络技术ZIP29.19MB7月前
      资源摘要: 功能 登陆注册 单聊 群聊 历史消息 图片发送 文件发送 代码片段发送 表情发送 白板协作 音视频聊天 消息已读提醒 好友分组 好友备注 好友上线提醒 在线用户头像高亮 添加好友 添加群聊 朋友圈功能 发表朋友圈 好友朋友圈 朋友圈动态点赞 朋友圈动态评论 朋友圈动态回复评论 日程设置

    6. 一个联邦平均框架，并基于ST-GCN模型进行实验，在Kinetics和NTU60数据集上验证

      2301_80488214#数据库ZIP30.2MB7月前
      资源摘要:通信模型: NOTE:分成两个循环： 客户端 listen -> 接收模型 -> 训练一个epoch -> 发起通信 -> 上传模型 -> (重复)listen 服务器 发起通信 -> 下放模型 -> listen -> 接收模型 -> 聚合 -> (重复)发起通信

    7. Lsky Pro 是一个用于在线上传、管理图片的图床程序

      2401_86632118#前端ZIP21.82MB7月前
      资源摘要:源码下载地址：00818.cnLsky Pro 是一个用于在线上传、管理图片的图床程序，中文名：兰空图床，你可以将它作为自己的云上相册，亦可以当作你的写作贴图库。该程序的最初版本诞生于2017年10月，由ThinkPHP 5框架精心打造而成。在历经数次迭代升级之后，终于在2022年3月迎来了全新的2.0版本。Lsky Pro具备广泛的存储支持能力，不仅包括本地存储，还涵盖了多种第三方云存储服务，如AWS S3、阿里云OSS、腾讯云COS、七牛云、又拍云、SFTP、FTP、WebDav以及Minio等。

    8. Aristosoft more Windows

      m0_70483194#操作系统ZIP295.8KB7月前
      资源摘要:More Windows 是一款视频驱动程序，可为您提供大的可滚动功能 现有 VGA 或 EGA 系统上的虚拟桌面。它提供了几个 虚拟桌面大小，包括纵向和横向 - 可能是 对于不同类型的任务更可取。它还提供一份“论文 白色“单色模式，使用较少的内存并具有”鸟” 眼睛“视图。其他视频模式将消耗额外的扩展内存。这是搬运过来的，国外的网站下载太慢了
    站长QQ:九六九五七三七