[NEW][freetokenqwen3635ba3bnvfp4][1.0.4]

freetokenqwen3635ba3bnvfp4/.helmignore

@@ -0,0 +1,3 @@
+tests/
+__pycache__/
+*.pyc

freetokenqwen3635ba3bnvfp4/Chart.yaml

@@ -0,0 +1,6 @@
+apiVersion: v2
+appVersion: 0.1.3
+description: Qwen3.6-35B-A3B vision-language model running locally with FreeToken
+name: freetokenqwen3635ba3bnvfp4
+type: application
+version: 1.0.4

freetokenqwen3635ba3bnvfp4/OlaresManifest.yaml

@@ -0,0 +1,215 @@
+olaresManifest.version: '0.12.0'
+olaresManifest.type: app
+apiVersion: 'v3'
+metadata:
+  name: freetokenqwen3635ba3bnvfp4
+  icon: https://app.cdn.olares.com/appstore/freetoken/icon.png
+  description: "FreeToken-powered Qwen3.6-35B-A3B for text, image understanding, and tool calling"
+  appid: freetokenqwen3635ba3bnvfp4
+  title: Qwen3.6-35B-A3B (FreeToken)
+  version: '1.0.4'
+  categories:
+  - AI
+  - models
+  tags:
+  - llm
+sharedEntrances:
+  - name: freetokenqwen3635ba3bnvfp4
+    host: sharedentrances-api
+    port: 0
+    title: Qwen3.6-35B-A3B
+    invisible: true
+    authLevel: internal
+    icon: https://app.cdn.olares.com/appstore/freetoken/icon.png
+entrances:
+  - name: freetokenclient
+    port: 8090
+    host: download-svc
+    title: Qwen3.6-35B-A3B
+    icon: https://app.cdn.olares.com/appstore/freetoken/icon.png
+    openMethod: window
+    authLevel: internal
+#   freetokenqwen3635ba3bnvfp4 — engine Deployment (== appid / Release.Name)
+#   llminit                    — llm-init / entrance Deployment
+workloadReplicas:
+  freetokenqwen3635ba3bnvfp4: 1
+  llminit: 1
+spec:
+  onlyAdmin: true
+  versionName: 'nvidia/Qwen3.6-35B-A3B-NVFP4'
+  fullDescription: |
+    Qwen3.6-35B-A3B is a vision-language model from Alibaba's Qwen team for text generation, image understanding, thinking, and tool calling in compatible applications. It uses a Mixture-of-Experts architecture with 35 billion parameters in total and about 3 billion active per token. This app runs NVIDIA's 4-bit NVFP4 quantized edition locally.
+
+    FreeToken is FlashML's inference engine for running Mixture-of-Experts models on personal hardware. It keeps expert weights in system memory and caches a working set on the GPU, reducing the need to fit the whole model in GPU memory. Model Console connects the engine to Olares and lets you download the model, check its status, and adjust settings.
+
+    This edition accepts text and images, with a default context limit of 131,072 tokens and one running request at a time. Available context depends on memory and engine settings.
+
+    First startup downloads about 23.5 GB of model files and converts them to FreeToken's reusable weight format. Allow additional storage and preparation time for this conversion. The preset requires an NVIDIA GPU with 23 GiB of available GPU memory, system memory as listed in the resource requirements, and an NVIDIA R580 or newer driver.
+
+    [Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) · [NVFP4](https://huggingface.co/nvidia/Qwen3.6-35B-A3B-NVFP4) · [FreeToken](https://github.com/FlashML-org/FreeToken)
+
+  developer: Alibaba
+  website: https://qwen.ai/home
+  sourceCode: https://github.com/FlashML-org/FreeToken
+  submitter: Olares
+  locale:
+    - en-US
+    - zh-CN
+    - de-DE
+    - es-ES
+    - it-IT
+    - fr-FR
+    - ja-JP
+  license:
+    - text: Apache-2.0
+      url: https://huggingface.co/nvidia/Qwen3.6-35B-A3B-NVFP4
+
+  accelerator:
+    # nvidia — discrete GPU; FREETOKEN_REQUIRED_GPU_MEMORY -> nvidia.com/gpumem.
+    # No nvidia-gb10 entry: the FreeToken image is amd64-only (CUDA 13
+    # devel base -- tag 0.1.3-cu132; the kernels are compiled on first use).
+    - mode: nvidia
+      limitedCpu: "15360m"
+      requiredCpu: "1100m"
+      requiredDisk: 50Mi
+      limitedDisk: 500Gi
+      # Engine MEMORY_REQUEST 24Gi + llm-init 256Mi <= requiredMemory 25Gi;
+      # MEMORY_LIMIT 32Gi + llm-init 2Gi <= limitedMemory 34Gi.
+      #
+      # The request has to cover the model itself: under MoE offload the NVFP4
+      # expert banks (21.8Gi, measured) sit in host RAM for the life of the
+      # process -- they are the model, not a transient. Steady state is ~25Gi
+      # with the ~3Gi python/torch/CUDA context on top, and --expert-load
+      # serial keeps the startup read bounded, so there is no load-time spike
+      # to leave room for: the 32Gi cap is ~7Gi of headroom over steady state,
+      # and an OOMKill under it would mean something is actually growing.
+      limitedMemory: "34Gi"
+      requiredMemory: "25Gi"
+      requiredGPUMemory: "23Gi"
+      limitedGPUMemory: "24Gi"
+
+  supportArch:
+    - amd64
+permission:
+  appCache: true
+  appData: true
+  appCommon: true
+options:
+  apiTimeout: 0
+  LLMGatewaySupported: true
+  shared: true
+  templateOnly: false
+  dependencies:
+    - name: olares
+      version: '>=1.12.7-0'
+      type: system
+envs:
+  # Model is preset for this app; these values are fixed and not user-editable.
+  - envName: MODEL_SOURCE
+    required: true
+    type: string
+    editable: false
+    applyOnChange: false
+    default: "hf://nvidia/Qwen3.6-35B-A3B-NVFP4"
+    description:  "Example: hf://nvidia/Qwen3.6-35B-A3B-NVFP4. The source to download your model. FreeToken loads a Hugging Face checkpoint directory, so the whole repo is fetched — no `--include` filter."
+    regex: "^hf://"
+  - envName: MODEL_NAME          # served model id, also the name Router shows
+    required: true
+    type: string
+    editable: false
+    applyOnChange: false
+    default: "nvidia/Qwen3.6-35B-A3B-NVFP4"
+    description:  "Example: nvidia/Qwen3.6-35B-A3B-NVFP4. The model id this app serves, passed to FreeToken as --served-model-name."
+  - envName: MODEL_MODE
+    required: true
+    type: string
+    editable: false
+    applyOnChange: false
+    default: "chat"
+    options:
+    - title: Chat
+      value: chat
+    - title: Embedding
+      value: embedding
+  - envName: MODEL_SUPPORTS
+    required: true
+    type: string
+    editable: false
+    applyOnChange: false
+    default: "thinking,tools,vision"
+    multiSelect: true
+    splitter: ","
+    options:
+    - title: Vision
+      value: vision
+    - title: Tools
+      value: tools
+    - title: Thinking
+      value: thinking
+    - title: None
+      value: none
+    description:  "The model capabilities. Pick what your model supports, if no extra capabilities, pick None."
+  - envName: ENGINE_ARGS
+    required: true
+    default: "--moe-strategy auto --expert-load serial --max-seq-len-override 131072 --kv-reserve-tokens 131072 --max-prefill-length 8192 --max-running-requests 1"
+    type: string
+    editable: true
+    applyOnChange: true
+    description:  "Raw `ft serve` flags. `--moe-strategy auto` picks the MoE placement (offload on this checkpoint), `--max-seq-len-override` sets the context window, `--kv-reserve-tokens` floors the KV pool before the expert cache takes the rest of the VRAM, `--max-running-requests` is the concurrency, `--expert-load serial` bounds how much host RAM the expert-bank read needs (see below). The wrapper also accepts SGLang's `--context-length N` as an alias for `--max-seq-len-override N`, which is the only spelling the model card can read the window from; a native `--max-seq-len-override` always wins over it."
+  - envName: LOG_LEVEL           # single-select: debug | info | warn | error
+    required: false
+    type: string
+    editable: true
+    applyOnChange: true
+    default: "info"
+    options:
+    - title: Debug
+      value: debug
+    - title: Info
+      value: info
+    - title: Warn
+      value: warn
+    - title: Error
+      value: error
+  - envName: HF_ENDPOINT
+    required: false
+    applyOnChange: true
+    valueFrom:
+      envName: OLARES_SYSTEM_HUGGINGFACE_SERVICE
+  - envName: HF_TOKEN
+    required: false
+    applyOnChange: true
+    valueFrom:
+      envName: OLARES_SYSTEM_HUGGINGFACE_TOKEN
+  - envName: FREETOKEN_CPU_REQUEST    # default: 500m
+    required: true
+    default: "500m"
+    type: string
+    editable: false
+    applyOnChange: false
+  - envName: FREETOKEN_MEMORY_REQUEST # sum with llm-init <= requiredMemory 25Gi
+    required: true
+    default: "24Gi"
+    type: string
+    editable: false
+    applyOnChange: false
+  - envName: FREETOKEN_CPU_LIMIT      # default: 8
+    required: true
+    default: "8"
+    type: string
+    editable: false
+    applyOnChange: false
+  - envName: FREETOKEN_MEMORY_LIMIT # 32Gi + llm-init 2Gi <= limitedMemory 34Gi
+    required: true
+    default: "32Gi"
+    type: string
+    editable: false
+    applyOnChange: false
+  - envName: FREETOKEN_REQUIRED_GPU_MEMORY  # 23Gi/23552/23552Mi -> bare-MiB gpumem
+    required: true
+    type: string
+    editable: false
+    applyOnChange: false
+    default: "23Gi"
+    description:  "Example: `23Gi`, `23552`, or `23552Mi`. GPU memory required for the model. On NVIDIA this sets `nvidia.com/gpumem` in MiB."
+    regex: "^[0-9]+(Gi|Mi)?$"

freetokenqwen3635ba3bnvfp4/i18n/de-DE/OlaresManifest.yaml

@@ -0,0 +1,15 @@
+metadata:
+  description: "FreeToken-basierte Qwen3.6-35B-A3B-Inferenz für Text, Bildverständnis und Werkzeugaufrufe"
+  title: Qwen3.6-35B-A3B (FreeToken)
+
+spec:
+  fullDescription: |
+    Qwen3.6-35B-A3B ist ein Vision-Language-Modell des Qwen-Teams von Alibaba für Textgenerierung, Bildverständnis, Denkprozesse und Werkzeugaufrufe in kompatiblen Anwendungen. Seine Mixture-of-Experts-Architektur umfasst 35 Milliarden Parameter, von denen etwa 3 Milliarden pro Token aktiv sind. Diese App führt NVIDIAs mit 4 Bit quantisierte NVFP4-Ausgabe lokal aus.
+
+    FreeToken ist die Inferenz-Engine von FlashML für Mixture-of-Experts-Modelle auf eigener Hardware. Sie hält die Expertengewichte im Arbeitsspeicher und speichert einen benötigten Teil auf der GPU zwischen. Dadurch muss nicht das gesamte Modell in den GPU-Speicher passen. Model Console verbindet die Engine mit Olares und ermöglicht das Herunterladen des Modells, Statusabfragen und Einstellungsänderungen.
+
+    Diese Ausgabe akzeptiert Text und Bilder, mit einer standardmäßigen Kontextgrenze von 131.072 Tokens und jeweils einer laufenden Anfrage. Der nutzbare Kontext hängt vom Speicher und den Engine-Einstellungen ab.
+
+    Beim ersten Start werden etwa 23,5 GB Modelldateien heruntergeladen und in das wiederverwendbare Gewichtsformat von FreeToken konvertiert. Planen Sie zusätzlichen Speicherplatz und Vorbereitungszeit ein. Diese Voreinstellung benötigt eine NVIDIA-GPU mit 23 GiB verfügbarem GPU-Speicher, den in den Ressourcenanforderungen angegebenen Arbeitsspeicher und einen NVIDIA-Treiber ab R580.
+
+    [Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) · [NVFP4](https://huggingface.co/nvidia/Qwen3.6-35B-A3B-NVFP4) · [FreeToken](https://github.com/FlashML-org/FreeToken)

freetokenqwen3635ba3bnvfp4/i18n/en-US/OlaresManifest.yaml

@@ -0,0 +1,15 @@
+metadata:
+  description: "FreeToken-powered Qwen3.6-35B-A3B for text, image understanding, and tool calling"
+  title: Qwen3.6-35B-A3B (FreeToken)
+
+spec:
+  fullDescription: |
+    Qwen3.6-35B-A3B is a vision-language model from Alibaba's Qwen team for text generation, image understanding, thinking, and tool calling in compatible applications. It uses a Mixture-of-Experts architecture with 35 billion parameters in total and about 3 billion active per token. This app runs NVIDIA's 4-bit NVFP4 quantized edition locally.
+
+    FreeToken is FlashML's inference engine for running Mixture-of-Experts models on personal hardware. It keeps expert weights in system memory and caches a working set on the GPU, reducing the need to fit the whole model in GPU memory. Model Console connects the engine to Olares and lets you download the model, check its status, and adjust settings.
+
+    This edition accepts text and images, with a default context limit of 131,072 tokens and one running request at a time. Available context depends on memory and engine settings.
+
+    First startup downloads about 23.5 GB of model files and converts them to FreeToken's reusable weight format. Allow additional storage and preparation time for this conversion. The preset requires an NVIDIA GPU with 23 GiB of available GPU memory, system memory as listed in the resource requirements, and an NVIDIA R580 or newer driver.
+
+    [Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) · [NVFP4](https://huggingface.co/nvidia/Qwen3.6-35B-A3B-NVFP4) · [FreeToken](https://github.com/FlashML-org/FreeToken)

freetokenqwen3635ba3bnvfp4/i18n/es-ES/OlaresManifest.yaml

@@ -0,0 +1,15 @@
+metadata:
+  description: "FreeToken como motor de Qwen3.6-35B-A3B para texto, comprensión de imágenes y uso de herramientas"
+  title: Qwen3.6-35B-A3B (FreeToken)
+
+spec:
+  fullDescription: |
+    Qwen3.6-35B-A3B es un modelo de visión y lenguaje del equipo Qwen de Alibaba para generar texto, comprender imágenes, razonar y usar herramientas en aplicaciones compatibles. Su arquitectura de mezcla de expertos tiene 35 mil millones de parámetros, con unos 3 mil millones activos por token. Esta aplicación ejecuta localmente la edición cuantizada NVFP4 de 4 bits de NVIDIA.
+
+    FreeToken es el motor de inferencia de FlashML para ejecutar modelos de mezcla de expertos en equipos personales. Mantiene los pesos de los expertos en la memoria del sistema y almacena una parte en caché en la GPU, reduciendo la necesidad de cargar todo el modelo en la memoria de la GPU. Model Console conecta el motor a Olares y permite descargar el modelo, consultar su estado y ajustar la configuración.
+
+    Esta edición admite texto e imágenes, con un límite de contexto predeterminado de 131.072 tokens y una solicitud en ejecución a la vez. El contexto disponible depende de la memoria y de la configuración del motor.
+
+    El primer inicio descarga unos 23,5 GB de archivos del modelo y los convierte al formato de pesos reutilizable de FreeToken. Reserva almacenamiento y tiempo de preparación adicionales para la conversión. Este ajuste requiere una GPU NVIDIA con 23 GiB de memoria GPU disponible, la memoria del sistema indicada en los requisitos de recursos y un controlador NVIDIA R580 o posterior.
+
+    [Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) · [NVFP4](https://huggingface.co/nvidia/Qwen3.6-35B-A3B-NVFP4) · [FreeToken](https://github.com/FlashML-org/FreeToken)

freetokenqwen3635ba3bnvfp4/i18n/fr-FR/OlaresManifest.yaml

@@ -0,0 +1,15 @@
+metadata:
+  description: "FreeToken pour exécuter Qwen3.6-35B-A3B : texte, compréhension d’images et appels d’outils"
+  title: Qwen3.6-35B-A3B (FreeToken)
+
+spec:
+  fullDescription: |
+    Qwen3.6-35B-A3B est un modèle vision-langage de l'équipe Qwen d'Alibaba pour la génération de texte, la compréhension d'images, la réflexion et les appels d'outils dans les applications compatibles. Son architecture à mélange d'experts compte 35 milliards de paramètres, dont environ 3 milliards actifs par token. Cette application exécute localement l'édition quantifiée NVFP4 sur 4 bits de NVIDIA.
+
+    FreeToken est le moteur d'inférence de FlashML pour exécuter des modèles à mélange d'experts sur du matériel personnel. Il conserve les poids des experts en mémoire système et en met une partie en cache sur le GPU, limitant le besoin de charger tout le modèle en mémoire GPU. Model Console relie le moteur à Olares et permet de télécharger le modèle, de consulter son état et d'ajuster les réglages.
+
+    Cette édition accepte du texte et des images, avec une limite de contexte par défaut de 131 072 tokens et une seule requête en cours d'exécution. Le contexte disponible dépend de la mémoire et des réglages du moteur.
+
+    Le premier démarrage télécharge environ 23,5 GB de fichiers de modèle et les convertit au format de poids réutilisable de FreeToken. Prévoyez du stockage et du temps de préparation supplémentaires pour cette conversion. Ce préréglage nécessite un GPU NVIDIA avec 23 GiB de mémoire GPU disponible, la mémoire système indiquée dans les exigences de ressources et un pilote NVIDIA R580 ou ultérieur.
+
+    [Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) · [NVFP4](https://huggingface.co/nvidia/Qwen3.6-35B-A3B-NVFP4) · [FreeToken](https://github.com/FlashML-org/FreeToken)

freetokenqwen3635ba3bnvfp4/i18n/it-IT/OlaresManifest.yaml

@@ -0,0 +1,15 @@
+metadata:
+  description: "FreeToken per eseguire Qwen3.6-35B-A3B: testo, comprensione delle immagini e chiamate a strumenti"
+  title: Qwen3.6-35B-A3B (FreeToken)
+
+spec:
+  fullDescription: |
+    Qwen3.6-35B-A3B è un modello visivo-linguistico del team Qwen di Alibaba per la generazione di testo, la comprensione delle immagini, il ragionamento e le chiamate a strumenti nelle applicazioni compatibili. Usa un'architettura Mixture-of-Experts con 35 miliardi di parametri totali, di cui circa 3 miliardi attivi per token. Questa app esegue localmente l'edizione NVFP4 quantizzata a 4 bit di NVIDIA.
+
+    FreeToken è il motore di inferenza di FlashML per eseguire modelli Mixture-of-Experts su hardware personale. Mantiene i pesi degli esperti nella memoria di sistema e ne conserva una parte nella cache della GPU, riducendo la necessità di caricare l'intero modello nella memoria della GPU. Model Console collega il motore a Olares e permette di scaricare il modello, verificarne lo stato e regolare le impostazioni.
+
+    Questa edizione accetta testo e immagini, con un limite di contesto predefinito di 131.072 token e una richiesta in esecuzione alla volta. Il contesto disponibile dipende dalla memoria e dalle impostazioni del motore.
+
+    Il primo avvio scarica circa 23,5 GB di file del modello e li converte nel formato di pesi riutilizzabile di FreeToken. Prevedere spazio di archiviazione e tempo di preparazione aggiuntivi per la conversione. Questa configurazione richiede una GPU NVIDIA con 23 GiB di memoria GPU disponibile, la memoria di sistema indicata nei requisiti delle risorse e un driver NVIDIA R580 o successivo.
+
+    [Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) · [NVFP4](https://huggingface.co/nvidia/Qwen3.6-35B-A3B-NVFP4) · [FreeToken](https://github.com/FlashML-org/FreeToken)

freetokenqwen3635ba3bnvfp4/i18n/ja-JP/OlaresManifest.yaml

@@ -0,0 +1,15 @@
+metadata:
+  description: "FreeTokenで動くQwen3.6-35B-A3Bによるテキスト生成・画像理解・ツール呼び出し"
+  title: Qwen3.6-35B-A3B (FreeToken)
+
+spec:
+  fullDescription: |
+    Qwen3.6-35B-A3Bは、AlibabaのQwenチームによる視覚言語モデルです。対応するアプリでテキスト生成、画像理解、思考機能、ツール呼び出しを利用できます。Mixture-of-Experts構造を採用し、総パラメータ数は350億、トークンごとに約30億が有効になります。このアプリはNVIDIAの4ビットNVFP4量子化版をローカルで実行します。
+
+    FreeTokenは、個人のハードウェアでMixture-of-Expertsモデルを実行するためのFlashMLの推論エンジンです。エキスパートの重みをシステムメモリに保持し、使用する一部をGPUにキャッシュすることで、モデル全体をGPUメモリに載せる必要性を減らします。Model Consoleを通じてOlaresに接続し、モデルのダウンロード、状態の確認、設定の調整を行えます。
+
+    この構成はテキストと画像を入力でき、標準のコンテキスト上限は131,072トークンです。同時に実行するリクエストは1件です。利用できるコンテキストはメモリとエンジン設定に依存します。
+
+    初回起動時に約23.5 GBのモデルファイルをダウンロードし、FreeTokenの再利用可能な重み形式に変換します。変換のための追加ストレージと準備時間を確保してください。この構成には、23 GiBの空きGPUメモリがあるNVIDIA GPU、リソース要件に記載されたシステムメモリ、R580以降のNVIDIAドライバが必要です。
+
+    [Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) · [NVFP4](https://huggingface.co/nvidia/Qwen3.6-35B-A3B-NVFP4) · [FreeToken](https://github.com/FlashML-org/FreeToken)