[Question]How can I use the Madlad model?

[DHLee-94]

I would like to try using Madlad models or hugging face models.

Most of the Hugging Face models only have encoder.onnx and decoder.onnx, but I don't know what to do.
Please help.

[niedev]

Hi @DHLee-94,
I'm having a conversation about the same topic here.
For Madlad the official support will come in the future (as soon as I will have enough time to work on the app), probably it will be much easier for you to wait the official support.

[nwdxlgzs]

I would like to try using Madlad models or hugging face models.

Most of the Hugging Face models only have encoder.onnx and decoder.onnx, but I don't know what to do. Please help.

#153 I implemented a framework can run optimum export model(encoder.onnx+decoder_merge.onnx+tokenizer.json), may be it can help you (with some modify?i guess). onnx-runtime is too slow, if you want to enjoy fast infer, select a good runtime(with more inference acceleration) is very imporant.

[lesca]

+1
The current NLLB model is inaccurate in translation in some cases, and even completely wrong.

For example, in Chinese, 我要下班了 ("I'm off work"), it translates into English is "I'm going to work right now." The result of the translation is completely opposite to the original input.

I think it's better to add an option of another model, or add some parameters to adjust current model.

[niedev]

@lesca yeah, the NLLB has very low quality with very short sentences, the Madlad support (and Mozilla models) will come with RTranslator 3.0, I'm currently working on it full time, the release will be out in a few months.

[ksitetar315-hue]

@lesca yeah, the NLLB has very low quality with very short sentences, the Madlad support (and Mozilla models) will come with RTranslator 3.0, I'm currently working on it full time, the release will be out in a few months.

Thank you very much for your work, and I’m really looking forward to the 3.0 release coming soon. In fact, I’ve already tried using AI to develop an ASR + MT pipeline myself: for ASR I’m using ggml-large-v3-turbo-q5_0.bin, and for MT I’m using tencent.Hunyuan-MT-7B.Q5_K_M.gguf. However, as you can imagine, running this setup on an 8 Gen 2 platform is extremely slow—so slow that it’s hard to tolerate.

I have to say, the bottleneck doesn’t seem to be memory capacity; it’s more likely about optimization, model size, and the device’s performance. Either way, I’m hoping your 3.0 version can bring some improvements and real changes—because not everywhere has network coverage as good as China’s