Repository navigation
fix: SDXL model crash on iOS - use correct pipeline type - #88
Conversation
…bump to 0.0.61 SDXL models crashed on iOS because CoreMLDiffusionModule always created StableDiffusionPipeline, which uses TextEncoder expecting "last_hidden_state" output. SDXL TextEncoder outputs "hidden_embeds" instead, causing a nil force-unwrap crash. Now detects SDXL models by checking for TextEncoder2.mlmodelc and creates StableDiffusionXLPipeline accordingly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request delivers a crucial fix for iOS users experiencing crashes when interacting with SDXL image models. By introducing dynamic pipeline selection based on model type, the application now correctly handles both standard and SDXL models, preventing runtime errors and improving stability. The change ensures that the correct underlying CoreML pipeline is utilized, addressing a fundamental incompatibility without altering the user interface or core generation flow. Additionally, the application's version has been incremented to reflect these updates. Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request correctly fixes a crash on iOS when using SDXL models by introducing logic to detect the model type and instantiate the appropriate pipeline (StableDiffusionXLPipeline for SDXL, StableDiffusionPipeline for others). The change to use the StableDiffusionPipelineProtocol is a good approach to handle different pipeline implementations. The version numbers are also consistently updated across platforms. My main feedback is a suggestion to refactor the pipeline creation logic into a separate helper function to improve code organization and maintainability.
| if self.isXLModel(at: url) { | ||
| // SDXL models need the XL pipeline which uses TextEncoderXL | ||
| // (expects "hidden_embeds" output instead of "last_hidden_state") | ||
| pipe = try StableDiffusionXLPipeline( | ||
| resourcesAt: url, | ||
| configuration: config, | ||
| reduceMemory: true | ||
| ) | ||
| } else { | ||
| pipe = try StableDiffusionPipeline( | ||
| resourcesAt: url, | ||
| controlNet: [], | ||
| configuration: config, | ||
| reduceMemory: true | ||
| ) | ||
| } |
There was a problem hiding this comment.
This conditional logic for creating the pipeline is correct. For better code organization and to improve the readability of loadModel, consider extracting this logic into a separate private factory method. This would encapsulate the details of pipeline creation and make the loadModel function's primary responsibility clearer.
For example, you could introduce a method like this:
private func createPipeline(for url: URL, with config: MLModelConfiguration) throws -> StableDiffusionPipelineProtocol {
if isXLModel(at: url) {
// SDXL models need the XL pipeline
return try StableDiffusionXLPipeline(
resourcesAt: url,
configuration: config,
reduceMemory: true
)
} else {
return try StableDiffusionPipeline(
resourcesAt: url,
controlNet: [],
configuration: config,
reduceMemory: true
)
}
}And then call it from loadModel:
let pipe = try self.createPipeline(for: url, with: config)This change would make the code more modular and easier to extend if more pipeline types are added in the future.
Summary
Fixes iOS crash when loading/generating with SDXL image models. The app was always using
StableDiffusionPipelinefor all CoreML models, but SDXL models requireStableDiffusionXLPipeline. The regular pipeline'sTextEncoderexpects an output feature namedlast_hidden_state, but SDXL's TextEncoder outputshidden_embeds— causing a nil force-unwrap crash atTextEncoder.swift:89.Now detects SDXL models by checking for
TextEncoder2.mlmodelcin the model directory and creates the appropriate pipeline type.Also bumps version to 0.0.61 for both platforms.
Type of Change
Screenshots / Screen Recordings
N/A — no UI changes, fix is in native pipeline selection logic.
Checklist
General
Testing
npm test)React Native Specific
project.pbxproj)Security
Additional Notes
StableDiffusionPipelineusesTextEncoderwhich looks for output"last_hidden_state". SDXL TextEncoder outputs"hidden_embeds"instead. The library already hasStableDiffusionXLPipelinewithTextEncoderXLthat handles this correctly — we just weren't using it.StableDiffusionPipelineProtocol, so the rest of the code (generate, unload, etc.) works unchanged.