smitebot: implement reproduce command - #207
Conversation
e517d04 to
16e0be5
Compare
fb4aa11 to
d190ab6
Compare
|
|
||
| ### smitebot reproduce | ||
|
|
||
| Replays a single input against a campaign's target by running its Docker image once, with the input fed to the target the same way the fuzzer's `LocalRunner` does. The target's output is streamed to the terminal; `reproduce` interprets nothing — read the logs to see what happened. Resolves the image and target from the campaign's `state.json`; no live campaign required. |
There was a problem hiding this comment.
nit: could remove the implementation details here, just like the semantics followed above
| let Some(runs_dir) = CampaignState::runs_dir() else { | ||
| log::error!("unable to determine home directory"); | ||
| return false; | ||
| }; | ||
|
|
||
| let state_path = runs_dir.join(&args.campaign_id).join("state.json"); | ||
| let state = match CampaignState::load(&state_path) { | ||
| Ok(s) => s, | ||
| Err(e) => { | ||
| log::error!("{e}"); | ||
| log::error!( | ||
| "campaign '{}' not found; list campaigns with: ls {}", | ||
| args.campaign_id, | ||
| runs_dir.display() | ||
| ); | ||
| return false; | ||
| } | ||
| }; |
There was a problem hiding this comment.
This part of the code is repeated across many commands, I think we could refactor it into a helper in a follow-up PR
| if !docker_image_exists(&state.image) { | ||
| log::error!( | ||
| "Docker image '{}' not found; build it with: smitebot build --target {} --scenario {}", | ||
| state.image, | ||
| state.target, | ||
| state.scenario | ||
| ); | ||
| return false; | ||
| } |
There was a problem hiding this comment.
I'm not sure about others, but I often run a campaign with a specific docker image and then continue development alongside it, creating new docker images with the same name. So, if this command runs a docker image referenced by the same name, I might get false results without realizing it
I think we should compare the found docker image's digest with the one stored in image_digest in state.json and log a warning if they don't match, saying that the results might differ from what was observed during the campaign. We could also log the smite git hash used for the campaign, so the user can rebuild the image from that commit to reproduce the same results.
| } | ||
|
|
||
| let run_args = docker_run_args(&state.image, state.target, parent, &basename, args.attach); | ||
| log::info!( |
There was a problem hiding this comment.
This log should come before let run_args...
| let parent = PathBuf::from("/tmp"); | ||
| let args = docker_run_args("myimage", Target::Ldk, &parent, "in", false); | ||
| let image_pos = args.iter().position(|a| a == "myimage").unwrap(); | ||
| let v_pos = args.iter().position(|a| a == "-v").unwrap(); | ||
| let e_pos = args.iter().position(|a| a == "-e").unwrap(); | ||
| assert!(v_pos < image_pos, "-v must appear before image"); | ||
| assert!(e_pos < image_pos, "-e must appear before image"); | ||
| assert_eq!(args.last().unwrap(), "/ldk-scenario"); | ||
| assert!(args.contains(&"--rm".to_string())); |
There was a problem hiding this comment.
nit: We could define flags upfront, like ["--rm", "-v", "-e"], and then iterate over them to get their positions in args and assert their positions relative to image_pos, this would also allow us to add the attach flag and test both cases
| /// Allocate an interactive TTY so the run can be watched live. | ||
| #[arg(long)] | ||
| attach: bool, |
There was a problem hiding this comment.
I don't think we currently have any observable behavior with the -it option? It might be useful in the future, but I don't see any use for it currently. If we do need it later, I think that change should be paired with the change that introduces the relevant use case
2 participants
Description
What
Adds
smitebot reproduce <campaign-id> -i <input> [--attach]: replays a single input against a campaign's target by running its Docker image once, the same wayscripts/coverage-report.shfeeds inputs toLocalRunner. The target's output streams to the terminal; the command interprets nothing as the logs are the deliverable.Mechanism
Resolves image + target from the campaign's state.json (no live campaign required), then runs:
trueonce the container runs to completion regardless of its exit code; false only on operational failure (unknown campaign, missing input, missing image, docker spawn error). Errors name the offending path/value and the fix, fore.g. a missing image points at smitebot build .Scope for v1
Local-Docker reproduction only. Deliberately deferred for later PR:
Ref. #70