Batch Processing

batch_find_untargeted_attack(nn, dataset, target_indices, main_solver; save_path, solve_rerun_option, pp, norm_order, tolerance, rebuild, tightening_algorithm, tightening_solver, cache_model, solve_if_predicted_in_targeted, adversarial_example_objective)

Runs find_adversarial_example for the specified neural network nn and dataset for samples identified by the target_indices, with the target labels for each sample set to the complement of the true label.

It creates a named directory in save_path, with the name summarizing

1. the name of the network in nn,

2. the perturbation family pp,

3. the norm_order

4. the tolerance.

Within this directory, a summary of all the results is stored in summary.csv, and results from individual runs are stored in the subfolder run_results.

This functioned is designed so that it can be interrupted and restarted cleanly; it relies on the summary.csv file to determine what the results of previous runs are (so modifying this file manually can lead to unexpected behavior.)

If the summary file already contains a result for a given target index, the solve_rerun_option determines whether we rerun find_adversarial_example for this particular index.

main_solver specifies the solver used to solve the MIP problem once it has been built.

Named Arguments:

• save_path: Directory where results will be saved. Defaults to current directory.

• pp, norm_order, tolerance, rebuild, tightening_algorithm, tightening_solver, cache_model, solve_if_predicted_in_targeted are passed through to find_adversarial_example and have the same default values; see documentation for that function for more details.

• solve_rerun_option::MIPVerify.SolveRerunOption: Options are never, always, resolve_ambiguous_cases, and refine_insecure_cases. See run_on_sample_for_untargeted_attack for more details.

batch_find_targeted_attack(nn, dataset, target_indices, main_solver; save_path, solve_rerun_option, target_labels, pp, norm_order, tolerance, rebuild, tightening_algorithm, tightening_solver, cache_model, solve_if_predicted_in_targeted)

Runs find_adversarial_example for the specified neural network nn and dataset for samples identified by the target_indices, with each of the target labels in target_labels individually targeted.

Otherwise same parameters as batch_find_untargeted_attack.

run_on_sample_for_targeted_attack(sample_number, target_label, summary_dt, solve_rerun_option)

Determines whether to run a solve on a sample depending on the solve_rerun_option by looking up information on the most recent completed solve recorded in summary_dt matching sample_number.

summary_dt is expected to be a DataFrame with columns :SampleNumber, :TargetIndexes, :SolveStatus, and :ObjectiveValue.

run_on_sample_for_untargeted_attack(sample_number, summary_dt, solve_rerun_option)

Determines whether to run a solve on a sample depending on the solve_rerun_option by looking up information on the most recent completed solve recorded in summary_dt matching sample_number.

summary_dt is expected to be a DataFrame with columns :SampleNumber, :SolveStatus, and :ObjectiveValue.

Behavior for different choices of solve_rerun_option:

• never: true if and only if there is no previous completed solve.

• always: true always.

• resolve_ambiguous_cases: true if there is no previous completed solve, or if the most recent completed solve a) did not find a counter-example BUT b) the optimization was not demosntrated to be infeasible.

• refine_insecure_cases: true if there is no previous completed solve, or if the most recent complete solve a) did find a counter-example BUT b) we did not reach a provably optimal solution.