FL-Security/Poisoning_Resilient_Federated_Learning_Playground

FL Security Experiment Results

This repository contains the experiment outputs used for the FL Security / FLPoison federated learning poisoning benchmark. The archive is intended for readers who want to inspect the raw training logs, reuse the aggregated curves and tables, or reproduce the paper figures without rerunning the full Compute Canada workload.

The uploaded artifact is:

exp_data.tar.gz  # about 241 MB

After extraction, the archive keeps the original Compute Canada scratch layout:

scratch/FL_Poison/
  logs_from_slurm_split_20260312/
  replot_from_slurm_no300_noadv04_20260312/
  tables_test_loss_from_slurm_split_cleaned_merged_ranked_epochmerged_nomean_withbase/
  custom_compare_fangattack_mean_vs_noattack/

The FL_Poison directory name is a legacy path from the experiment runs. The code repository has since been renamed to FL_Security.

What Is Included

• —logs_from_slurm_split_20260312/: split raw result logs recovered from Slurm outputs.
• —replot_from_slurm_no300_noadv04_20260312/: regenerated plots and plot indexes after filtering known incomplete or out-of-scope runs.
• —tables_test_loss_from_slurm_split_cleaned_merged_ranked_epochmerged_nomean_withbase/: LaTeX tables and an index for final test-loss comparisons.
• —custom_compare_fangattack_mean_vs_noattack/: focused FangAttack vs. NoAttack comparisons for selected settings.

The archive contains result files only. It does not include the original training datasets such as CIFAR10, CIFAR100, or TinyImageNet.

Experiment Scope

The main result set covers federated learning poisoning experiments with:

• —Algorithms: FedAvg
• —Datasets/models: CIFAR10 + VGG19, CIFAR100 + ResNet18
• —Data splits: iid, non-iid
• —Non-IID Dirichlet alpha values: 0.1, 0.5, 1
• —Adversarial client fractions: 0.1, 0.2, 0.3, with some raw logs also containing 0.4
• —Seeds used in the regenerated plots: 42, 43, 44, 45, 46
• —Attacks in the main plots: ALIE, FangAttack, MinMax, MinSum

Most regenerated figures target the 200-round configuration and use epoch 199 / 200 as the final evaluation point, depending on whether the file is indexed by zero-based epoch or by configured training rounds.

Important Files

The most useful entry points are:

scratch/FL_Poison/logs_from_slurm_split_20260312/extraction_summary.txt
scratch/FL_Poison/logs_from_slurm_split_20260312/extraction_manifest.tsv

scratch/FL_Poison/replot_from_slurm_no300_noadv04_20260312/accuracy_1x4/summary.txt
scratch/FL_Poison/replot_from_slurm_no300_noadv04_20260312/accuracy_1x4/plot_index.csv

scratch/FL_Poison/replot_from_slurm_no300_noadv04_20260312/adv_sensitivity_0.1-0.3_epoch199/plot_index.csv
scratch/FL_Poison/replot_from_slurm_no300_noadv04_20260312/alpha_non_iid_epoch199/plot_index.csv
scratch/FL_Poison/replot_from_slurm_no300_noadv04_20260312/num_clients_sensitivity_epoch199/figure_index.csv
scratch/FL_Poison/replot_from_slurm_no300_noadv04_20260312/triguard_reputation_curves/plot_index.csv
scratch/FL_Poison/replot_from_slurm_no300_noadv04_20260312/triguard_fp_fn/summary.txt

scratch/FL_Poison/tables_test_loss_from_slurm_split_cleaned_merged_ranked_epochmerged_nomean_withbase/table_index.csv
scratch/FL_Poison/tables_test_loss_from_slurm_split_cleaned_merged_ranked_epochmerged_nomean_withbase/summary.txt

The index CSV files are the easiest way to locate a figure or table by dataset, model, IID mode, adversarial fraction, alpha, client count, and seed coverage.

Quick Start

Download and extract the archive:

tar -xzf exp_data.tar.gz
cd scratch/FL_Poison

Inspect the raw extraction summary:

sed -n '1,80p' logs_from_slurm_split_20260312/extraction_summary.txt

List available regenerated plots:

python - <<'PY'
import pandas as pd

idx = pd.read_csv(
    "replot_from_slurm_no300_noadv04_20260312/accuracy_1x4/plot_index.csv"
)
print(idx[["iid", "dataset", "model", "num_clients", "adv", "alpha", "seeds_used", "figure_png"]].head())
PY

Open a generated plot directly from the figure_png or figure_pdf path recorded in the index. If you extracted the archive outside /home/fengye, replace the absolute prefix in the index paths with your local extraction path.

Directory Details

Raw Slurm-Split Logs

logs_from_slurm_split_20260312/ contains split result logs extracted from Compute Canada Slurm output files. The summary reports:

out_files=2140
total_seed_segments=10089
result_txt_total=9622
dataset_seen={'CIFAR10': 4351, 'CIFAR100': 5738}
iid_seen={'iid': 2521, 'non-iid': 7568}
adv_seen={'0.1': 2383, '0.2': 3105, '0.3': 2831, '0.4': 1770}

Use extraction_manifest.tsv to map extracted text files back to their Slurm source segments.

Regenerated Figures

replot_from_slurm_no300_noadv04_20260312/ contains PNG/PDF figures and CSV indexes. The main subdirectories are:

• —accuracy_1x4/: attack/defense accuracy curves grouped into 1x4 panels.
• —adv_sensitivity_0.1-0.3_epoch199/: sensitivity to adversarial client fraction.
• —alpha_non_iid_epoch199/: sensitivity to Dirichlet non-IID alpha.
• —num_clients_sensitivity_epoch199/: sensitivity to number of clients.
• —triguard_reputation_curves/: TriGuardFL reputation curves.
• —triguard_fp_fn/: TriGuardFL false-positive/false-negative summaries.
• —all_figures/: a flat collection of generated figure files.

The accuracy_1x4/summary.txt file records the filtering used for the main regenerated plots:

parsed_after_seed_filter=6901
filtered_by_adv=1598
filtered_by_epoch_cfg=1123
groups_total=48
generated_figures=48
max_epoch=200
attacks=ALIE,FangAttack,MinMax,MinSum
seed_filter=42,43,44,45,46
adv_filter=0.1,0.2,0.3
exclude_config_epochs=300
filter_buggy_fltrust=True

Tables

tables_test_loss_from_slurm_split_cleaned_merged_ranked_epochmerged_nomean_withbase/ contains LaTeX tables for final test-loss comparisons. table_index.csv gives the table path and metadata fields:

table_path,iid,dataset,model,num_clients,lr,algo,adv,alpha,cfg,epochs_tag,loss_stat

Tables are grouped by dataset and adversarial fraction, for example:

CIFAR100/adv_0.2/
CIFAR100/adv_0.3/
CIFAR100/adv_0.4/
CIFAR10/adv_0.2/
CIFAR10/adv_0.3/
CIFAR10/adv_0.4/

Reproducing or Extending the Analysis

The code used to run and post-process experiments is available in the GitHub repository:

https://github.com/fye97/FL_Security

Relevant local entry points in that repository include:

exps/run_cc.sh
exps/run_cc_carat.sh
exps/launch.py
plot_num_clients_accuracy_sensitivity.py

For new Compute Canada runs, use the experiment launcher rather than submitting worker scripts directly:

./exps/run_cc.sh <spec> --dry-run
./exps/run_cc.sh <spec>

Notes and Caveats

• —Some raw logs include adversarial fraction 0.4, but the main regenerated figures filter to 0.1, 0.2, and 0.3.
• —Some 300-round configurations were excluded from the regenerated figure set to keep the plotted results aligned with the 200-round analysis.
• —The result indexes may contain absolute paths from the original machine. Treat those paths as metadata; replace the /home/fengye/scratch/FL_Poison prefix with your local extracted path when opening files programmatically.
• —The raw text logs are useful for auditability, but most readers should start from the plot_index.csv, figure_index.csv, and table_index.csv files.