src/Stats.jl

Thu, 25 Apr 2024 13:05:40 -0500

author
Tuomo Valkonen <tuomov@iki.fi>
date
Thu, 25 Apr 2024 13:05:40 -0500
changeset 36
e4a8f662a1ac
parent 34
aca9c90f151c
child 41
3e54b2b54029
permissions
-rw-r--r--

Reduce code duplication.

34
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
1 __precompile__()
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
2
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
3 module Stats
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
4
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
5
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
6 ########################
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
7 # Load external modules
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
8 ########################
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
9
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
10 using CSV, DataFrames
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
11 using Statistics
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
12
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
13 export calculate_statistics
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
14
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
15 function calculate_statistics()
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
16 ImName = ("lighthouse200x300", "shepplogan256x256", "brainphantom256x256")
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
17 AlgName = ("dualscaling", "greedy","noprediction","primalonly","proximal","rotation","zerodual")
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
18 mystart = 41 # Corresponds to the 500th iterate
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
19
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
20 # Define an array to store results
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
21 results = DataFrame(experiment = String[], α = Float64[], algorithm = String[], psnr_mean1 = Float64[], psnr_mean500 = Float64[], psnr_ci = String[], ssim_mean1 = Float64[], ssim_mean500 = Float64[], ssim_ci = String[])
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
22
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
23 for imname in ImName
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
24 for algname in AlgName
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
25 directory_path = "./img/"
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
26 files = readdir(directory_path)
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
27 filtered_files = filter(file -> startswith(file, "$(imname)_pdps_known_$(algname)") && endswith(file, ".txt"), files)
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
28
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
29 for file in filtered_files
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
30 filename = directory_path * file
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
31 data = CSV.File(filename, delim='\t', header=2) |> DataFrame
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
32
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
33 # Extract α from filename
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
34 α, _, _ = extract_parameters(filename)
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
35
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
36
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
37 # Extract SSIM and PSNR columns starting from 1st iteration
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
38 ssim_values1 = Float64.(data[:, :ssim])
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
39 psnr_values1 = Float64.(data[:, :psnr])
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
40
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
41 # Extract SSIM and PSNR columns starting from 500th iteration
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
42 ssim_values500 = Float64.(data[mystart:end, :ssim])
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
43 psnr_values500 = Float64.(data[mystart:end, :psnr])
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
44
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
45 # Calculate mean and confidence intervals
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
46 ssim_mean1 = round(mean(ssim_values1), digits=4)
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
47 psnr_mean1 = round(mean(psnr_values1), digits=4)
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
48
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
49 ssim_mean500 = round(mean(ssim_values500), digits=4)
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
50 psnr_mean500 = round(mean(psnr_values500), digits=4)
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
51 ssim_std500 = round(std(ssim_values500), digits=4)
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
52 psnr_std500 = round(std(psnr_values500), digits=4)
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
53 n = length(ssim_values500)
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
54
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
55 ssim_ci_lower = round(ssim_mean500 - 1.96 * ssim_std500 / sqrt(n), digits=4)
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
56 ssim_ci_upper = round(ssim_mean500 + 1.96 * ssim_std500 / sqrt(n), digits=4)
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
57 psnr_ci_lower = round(psnr_mean500 - 1.96 * psnr_std500 / sqrt(n), digits=4)
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
58 psnr_ci_upper = round(psnr_mean500 + 1.96 * psnr_std500 / sqrt(n), digits=4)
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
59
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
60 ssim_ci = "$(ssim_ci_lower) - $(ssim_ci_upper)"
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
61 psnr_ci = "$(psnr_ci_lower) - $(psnr_ci_upper)"
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
62 experiment = "$(imname)"
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
63 algorithm = "$(algname)"
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
64
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
65 # Append results to DataFrame
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
66 push!(results, (experiment, α, algorithm, psnr_mean1, psnr_mean500, psnr_ci, ssim_mean1, ssim_mean500, ssim_ci))
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
67 end
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
68 end
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
69 end
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
70 sort!(results, [:experiment, :α])
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
71 csv_path = "./img/summarystats.csv"
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
72 if isfile(csv_path)
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
73 rm(csv_path)
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
74 end
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
75 CSV.write(csv_path, results)
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
76 end
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
77
aca9c90f151c Reduce dependencies; separate Stats from PlotResults (disabled by default to avoid heavy deps)
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
78 end

mercurial