borgend.py

Sun, 28 Jan 2018 19:27:34 +0000

author
Tuomo Valkonen <tuomov@iki.fi>
date
Sun, 28 Jan 2018 19:27:34 +0000
changeset 87
a214d475aa28
parent 85
56a000d15965
child 89
51cc2e25af38
permissions
-rwxr-xr-x

Better recovery from errors; fixes to potential race conditions in scheduler and repository queue

0
f5aecaad0bcf Some rough drafting
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
1 #!/usr/local/bin/python3
f5aecaad0bcf Some rough drafting
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
2
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
3 # Common modules
41
e2641cb9ca6d --no-tray command line option for running on non-MacOS systems
Tuomo Valkonen <tuomov@iki.fi>
parents: 40
diff changeset
4 import os
45
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 43
diff changeset
5 import sys
41
e2641cb9ca6d --no-tray command line option for running on non-MacOS systems
Tuomo Valkonen <tuomov@iki.fi>
parents: 40
diff changeset
6 import argparse
43
8f3ac19f11b6 Use platform package to detect whether to:
Tuomo Valkonen <tuomov@iki.fi>
parents: 41
diff changeset
7 import platform
80
a409242121d5 Better package-like organisation
Tuomo Valkonen <tuomov@iki.fi>
parents: 79
diff changeset
8 import logging
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
9 # Own modules needed at this stage
80
a409242121d5 Better package-like organisation
Tuomo Valkonen <tuomov@iki.fi>
parents: 79
diff changeset
10 import borgend.branding as branding
a409242121d5 Better package-like organisation
Tuomo Valkonen <tuomov@iki.fi>
parents: 79
diff changeset
11 import borgend.locations as locations
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 46
diff changeset
12
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 46
diff changeset
13 #
41
e2641cb9ca6d --no-tray command line option for running on non-MacOS systems
Tuomo Valkonen <tuomov@iki.fi>
parents: 40
diff changeset
14 # Argument processing
e2641cb9ca6d --no-tray command line option for running on non-MacOS systems
Tuomo Valkonen <tuomov@iki.fi>
parents: 40
diff changeset
15 #
e2641cb9ca6d --no-tray command line option for running on non-MacOS systems
Tuomo Valkonen <tuomov@iki.fi>
parents: 40
diff changeset
16
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
17 parser=argparse.ArgumentParser(
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
18 description=branding.appname_stylised + ": BorgBackup scheduler, queue, and tray icon.",
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
19 epilog='Configuration file location:\n\n %s\n ' % locations.cfgfile,
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
20 formatter_class=argparse.RawDescriptionHelpFormatter)
41
e2641cb9ca6d --no-tray command line option for running on non-MacOS systems
Tuomo Valkonen <tuomov@iki.fi>
parents: 40
diff changeset
21
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
22 parser.add_argument(
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
23 '--no-tray',
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
24 dest='notray',
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
25 action='store_true',
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
26 help='Do not show the tray icon')
41
e2641cb9ca6d --no-tray command line option for running on non-MacOS systems
Tuomo Valkonen <tuomov@iki.fi>
parents: 40
diff changeset
27
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
28 parser.add_argument(
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
29 '--debug',
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
30 dest='debug',
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
31 action='store_true',
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
32 help='Set logging level to debug')
73
4f0e9cf8f230 Added --debug switch to enable debug logging level
Tuomo Valkonen <tuomov@iki.fi>
parents: 52
diff changeset
33
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
34 args=parser.parse_args()
41
e2641cb9ca6d --no-tray command line option for running on non-MacOS systems
Tuomo Valkonen <tuomov@iki.fi>
parents: 40
diff changeset
35
e2641cb9ca6d --no-tray command line option for running on non-MacOS systems
Tuomo Valkonen <tuomov@iki.fi>
parents: 40
diff changeset
36 #
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
37 # Done parsing args, import our own modules, and launch everything
41
e2641cb9ca6d --no-tray command line option for running on non-MacOS systems
Tuomo Valkonen <tuomov@iki.fi>
parents: 40
diff changeset
38 #
e2641cb9ca6d --no-tray command line option for running on non-MacOS systems
Tuomo Valkonen <tuomov@iki.fi>
parents: 40
diff changeset
39
80
a409242121d5 Better package-like organisation
Tuomo Valkonen <tuomov@iki.fi>
parents: 79
diff changeset
40 import borgend.config as config
a409242121d5 Better package-like organisation
Tuomo Valkonen <tuomov@iki.fi>
parents: 79
diff changeset
41 import borgend.dreamtime as dreamtime
a409242121d5 Better package-like organisation
Tuomo Valkonen <tuomov@iki.fi>
parents: 79
diff changeset
42 import borgend.loggers as loggers
a409242121d5 Better package-like organisation
Tuomo Valkonen <tuomov@iki.fi>
parents: 79
diff changeset
43 from borgend.scheduler import Scheduler
a409242121d5 Better package-like organisation
Tuomo Valkonen <tuomov@iki.fi>
parents: 79
diff changeset
44 from borgend.repository import Repository
a409242121d5 Better package-like organisation
Tuomo Valkonen <tuomov@iki.fi>
parents: 79
diff changeset
45 from borgend.backup import Backup
31
b4b4bb7a2ec5 More logging detail
Tuomo Valkonen <tuomov@iki.fi>
parents: 10
diff changeset
46
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
47 logger=loggers.mainlogger
34
9fce700d42de Log window and other logging improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 33
diff changeset
48
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
49 if args.debug:
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
50 logger.setLevel(logging.DEBUG)
34
9fce700d42de Log window and other logging improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 33
diff changeset
51
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
52 tray = None
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
53 repos=[]
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
54 backups=[]
1
4cdc9c1f6b28 basic scheduler structure draft, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 0
diff changeset
55
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
56 try:
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
57 dreamtime.start_monitoring()
76
4b08fca3ce34 Dreamtime scheduling: discount system sleep periods
Tuomo Valkonen <tuomov@iki.fi>
parents: 74
diff changeset
58
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
59 scheduler = Scheduler()
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
60 scheduler.start()
40
cfbeeec8cb82 Improved exception reporting etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 39
diff changeset
61
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
62 repoconfigs=config.settings['repositories']
74
4f56142e7497 Separated repository configuration form backup configuration;
Tuomo Valkonen <tuomov@iki.fi>
parents: 73
diff changeset
63
87
a214d475aa28 Better recovery from errors; fixes to potential race conditions in scheduler and repository queue
Tuomo Valkonen <tuomov@iki.fi>
parents: 85
diff changeset
64 logger.info('Initialising repositories')
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
65 for i in range(len(repoconfigs)):
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
66 r=Repository(i, repoconfigs[i])
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
67 repos.append(r)
74
4f56142e7497 Separated repository configuration form backup configuration;
Tuomo Valkonen <tuomov@iki.fi>
parents: 73
diff changeset
68
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
69 backupconfigs=config.settings['backups']
1
4cdc9c1f6b28 basic scheduler structure draft, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 0
diff changeset
70
87
a214d475aa28 Better recovery from errors; fixes to potential race conditions in scheduler and repository queue
Tuomo Valkonen <tuomov@iki.fi>
parents: 85
diff changeset
71 logger.info('Initialising backups')
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
72 for i in range(len(backupconfigs)):
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
73 b=Backup(i, backupconfigs[i], scheduler)
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
74 backups.append(b)
3
4cad934aa9ce Can launch borg now; output not yet processed
Tuomo Valkonen <tuomov@iki.fi>
parents: 2
diff changeset
75
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
76 if args.notray or platform.system()!='Darwin':
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
77 # Wait for scheduler to finish
85
56a000d15965 On startup, for better scheduling, obtain previous backup time with 'borg list'
Tuomo Valkonen <tuomov@iki.fi>
parents: 82
diff changeset
78 run=scheduler.join
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
79 else:
82
4e7678dd7b42 Enable Ctrl+C in MacOS UI mode.
Tuomo Valkonen <tuomov@iki.fi>
parents: 80
diff changeset
80 # This is needed for Ctrl+C to work.
4e7678dd7b42 Enable Ctrl+C in MacOS UI mode.
Tuomo Valkonen <tuomov@iki.fi>
parents: 80
diff changeset
81 # TODO: proper exit handler, which seems to require
4e7678dd7b42 Enable Ctrl+C in MacOS UI mode.
Tuomo Valkonen <tuomov@iki.fi>
parents: 80
diff changeset
82 # ditching/forking/extending rumps to extend the NSApp class
4e7678dd7b42 Enable Ctrl+C in MacOS UI mode.
Tuomo Valkonen <tuomov@iki.fi>
parents: 80
diff changeset
83 from PyObjCTools.AppHelper import installMachInterrupt
4e7678dd7b42 Enable Ctrl+C in MacOS UI mode.
Tuomo Valkonen <tuomov@iki.fi>
parents: 80
diff changeset
84 installMachInterrupt()
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
85 # Start UI, and let it handle exit control
80
a409242121d5 Better package-like organisation
Tuomo Valkonen <tuomov@iki.fi>
parents: 79
diff changeset
86 from borgend.ui import BorgendTray
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
87 tray=BorgendTray(backups);
85
56a000d15965 On startup, for better scheduling, obtain previous backup time with 'borg list'
Tuomo Valkonen <tuomov@iki.fi>
parents: 82
diff changeset
88 run=tray.run
56a000d15965 On startup, for better scheduling, obtain previous backup time with 'borg list'
Tuomo Valkonen <tuomov@iki.fi>
parents: 82
diff changeset
89
56a000d15965 On startup, for better scheduling, obtain previous backup time with 'borg list'
Tuomo Valkonen <tuomov@iki.fi>
parents: 82
diff changeset
90 for r in repos:
56a000d15965 On startup, for better scheduling, obtain previous backup time with 'borg list'
Tuomo Valkonen <tuomov@iki.fi>
parents: 82
diff changeset
91 r.start()
56a000d15965 On startup, for better scheduling, obtain previous backup time with 'borg list'
Tuomo Valkonen <tuomov@iki.fi>
parents: 82
diff changeset
92
56a000d15965 On startup, for better scheduling, obtain previous backup time with 'borg list'
Tuomo Valkonen <tuomov@iki.fi>
parents: 82
diff changeset
93 for b in backups:
56a000d15965 On startup, for better scheduling, obtain previous backup time with 'borg list'
Tuomo Valkonen <tuomov@iki.fi>
parents: 82
diff changeset
94 b.start()
56a000d15965 On startup, for better scheduling, obtain previous backup time with 'borg list'
Tuomo Valkonen <tuomov@iki.fi>
parents: 82
diff changeset
95
56a000d15965 On startup, for better scheduling, obtain previous backup time with 'borg list'
Tuomo Valkonen <tuomov@iki.fi>
parents: 82
diff changeset
96 run()
9
aa121291eb0e Rumps/Mac UI stuff is fucked and disables ^C etc.; threading doesn't help
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
97
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
98 except Exception as err:
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
99 # TODO: Should write errors here to stderr;
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
100 # perhaps add an extra stderr logger for error level messages
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
101 logger.exception("Exception fell through: exiting")
9
aa121291eb0e Rumps/Mac UI stuff is fucked and disables ^C etc.; threading doesn't help
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
102
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
103 finally:
82
4e7678dd7b42 Enable Ctrl+C in MacOS UI mode.
Tuomo Valkonen <tuomov@iki.fi>
parents: 80
diff changeset
104 logger.debug("Exiting")
4e7678dd7b42 Enable Ctrl+C in MacOS UI mode.
Tuomo Valkonen <tuomov@iki.fi>
parents: 80
diff changeset
105
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
106 for b in backups:
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
107 b.terminate()
9
aa121291eb0e Rumps/Mac UI stuff is fucked and disables ^C etc.; threading doesn't help
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
108
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
109 for r in repos:
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
110 r.terminate()
9
aa121291eb0e Rumps/Mac UI stuff is fucked and disables ^C etc.; threading doesn't help
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
111
79
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
112 if tray:
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
113 tray.quit()
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
114 else:
b075b3db3044 Cleaned up module organisation to simplify borgend.py and not have to import it in other modules.
Tuomo Valkonen <tuomov@iki.fi>
parents: 78
diff changeset
115 logging.shutdown()
9
aa121291eb0e Rumps/Mac UI stuff is fucked and disables ^C etc.; threading doesn't help
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
116

mercurial