backup.py

Wed, 24 Jan 2018 09:19:42 +0000

author
Tuomo Valkonen <tuomov@iki.fi>
date
Wed, 24 Jan 2018 09:19:42 +0000
changeset 60
10bd7e3906d9
parent 58
170d69da51bb
child 61
bc6c3d74e6ea
permissions
-rw-r--r--

py2app fails on several dependencies (xdg, pyyaml) that are satisfied
(pip reports installed), so remove them; need to be installed manually

1
4cdc9c1f6b28 basic scheduler structure draft, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
1 #
4cdc9c1f6b28 basic scheduler structure draft, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
2 # Borgend Backup instance
4cdc9c1f6b28 basic scheduler structure draft, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
3 #
4cdc9c1f6b28 basic scheduler structure draft, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
4
2
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
5 import config
4
d72c4844e791 Better borg output processing and some logging
Tuomo Valkonen <tuomov@iki.fi>
parents: 3
diff changeset
6 import logging
5
4c5514b2fa76 basic schedule calculation
Tuomo Valkonen <tuomov@iki.fi>
parents: 4
diff changeset
7 import time
20
fdfbe5d7b677 Keychain support and random fixes
Tuomo Valkonen <tuomov@iki.fi>
parents: 17
diff changeset
8 import keyring
34
9fce700d42de Log window and other logging improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 32
diff changeset
9 import borgend
54
cfcaa5f6ba33 Basic repository queue
Tuomo Valkonen <tuomov@iki.fi>
parents: 50
diff changeset
10 import repository
2
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
11 from instance import BorgInstance
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
12 from threading import Thread, Lock, Condition
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
13 from scheduler import TerminableThread
2
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
14
34
9fce700d42de Log window and other logging improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 32
diff changeset
15 logger=borgend.logger.getChild(__name__)
31
b4b4bb7a2ec5 More logging detail
Tuomo Valkonen <tuomov@iki.fi>
parents: 30
diff changeset
16
45
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
17 # State
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
18 INACTIVE=0
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
19 SCHEDULED=1
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
20 QUEUED=2
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
21 ACTIVE=3
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
22 BUSY=4
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
23 OFFLINE=5
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
24 ERRORS=6
45
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
25
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
26 def combine_state(state1, state2):
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
27 return max(state1, state2)
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
28
6
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
29 loglevel_translation={
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
30 'CRITICAL': logging.CRITICAL,
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
31 'ERROR': logging.ERROR,
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
32 'WARNING': logging.WARNING,
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
33 'DEBUG': logging.DEBUG,
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
34 'INFO': logging.INFO
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
35 }
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
36
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
37 def translate_loglevel(x):
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
38 if x in loglevel_translation:
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
39 return loglevel_translation[x]
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
40 else:
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
41 return logging.ERROR
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
42
15
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
43 def safe_get_int(t, x):
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
44 if x in t:
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
45 tmp=t[x]
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
46 if isinstance(tmp, int):
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
47 return tmp
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
48 return None
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
49
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
50
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
51 class Backup(TerminableThread):
1
4cdc9c1f6b28 basic scheduler structure draft, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
52
2
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
53 def __decode_config(self, cfg):
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
54 loc0='backup target %d' % self.identifier
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
55
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
56 self._name=config.check_string(cfg, 'name', 'Name', loc0)
2
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
57
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
58 self.logger=logger.getChild(self._name)
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
59
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
60 self.loc='backup target "%s"' % self._name
2
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
61
54
cfcaa5f6ba33 Basic repository queue
Tuomo Valkonen <tuomov@iki.fi>
parents: 50
diff changeset
62 reponame=config.check_string(cfg, 'repository',
cfcaa5f6ba33 Basic repository queue
Tuomo Valkonen <tuomov@iki.fi>
parents: 50
diff changeset
63 'Target repository', self.loc)
cfcaa5f6ba33 Basic repository queue
Tuomo Valkonen <tuomov@iki.fi>
parents: 50
diff changeset
64
cfcaa5f6ba33 Basic repository queue
Tuomo Valkonen <tuomov@iki.fi>
parents: 50
diff changeset
65 self.repository=repository.get_controller(reponame)
2
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
66
3
4cad934aa9ce Can launch borg now; output not yet processed
Tuomo Valkonen <tuomov@iki.fi>
parents: 2
diff changeset
67 self.archive_prefix=config.check_string(cfg, 'archive_prefix',
4cad934aa9ce Can launch borg now; output not yet processed
Tuomo Valkonen <tuomov@iki.fi>
parents: 2
diff changeset
68 'Archive prefix', self.loc)
4cad934aa9ce Can launch borg now; output not yet processed
Tuomo Valkonen <tuomov@iki.fi>
parents: 2
diff changeset
69
2
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
70 self.archive_template=config.check_string(cfg, 'archive_template',
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
71 'Archive template', self.loc)
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
72
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
73 self.backup_interval=config.check_nonneg_int(cfg, 'backup_interval',
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
74 'Backup interval', self.loc,
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
75 config.defaults['backup_interval'])
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
76
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
77 self.retry_interval=config.check_nonneg_int(cfg, 'retry_interval',
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
78 'Retry interval', self.loc,
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
79 config.defaults['retry_interval'])
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
80
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
81 self.paths=config.check_nonempty_list_of_strings(cfg, 'paths', 'Paths', self.loc)
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
82
6
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
83 self.common_parameters=config.check_list_of_dicts(cfg, 'common_parameters',
3
4cad934aa9ce Can launch borg now; output not yet processed
Tuomo Valkonen <tuomov@iki.fi>
parents: 2
diff changeset
84 'Borg parameters', self.loc,
4cad934aa9ce Can launch borg now; output not yet processed
Tuomo Valkonen <tuomov@iki.fi>
parents: 2
diff changeset
85 default=[])
4cad934aa9ce Can launch borg now; output not yet processed
Tuomo Valkonen <tuomov@iki.fi>
parents: 2
diff changeset
86
6
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
87 self.create_parameters=config.check_list_of_dicts(cfg, 'create_parameters',
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
88 'Create parameters', self.loc,
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
89 default=[])
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
90
3
4cad934aa9ce Can launch borg now; output not yet processed
Tuomo Valkonen <tuomov@iki.fi>
parents: 2
diff changeset
91 self.prune_parameters=config.check_list_of_dicts(cfg, 'prune_parameters',
6
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
92 'Prune parameters', self.loc,
3
4cad934aa9ce Can launch borg now; output not yet processed
Tuomo Valkonen <tuomov@iki.fi>
parents: 2
diff changeset
93 default=[])
2
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
94
21
c36e549a7f12 Errors as rumps notifications
Tuomo Valkonen <tuomov@iki.fi>
parents: 20
diff changeset
95 self.__keychain_account=config.check_string(cfg, 'keychain_account',
c36e549a7f12 Errors as rumps notifications
Tuomo Valkonen <tuomov@iki.fi>
parents: 20
diff changeset
96 'Keychain account', self.loc,
c36e549a7f12 Errors as rumps notifications
Tuomo Valkonen <tuomov@iki.fi>
parents: 20
diff changeset
97 default='')
c36e549a7f12 Errors as rumps notifications
Tuomo Valkonen <tuomov@iki.fi>
parents: 20
diff changeset
98
32
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
99 self.__passphrase=None
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
100
30
3dd525652dc8 Added no_quit_menu_entry option
Tuomo Valkonen <tuomov@iki.fi>
parents: 21
diff changeset
101 if config.settings['extract_passphrases_at_startup']:
32
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
102 try:
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
103 self.extract_passphrase()
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
104 except Exception:
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
105 pass
20
fdfbe5d7b677 Keychain support and random fixes
Tuomo Valkonen <tuomov@iki.fi>
parents: 17
diff changeset
106
21
c36e549a7f12 Errors as rumps notifications
Tuomo Valkonen <tuomov@iki.fi>
parents: 20
diff changeset
107 def extract_passphrase(self):
c36e549a7f12 Errors as rumps notifications
Tuomo Valkonen <tuomov@iki.fi>
parents: 20
diff changeset
108 acc=self.__keychain_account
32
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
109 if not self.__passphrase:
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
110 if acc and acc!='':
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
111 self.logger.debug('Requesting passphrase')
32
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
112 try:
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
113 pw=keyring.get_password("borg-backup", acc)
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
114 except Exception as err:
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
115 self.logger.error('Failed to retrieve passphrase')
32
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
116 raise err
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
117 else:
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
118 self.logger.debug('Received passphrase')
32
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
119 self.__passphrase=pw
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
120 else:
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
121 self.__passphrase=None
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
122 return self.__passphrase
2
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
123
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
124 def __init__(self, identifier, cfg, scheduler):
2
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
125 self.identifier=identifier
1
4cdc9c1f6b28 basic scheduler structure draft, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents:
diff changeset
126 self.config=config
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
127 self.__status_update_callback=None
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
128 self.scheduler=scheduler
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
129 self.logger=None # setup up __decode_config once backup name is known
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
130
3
4cad934aa9ce Can launch borg now; output not yet processed
Tuomo Valkonen <tuomov@iki.fi>
parents: 2
diff changeset
131 self.borg_instance=None
7
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
132 self.thread_log=None
8
7b2d2eac6a48 basic scheduling
Tuomo Valkonen <tuomov@iki.fi>
parents: 7
diff changeset
133 self.thread_res=None
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
134
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
135 self.current_operation=None
10
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
136 self.scheduled_operation=None
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
137 self.lastrun_when=None
45
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
138 self.state=INACTIVE
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
139
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
140 self.__decode_config(cfg)
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
141
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
142 super().__init__(target = self.__main_thread, name = self._name)
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
143 self.daemon=True
3
4cad934aa9ce Can launch borg now; output not yet processed
Tuomo Valkonen <tuomov@iki.fi>
parents: 2
diff changeset
144
7
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
145 def is_running(self):
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
146 with self._cond:
8
7b2d2eac6a48 basic scheduling
Tuomo Valkonen <tuomov@iki.fi>
parents: 7
diff changeset
147 running=self.__is_running_unlocked()
7b2d2eac6a48 basic scheduling
Tuomo Valkonen <tuomov@iki.fi>
parents: 7
diff changeset
148 return running
7b2d2eac6a48 basic scheduling
Tuomo Valkonen <tuomov@iki.fi>
parents: 7
diff changeset
149
7b2d2eac6a48 basic scheduling
Tuomo Valkonen <tuomov@iki.fi>
parents: 7
diff changeset
150 def __is_running_unlocked(self):
7b2d2eac6a48 basic scheduling
Tuomo Valkonen <tuomov@iki.fi>
parents: 7
diff changeset
151 running=self.current_operation
7b2d2eac6a48 basic scheduling
Tuomo Valkonen <tuomov@iki.fi>
parents: 7
diff changeset
152 if not running:
7b2d2eac6a48 basic scheduling
Tuomo Valkonen <tuomov@iki.fi>
parents: 7
diff changeset
153 # Consistency check
7b2d2eac6a48 basic scheduling
Tuomo Valkonen <tuomov@iki.fi>
parents: 7
diff changeset
154 assert((not self.borg_instance and not self.thread_log and
7b2d2eac6a48 basic scheduling
Tuomo Valkonen <tuomov@iki.fi>
parents: 7
diff changeset
155 not self.thread_res))
7
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
156 return running
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
157
3
4cad934aa9ce Can launch borg now; output not yet processed
Tuomo Valkonen <tuomov@iki.fi>
parents: 2
diff changeset
158 def __block_when_running(self):
7
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
159 running=self.is_running()
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
160 assert(not running)
2
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
161
7
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
162 def __log_listener(self):
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
163 self.logger.debug('Log listener thread waiting for entries')
7
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
164 success=True
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
165 for status in iter(self.borg_instance.read_log, None):
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
166 self.logger.debug(str(status))
4
d72c4844e791 Better borg output processing and some logging
Tuomo Valkonen <tuomov@iki.fi>
parents: 3
diff changeset
167 t=status['type']
15
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
168
21
c36e549a7f12 Errors as rumps notifications
Tuomo Valkonen <tuomov@iki.fi>
parents: 20
diff changeset
169 errors_this_message=None
15
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
170 callback=None
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
171
4
d72c4844e791 Better borg output processing and some logging
Tuomo Valkonen <tuomov@iki.fi>
parents: 3
diff changeset
172 if t=='progress_percent':
15
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
173 current=safe_get_int(status, 'current')
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
174 total=safe_get_int(status, 'total')
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
175 if current is not None and total is not None:
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
176 with self._cond:
17
5f6aa8830cea Python is a mess..
Tuomo Valkonen <tuomov@iki.fi>
parents: 16
diff changeset
177 self.current_operation['progress_current']=current
5f6aa8830cea Python is a mess..
Tuomo Valkonen <tuomov@iki.fi>
parents: 16
diff changeset
178 self.current_operation['progress_total']=total
14
5a988a2c2624 UI: progress percentange support (Borg doesn't seem to be reporting) + error indicator in tray: B?
Tuomo Valkonen <tuomov@iki.fi>
parents: 12
diff changeset
179 status, callback=self.__status_unlocked()
15
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
180
4
d72c4844e791 Better borg output processing and some logging
Tuomo Valkonen <tuomov@iki.fi>
parents: 3
diff changeset
181 elif t=='archive_progress':
15
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
182 original_size=safe_get_int(status, 'original_size')
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
183 compressed_size=safe_get_int(status, 'compressed_size')
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
184 deduplicated_size=safe_get_int(status, 'deduplicated_size')
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
185 if original_size is not None and original_size is not None and deduplicated_size is not None:
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
186 with self._cond:
17
5f6aa8830cea Python is a mess..
Tuomo Valkonen <tuomov@iki.fi>
parents: 16
diff changeset
187 self.current_operation['original_size']=original_size
5f6aa8830cea Python is a mess..
Tuomo Valkonen <tuomov@iki.fi>
parents: 16
diff changeset
188 self.current_operation['compressed_size']=compressed_size
5f6aa8830cea Python is a mess..
Tuomo Valkonen <tuomov@iki.fi>
parents: 16
diff changeset
189 self.current_operation['deduplicated_size']=deduplicated_size
15
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
190 status, callback=self.__status_unlocked()
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
191
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
192 elif t=='progress_message':
4
d72c4844e791 Better borg output processing and some logging
Tuomo Valkonen <tuomov@iki.fi>
parents: 3
diff changeset
193 pass
15
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
194
4
d72c4844e791 Better borg output processing and some logging
Tuomo Valkonen <tuomov@iki.fi>
parents: 3
diff changeset
195 elif t=='file_status':
d72c4844e791 Better borg output processing and some logging
Tuomo Valkonen <tuomov@iki.fi>
parents: 3
diff changeset
196 pass
15
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
197
4
d72c4844e791 Better borg output processing and some logging
Tuomo Valkonen <tuomov@iki.fi>
parents: 3
diff changeset
198 elif t=='log_message':
6
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
199 if 'levelname' not in status:
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
200 status['levelname']='ERROR'
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
201 if 'message' not in status:
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
202 status['message']='UNKNOWN'
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
203 if 'name' not in status:
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
204 status['name']='borg'
15
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
205 lvl=translate_loglevel(status['levelname'])
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
206 self.logger.log(lvl, status['name'] + ': ' + status['message'])
15
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
207 if lvl>=logging.WARNING:
21
c36e549a7f12 Errors as rumps notifications
Tuomo Valkonen <tuomov@iki.fi>
parents: 20
diff changeset
208 errors_this_message=status
45
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
209 state=ERRORS
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
210 if ('msgid' in status and
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
211 (status['msgid']=='LockTimeout' or # observed in reality
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
212 status['msgid']=='LockErrorT' or # in docs
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
213 status['msgid']=='LockErrorT')): # in docs
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
214 state=BUSY
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
215 with self._cond:
45
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
216 self.state=combine_state(self.state, state)
21
c36e549a7f12 Errors as rumps notifications
Tuomo Valkonen <tuomov@iki.fi>
parents: 20
diff changeset
217 status, callback=self.__status_unlocked()
c36e549a7f12 Errors as rumps notifications
Tuomo Valkonen <tuomov@iki.fi>
parents: 20
diff changeset
218 else:
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
219 self.logger.debug('Unrecognised log entry %s' % str(status))
15
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
220
aaf1a281a3fe More error reporting
Tuomo Valkonen <tuomov@iki.fi>
parents: 14
diff changeset
221 if callback:
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
222 callback(status, errors=errors_this_message)
6
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
223
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
224 self.logger.debug('Waiting for borg subprocess to terminate in log thread')
4
d72c4844e791 Better borg output processing and some logging
Tuomo Valkonen <tuomov@iki.fi>
parents: 3
diff changeset
225
6
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
226 self.borg_instance.wait()
46c89e5a219f subprocess improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 5
diff changeset
227
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
228 self.logger.debug('Borg subprocess terminated; terminating log listener thread')
7
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
229
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
230 def __result_listener(self):
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
231 # self.state=ACTIVE
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
232 # with self._cond:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
233 # status, callback=self.__status_unlocked()
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
234 # if callback:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
235 # callback(status)
10
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
236
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
237 self.logger.debug('Result listener thread waiting for result')
7
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
238
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
239 res=self.borg_instance.read_result()
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
240
45
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
241 # Finish processing remaining errors
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
242 self.thread_log.join()
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
243
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
244 with self._cond:
45
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
245 state=self.state
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
246
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
247 # If there were no errors, reset back to INACTIVE state
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
248 if state==ACTIVE:
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
249 state=INACTIVE
7
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
250
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
251 self.logger.debug('Borg result: %s' % str(res))
7
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
252
45
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
253 if res is None and state==INACTIVE:
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
254 self.logger.error('No result from borg despite no error in log')
45
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
255 state=ERRORS
7
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
256
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
257 self.logger.debug('Waiting for borg subprocess to terminate in result thread')
7
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
258
45
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
259 if not self.borg_instance.wait():
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
260 self.logger.error('Borg subprocess did not terminate')
45
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
261 state=combine_state(state, ERRORS)
7
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
262
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
263 self.logger.debug('Borg subprocess terminated (end state %s); terminating result listener thread' % str(state))
12
16a8c63344c0 Fixed retry scheduling
Tuomo Valkonen <tuomov@iki.fi>
parents: 10
diff changeset
264
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
265 with self._cond:
10
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
266 if self.current_operation['operation']=='create':
32
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
267 self.lastrun_when=self.current_operation['when_monotonic']
7
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
268 self.thread_res=None
12
16a8c63344c0 Fixed retry scheduling
Tuomo Valkonen <tuomov@iki.fi>
parents: 10
diff changeset
269 self.thread_log=None
16a8c63344c0 Fixed retry scheduling
Tuomo Valkonen <tuomov@iki.fi>
parents: 10
diff changeset
270 self.borg_instance=None
16a8c63344c0 Fixed retry scheduling
Tuomo Valkonen <tuomov@iki.fi>
parents: 10
diff changeset
271 self.current_operation=None
48
be3ed25df789 reorder-fix
Tuomo Valkonen <tuomov@iki.fi>
parents: 46
diff changeset
272 self.state=state
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
273 self.__update_status()
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
274 self._cond.notify()
7
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
275
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
276 def __do_launch(self, op, archive_or_repository, *args):
32
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
277 passphrase=self.extract_passphrase()
21
c36e549a7f12 Errors as rumps notifications
Tuomo Valkonen <tuomov@iki.fi>
parents: 20
diff changeset
278
10
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
279 inst=BorgInstance(op['operation'], archive_or_repository, *args)
32
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
280 inst.launch(passphrase=passphrase)
3
4cad934aa9ce Can launch borg now; output not yet processed
Tuomo Valkonen <tuomov@iki.fi>
parents: 2
diff changeset
281
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
282 self.logger.debug('Creating listener threads')
31
b4b4bb7a2ec5 More logging detail
Tuomo Valkonen <tuomov@iki.fi>
parents: 30
diff changeset
283
7
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
284 t_log=Thread(target=self.__log_listener)
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
285 t_log.daemon=True
2
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
286
7
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
287 t_res=Thread(target=self.__result_listener)
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
288 t_res.daemon=True
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
289
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
290 self.thread_log=t_log
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
291 self.thread_res=t_res
3
4cad934aa9ce Can launch borg now; output not yet processed
Tuomo Valkonen <tuomov@iki.fi>
parents: 2
diff changeset
292 self.borg_instance=inst
4cad934aa9ce Can launch borg now; output not yet processed
Tuomo Valkonen <tuomov@iki.fi>
parents: 2
diff changeset
293
7
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
294 t_log.start()
e189d4a6cb8c Also listen to stdout
Tuomo Valkonen <tuomov@iki.fi>
parents: 6
diff changeset
295 t_res.start()
3
4cad934aa9ce Can launch borg now; output not yet processed
Tuomo Valkonen <tuomov@iki.fi>
parents: 2
diff changeset
296
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
297 def __launch(self, op):
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
298 self.logger.debug("Launching '%s'" % op['operation'])
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
299
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
300 if op['operation']=='create':
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
301 archive="%s::%s%s" % (self.repository.repository_name,
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
302 self.archive_prefix,
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
303 self.archive_template)
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
304
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
305 self.__do_launch(op, archive,
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
306 self.common_parameters+self.create_parameters,
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
307 self.paths)
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
308 elif op['operation']=='prune':
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
309 self.__do_launch(op, self.repository.repository_name,
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
310 ([{'prefix': self.archive_prefix}] +
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
311 self.common_parameters +
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
312 self.prune_parameters))
8
7b2d2eac6a48 basic scheduling
Tuomo Valkonen <tuomov@iki.fi>
parents: 7
diff changeset
313 else:
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
314 raise NotImplementedError("Invalid operation '%s'" % op['operation'])
2
e343594c0014 basic config processing
Tuomo Valkonen <tuomov@iki.fi>
parents: 1
diff changeset
315
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
316 def __launch_check(self):
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
317 op=self.scheduled_operation
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
318 if not op:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
319 self.logger.debug("Queued operation aborted")
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
320 else:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
321 self.scheduled_operation=None
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
322
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
323 self.__launch(op)
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
324
58
170d69da51bb launch error handling fix
Tuomo Valkonen <tuomov@iki.fi>
parents: 55
diff changeset
325 self.current_operation=op
170d69da51bb launch error handling fix
Tuomo Valkonen <tuomov@iki.fi>
parents: 55
diff changeset
326 self.current_operation['when_monotonic']=time.monotonic()
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
327 self.state=ACTIVE
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
328 self.__update_status()
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
329
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
330
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
331 def __main_thread(self):
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
332 with self._cond:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
333 try:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
334 while not self._terminate:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
335 self.__main_thread_wait_finish()
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
336 if not self._terminate:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
337 self.__main_thread_wait_schedule()
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
338 if not self._terminate:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
339 self.__main_thread_queue_and_launch()
32
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
340 except Exception as err:
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
341 self.logger.exception("Error with backup '%s'" % self._name)
32
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
342 self.lastrun_when=time.monotonic()
45
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
343 self.state=ERRORS
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
344 self.scheduled_operation=None
3
4cad934aa9ce Can launch borg now; output not yet processed
Tuomo Valkonen <tuomov@iki.fi>
parents: 2
diff changeset
345
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
346 # Clean up to terminate: kill borg instance and communication threads
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
347 if self.borg_instance:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
348 self.logger.debug("Terminating a borg instance")
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
349 self.borg_instance.terminate()
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
350
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
351 # Store threads to use outside lock
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
352 thread_log=self.thread_log
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
353 thread_res=self.thread_res
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
354
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
355 self.logger.debug("Waiting for log and result threads to terminate")
8
7b2d2eac6a48 basic scheduling
Tuomo Valkonen <tuomov@iki.fi>
parents: 7
diff changeset
356
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
357 if thread_log:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
358 thread_log.join()
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
359
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
360 if thread_res:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
361 thread_res.join()
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
362
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
363
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
364 # Main thread/1. Wait while a current operation is running
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
365 def __main_thread_wait_finish(self):
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
366 while self.current_operation and not self._terminate:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
367 self.logger.debug("Waiting for current operation to finish")
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
368 self._cond.wait()
3
4cad934aa9ce Can launch borg now; output not yet processed
Tuomo Valkonen <tuomov@iki.fi>
parents: 2
diff changeset
369
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
370 # Main thread/2. Schedule next operation if there is no manually
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
371 # requested one
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
372 def __main_thread_wait_schedule(self):
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
373 op=None
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
374 if not self.scheduled_operation:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
375 op=self.__next_operation_unlocked()
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
376 if op:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
377 now=time.monotonic()
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
378 delay=max(0, op['when_monotonic']-now)
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
379 self.logger.info("Scheduling '%s' (detail: %s) in %d seconds" %
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
380 (op['operation'], op['detail'], delay))
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
381
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
382 self.scheduled_operation=op
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
383 self.state=combine_state(self.state, SCHEDULED)
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
384 self.__update_status()
3
4cad934aa9ce Can launch borg now; output not yet processed
Tuomo Valkonen <tuomov@iki.fi>
parents: 2
diff changeset
385
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
386 # Wait under scheduled wait
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
387 self.scheduler.wait_until(now+delay, self._cond, self._name)
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
388 else:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
389 # Nothing scheduled - just wait
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
390 self.logger.debug("Waiting for manual scheduling")
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
391 self._cond.wait()
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
392
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
393 # Main thread/3. If there is a scheduled operation (it might have been
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
394 # changed manually from 'op' created in __main_thread_wait_schedule above),
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
395 # queue it on the repository, and launch the operation once repository
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
396 # available
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
397 def __main_thread_queue_and_launch(self):
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
398 if self.scheduled_operation:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
399 self.logger.debug("Queuing")
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
400 self.state=combine_state(self.state, QUEUED)
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
401 self.__update_status()
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
402 self.repository.queue_action(self._cond,
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
403 action=self.__launch_check,
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
404 name=self._name)
8
7b2d2eac6a48 basic scheduling
Tuomo Valkonen <tuomov@iki.fi>
parents: 7
diff changeset
405
7b2d2eac6a48 basic scheduling
Tuomo Valkonen <tuomov@iki.fi>
parents: 7
diff changeset
406 def __next_operation_unlocked(self):
7b2d2eac6a48 basic scheduling
Tuomo Valkonen <tuomov@iki.fi>
parents: 7
diff changeset
407 # TODO: pruning as well
5
4c5514b2fa76 basic schedule calculation
Tuomo Valkonen <tuomov@iki.fi>
parents: 4
diff changeset
408 now=time.monotonic()
32
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
409 if not self.lastrun_when:
10
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
410 initial_interval=self.retry_interval
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
411 if initial_interval==0:
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
412 initial_interval=self.backup_interval
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
413 if initial_interval==0:
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
414 return None
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
415 else:
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
416 return {'operation': 'create',
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
417 'detail': 'initial',
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
418 'when_monotonic': now+initial_interval}
45
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
419 elif self.state>=BUSY:
10
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
420 if self.retry_interval==0:
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
421 return None
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
422 else:
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
423 return {'operation': 'create',
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
424 'detail': 'retry',
32
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
425 'when_monotonic': self.lastrun_when+self.retry_interval}
5
4c5514b2fa76 basic schedule calculation
Tuomo Valkonen <tuomov@iki.fi>
parents: 4
diff changeset
426 else:
4c5514b2fa76 basic schedule calculation
Tuomo Valkonen <tuomov@iki.fi>
parents: 4
diff changeset
427 if self.backup_interval==0:
10
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
428 return None
5
4c5514b2fa76 basic schedule calculation
Tuomo Valkonen <tuomov@iki.fi>
parents: 4
diff changeset
429 else:
10
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
430 return {'operation': 'create',
38
085a635f23f5 Improved error indicators
Tuomo Valkonen <tuomov@iki.fi>
parents: 34
diff changeset
431 'detail': 'normal',
32
06fc14211ba9 Error handling improvements
Tuomo Valkonen <tuomov@iki.fi>
parents: 31
diff changeset
432 'when_monotonic': self.lastrun_when+self.backup_interval}
10
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
433
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
434 def __status_unlocked(self):
21
c36e549a7f12 Errors as rumps notifications
Tuomo Valkonen <tuomov@iki.fi>
parents: 20
diff changeset
435 callback=self.__status_update_callback
38
085a635f23f5 Improved error indicators
Tuomo Valkonen <tuomov@iki.fi>
parents: 34
diff changeset
436
10
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
437 if self.current_operation:
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
438 status=self.current_operation
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
439 status['type']='current'
38
085a635f23f5 Improved error indicators
Tuomo Valkonen <tuomov@iki.fi>
parents: 34
diff changeset
440 # Errors should be set by listeners
10
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
441 else:
38
085a635f23f5 Improved error indicators
Tuomo Valkonen <tuomov@iki.fi>
parents: 34
diff changeset
442 if self.scheduled_operation:
085a635f23f5 Improved error indicators
Tuomo Valkonen <tuomov@iki.fi>
parents: 34
diff changeset
443 status=self.scheduled_operation
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
444 if self.state==QUEUED:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
445 status['type']='queued'
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
446 else:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
447 status['type']='scheduled'
38
085a635f23f5 Improved error indicators
Tuomo Valkonen <tuomov@iki.fi>
parents: 34
diff changeset
448 else:
085a635f23f5 Improved error indicators
Tuomo Valkonen <tuomov@iki.fi>
parents: 34
diff changeset
449 status={'type': 'nothing'}
10
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
450
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
451 status['name']=self._name
45
aa2a95dc6093 Further improvements to state reporting; indicate busyness if repository lock cannot be acquired
Tuomo Valkonen <tuomov@iki.fi>
parents: 38
diff changeset
452 status['state']=self.state
38
085a635f23f5 Improved error indicators
Tuomo Valkonen <tuomov@iki.fi>
parents: 34
diff changeset
453
085a635f23f5 Improved error indicators
Tuomo Valkonen <tuomov@iki.fi>
parents: 34
diff changeset
454 if 'detail' not in status:
085a635f23f5 Improved error indicators
Tuomo Valkonen <tuomov@iki.fi>
parents: 34
diff changeset
455 status['detail']='NONE'
10
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
456
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
457 if 'when_monotonic' in status:
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
458 status['when']=(status['when_monotonic']
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
459 -time.monotonic()+time.time())
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
460
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
461 return status, callback
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
462
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
463 def __update_status(self):
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
464 status, callback = self.__status_unlocked()
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
465 if callback:
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
466 #self._cond.release()
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
467 try:
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
468 callback(status)
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
469 except Exception:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
470 self.logger.exception("Status update error")
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
471 #finally:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
472 # self._cond.acquire()
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
473
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
474 #
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
475 # Interface functions
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
476 #
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
477
21
c36e549a7f12 Errors as rumps notifications
Tuomo Valkonen <tuomov@iki.fi>
parents: 20
diff changeset
478 def set_status_update_callback(self, callback):
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
479 with self._cond:
21
c36e549a7f12 Errors as rumps notifications
Tuomo Valkonen <tuomov@iki.fi>
parents: 20
diff changeset
480 self.__status_update_callback=callback
10
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
481
49
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
482 def status(self):
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
483 with self._cond:
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
484 res=self.__status_unlocked()
db33dfa64ad6 Improved scheduler
Tuomo Valkonen <tuomov@iki.fi>
parents: 48
diff changeset
485 return res[0]
10
76dbfb06eba0 Semi-working menu items.
Tuomo Valkonen <tuomov@iki.fi>
parents: 8
diff changeset
486
55
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
487 def create(self):
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
488 op={'operation': 'create', 'detail': 'manual'}
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
489 with self._cond:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
490 self.scheduled_operation=op
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
491 self._cond.notify()
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
492
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
493 def prune(self):
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
494 op={'operation': 'prune', 'detail': 'manual'}
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
495 with self._cond:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
496 self.scheduled_operation=op
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
497 self._cond.notify()
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
498
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
499 # TODO: Decide exact (manual) abort mechanism. Perhaps two stages
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
500 def abort(self):
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
501 with self._cond:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
502 if self.borg_instance:
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
503 self.borg_instance.terminate()
407af23d16bb Improved backup main thread loop, etc.
Tuomo Valkonen <tuomov@iki.fi>
parents: 54
diff changeset
504

mercurial