Bugzilla – Bug 860
waf sometimes dies while executing ns3header or gen_ns3_module_header tasks in case of parallel jobs
Last modified: 2010-04-06 03:39:33 UTC
The reason for this is that several headers are listed several times as source for some ns3header. For example, in "src/devices/wifi/wscript" (as of revision b920710704c9): 65 headers = bld.new_task_gen('ns3header') 66 headers.module = 'wifi' 67 headers.source = [ ... 75 'yans-wifi-channel.h', 76 'wifi-phy.h', <-- first ... 92 'nqap-wifi-mac.h', 93 'wifi-phy.h', <-- second ... 118 ] Waf tries to copy a new ns3header in one thread, while another thread already copied it and marked as read-only. In ns-3-dev there are a few such cases, so buildbot doesn't detect such failures. Example waf traceback: Build failed Traceback (most recent call last): File "/home/me/ns3nonfree/.waf-1.5.13-4340ef810f7db5d61c618fdf96cb5695/wafadmin/Runner.py", line 42, in loop else:ret=tsk.call_run() File "/home/me/ns3nonfree/.waf-1.5.13-4340ef810f7db5d61c618fdf96cb5695/wafadmin/Task.py", line 258, in call_run return self.run() File "<string>", line 177, in run File "/usr/lib64/python2.6/shutil.py", line 99, in copy2 copyfile(src, dst) File "/usr/lib64/python2.6/shutil.py", line 53, in copyfile fdst = open(dst, 'wb') IOError: [Errno 13] Permission denied: 'debug/ns3/ipv4-l3-protocol.h' There are at least 2 possible fixes: 1) remove all such duplicates from all wscripts 2) remove all such duplicates in ns3header_taskgen::apply() and in ns3moduleheader_taskgen::apply() functions I'd prefer 2), because it allows to legally do things like: headers.source = [ # required by module1 'header1.h', 'header2.h', # required by module2 'header2.h', 'header3.h', ] to better keep track of dependencies and then unexport unused headers.
Created attachment 807 [details] proposed fix
(In reply to comment #1) > Created an attachment (id=807) [details] > proposed fix I am fine with the proposed change, but I think you could use the builtin (python 2.4+) set type for a simpler fix. E.g.: - for filename in self.to_list(self.source): + for filename in set(self.to_list(self.source)): I don't think at this point we really need Python 2.3 compatibility. If we did, we'd just need to add this to the wscript: try: set except NameError: from sets import Set as set # Python 2.3 fallback
(In reply to comment #2) > I am fine with the proposed change, but I think you could use the builtin > (python 2.4+) set type for a simpler fix. E.g.: > > - for filename in self.to_list(self.source): > + for filename in set(self.to_list(self.source)): > > I don't think at this point we really need Python 2.3 compatibility. If we > did, we'd just need to add this to the wscript: > > try: > set > except NameError: > from sets import Set as set # Python 2.3 fallback Thank you for a quick reply! I agree with your simpler fix: it's looks clearer. I've never thought about python-2.3 compatibility. But waf states, that it is compatible with it, so we'd better keep the compatibility too.
Created attachment 808 [details] proposed fix based on sets
(In reply to comment #4) > Created an attachment (id=808) [details] > proposed fix based on sets Looks good. Can you commit?
(In reply to comment #5) > (In reply to comment #4) > > Created an attachment (id=808) [details] [details] > > proposed fix based on sets > > Looks good. Can you commit? Ok. Yes, I can. According to release schedule, ns-3-dev is open for such commits, so changeset c737d0a0e9a0.