Comments on: Direct IO in Python http://www.alexonlinux.com/direct-io-in-python Sun, 05 Feb 2012 21:17:46 +0000 hourly 1 http://wordpress.org/?v= By: Alexander Sandler http://www.alexonlinux.com/direct-io-in-python/comment-page-1#comment-28437 Alexander Sandler Sun, 05 Feb 2012 21:05:35 +0000 http://www.alexonlinux.com/?p=1684#comment-28437 <a href='#comment-27746' rel="nofollow">@Sebastian</a> My pleasure. Please come again :-)<div class="comment-remix-meta"><a href="#" class="replyto" onclick="replyto('28437','Alexander Sandler'); return false;">Reply</a> - <a href="#" class="quote" onclick="quote('28437','Alexander Sandler','<a href=\'#comment-27746\' rel=\"nofollow\">@Sebastian<\/a>\r\nMy pleasure. Please come again :-)'); return false;">Quote</a></div> @Sebastian
My pleasure. Please come again :-)
Reply – Quote
]]> By: Sebastian http://www.alexonlinux.com/direct-io-in-python/comment-page-1#comment-27746 Sebastian Sat, 19 Nov 2011 15:24:27 +0000 http://www.alexonlinux.com/?p=1684#comment-27746 Thanks for the hint to alignment as a reason for "Invalid argument"!<div class="comment-remix-meta"><a href="#" class="replyto" onclick="replyto('27746','Sebastian'); return false;">Reply</a> - <a href="#" class="quote" onclick="quote('27746','Sebastian','Thanks for the hint to alignment as a reason for \"Invalid argument\"!'); return false;">Quote</a></div> Thanks for the hint to alignment as a reason for “Invalid argument”!
Reply – Quote
]]> By: Alexander Sandler http://www.alexonlinux.com/direct-io-in-python/comment-page-1#comment-27187 Alexander Sandler Mon, 19 Sep 2011 16:27:02 +0000 http://www.alexonlinux.com/?p=1684#comment-27187 <a href='#comment-27184' rel="nofollow">@Evan Jones</a> Good. Very interesting reading. Thanks a lot.<div class="comment-remix-meta"><a href="#" class="replyto" onclick="replyto('27187','Alexander Sandler'); return false;">Reply</a> - <a href="#" class="quote" onclick="quote('27187','Alexander Sandler','<a href=\'#comment-27184\' rel=\"nofollow\">@Evan Jones<\/a>\r\nGood. Very interesting reading. Thanks a lot.'); return false;">Quote</a></div> @Evan Jones
Good. Very interesting reading. Thanks a lot.
Reply – Quote
]]> By: Evan Jones http://www.alexonlinux.com/direct-io-in-python/comment-page-1#comment-27184 Evan Jones Mon, 19 Sep 2011 11:11:39 +0000 http://www.alexonlinux.com/?p=1684#comment-27184 <a href='#comment-27168' rel="nofollow">@Alexander Sandler</a> - I actually did, but forgot to post something here. See <a href="http://www.evanjones.ca/durable-writes.html" rel="nofollow">Making Writes Durable</a>. This is unfortunately written from the perspective of the underlying C system calls, and doesn't discuss Python at all. However, it is relevant provided that you use the correct Python function calls. http://www.evanjones.ca/durable-writes.html<div class="comment-remix-meta"><a href="#" class="replyto" onclick="replyto('27184','Evan Jones'); return false;">Reply</a> - <a href="#" class="quote" onclick="quote('27184','Evan Jones','<a href=\'#comment-27168\' rel=\"nofollow\">@Alexander Sandler<\/a> - I actually did, but forgot to post something here. See <a href=\"http:\/\/www.evanjones.ca\/durable-writes.html\" rel=\"nofollow\">Making Writes Durable<\/a>. This is unfortunately written from the perspective of the underlying C system calls, and doesn\'t discuss Python at all. However, it is relevant provided that you use the correct Python function calls.\r\n\r\nhttp:\/\/www.evanjones.ca\/durable-writes.html'); return false;">Quote</a></div> @Alexander Sandler – I actually did, but forgot to post something here. See Making Writes Durable. This is unfortunately written from the perspective of the underlying C system calls, and doesn’t discuss Python at all. However, it is relevant provided that you use the correct Python function calls.

http://www.evanjones.ca/durable-writes.html

Reply – Quote
]]> By: Alexander Sandler http://www.alexonlinux.com/direct-io-in-python/comment-page-1#comment-27168 Alexander Sandler Sun, 18 Sep 2011 08:12:22 +0000 http://www.alexonlinux.com/?p=1684#comment-27168 <a href='#comment-24939' rel="nofollow">@Evan Jones</a> Please do so. I'll post a link if you do.<div class="comment-remix-meta"><a href="#" class="replyto" onclick="replyto('27168','Alexander Sandler'); return false;">Reply</a> - <a href="#" class="quote" onclick="quote('27168','Alexander Sandler','<a href=\'#comment-24939\' rel=\"nofollow\">@Evan Jones<\/a>\r\nPlease do so. I\'ll post a link if you do.'); return false;">Quote</a></div> @Evan Jones
Please do so. I’ll post a link if you do.
Reply – Quote
]]> By: Things to do in Phuket http://www.alexonlinux.com/direct-io-in-python/comment-page-1#comment-27111 Things to do in Phuket Tue, 13 Sep 2011 03:47:53 +0000 http://www.alexonlinux.com/?p=1684#comment-27111 I together with my friends came analyzing the good tips located on your website and so instantly got a terrible suspicion I had not thanked the website owner for those strategies. Those guys were definitely glad to study all of them and now have truly been making the most of them. Many thanks for indeed being quite helpful and then for obtaining this sort of fine issues millions of individuals are really desirous to be aware of. Our honest regret for not expressing appreciation to you earlier.<div class="comment-remix-meta"><a href="#" class="replyto" onclick="replyto('27111','Things to do in Phuket'); return false;">Reply</a> - <a href="#" class="quote" onclick="quote('27111','Things to do in Phuket','I together with my friends came analyzing the good tips located on your website and so instantly got a terrible suspicion I had not thanked the website owner for those strategies. Those guys were definitely glad to study all of them and now have truly been making the most of them. Many thanks for indeed being quite helpful and then for obtaining this sort of fine issues millions of individuals are really desirous to be aware of. Our honest regret for not expressing appreciation to you earlier.'); return false;">Quote</a></div> I together with my friends came analyzing the good tips located on your website and so instantly got a terrible suspicion I had not thanked the website owner for those strategies. Those guys were definitely glad to study all of them and now have truly been making the most of them. Many thanks for indeed being quite helpful and then for obtaining this sort of fine issues millions of individuals are really desirous to be aware of. Our honest regret for not expressing appreciation to you earlier.
Reply – Quote
]]> By: Saving file data using Python in an embedded system in an safe and fast way http://www.alexonlinux.com/direct-io-in-python/comment-page-1#comment-25658 Saving file data using Python in an embedded system in an safe and fast way Sat, 23 Apr 2011 13:11:46 +0000 http://www.alexonlinux.com/?p=1684#comment-25658 [...] only works for raw system file descriptors - not python file objects. There is a interesting post here on the subject. Changing the filesystem, as suggested by orgcandman, could help, a good [...]<div class="comment-remix-meta"><a href="#" class="replyto" onclick="replyto('25658','Saving file data using Python in an embedded system in an safe and fast way'); return false;">Reply</a> - <a href="#" class="quote" onclick="quote('25658','Saving file data using Python in an embedded system in an safe and fast way','[...] only works for raw system file descriptors - not python file objects. There is a interesting post here on the subject. Changing the filesystem, as suggested by orgcandman, could help, a good [...]'); return false;">Quote</a></div> [...] only works for raw system file descriptors – not python file objects. There is a interesting post here on the subject. Changing the filesystem, as suggested by orgcandman, could help, a good [...]
Reply – Quote
]]> By: Evan Jones http://www.alexonlinux.com/direct-io-in-python/comment-page-1#comment-24939 Evan Jones Fri, 18 Feb 2011 12:52:01 +0000 http://www.alexonlinux.com/?p=1684#comment-24939 The comments here are ignoring the fact that O_DIRECT writes *do* write directly to the device, but *don't* issue "cache flush" commands to ensure that the data is in fact on disk. In other words: even when using O_DIRECT writes you *still* need fsync, or you need to use the O_DSYNC or O_SYNC flags as well, when calling open(). I've done a bunch of power failure testing with this stuff. I really should write a blog post about it ...<div class="comment-remix-meta"><a href="#" class="replyto" onclick="replyto('24939','Evan Jones'); return false;">Reply</a> - <a href="#" class="quote" onclick="quote('24939','Evan Jones','The comments here are ignoring the fact that O_DIRECT writes *do* write directly to the device, but *don\'t* issue \"cache flush\" commands to ensure that the data is in fact on disk. In other words: even when using O_DIRECT writes you *still* need fsync, or you need to use the O_DSYNC or O_SYNC flags as well, when calling open().\r\n\r\nI\'ve done a bunch of power failure testing with this stuff. I really should write a blog post about it ...'); return false;">Quote</a></div> The comments here are ignoring the fact that O_DIRECT writes *do* write directly to the device, but *don’t* issue “cache flush” commands to ensure that the data is in fact on disk. In other words: even when using O_DIRECT writes you *still* need fsync, or you need to use the O_DSYNC or O_SYNC flags as well, when calling open().

I’ve done a bunch of power failure testing with this stuff. I really should write a blog post about it …

Reply – Quote
]]> By: jpetazzo http://www.alexonlinux.com/direct-io-in-python/comment-page-1#comment-24882 jpetazzo Sat, 05 Feb 2011 03:11:18 +0000 http://www.alexonlinux.com/?p=1684#comment-24882 @Lyubomir About the CentOS 5 issue: maybe you can run your test program within ltrace to actually see the calls issued. Using malloc debugging macros might also give some information, but I'm afraid that the information would be in the middle of other Python malloc/free, which would make debugging quite hard. About the OS X port: I don't use OS X, but I would be happy to know if it works (and/or what fixes were needed)!<div class="comment-remix-meta"><a href="#" class="replyto" onclick="replyto('24882','jpetazzo'); return false;">Reply</a> - <a href="#" class="quote" onclick="quote('24882','jpetazzo','@Lyubomir\r\n\r\nAbout the CentOS 5 issue: maybe you can run your test program within ltrace to actually see the calls issued. Using malloc debugging macros might also give some information, but I\'m afraid that the information would be in the middle of other Python malloc\/free, which would make debugging quite hard.\r\n\r\nAbout the OS X port: I don\'t use OS X, but I would be happy to know if it works (and\/or what fixes were needed)!'); return false;">Quote</a></div> @Lyubomir

About the CentOS 5 issue: maybe you can run your test program within ltrace to actually see the calls issued. Using malloc debugging macros might also give some information, but I’m afraid that the information would be in the middle of other Python malloc/free, which would make debugging quite hard.

About the OS X port: I don’t use OS X, but I would be happy to know if it works (and/or what fixes were needed)!

Reply – Quote
]]> By: Alexander Sandler http://www.alexonlinux.com/direct-io-in-python/comment-page-1#comment-24872 Alexander Sandler Sat, 29 Jan 2011 21:10:22 +0000 http://www.alexonlinux.com/?p=1684#comment-24872 <a href='#comment-24828' rel="nofollow">@Lyubomir</a> I missed that point. You're right of course. And.. Thanks :-)<div class="comment-remix-meta"><a href="#" class="replyto" onclick="replyto('24872','Alexander Sandler'); return false;">Reply</a> - <a href="#" class="quote" onclick="quote('24872','Alexander Sandler','<a href=\'#comment-24828\' rel=\"nofollow\">@Lyubomir<\/a>\r\nI missed that point. You\'re right of course. And.. Thanks :-)'); return false;">Quote</a></div> @Lyubomir
I missed that point. You’re right of course. And.. Thanks :-)
Reply – Quote
]]>