[05:15:47] ** apoirier has joined us [05:56:16] [connected at Wed Jan 16 05:56:16 2008] [05:56:16] <> *** Looking up your hostname... [05:56:16] <> *** Checking ident [05:56:16] <> *** No identd (auth) response [05:56:16] <> *** Found your hostname [05:56:16] <> *** Your host is clarke.freenode.net[clarke.freenode.net/6667], running version hyperion-1.0.2b [05:56:18] [I have joined #peak] [05:56:18] ** clarke.freenode.net set the topic to does anyone remember the real topic ? - not really ... [08:17:32] ** ronny has joined us [08:17:35] yo [08:18:05] i there a way to get setuptools to include another package into srcdists/bdists [11:00:17] I ship a package with bundled dependencies. [11:00:56] Try "easy_install --always-unzip --editable allmydata-tahoe" [11:01:10] And you'll get the sdist that I distribute which contains sdists of some of its dependencies. [12:03:31] ronny: svn:externals + find_packages() is a good place to start [12:08:20] ** pje has joined us [12:17:08] coderanger: we dont use svn [12:17:30] zooko: sounds like a good way [12:30:08] Howdy pje. [12:30:32] 'lo [12:32:00] btw, I don't really recommend direct inclusion of the type zooko is doing [12:32:12] I think it's better to have a separate all-in-one distribution if you're going to do that. [12:32:26] i.e., just tarball all the tarballs or eggs together and make an installer for it. [12:32:34] But that's just me. [12:33:42] ** apoirier has left IRC (Remote closed the connection) [12:33:46] pje: why? [12:34:19] Modularity & DRY, for starters. Overhead, for another. [12:34:42] e.g. overhead of svn checkouts, which you've had some people complain about IIRC [12:34:54] duplication if I already have the dependencies [12:35:35] more narrowly focused within what setuptools is designed to support (i.e., less chance of oddball edge cases) [12:36:13] I'm not at all opposed to all-in-one distributions; I just think that sdist is the wrong vehicle for it. [12:36:25] We need something like a "bdist_all" or "bdist_basket" [12:36:38] or even sdist_all would be fine. [12:38:12] What's DRY? [12:38:21] You're right about overhead and duplication. [12:38:27] But I've had no problems with setuptools supporting it. [12:38:41] Except for my problems with trying to get setuptools to install them unzipped so that py2exe could find them. [12:38:52] "Don't Repeat Yourself" :) [12:38:54] I would like a ?dist_all [12:39:26] Yeah. My partner Brian Warner is bothered by my bundling trick because of, I suppose, the same sorts of aesthetics that you are talking about. [12:39:43] I sympathize, but for the moment I'm happy with the resulting convenience, despite the aesthetics. [12:40:24] Yep. Well, easy_install certainly has all the requisite pieces to fetch source distributions and bundle them together. [12:40:46] i.e., calling --editable on all the sources. [12:42:20] That's an intriguing feature. [12:42:28] I value making it easier for people to hack source. [12:58:05] oO [12:58:20] i realyl could use uch a sdist_all/bdist_all thing [12:58:31] got a few projects i want to split up [12:58:50] right now anything to get all-in-one dists would end up messy [13:29:29] zooko is now known as zookonap [14:00:04] zookonap is now known as zooko [14:03:51] pje is now known as pje_mtg [15:03:30] ronny: what revision control tool do you use? [15:03:43] Ronny: I'm pretty happy with my way of making "sumo" dists. [15:04:00] It is quite flexible -- if you simply leave one of the source tarballs out of the misc/dependencies/ subdirectory then the normal easy_install thing happens. [15:04:08] Leave them all out and then you have the normal use case. [15:04:15] Bundle them all in and you have a full sumo distribution. [15:04:31] pje: I have a problem with the setuptools_darcs plugin that I wrote. [15:04:59] I can't reliably warn the user if the plugin is incapable of doing its thing (for example, because it can't execute darcs), while also leaving the user alone in the case that my plugin isn't needed. [15:05:30] Currently my users are complaining that they don't want to see unnecessary warning messages when they are (a) not making dist anyway so they don't need a list of files, or (b) building from an sdist that has a good SOURCES.txt. [15:05:53] But if I turn off the warning, then in the case that someone builds a dist without having darcs executable and a darcs repository, then they will silently get a broken dist. [15:06:20] So having thought about it awhile, I suggest that rev ctrl file finder plugins should be able to raise an exception (or otherwise return a distinguished value) meaning that they couldn't do their thing. [15:06:52] Then setuptools could, in the case that there was no SOURCES.txt and no successful revctrl plugins, warn the user or otherwise deal with issue. [15:06:55] What do you think? [15:15:35] re [15:16:07] zooko: i use hg [15:16:26] i wont think about that now [15:16:27] bbl [15:43:44] pje_mtg is now known as pje [15:45:12] zooko, SOURCES.txt is rebuilt constantly. [15:45:32] and it happens whenever you do any sort of build operation, not just sdist [15:46:02] so, your hypothetical "warn the user" would happen just as often as it does now. [15:46:54] How does your file finder determine whether darcs is even in use to start with? [15:51:15] pje: it doesn't -- it just tries to invoke darcs to get a list, and if that doesn't work it returns... [15:51:38] Then it generates zero results. [15:51:46] Which is the recommended protocol, I believe. [15:52:26] So, I think that there is a case that we could detect, which is that there is not already a SOURCES.txt, and all of the revision control plugins raise the "CouldNotListFilesException". [15:52:53] In that case, I think we can safely warn the user that they are not building with a full file list. Probably. [15:52:55] What do you think? [15:57:47] I don't really see what the problem is. If somebody doesn't have darcs, why did they install the plugin? [15:58:16] Is it that some directories aren't under revision control? If so, can't you detect that? [15:58:51] What would I do if I did detect that? [15:58:56] It is doable, but a bit messy. [15:59:38] Actually it is clean enough -- I can invoke darcs and ask it if a given dir is under its purview. [15:59:43] But then what do I do with this information? [16:00:30] So yes, the case that I am thinking of is that someone gets a copy of the source code which does not include a correct SOURCES.txt, and then they use setuptools and it generates a SOURCES.txt, but the one that it generates is incomplete, because there is no darcs repository present, or else there is no darcs executable. [16:15:01] the only way they'd get a copy of the source that doesn't have SOURCES.txt is if they checked it out... [16:15:07] of darcs, presumably. :) [16:16:07] I would say a darcs plugin should warn if it can't execute darcs, period. There's no reason to install it unless you have darcs. [16:16:09] Okay, good enough. [16:16:27] sdists always contain a SOURCES.txt that's good enough to regenerate the original sdist [16:16:34] Right. [16:16:45] I went through heck to make sure that you don't have to have any revision control on the machine where you unpack an sdist [16:17:00] so as long as distribution is via sdist or darcs, it should be pretty well covered [16:17:13] I don't like silent failure modes, even if the only way that I can think for someone to get here is to do something that they oughtn't have, such as doing a "darcs dist" (which is like an "svn export") and getting a copy of the source code without invoking setup.py sdist. [16:17:18] And if the directory isn't under darcs, then not detecting any files is the right thing to do. [16:17:54] Again: the purpose is to detect files that are under revision control, so the plugin is doing the right thing. [16:18:00] So, I don't like silent failure, but a noisy warning that is almost always a false alarm is no better. [16:18:06] So I'll turn off the warning. [16:18:28] Oh, I think your docs suggest that we could use distutils.log.warn() for this case. [16:18:29] * zooko looks. [16:18:32] well, warning (or error) for no executable makes sense [16:19:09] Well, that is wht I currently do, but it makes for noise and my users are complaining. [16:20:38] So unless you can tell me how I could use distutils.log.warn() in a way that can be quiet in the common case or something, I'll just remove the warning code entirely. [17:09:02] zooko, why are people installing the plugin if they don't have the executable? that makes no sense [17:09:48] If it were me, I'm not sure I'd make it a warning - I'd be likely to just have it error out and say "either install darcs or uninstall the plugin" :) [17:37:42] pje: if my project, e.g. allmydata-tahoe, doesn't setup_require the plugin, then there is a very common failure mode, in which a user gets the project through darcs but doesn't know to install the plugin. [17:37:49] So currently, my project setup_requires the plugin. [17:38:52] And by the way, you told me to use distutils.log instead of logging.log so that the log messages would respect the distutils verbosity configuration, but when I subsequently wanted to use this configuration of verbosity I didn't see how. [17:39:04] It's the -v/-q options to setup.py [17:39:11] that determine verbosity [17:39:24] That didn't change the way that log.warn() emitted things to stdout/stderr. I'll try again to be sure. [17:39:38] So, what if you could drop the setup_requires when building from an sdist? [17:39:39] So, in fact, I added a "if '-q' in sys.argv:" type hack.. :-( [17:39:47] ew [17:39:48] How do I determined that I'm building from an sdist? [17:40:02] I started to do it by checking for a SOURCES.txt, but then I realized that there will be a SOURCES.txt, [17:40:02] hang on a sec... let me check something [17:40:08] even if it is an incomplete, broken one. :-( [17:40:41] So at this point, one thing that would make me plenty happy is a way to determine, at setup.py import time, whether the current distribution has a complete SOURCES.txt or not. [17:40:50] If it has, I don't setup_require the setuptools-darcs plugin. [17:40:58] Aha -- presence of PKG-INFO file alongside setup.py [17:41:11] if it's there, you're an sdist [17:41:18] Cool. Thanks. [17:41:21] or at least were unpacked from one at some point. :) [17:41:38] absence means you didn't, so you could require the plugin in that case. [17:43:42] Right-o. [17:43:55] And I'll just remove this warning stuff, in the case that "darcs" can't be executed. [20:25:30] ** pje has left IRC ("Client exiting")