[01:37:25] <[Outcast]> I have just start working custom function. I am fine getting thing working on the CLI. I have two questions while working with pymongo and python:
[01:38:36] <[Outcast]> 1. is there to get pymongo to execute the db.loadServerScripts() function?
[01:39:19] <[Outcast]> 2. How do you get pymongo to use the function that where loaded?
[01:59:20] <[Outcast]> look like I am going to have to use the eval function for which can make thing insecure.
[08:04:54] <Xedecimal> I don't think I fully understand this, I have a unique column set in my ensureIndex, then I run save(vals, {safe: 1}), as that's a shortcut to do an upsert, udpate or create new, then I get an error saying "duplicate key" ... Isn't the entire point of this to update if your key overlaps ?
[08:05:31] <Xedecimal> Something tells me it's specifically because one of those 3 keys that was found as a duplicate is 'null'
[08:08:17] <Xedecimal> I've read about null values and using 'sparse' in my index generation... Yet I don't think I actually need that because the two entries that have 'null' really are the same thing and should be updated ?
[08:26:43] <Xedecimal> NodeX: It should fail to insert, but what about save()? This is supposed to naturally upsert right?
[08:27:00] <NodeX> auth only takes care of read/write iirc
[08:27:49] <NodeX> well if it already exists then it will update
[08:28:16] <Xedecimal> that's my problem, I'm using save() and it's telling me duplicate key, this is through php too by the way in case that has any relevance
[08:28:20] <oskie> there is something wrong here... I will make a script
[08:30:47] <Xedecimal> I have 3 keys, path, parent and index, both have same index, parent and null for path... I'm starting to think there are other indexes that didn't get removed from the past
[08:31:21] <NodeX> look for those keys in a find and see what they match
[08:33:56] <Xedecimal> if I can get it to do it again, very hard to reproduce
[09:18:17] <Init--WithStyle-> I want to bring a binary file into my mongoDB and sort it into a different structure
[09:18:22] <Init--WithStyle-> Any idea how I might begin?
[09:20:17] <Init--WithStyle-> I just want to convert it from a big flat binary file into a 2d array of data entries with multiple pieces of data on them
[09:20:32] <Init--WithStyle-> eg. right now: 0x23, 0x44, 0x57 <-- the binary file
[09:33:07] <mids> Init--WithStyle-: that is not really something mongodb can help you with though. you'll need to implement some conversion code before inserting the data into mongo
[09:33:23] <mids> Init--WithStyle-: what language will you implement this in?
[09:34:00] <Init--WithStyle-> javascript.. but I only need the data from this binary file converted once every few months
[09:34:00] <cmex> can we put an 2.2.0 at production . som1 tryed it yet?
[09:36:37] <jhsto> I'm using nodejs native driver for mongodb and I would like to search for data stored in my database under 'url' identifier, which has to be then compared with a string... Any help on how to retrieve the value only, so that it can be compared?
[09:37:24] <Init--WithStyle-> mids would I need to create a utility on my node.js server that somehow takes the binary file as an upload then puts it into my mongoDB in the structure I want?
[09:37:38] <jhsto> If I try to print out the document, I get all this information about what I dont know anything.
[09:37:44] <mids> Init--WithStyle-: yeah, that would be my suggestion
[09:37:57] <Init--WithStyle-> Seems like a bit of a waste since I will only be doing this every half year or so..
[09:38:54] <mids> Init--WithStyle-: what is a waste?
[09:39:08] <Init--WithStyle-> coding a small utility
[09:39:38] <wereHamster> Init--WithStyle-: uhm.. so what is your suggestion? Hack the mongodb source to understand your custom binary format?
[09:47:22] <jhsto> The script is supposed to first try to find a url value, and if found, check another value of the same document. If else, the script will continue to create new id
[10:08:32] <jhsto> mids, its now responding me with [ { answers: 5 } ] - however, i cant get it parsed so that i would only have the answers value, in this case 5?
[10:52:21] <riot> ohai everyone. I'm quite new to mongodb and i'm aiming to use it as storage for map data (openstreetmap) to render via mapnik. All written in python. Oh, i have a nice MongoDB coffee mug =) Anyone playing around with GIS? Maps? Mapnik?
[10:53:18] <riot> oh, and i'm totally interested (as in *NEEDS*) in an armhf port. Is this already possible to build?
[10:53:57] <riot> i saw some patches and the ticket having some higher priority.. but took only a short (promising) glance
[11:26:38] <SisterArrow> Im trying to figure out how mongo stores stuff on disk. I have lots and lots of documents (~100 000 000) with an avarage size of 4kb each. Each of these documents have a "product_hash". Every day i insert a new document which may or may not have a previous document with the same product_hash.
[11:26:56] <SisterArrow> I query the database all the time for product_hash. It may return 1 document or 1000.
[11:27:02] <SisterArrow> Im trying to tune the read ahead.
[11:27:12] <SisterArrow> I have an index on product_hash
[11:27:33] <SisterArrow> So say I have 300 documents for product_hash:blargh.
[11:27:54] <SisterArrow> Will mongo store these 300 documents sequentially on disc since I have a index for product_hash?
[11:28:00] <SisterArrow> Or will it be spread across randomly?
[13:00:14] <NodeX> Hello nebojsa_kamber : welcome to MongoDB, we are your pleasure
[13:00:59] <nebojsa_kamber> I'm having trouble installing the Mongo driver for PHP, does someone have the time to help me?
[13:01:11] <Derick> nebojsa_kamber: state the problem, and we perhaps can
[13:02:52] <nebojsa_kamber> I tried to install the PHP driver through PECL, as stated in many tutorials, but whenever PHP tries to connect it fails with the following error: You have too many open sockets (7035) to fit in the FD_SETSIZE (1024). The extension can't work around that.
[13:03:18] <nebojsa_kamber> I managed to digg out that it has something to do with Apache
[13:03:37] <Derick> does apache really have that many open sockets?
[13:16:23] <Derick> algernon: only with a hack, and I'd rather fix it properly than hacking it
[13:17:04] <algernon> well, on BSD, it's not even hackish, as far as I remember. But yes, a proper fix is a thousand times better, but that also takes considerably more time.
[13:17:35] <nebojsa_kamber> I understand. Is there a workaround to get it to work? When I installed the PHP driver on my local Fedora box from a RPM, it worked like a charm.. was hoping Ubuntu would be just as easy..
[13:20:15] <Derick> it should be if you don't have some many open files/sockets
[13:21:53] <nebojsa_kamber> Shouldn't raising the open_files with ulimit -n help?
[13:32:41] <nebojsa_kamber> Is there a way to check if the FD_SETSIZE is actually 1024? Because our sysadmin has strong belief that the limit is raised to 65.000
[13:33:04] <Derick> you can't FD_SETSIZE like that
[13:33:18] <Derick> it's not linked to the filedescriptor limit
[13:34:26] <nebojsa_kamber> Too bad.. I was hoping that our company could give MongoDB a try..
[13:39:31] <nebojsa_kamber> Well, we're mainly PHP devs, so it'd be hard getting my colleguages to use driver other than PHP.. I'll guess I'll write JS or something..
[15:38:48] <souza> Hello guys, i'm have to iterate in an array in C language, but i've no idea how to do this, someone knows how can i achieve this, or link me to some site? =)
[15:45:38] <Bartzy> If I have an ObjectId as _id in a comments collection for example, where I need the date and time of each comment - do I need a datetime field too ?
[15:45:45] <Bartzy> or the timestamp in the _id is sufficient ?
[15:51:48] <algernon> depends on how much resolution you want for the timestamp, and whether you trust your oids
[16:00:07] <bjori> Bartzy: you would probably want an additional datetime field too, say if you would like to get all comments from one day.. you can't really do such queries against the id :)
[16:03:35] <joeljohnson> hey guys, I have a ~100MB json file that I want to import. It looks like this: http://pastie.org/4434504
[16:04:05] <joeljohnson> and I try to import it like this: mongoimport --file toImport.json -c myData
[16:04:21] <Bartzy> bjori: If I only want to sort by them ?
[16:04:27] <Bartzy> And show the date time on the comment
[16:04:31] <joeljohnson> and I get this error: exception:unknown error reading file
[16:04:36] <Bartzy> bjori: Why would I want to get all comments from one day ? :P
[16:12:03] <bjori> joeljohnson: have you validated the file?
[16:12:57] <bjori> Bartzy: idk... I would still use a separate datetime field
[16:14:02] <joeljohnson> bjori: I've done this: cat toImport.json | python -mjson.tool to look at it formatted, and it didn't have a problem. Do you know of a quick way to do full json validation?
[16:15:28] <bjori> joeljohnson: I usually use jsonlint.com.. but for a 100mb file that probably isn't an option :)
[16:31:45] <joeljohnson> so this json is generated by java
[16:37:24] <joeljohnson> I don't think it's invalid UTF8 characters. I just used a tool to strip out all invalid UTF8 and it gives me the same error
[16:39:53] <tunele> hallo everyone. I have a problem with replica set and "couldnt determine master". I have searched both in the official mongo google group and jiira, and found similar problems, but none of the workaround proposed seems to fix my problem.
[16:42:25] <tunele> I have three mongo nodes, 1 of them is arbiter. I'm running latest mongo version and latest mongo php driver. rs.status() on any node tells me that everything is working fine. But when I connect with php, I get the following error: couldn't determine master".
[16:49:56] <linsys> tunele: Does rs.status() show a node as a master? Also in your php config are you listing all of the mongodb nodes? or just one?
[17:11:50] <skot> Generally you want a seed list of a few.
[17:12:10] <skot> rs.status() show which node is primary, but db.isMaster() show this more clearly/succinctly
[17:21:12] <cedrichurst> mapreduce question, let's saying i'm reducing a sales collection into a customer collection
[17:21:44] <cedrichurst> sales has the structure {_id: …, customerId: 1234, price: 102.39}
[17:22:06] <cedrichurst> and customer has the structure {_id: 1234, name: 'ABC Widgets', price: 0}
[17:22:33] <cedrichurst> i want to do a mapreduce from sales that repopulates only the price value, without replacing the whole key
[20:29:03] <elarson> the and expression should have been: {$and: [{x: 1}, {y:2}]}
[20:37:47] <ashley_w> skot: thanks, and sorry about that elarson. but when i was first learning (which was pretty recent), not using $and wasn't working for me. dunno what i did wrong then.
[20:38:44] <elarson> ashley_w: I think they are not equivalent
[20:39:10] <ashley_w> they might not be, but i was still wrong. :)
[20:39:28] <elarson> or at least I could see how your data could have made it seem like the latter is like an OR
[21:40:11] <cedrichurst> is there any way to get rid of the 'value' property in mapreduce and write straight to the underlying collection object?
[21:47:30] <crudson> cedrichurst: vote for this https://jira.mongodb.org/browse/SERVER-2517
[21:48:38] <crudson> and read discussion for how some of us are working with it for now
[21:50:33] <sapht> what's the preferred way to sort a query based on a numerical field? is it fast enough to use .sort in a >10000 document collectino or should i create capped aggregates and sort using $natural?
[21:51:15] <sapht> i could limit the scan to a maximum of maybe 1000 items but no less
[21:57:20] <cedrichurst> crudson: i couldn't really find many examples of how people are working with it for now
[21:57:39] <cedrichurst> unless you're referring to the eval thing
[22:00:55] <crudson> cedrichurst: some for each loop that merges _id and value to top level document attributes, either eval or across a client driver. There is no way currently to not have reduce value embedded in 'value'.
[22:47:14] <emperorcezar> Can I declare fields unique together?