pmxbot IRC Log Viewer

[03:05:03] <Evil_Bun_Bun> anyone have 10gen support contact info (phone) we have a support contract but one dba is on an international flight another is camping and we don't have it handy

[03:12:16] <cheeser> if you have a jira login you can file a commercial support ticket and get in touch with someone

[03:12:58] <Evil_Bun_Bun> yeah i don't think we have that either with both dbas out.. normally thats prohibited but one had a camping trip planned months ago another had a death in the family and had to fly to england for the weekend

[03:13:08] <Evil_Bun_Bun> hmm tho....

[03:13:21] <cheeser> ouch

[03:13:23] <cheeser> sorry to hear that.

[03:13:31] <Evil_Bun_Bun> i think one of our sysadmins might have a jira login escalating now :)

[03:13:57] <cheeser> or you can try creating one with your company email and try to get help that way.

[03:15:24] <Evil_Bun_Bun> hmm actually _I_ have one someone else signed me up :)

[03:15:37] <Evil_Bun_Bun> just have to password recover :)

[03:15:41] <Evil_Bun_Bun> thanks

[03:26:45] <cheeser> good luck :)

[03:49:16] <Evil_Bun_Bun> thanks got the right team members started on it temp gave them my account :)

[03:51:44] <cheeser> excellent

[04:21:56] <danlamanna1> can anyone assist with this basic pymongo issue? https://gist.github.com/anonymous/7407804

[04:37:53] <danlamanna> is having a date always inserted with a document something people typically do in mongo, or their application(s)?

[04:38:07] <joannac_> Does the list(...) actually give you a list?

[04:39:11] <danlamanna> joannac_: yes, the list gives me a python list of dict objects

[04:46:44] <regreddit> anyone run replicated mongodb on AWS EC2 instances?

[04:47:02] <regreddit> im having issues with DNS

[04:47:51] <joannac_> danlamanna: I can't reproduce it on my end, could you provide a small test case that fails for you?

[04:50:29] <danlamanna> joannac_: like this? https://gist.github.com/anonymous/7408035

[04:56:32] <danlamanna> joannac_: i think i solved it

[04:56:56] <danlamanna> joannac_: i'm doing very weird things

[04:57:23] <jkitchen> anonymous gist with password hashes :(

[04:59:12] <danlamanna> letmein

[04:59:27] <danlamanna> is the password :P - on an application thats local to my machine

[04:59:49] <RaviTezu> cheeser: iirc, you're the one who replied, when I asked the question about syncing a secondary node from another secondary. I did some digging ... and here's the answer:

[04:59:50] <RaviTezu> "rs.syncFrom(hostportstr) make a secondary to sync from the given member"

[05:00:09] <RaviTezu> will let you sync from a specified source. Thanks

[05:00:33] <jkitchen> danlamanna: fair enough :)

[05:00:50] <joannac_> danlamanna: great. what was the solution?

[05:01:55] <danlamanna> joannac_: not modeling my data like an idiot. the author segment of the tweet didn't have the following element.

[05:19:31] <aendruk> I have a large map-reduce running. Is there a way to pause it and resume later?

[05:25:54] <cheeser> no

[05:26:03] <cheeser> you can kill it but you'll have to restart

[05:27:09] <voidhouse> Hey, how can I get objects that have field x lenght gt than y. Say I want '{ x: 'abc' }, { x: 'ab' }, {x: 'cdef'}' how can I just '.find()' '{x: 'cdef'}'? I want obj.x > 3

[05:27:54] <voidhouse> I know, `.find({ x: { $lg: 3}})` wouldn't work, since it would be looking for a number.

[05:28:09] <voidhouse> s/lg/gt/

[05:29:49] <voidhouse> `find({'x.3': {$exists: true}})`

[05:29:59] <voidhouse> Any better ideas?

[05:34:12] <joannac_> Store the length of the field in a different field?

[05:35:30] <joannac_> You would use a $where clause but that's going to be slow

[05:36:02] <joannac_> db.foo.find({$where: "this.x.length > 3"})

[05:37:30] <joannac_> s/would/could/

[06:26:43] <dnsdds> I know text search isn't meant for production, but... what would be the implications if I big fat did it anyway?

[08:37:27] <[AD]Turbo> hola

[11:44:57] <Nodex> anyone know how to do synchronous calls with mongodb native? . I have to get data from 3 collections to form some output and I don't want madness callback spaghetti

[11:45:21] <kali> Nodex: mouhahaha

[11:46:27] <Nodex> I know :/

[11:46:38] <Nodex> tbh, really not happy about the whole driver, it's a bit of a mess

[11:54:19] <lpin> Nodex async or Q usually helps

[12:14:44] <voidhouse> Probably a stupid question, but it is not possible to have 'null' objects?

[12:15:16] <Nodex> eh?

[12:27:59] <voidhouse> I am using node-native-mongodb and I have `items = db.collections('mycollection').find()' and if I call `items.nextObject` multiple times very fast, I get null objects.

[12:28:11] <voidhouse> And I am pretty sure I am nowhere near the end of the 'collection'

[12:35:30] <Nodex> that makes more sense

[12:35:44] <Nodex> and it shouldn't happen because it's async

[13:00:31] <voidhouse> Nodex: Exactly, as I pass a callback to it, so it can take it's time, but I am not sure what is the error.

[13:06:03] <platzhirsch> The operation "GET MORE" fails after a long time. How can I increase the cursor's timeout on the server side?

[13:10:37] <durre> what are good tools to monitor a local mongodb instance, for the purpose of load testing and understanding bottlenecks

[13:11:22] <Number6> durre: MMS is good, but requires your machine to have internet access

[13:12:01] <durre> MMS is the mongodb pay-tool?

[13:14:13] <Number6> durre: It is free

[13:14:31] <Number6> durre: The backup, however, is paid. But you don't need the backup, for testing

[13:21:56] <voidhouse> Any hints or tools that can help me? I just randomly get a batch of 'null'.

[13:28:26] <x0f> voidhouse, does it happen if you throttle your calls to nextObject()? (like in sleeping)

[13:28:56] <voidhouse> x0f: Yes, if I just have one single loop it works.

[13:29:21] <voidhouse> But if I want to have multiple loops, it will give me nulls.

[13:30:45] <x0f> i'm not familiar with node.js, but is uses v8, right? and it is fully multithreading capable, right? could it be a race-condition issue with the underlying mongo-driver?

[13:32:01] <voidhouse> x0f: No, it is actually single threaded and uses an event loop, libuv to be exact.

[13:32:38] <voidhouse> But it seems like a race issue. Maybe I will create an issue over at Github.

[13:32:42] <RaviTezu> maverick

[13:32:56] <RaviTezu> sorry wrong window :(

[13:36:18] <x0f> voidhouse, so it's callback driven, but your callback method gets an "empty" instead of an "instance" passed?

[13:37:00] <voidhouse> x0f: It gets a 'null' instead of a document.

[13:41:22] <x0f> voidhouse, even the example includes a null-check: https://github.com/mongodb/node-mongodb-native/blob/e5c3ea13a018e5603df3963ac2dae4839bccfb24/examples/cursor.js#L40-L43

[13:43:50] <voidhouse> x0f: Yeah, the null is used to indicte the end of the cursor. But I had another silly mistake. I figured it.

[13:45:42] <voidhouse> instead of creating n loops that repeated until end, I was creating n loops that were creating n loops for ever.

[13:46:10] <voidhouse> That is sure a good recepie for race condtion.

[13:49:31] <x0f> voidhouse, heh, live and learn, i guess.

[13:52:21] <voidhouse> x0f: Such is life. :)

[14:12:10] <barisc> hi

[14:12:26] <barisc> id there channel for mongoose ?

[14:12:32] <barisc> is there a channel for mongoose?

[14:31:33] <zymogens> Anyone know how you generate a mongo ObjectId in the browser? Are there javascript libraries for that?

[14:32:59] <tellnes> zymogens: there is some for node, which you probably can make work in the browser

[14:33:27] <zymogens> tellies.. thanks.. will take a look.

[14:33:41] <zymogens> tellnes

[14:37:17] <tellnes> zymogens: https://github.com/mongodb/js-bson/blob/master/lib/bson/objectid.js

[14:37:29] <zymogens> thanks tellnes

[14:37:32] <tellnes> and https://github.com/mongodb/js-bson/blob/master/lib/bson/binary_parser.js

[14:54:45] <x0f> i need some help with queryable structed time series data. here is my current case, helpful thoughts/ideas appreciated: http://pastie.org/private/79omiqxkowajzsshrli75g

[17:11:44] <Tomasso_> Im trying the 2dsphere index and when I query , it returns Nothing. I have tried querying on my own collections, and also the following example, that i cannot make it work.. can somebody take a look ? http://pastebin.com/vw22WLx5

[17:16:39] <tham> I have a collection that has to be re-written every once in a while. What I do is db.coll.remove() and db.coll.insert() in that order. But that makes a state where the coll is empty.

[17:16:50] <tham> What's the best way to sort this out?

[17:20:15] <Nodex> you can copy the collection if you want to retain the data

[17:22:35] <tham> Nodex: That sounds good. is there a specific command to copy collections, or do I use the cloneCollection?

[17:23:43] <Nodex> cloneCollection iirc

[17:24:03] <Nodex> http://docs.mongodb.org/manual/reference/command/cloneCollection/

[17:25:37] <tham> Nodex: cool, thanks

[17:26:00] <tham> wonder what happens if the collection I'm copying to exists though

[17:27:21] <tham> perhaps that I should check myself :)

[17:31:27] <tham> perhaps this might be a good idea: db.mycopy.renameCollection("myoriginal", true);

[17:42:20] <Tomasso_> Im trying the 2dsphere index and when I query , it returns Nothing. I have tried querying on my own collections, and also the following example, that i cannot make it work.. can somebody take a look ? http://pastebin.com/vw22WLx5

[18:03:14] <regreddit> trying to do replication between an AWS EC2 instance and a NON EC2 VM, but the maste keeps inserting his private hostname into the config

[18:03:38] <regreddit> how do I get him to insert a public DNS name in the config, so that the other instace can find him?

[18:04:30] <regreddit> as background, AWS instances have private and public IPs and DNS names, but only see their private IP

[18:04:49] <regreddit> so it's not like the mongo instance sees more than one network interface

[18:05:14] <regreddit> I tried forcing it, but it refused the change

[18:15:47] <chaotic_good> how long should rs.inititate() take?

[18:17:49] <Honeyman> Hello. I'm stuck on the problem how to update the field only if it exists.

[18:18:01] <Honeyman> I need to implement the collection of keys, which may or may not have the "count" field. When it has the field, this key is "consumable".

[18:18:09] <Honeyman> And I need to make a query to check if some key is valid.

[18:18:15] <Honeyman> The problem is that when I make findAndModify that validates some key (on "find" phase), what can I do on "modify" phase to decrease the "count" field by one, but ONLY if it exists?

[18:20:24] <Honeyman> When I do findAndModify with "update": {"$inc": {"count": -1}}, it works great with the keys which have the "count" field. But for fields which don't have one, it assumes the value was 0, and sets it to -1.

[18:28:57] <Honeyman> Or maybe, how can I do $inc which doesn't affect the value if it is missing...

[18:32:38] <Derick> Honeyman: do a conditional update?

[18:33:16] <Derick> update( { field: { $exists: 'false' }, _id: … }, { field: { '$inc': 1 } } );

[18:33:17] <Honeyman> Derick: this will be a separate transaction then, with a chance of race condition with the key validation.

[18:33:22] <Derick> nope

[18:33:59] <Nodex> or $gte :1 ?

[18:34:20] <Derick> "it is missing"

[18:34:37] <Nodex> $gte should not match on missing no?

[18:34:57] <Derick> Nodex: correct, but it wasn't the question :P

[18:35:02] <Derick> although, that can be indexed...

[18:35:21] <Nodex> fair point

[18:36:05] <Honeyman> My current spec (to check if the key is valid) contains now '$or': [{'count': {'$exists': False}}, {'count': {'$gt': 0}}]

[18:36:35] <Honeyman> So either 'count' doesn't exist (then the key is valid), or the "count" still not fully consumed.

[18:36:35] <Derick> Honeyman: no need to do that with $or

[18:36:51] <Nodex> $exists will catch it, unless you ahve values with zero alread

[18:36:51] <Derick> just: count: { $gt: 0 } works

[18:36:52] <Nodex> +y

[18:37:09] <Nodex> have*

[18:39:18] <Honeyman> Derick: count: ($gt: 0) doesn't work for me, or I am missing something.

[18:40:26] <Honeyman> Cause I want the keys which either not yet consumed (count > 0), or are not even consumable (count: {$exists: False})

[18:40:52] <Honeyman> count: { $gt: 0 } doesn't seem to catch the second condition.

[18:41:59] <Honeyman> So that {key: 1, count: 1} and {key: 2} are both valid auth keys, but {key: 3, count: 0} is not

[18:42:31] <Nodex> $gte ?

[18:44:11] <Honeyman> And I need somehow to findAndModify the keys (in a race-condition-prone way) that first and second keys are valid, and the third is not. And after this findAndModify the first key should become (key: 1, count: 0) i.e. invalid,... and the second key must stay {key: 2}, and not become {key: 2, count: -1}

[18:45:20] <Honeyman> count: {$gte: 0} still doesn't match the docs where "count" is missing

[18:45:33] <Nodex> it shoouldn't match it

[18:45:38] <Nodex> it's a negative match

[18:46:59] <cheeser> correct. "nothing" is not gte anything.

[18:47:21] <Nodex> this is getting metaphysical haha

[18:47:52] <cheeser> you could always update with a $set to 0 for documents where that field doesn't esist

[18:47:57] <cheeser> exist

[18:50:23] <Honeyman> count: 0 is metaphysically the opposition to "count doesn't exist" :)

[18:51:28] <Honeyman> Cause count: 0 means counter is depleted and the key should not be used, while "count doesn't exist" means there is no counter, and this key never can be depleted...

[18:59:31] <Honeyman> For now, I have the only idea. To consider the key valid if '$or': [{'count': {'$exists': False}}, {'count': {'$ne': 0}}] ($ne rather than $gt). So that the counter of consumable keys will go like 3, 2, 1, 0 (invalid and won't match anymore) and the counter for the infinite keys will go like "doesn't exist", -1, -2, -3... and to add a separate audit the finds all the "count < 0" docs and unsets the "count" field for them...

[19:27:52] <Tomasso_> i cant make find return any results querying on a 2dsphere index.. neither my code, nor in the examples i found.. Im using mongo 2.4.6 , has 2dsphere index support

[19:42:33] <joannac> Tomasso_: Pastebin an example?

[19:44:09] <Tomasso_> joannac: sure, http://pastebin.com/vw22WLx5

[19:45:34] <Tomasso_> i inserted 3 documents to the features collection, created 2dsphere index, ,checked it was created, and then run the .find with no results :'(

[19:46:21] <Tomasso_> and te same happens with a bigger collection of places formatted using GeoJSON Point format that i have

[19:48:17] <joannac> Hmmm

[19:48:44] <Tomasso_> should that be a defect from that version? :S

[19:49:19] <joannac> No

[19:49:29] <Tomasso_> if i do .find() without the location stuff i get results

[19:49:49] <joannac> "If you query for a GeoJSON point, specify $maxDistance in meters."

[19:50:17] <joannac> I think the closest thing you have is still multiple latitude degrees away

[19:50:29] <joannac> a lot more than 50000 metres :)

[19:51:40] <Tomasso_> currently in my other collection i query by : db.mystores.find({location: {$near: {$geometry: casa}, $maxDistance: 50000 } }).pretty()

[19:51:56] <Tomasso_> where casa = { "type" : "Point", "coordinates" : [ -58.417016, -34.603585 ] }

[19:52:12] <Tomasso_> they are real earth locations

[19:53:27] <Tomasso_> i tought it could be the location field, that was not on the json root.. but it isnt, else it would have failed to create the index

[19:55:27] <Tomasso_> that if i check it is there

[20:20:21] <tystr_> what's an acceptable backfround flush avg value under heavy writes

[20:21:33] <tystr_> also, what tools/strategies do you guys typically use to benchmark/stress test a mongodb replica set

[21:45:32] <regreddit> philosophical question: I was told on #node a few days a go "If you are querying a nosql db that requires and index other than _id you're doing it wrong"

[21:45:45] <regreddit> any comments on that opinion?

[21:46:01] <regreddit> s/and/an/

[21:48:08] <_Sawbones_> Can anyone recommend a visual mongo database editor?

[21:48:56] <regreddit> _Sawbones_, I cant imagine one exists

[21:49:03] <regreddit> with mongo being schemaless

[21:49:12] <chaotic_good> vi

[21:49:31] <chaotic_good> vi is a good mongo editor

[21:49:36] <chaotic_good> it shows characters visually

[21:49:39] <chaotic_good> in a terminal

[21:49:43] <chaotic_good> yeah

[21:51:03] <regreddit> that said, I have used mongo-express, which does nothing but let you 'browse' a db, but editing is still json documents

[21:52:52] <jkbbwr> Say I have a User and they have a list of posts, how do I get the last 10 posts made ever

[21:52:58] <jkbbwr> over any number of users

[21:56:32] <regreddit> jkbbwr, have ypu looked at aggregation?

[21:57:03] <jkbbwr> regreddit: treat me like an sql baby

[21:57:09] <jkbbwr> regreddit: care to explain?

[21:57:34] <regreddit> 1: mongo is not sql :)

[21:57:37] <regreddit> thats your first lesson

[21:58:17] <regreddit> 2: aggregagtion is the process by which you filter, group, and create complex queries

[21:58:54] <jkbbwr> regreddit: my idea was, get the last 10 posts of every user, and then get the last ten of that collection sorted by datetime

[21:59:08] <regreddit> so, using aggregation: http://docs.mongodb.org/manual/reference/operator/aggregation/limit/

[21:59:44] <regreddit> youre going to do a query that select the records that meet your criteria, with sorting, etc

[21:59:53] <jkbbwr> regreddit: but wwouldnt it be super inefficient

[22:00:27] <regreddit> maybe, it really depends on your data set

[22:00:40] <jkbbwr> regreddit: trying to design a forum

[22:02:17] <regreddit> here is some background on aggregation: http://docs.mongodb.org/manual/core/aggregation-pipeline/

[22:04:59] <jkbbwr> regreddit: how would you compose data in a forum

[22:05:09] <jkbbwr> regreddit: it always feels like im being "too relational"

[22:51:41] <regreddit> forums are quite relational, but each user has an array of posts, which have post_id, etc

[22:53:09] <regreddit> i would just have a posts collection and a user collection, and relate then via _id references

Log file Viewer

Help | Karma | Search:

#mongodb logs for Monday the 11th of November, 2013