PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Thursday the 11th of June, 2015

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[01:03:12] <svm_invictvs> Datastore.save doesn't actually modify the objcct you pass in does it?
[01:10:24] <brotatochip> hey Boomtime, I wonder if you can explain why my secondary replica set member only has 6gb of journals whereas the primary has 10
[01:10:41] <brotatochip> s/journals/indexes/g
[01:11:35] <brotatochip> Nevermind... It must still be indexing in the background
[01:12:12] <Boomtime> journals come and go, you really shouldn't read too much into them drifting around
[01:12:28] <brotatochip> Not journals, indexes
[01:13:27] <Boomtime> ok, that's a bit more interesting, though there are a few ways that can happen too
[01:13:37] <Boomtime> has the secondary been resync'ed ever?
[01:13:39] <brotatochip> Also, it does seem that the indexing is not happening in the background, but when there is a change in the size on the primary, i'm seeing a similar change on the secondary, although the primary is still just under 10gb and the secondary is 6
[01:13:52] <brotatochip> This is after an initial sync
[01:14:00] <brotatochip> Completed under 10 minutes ago
[01:14:21] <Boomtime> right, that's your answer
[01:14:45] <Boomtime> your index is wonderfully compressed right now, it has no gaps in any of the buckets
[01:15:04] <brotatochip> Ahh, awesome
[01:15:06] <Boomtime> the index on the primary has had some random keys deleted in the buckets, and those haven't been refilled yet
[01:15:42] <brotatochip> Is it possible for those keys to be deleted on their own? I have made zero writes to this DB since I morgorestore'd it
[01:16:26] <brotatochip> Or maybe the keys were deleted before the mongodump
[01:17:53] <Boomtime> mongorestore causes indexes to be rebuilt - it should be similar to a resync
[01:18:12] <Boomtime> though there can be differences depending on if the index is background or not
[01:18:28] <Boomtime> when an index is built in the background the result is less effecient
[01:18:42] <Boomtime> when an index is built by resync it is always built in the foreground
[01:18:56] <Boomtime> thus, more efficient space-wise
[06:06:16] <shlant> GothAlice: you around?
[06:07:12] <shlant> nm i think I got it ;)
[06:29:42] <svm_invictvs> So, with Morphia
[06:29:50] <svm_invictvs> It seems to be ignoring the limit on the query.
[06:40:26] <nawi> is index { name: 1, address: 1 } used for a query { name: "foo" }? more specifically, I want to make queries with 2 to 6 properties, do I need to make index for each combination of properties used?
[06:47:46] <Boomtime> @nawi: to your first question; yes. to you second question; no
[06:49:04] <Boomtime> note that if you index { name: 1, address: 1 } then search for { address: "N blah street" } may not use that index because it requires the full scan of the first field which isn't very efficient
[06:49:52] <nawi> to be more specific, my queries are like { time: 1234, "foo.bar": true, "foo.bar2": true } or { time: 1234, "foo.bar3": true } and so on
[06:50:02] <Boomtime> but a search for { name: "foo", state: "sunnyville" } would likely use that index
[06:50:33] <nawi> so I guess I can just make an index { time: 1, "foo.bar": 1, "foo.bar2": 1, "foo.bar3": 1 } and the index should be utilized for all the queries?
[06:51:06] <Boomtime> in the examples given, yes, but with varying levels of efficiency
[06:52:04] <Boomtime> any query containing 'time' field would use the index - any query without that field is unlikely to ise that index because it occurs first, and is thus not likely to be useful
[06:52:27] <nawi> the time is always used here, but the "foo.bar" properties depends on the search
[06:53:01] <nawi> thank you for your help
[06:53:02] <Boomtime> then the efficiency of the index use will depend on the presence of those fields
[07:33:49] <auzty> how to mongorestore except 1 collection?
[08:24:18] <kas84> Hi, I’m trying mongo 3.0.3 before migrating my existing app and the mongod instance crashed but there was no sign of it on the log
[08:24:21] <kas84> any ideas?
[08:26:35] <vagelis> Hello everybody
[08:28:23] <vagelis> Can we use $set and and $unset for the same <field> at the same time? Well it returns that it cant do that but actually I would like to know if this was the behavior before or it was from the start like this.
[08:29:11] <vagelis> I found a python test that has: {'$set': {'a': 3}, {'$unset': {'a': 4}}
[08:29:23] <vagelis> And it expects a to be 3 after the test
[08:29:46] <vagelis> (lets say a = 2)
[08:40:14] <vagelis> I realized what i wrote let me re-write it. I hope someone sees it
[08:40:20] <vagelis> Lets say a = 2
[08:40:55] <vagelis> { {'$set': {'a': 3} }, {'$unset': {'a': True} } }
[08:41:17] <vagelis> Can u please tell me the value of a IF it exists?
[08:48:54] <vagelis> Does anyone see my messages?
[09:22:09] <Garito> hi!
[09:22:21] <Garito> I'm trying to connect mongodb with elasticsearch
[09:22:29] <Garito> I know I have, at least, two options:
[09:22:42] <Garito> elasticsearch rivers (which are deprecated apparently)
[09:22:54] <Garito> mongo-connector which fails like a champ
[09:22:58] <Garito> any other alternatives?
[09:22:59] <Garito> thanks!
[09:29:47] <Garito> hello?
[09:33:46] <kas84> When using nodejs mongo driver 2.x
[09:33:53] <kas84> how do I tell if a collection exists?
[09:34:04] <kas84> db.colectionNames method no longer exists
[09:42:30] <Derick> kas84: there should be a method for it: http://mongodb.github.io/node-mongodb-native/2.0/api/Db.html#listCollections
[09:46:52] <Garito> no lucky for on this today, isn't it?
[09:47:23] <Garito> ok
[09:47:24] <Garito> thanks
[11:04:14] <joannac> vagelis: i don't think you can use $set and $unset at the same time
[12:23:56] <symbol> I'm new to mongo and I'm a bit confused as to why the driver syntax is differnt than the shell?
[12:24:53] <StephenLynx> language used.
[12:25:02] <cheeser> symbol: which language are you using?
[12:25:05] <StephenLynx> runtime environment.
[12:25:24] <StephenLynx> they were developed by two different guys
[12:25:32] <cheeser> and gals
[12:25:33] <symbol> Sure but with postgres - sql is sql despite the language. There are different pg clients but the syntax remains the same.
[12:25:56] <symbol> I use ruby and javascript mostly
[12:26:02] <cheeser> ah
[12:26:08] <symbol> Well, node.
[12:26:53] <symbol> That makes sense though, different people developing the drivers.
[12:26:57] <StephenLynx> aaah
[12:27:00] <StephenLynx> theres another thing.
[12:27:09] <StephenLynx> with a sql db
[12:27:12] <StephenLynx> you are just using sql.
[12:27:21] <StephenLynx> mongo doesn't have its own language.
[12:27:35] <StephenLynx> drivers use the language of the runtime environment.
[12:27:37] <cheeser> not in the same way at least.
[12:28:01] <StephenLynx> it becomes much more intuitive when you get used to it.
[12:28:19] <StephenLynx> because you are not writing code and hope its right, you right actual commands.
[12:28:20] <cheeser> ultimately, mongo queries are (extended) json and each language/driver has different approaches to building those json documents.
[12:28:27] <symbol> Ah ok, that's really interesting! So it doesn't matter than the shell is a JS interpretter since the driver doesn't go through the shell?
[12:28:34] <StephenLynx> nope.
[12:28:39] <StephenLynx> the shell has its own driver.
[12:29:02] <cheeser> unfortunately.
[12:29:03] <StephenLynx> thats another difference, the shell isn't just parsing code.
[12:29:26] <StephenLynx> don't expect to carry much sql knowledge to mongo.
[12:29:54] <StephenLynx> you are not in kansas anymore.
[12:30:04] <symbol> Haha noted
[12:30:20] <cheeser> kansas is boring.
[12:31:08] <symbol> It's interesting that a lot of lessons I've come across teach using the shell. I guess that's done to demonstrate concepts.
[12:32:08] <StephenLynx> yeah, because the shell its the default way to do things.
[12:32:32] <StephenLynx> and doesn't hide anything from you.
[12:32:33] <cheeser> there's nothing else to install with the shell
[12:32:41] <StephenLynx> also that.
[12:32:46] <cheeser> makes it easy to follow along at home
[12:33:02] <StephenLynx> so when you learn the tool using the shell, you look at how the driver implements said tool.
[12:33:44] <cheeser> if you're using java, jongo lets you query using the shell syntax for the most part
[12:33:52] <symbol> True - it'd take much more work for the end user to get their env set up to use a driver.
[12:34:52] <symbol> So with drivers, if you want to write a script, you'd just use its syntax?
[12:34:59] <cheeser> mostly, yes.
[12:35:07] <symbol> That's neat.
[12:35:37] <StephenLynx> that is the best invention since orgasm.
[12:35:50] <StephenLynx> queries are much more developer friendly.
[12:36:02] <StephenLynx> and not a xbox hueg string.
[12:38:03] <symbol> Heh - I like it.
[12:39:05] <symbol> Thanks for the clarification!
[12:41:03] <StephenLynx> np
[13:47:20] <ArTiSTiX> Hi there ! I'm searching for some advice about custom aggregations... Is it a good idea (or possible) to make a $sort for each query by an aggregated field which is a linear function between $geoNear distance and time (like a score) ?
[13:48:22] <ArTiSTiX> (e.g. i search documents near a point, but i want to sort it by descending [X*distance + Y*timeDifference] )
[13:49:04] <ArTiSTiX> I need to explain my client it's a very bad idea...
[14:11:18] <koren> Does secondaries un a replica set replicate from the primary's oplog only? I want that only 1 secondary read primary oplog and then others secondaries read from secondaries oplog
[14:11:43] <cheeser> you can tell a secondary to sync from a different member, sure.
[14:11:52] <koren> ho nice
[14:12:12] <koren> thx
[14:12:20] <cheeser> np
[14:12:46] <cheeser> http://docs.mongodb.org/manual/tutorial/configure-replica-set-secondary-sync-target/
[14:14:51] <koren> you saves the day, traffic is not cheap our primary. I can't find it on mms, I have to configure the hard way or I'm blind?
[14:16:56] <cheeser> i'm not sure if that option is exposed via MMS's interface...
[14:17:41] <koren> I'll do it by hand thx a lot, I'm confident enough now :p
[14:17:47] <cheeser> sure
[14:17:48] <cheeser> good luck.
[14:27:28] <deathanchor> off-topic general question: workstation setup, facing toward or away from window?
[14:29:46] <jacksnipe> toward but there's a curtain for bad sun angles
[14:43:49] <deathanchor> I face away (so monitor glare is a problem), but if I turn around then my back is to a walking area that everyone passes by :(
[14:53:42] <chowes> Is there a way to sort by the # of matches found by the $in array operator? (http://docs.mongodb.org/manual/reference/operator/query/in/#use-the-in-operator-to-match-values-in-an-array)
[14:54:11] <chowes> something like $score for $text?
[14:59:10] <koren> chowes: would use aggregation framework, $group then $sort
[15:10:16] <Hanumaan> getting following error from the server "unauthorized db:tapari lock type:-1 client:156.211.156.175
[15:59:08] <hundredideas> I'm creating a simple ontology to represent semantic relations b/w a set of ~50 terms. Straightforward to serialize as json-ld or in xml/owl and I could probably write or find a little js DAG data structure to facilitate accessing the relations in the ontology... But should I just use mongo? Not sure if it's a terrible idea. The mongo docs explain how to set up a very simple tree structure (http://docs.mongodb.org/manual/tutorial/m
[15:59:08] <hundredideas> odel-tree-structures/)
[15:59:15] <hundredideas> http://docs.mongodb.org/manual/tutorial/model-tree-structures/
[16:00:48] <GothAlice> hundredideas: When storing graph data, it's generally best to use a real graph database.
[16:01:10] <hundredideas> yeah
[16:01:24] <GothAlice> MongoDB has no concept of joins or recursive queries server-side, so depending on your queries (find nodes two relations away, for example) may be either painful to work with, or impossible.
[16:05:45] <hundredideas> I am completely inexperienced w/ how ontologies expressed as rdf/xml or owl or json-ld are actually parsed. I guess they might not be stored in a db at all by many clients, just read into memory
[17:02:36] <deathanchor> hmmmm .sort({ time : -1}).limit(1) doesn't show me the last document by time.
[17:04:54] <deathanchor> doh.. wrong field name
[17:41:27] <foldedcat> I have a collection with a field that is an array of _id's from another collection. What's the best way to fill out that array with the data of each item in that array of _ids
[19:08:12] <greyTEO> GothAlice, have you ever had to exclude a field from the mongo-connector?
[19:08:24] <GothAlice> I technically don't use mongo-connector.
[19:08:40] <GothAlice> I rolled my own mongo-connector-alike prior to the development of mongo-connector, for my needs.
[19:08:42] <greyTEO> oh ok. My bad I thought you did
[19:09:25] <greyTEO> doesnt look like there is a way to do it yet.
[19:09:25] <GothAlice> It seems I frequently ran into needed features, and would add them myself, prior to them being added by the 10gen folks. Compression is another example, also full text indexing. ;)
[19:10:22] <greyTEO> do you manually create your indexes in ES?
[19:10:29] <greyTEO> or Solr?
[19:12:33] <GothAlice> It's a Neo4j + Jython abomination. ;)
[19:12:51] <greyTEO> lol shit....
[19:13:05] <GothAlice> And yeah, the Neo4j and MongoDB sides are basically completely separate, schema-wise.
[19:13:36] <greyTEO> Neo4j has always interested me but I can't find a good time to use it
[19:17:26] <greyTEO> looks like in ES I can just set the field to be ignored. https://www.elastic.co/guide/en/elasticsearch/guide/current/dynamic-mapping.html
[19:17:54] <greyTEO> But I would like to stop it more upstream
[19:58:04] <owen1> why is this insert fail? db.events.insert({ non : false, sfUserId: '54d41351db0349cf4b000008', times : { tz : "-07:00", end : ISODate("2015-06-12T14:00:00Z"), start : ISODate("2015-06-12T10:00:00Z") }, eventName : "Sangre app" });
[19:58:16] <owen1> (it's working on my local mongo, but not on my production server)
[19:59:58] <cheeser> what does the error message say?
[20:02:08] <owen1> cheeser: how to print the error message?
[20:04:37] <owen1> db.runCommand( { getLastError: 1}) ?
[20:04:39] <owen1> i am not sure ):
[20:10:07] <cheeser> in the shell it usually just shows up right there.
[20:21:08] <owen1> not showing me any output
[20:21:17] <owen1> it just not adding to that collection.
[20:21:28] <owen1> btw, it works also on my test db on production box
[20:45:00] <shlant> hi all. If I need to create a user for a node.js app to connect to my database, and do the usual create/delete/modify data, what kind of role should it have?
[20:46:02] <StephenLynx> writeread
[20:46:05] <StephenLynx> or something.
[20:51:25] <shlant> StephenLynx: can I create a user for a db that doesn't exist yet?
[20:51:33] <StephenLynx> yes.
[20:51:37] <GothAlice> shlant: Creating a user in the DB creates that DB.
[20:51:48] <StephenLynx> same with collections.
[20:51:50] <shlant> ah ok
[20:51:54] <shlant> that works
[20:52:22] <GothAlice> shlant: http://docs.mongodb.org/manual/reference/built-in-roles/#readWrite
[20:53:04] <shlant> yea that looks about right
[20:53:41] <shlant> thnaks
[22:06:19] <_ari> hi
[22:06:28] <_ari> how many connections to mongodb are usually good?
[22:06:42] <_ari> i have people making a new connection for ever collection they make
[22:06:49] <_ari> it seems crazy
[22:07:33] <_ari> this is on nodejs
[22:50:11] <diegoaguilar> Hello, anyone experienced with mongolab here?
[23:18:16] <rsully> how can I install mongo on ubuntu without it auto-starting and creating prealloc files? they fill up my hdd, I need to symlink the folder before it starts. I tried the runlevel=1 hack but it does not work for mongo
[23:18:33] <joannac> why don't you symlink the folder first?
[23:18:52] <rsully> because the folder doesn't exist until its installed
[23:19:05] <rsully> and mongodb user is not created
[23:19:57] <rsully> would symlinking /var/lib/mongodb to another drive work if that is owned by root? would the installer change the owner when it creates mongodb user?