Log file Viewer
#mongodb logs for Wednesday the 6th of February, 2013
[00:39:22] <themoebius_> Wed Feb 6 00:37:06 [cleanupOldData-51111624335e39b612de7d55] moveChunk deleted 48700 documents for pb3.hourly_stats from …..
[03:10:44] <ranmacar> Hello! i am having a bit of trouble with mongoose - the result of a query gets transformed into the query object it seems
[03:12:53] <ranmacar> the loadProfile function prints the correct json, but the app.post prints the query object somehow
[04:51:35] <preaction> I have a document with a list of tags, how do I query for all documents with at least one tag matching from an array of tags to search for?
[05:00:25] <preaction> nevermind. turns out if i actually read the docs instead of skim them, i'd know that if "field" is an array, it has to match only one of the array i give it
[06:52:16] <lledet> hi, is it possible to do something like owner: {$ne: null, this: 'some id'} in the same "clause"? (i'm brand new to mongo)
[09:42:07] <BadCodSmell> Are there any recent bugs that can cause importing bson to result in a segfaut?
[10:03:14] <fdf> Hi, if i`m adding new replica to existing sharded cluster with rs.add() the data will be auto rebalance to the new replica ? thanks
[11:21:05] <Seb> I have a replica set with one master, one slave, and one arbiter; my "foo" DB on the master is doing just fine, but somehow the slave chokes on replication with "replSet syncThread: 10320 BSONElement: bad type 104"
[11:25:29] <Seb> I've already tried to stop the slave, wipe out all the foo DB files, drop the DB on the master, restore from a dump, then restart the slave, but it doesn't do any good
[11:25:50] <Seb> db.repairDatabase() on the master, although it completes successfully, doesn't fix the problem either
[11:52:31] <kali> Seb: you can investigate by looking at the oplog positions to see what update breaks the replication
[11:54:26] <Seb> kali: I'm having trouble comprehending at all how a perfectly working master DB is failing to sync to a slave...
[11:55:57] <Seb> kali: but anyway, would I start with rs.status() on the slave and get its oplog position ?
[12:00:16] <Seb> kali: would you be so kind as how to telling what that query would look like ? I'm more on the Ops side of things than the Dev one ;)
[12:01:55] <Seb> kali: I'm perfectly comfortable querying mongo and all; I just ahve no clue what the local/oplor.rs collection is looking like
[12:02:35] <kali> i won't write commands for you. this kind of stuff is too tricky to be done this way.
[12:02:44] <Seb> kali: I mean, if this is an upstream bug, I'm willing to go the extra mile to extract information that will lead to its resolution
[12:03:44] <Seb> kali: I figured it would be a reasonable request to have someone sorta lead me along the way in order to extract said information :|
[12:04:34] <kali> Seb: if the hints i gave you are not enough for you, your best guess may be to get some support from 10gen
[12:54:52] <msch> hi, are the points made in this article true? http://hackingdistributed.com/2013/01/29/mongo-ft/
[13:14:05] <Nodex> you mean you paid attention past the first 3 babbling paragraphs to read the rest?
[13:15:51] <solars> does anyone know sources discussing this recent mongodb "broken by design" article?
[13:17:25] <msch> Nodex: is there a point-by-point refutation of his points? because some seem plausible
[13:18:32] <msch> Nodex: ok so "All bulk operations to a sharded collection run with ContinueOnError, which applications cannot disable."
[13:21:50] <msch> Nodex: also http://docs.mongodb.org/manual/core/write-operations/ clearly states that REPLICAS_SAFE just means that the write has been "propagated" to the other servers. that's not safe in any way
[13:22:29] <Nodex> it's written by yet another butt hurt developer who thought that Mongodb could save his app and was one shoe fits all only to find out it's not
[13:23:42] <msch> kwizzles: and REPLICAS_SAFE isn't safe in any way of the word and there's no way to specify that you want something to be really safe
[13:24:05] <msch> kwizzles: http://docs.mongodb.org/manual/core/write-operations/ clearly states that REPLICAS_SAFE just means that the write has been "propagated" to the other servers. that's not safe in any way
[13:24:33] <kwizzles> buyou szre that REPLICAS_SAFE is not safe? I doubt that, because even it its name there is the phrase "SAFE"!!??
[13:24:33] <msch> kwizzles: and "All bulk operations to a sharded collection run with ContinueOnError, which applications cannot disable." which due to the semantics of getLastError means you can't even know if any but the last operation failed
[13:25:17] <msch> kwizzles: yeah but read what the documentation says. REPLICAS_SAFE just means that 2 nodes received the write, not that they have persisted it.
[13:25:33] <msch> kwizzles: and there's no way to say "make sure 2 nodes received the write and have persisted it"
[13:29:56] <Nodex> "If the bulk insert process generates more than one error in a batch job, the client will only receive the most recent error. All bulk operations to a sharded collection run with ContinueOnError, which applications cannot disable. See Strategies for Bulk Inserts in Sharded Clusters section for more information on consideration for bulk inserts in sharded clusters."
[13:30:10] <Nodex> http://docs.mongodb.org/manual/tutorial/manage-chunks-in-sharded-cluster/#sharding-bulk-inserts
[13:30:43] <Nodex> http://docs.mongodb.org/manual/core/write-operations/#bulk-inserts <---- if you can't be bothered to read the page
[13:31:40] <msch> Nodex: i read that. i want bulk inserts because "Bulk insert can significantly increase performance by amortizing write concern costs". But in sharding-bulk-inserts it still says I can't have bulk inserts with getLastError.
[13:31:47] <Nodex> the large array of high profile users of MongoDB are null and void because some troll wrote an article, oh noes
[13:32:27] <msch> Nodex: can you please keep the the actual issues. i read that article and i'm only asking about those two issues. because they actually are important
[13:33:05] <Nodex> msch : as with most things Mongo .. if you dont like it or it doesn't suit your needs then don't use it
[13:34:36] <msch> Nodex: well yeah sure. but I would like to use mongo. it's only that the devs make decisions that I can't understand that make me complain. how hard can it be to introduce a getLastErrorsForBulkOperation or to call something by it's real name PROPAGATED instead of lying and saying it's SAFE?
[13:34:40] <Nodex> the very simple answer is dont use bulk inserts, they are NOT mandatory, they increase performance when loading data by trading off on write concerns
[13:36:23] <Nodex> the article writer has blown it out of context claiming that everything is unsafe
[13:36:32] <msch> Nodex: but why did they call it SAFE. that's dishonest. that's like if I offer you a "working thing" and in the fineprint it says "oh yeah and what I call "working" actually means "broken""
[13:37:24] <Nodex> "Bulk insert can significantly increase performance by amortizing write concern costs. In the drivers, you can configure write concern for batches rather than on a per-document level."
[13:37:49] <dorong> what does the "maintenanceMode" item with a value of -3 mean inside the primary member of rs.status() ? (meaning, when running db.status(), the following key members[0].maintenanceMode equals -3)
[13:39:03] <msch> kwizzles: https://stripe.com/blog/announcing-mosql seems perfect for migrating a live system off of mongo and to postgres
[13:41:50] <msch> kwizzles: http://www.postgresql.org/docs/9.2/static/warm-standby.html#SYNCHRONOUS-REPLICATION explains Postgres replication. it works like you'd expect it to.
[13:42:02] <msch> kwizzles: also read about the WAL because that's different from Oracle afaik http://www.postgresql.org/docs/9.2/static/wal.html
[13:42:12] <Nodex> aww poor me, mongodb doesn't do what I want... it must be the fault of the developers that wrote it not my design
[13:44:26] <msch> usestrict: it boils down to: if what you call a safe insert is not safe then you're dishonest
[13:44:45] <kwizzles> i think this link says everything: http://googlefight.com/index.php?lang=en_GB&word1=mongodb&word2=postgresql
[13:45:06] <usestrict> dishonest, or slightly optimistic, or slightly sloppy in naming your functions ;)
[13:48:19] <kwizzles> anyways, coworker just laughed at me and said he thought that already in 2012 everybody switched to Cassandra
[13:48:44] <msch> kwizzles: i know guys that switched to riak and are happy there. might want to add this to your list
[13:50:27] <kwizzles> ok, i am leaving, it gets boring, i dont use database, just wanted to troll a bit
[13:52:52] <usestrict> do people seem to be generally unaware that using nosql-stuff, which essentially is fancy kv-stores, is different from using RDBMS?
[13:53:13] <Nodex> I can imagine that even db's with crazy high integrity one can still lose data when a machine crashes or w/e and nothing has been flushed to disk
[13:53:57] <Nodex> they seem to think that MongoDB claims to replace *SQL / RDBMS and works for every solution
[13:54:10] <usestrict> slightly exaggerated by the fact that nosql-solutions are advertized as "databases"
[13:55:03] <Nodex> "A structured set of data held in a computer, esp. one that is accessible in various ways"
[13:56:40] <Nodex> I imagine a few more trolls over the next few days while that article does the rounds of the interwebs. No doubt being replicated in a few Mongo shards / Replica sets along the way
[13:59:21] <Nodex> now the article makes sense. the author is one of the main devs for a rival database/store
[15:48:25] <dorong> what does the "maintenanceMode" item with a value of -3 mean inside the primary member of rs.status() ? (meaning, when running db.status(), the following key members[0].maintenanceMode equals -3)
[15:51:42] <Zelest> couldn't care less how ruby handles it internally, just worried about wasting extra space for no reason, :)
[15:56:11] <Zelest> hmms, i've never used mongo that way, but can one use fields in the query? e.g, if I have date field and a integer, can i do something like find({date: {$gte: new Date(somefield_in_this_doc)}}) ?
[15:56:42] <Zelest> what I'm curious about is to have a "find things older than X minutes" where I wish to define X per doc
[16:06:42] <sander> When compiling mongoclient I want it to be written to a configuration folder (debug or release). The "--prefix=" flag doesn't work. Is this possible and if so, how could I approach this?
[17:02:28] <owen1> i want to add a compound index - app_id and user_id. i only have 3 different apps ids and many many user ids. does it mean the user_id should be the first index and the app_id second?
[17:04:53] <owen1> Nodex: "The order of fields in a compound index is very important. In the previous example, the index will contain references to documents sorted first by the values of the item field and, within each value of the item field, sorted by the values of location, and then sorted by values of the stock field."
[17:04:53] <Nodex> what you should be doing with indexes is creating them to make the best use of queries
[17:06:17] <owen1> Nodex: ok. so if i know i'll have some finds based on uid, i should have uid, app_id, right?
[17:06:22] <Nodex> it's not important for the reason you're thinking which is the number of possible permiatations of the index
[17:07:04] <Nodex> if you index on uid+aid then you can use uid+aid in a query and also uid on it's own
[17:09:49] <Nodex> I dont know if the exact params get passed over the wire and they all build together or what
[17:10:19] <Nodex> it would be more efficient to let the secondaries build first and shift reads / primary status to them on large indexes but I a not sure if that's built in
[17:11:11] <owen1> (not related) - using the node client from 10gen, how to set the read preference to 'nearest' ?
[17:11:43] <Nodex> I dont have a clue but the PHP driver you feed it a seed list so perhaps the same?
[17:14:05] <owen1> that's what i currently do (3 hosts) - 'mongodb://push.np.wc1.yellowpages.com,push2.np.wc1.yellowpages.com,push3.np.wc1.yellowpages.com/push'
[17:28:01] <Nodex> var mongoclient = new MongoClient(new Server("localhost", 27017, {readPreference:'nearest'}));
[17:32:41] <barroca> Hello, I wasn't able to find the answer searching in the Interwebs, I have a list, with saleid, and I have to update those with the same saleid and insert the new ones. update function do the trick, but I wasn't able to create a simple criteria
[17:33:36] <barroca> I'm iterating over the list and updating like this: db.collection.update({'@id':saleid},sale,True)
[17:36:28] <barroca> I'm iterating over the list and updating like this: db.collection.update({'@id':saleid},sale,True)
[17:38:30] <barroca> Nodex: thanks for the help, to update elements on a DB using the update fuction, if they were in a list, do I need to iterate over the elements of that list and use the field as the criteria?
[17:39:35] <owen1> Nodex: i think this should work: 'mongodb://push.np.wc1.yellowpages.com,push2.np.wc1.yellowpages.com,push3.np.wc1.yellowpages.com/push?readPreference=nearest'
[17:39:36] <barroca> Nodex: or do is there a better way on defining the criteria, let's that my list is [{saleid:1,data:DDD},{saleid:2,data:DDD}]
[17:39:44] <owen1> acccording to http://docs.mongodb.org/manual/reference/connection-string/#connections-standard-connection-string-format
[17:40:28] <barroca> Nodex: and I wish to add all the elements on this list, updating those on the DB that have the same saleid
[17:52:40] <barroca> Nodex: might be, but I'd like to update all documents that have the same saleid that i have on the list. I've actually solve this using http://pastebin.com/FiMPqQ6X but I don't if the processing is worser than one liner that might exist to solve the problem.
[17:55:14] <Nodex> if you want to update multiple documents with the SAME data then you can do it in one line else you HAVE to itterate
[19:12:26] <bobbytek> well, there is $size expression but for some odd reason that isn't available in this context.
[19:23:21] <Sharcho> [Ruby on Rails] I'm trying to setup errbit. Any idea why I get the error "db_name must be a string or symbol" when running from passenger, but not when running "rails server" with thin?
[19:59:57] <l2trace99> can anyone tell me how to db.collection.remove({_id: /pattern/ }) with the php driver
[20:04:00] <klusias> hello. sometimes value is stored as int, sometimes as string. how can i do proper find(), to get both - string and int values?
[20:48:38] <tab1293> how can i get all the ids of the torrents in a data structure like this {torrents: [ {id:1, name:1}, {id:2, name:2}]}
[21:28:22] <danshipper> hey guys I'm having a problem with Mongo's MMS (monitoring system). does anyone have experience with it?
[21:29:16] <danshipper> basically what's happening is the monitoring service is only picking up my database.sessions collection. none of the other connections are being picked up
[21:29:28] <danshipper> i have the same issue with newrelic, so I'm assuming it's some sort of mongo configuration issue
[21:39:23] <danshipper> does anyone have somewhere they could point me to read about how i could debug this?