Log file Viewer
#mongodb logs for Friday the 25th of April, 2014
[00:01:59] <ranman> jjsam: err don't think you had the full context for that, the act of serializing objects into json for v8 to be able to operate on them is slow
[01:29:54] <Favorlock> Hi there, can someone explain how to set an authenticated user with mongodb for the latest version? I've followed the manual but this is what happens: http://puu.sh/8mjXM.png This is user that I added: http://puu.sh/8mk3v.jpg Thanks to anyone who can help me :)
[03:51:08] <arrty> http://stackoverflow.com/questions/19446173/key-duplicates-in-the-array-in-mongodb-even-using-addtoset this is not working for me. I still have dupes with a unique index and doing an update with $addToSet
[08:10:37] <thearchitectnlog> guys i got mongo db segmentation error while trying to insert some special characters in the db with node js and the db gone crazy
[08:27:22] <thearchitectnlog> i dunno i get 51881 segmentation fault NODE_ENV=production node app.js
[08:28:28] <thearchitectnlog> omments.insert({'sdsdsd':"sdsds"}, {safe:true}, function(err, result) { => this works
[08:29:00] <thearchitectnlog> omments.insert({sth}, {safe:true}, function(err, result) { => this give segmentation error
[08:32:01] <thearchitectnlog> where is the problem if i do var foo={bar:1} .... db.foo.insert(foo,...)
[08:34:18] <blaubarschbube> hi folks. what is the common way of backing up mongodb collections? if possible i would prefer a solution with bacula. any suggestions?
[08:37:17] <Nodex> [09:32:11] <thearchitectnlog> when i inserted some special characters but id deleted the document
[08:43:08] <thearchitectnlog> nodex i dont know the difference betwen droping a collection and deleteing a document
[08:46:31] <thearchitectnlog> so anyone can help here the error is weird when i define in a var and insert it give segmentation error the same object i insert directly in the db i t get success
[08:56:53] <samotarnik> hi everyone. mongo newbie here. i'm scraping some pages and saving them into mongo in batches, doing a basic bulk insert with an array of documents. sometimes i hit the insertion limit of 16 MB and loose the data. so i have two questions now. firstly, does the limit hitting mean, that my whole batch (i.e array) exceeds 16 MB, or could it be that a *single document* is bigger than 16 MB alone? (i have no way of knowing that
[08:56:53] <samotarnik> up front). secondly, how does one generally handle such an error? could i do sth. on the db level, or do i have to take care of it on the app level?
[09:00:00] <samotarnik> nodex: are you saying, that a single document in the batch exceeded that limit? are you *sure* of that? (because this is very strange indeed)
[09:00:49] <samotarnik> nodex: in other words i could do an insert of let's say 30 MB as long as individual documents in the array would not exceed 16 MB?
[09:05:14] <domae> is this how i search by _id in PHP? $collection->findOne(array('_id' => new MongoId('535a23d0089056e81b000039'))) I get NULL as answer, the id exists.
[09:08:38] <samotarnik> Nodex: my question is: if i am doing an insertion of an array of documents---does the 16 MB limit apply to a single document in that array (i.e. the array itself could exceed e.g. 25 MB in a single insert) or to the array as a whole (the whole array must not exceed 16 MB) ?
[09:10:33] <samotarnik> Nodex: ok... do you maybe have a link or a source for that---i have to pass this information on?
[09:12:21] <Nodex> domae : the only thing I can think of is that the document did not hit the db before you query it. Try adding some write concern to your batch insert
[09:13:39] <rspijker> samotarnik: mongod has a message limit (48MB, iirc) and a dcoument size limit. The document size limit applies to documents that can be stored in the DB. The message size limit applies to messages sent to the DB. If your array exceeds 48MB, it will fail unless the driver you use splits it up. If one of the docs exceeds 16MB it can't be inserted and therefore it will fail as well
[09:16:01] <domae> Nodex, found the error, it was my own fault. I rewrite the testdata with the same script that reads it, resulting in different ids every time.
[09:19:24] <samotarnik> rspijker: so the array could be bigger than 16 MB as long as any of the docs don't exceed that limit. but it can't be bigger than the message size limit (48 MB)..
[09:21:34] <rspijker> samotarnik: can't find it anywhere... might be that the message limit no longer exists
[09:21:48] <rspijker> the limits page in the docs does say that bulk operations can't exceed 1000 ops though
[09:23:14] <rspijker> just write a javascript function that creates two 15MB docs, put them in an array and do a bulk insert in some test collection
[09:23:55] <samotarnik> rspijker: but my array has more than 1000 docs, i am certain of that. and the error i get is that i exceeded the limit of 16 MB
[10:22:46] <TeTeT> hi, discovered a problem with a query using the java driver mongo-java-driver-2.12.0.jar - but it is quite hard to reproduce. Isolated test programm did not trigger the bug, it seems to be related to previous events in a connection.
[10:23:44] <TeTeT> so the situation is: doing a query for the _id field in a collection. Works fine for any collection, but not for gridFS collection.
[10:24:46] <TeTeT> but it only happens when I ran tests on the db before, when i just launch a separate program, it works fine
[10:25:25] <TeTeT> any suggestions on how to isolate the problem further? I've been thinking of using @Before and @After with the code on the tests
[10:32:35] <Guest90553> enabled auth=true, created admin user in admin db with roles [ "dbAdminAnyDatabase", "clusterAdmin" ], after login to mongo with admin user, the admin user is 'not authorized' to execute command 'createUser'
[11:28:38] <TeTeT> on the problem I mentioned above, it seems that the exception is triggered when a GridFS file was ever stored in the collection. Then the driver seems to use GridFSFile, which throws the exception. Otherwise BasicDBObject seems to be used.
[11:50:23] <arussel> what operator should I use to find this doc based on the coll values ? should return if a1->x1,a2-> x2 but not if a1->x1, a2->y2
[12:01:29] <policerspnd> Hey guys, could you point me at some good tutorial for working with mongodb and php?
[12:05:09] <policerspnd> basically I want to get my hands dirty with it, we plan to use it on some project and I've gathered some opinions from different people, they told me mongo is good with handling massive amount of data, weaker in doing table operations, and good with location (gps) storage
[12:05:47] <Derick> policerspnd: le me know if there are any questions, I work on the PHP driver and maps and geo stuff is a hobby
[12:10:42] <policerspnd> I submit a query to mongo, I get a document in return - am I correct so far?
[12:11:10] <Derick> you get a cursor in return, iterating over a cursor (with foreach) returns the results as documents
[12:11:48] <policerspnd> the cursor just goes through the collection of the documents I got in result
[12:12:54] <policerspnd> I see, well I wont bore you with the more detailed and techincal questions on how to read the document data itself, I assume there's plenty of info about it
[12:26:43] <Nodex> Collection names should begin with an underscore or a letter character, and cannot: ...
[12:27:20] <blaubarschbube> hi. i am backing up collections with mongodump and bacula and that seems to work great. but what if i have to backup a real huge amount of data from a mongodb? is it possible to backup just the delta (incremental, differential) and to get that data from the slave to not to harm the performance of the system?
[12:30:58] <Nodex> you can do things like $gt :{_id:ObjectId(".....")} and it saves you creatin a new index
[12:31:26] <Industrial> Can I set the value of an object id to the value of another property on the document?
[12:50:35] <toastedpenguin> fairly new to mongodb, looking to increase the size of a capped collection is this possible and if so how is it done?
[13:14:01] <foofoobar> Hi. I have a page where users can upload photos. The filename is generated with group.objectid + photo.objectid. I then palce this file on amazon s3.
[13:14:51] <foofoobar> As I read the mongoid is kind of a timestamp + machine id + X, so most of the bits in this id are in fact not random (which I want them to be so they are not guessable)
[13:15:32] <foofoobar> So in my understanding its not the best idea to use two objectids as a "random filename", is that correct?
[13:40:14] <blaubarschbube> Nodex, i dont get it. how can i with mongoexpert json query backup only delta?
[13:42:18] <blaubarschbube> but in the moment where i make a query and get delta, i could also make a full backup
[13:43:21] <blaubarschbube> okay, i have a benefit in storage. but the main point for me is performance
[13:50:17] <blaubarschbube> my problem: i want to backup some really big collections without harming the performance. i thought it might be a bad idea to make a full backup once a day. its a production server and i dont want to make the developers in my company getting angry
[13:51:52] <blaubarschbube> so i hoped that there exists a possibility to just backup delta. im quite new to mongodb and dont really know the mechanisms.
[13:53:58] <Nodex> I use two methods currently. 1. Every time a record is updated I push the database + collection + _id to a redis hash which gets looped at intervals in a queue reading the document into a json file (or json array), zips it up and put's it offsite
[13:54:29] <Nodex> 2. in my driver wrapper I append a "_last_updated" timestamp so I can delta anything I like from any point in time
[14:13:39] <blaubarschbube> Nodex, sounds nice. but for now i think that will be to complex for me. but this could be the next level.
[14:15:33] <blaubarschbube> is mongodump using slaveok by default? i cant find a command line parameter
[14:28:38] <Derick> blaubarschbube: no, it does not know anything about replicasets. You just need to point it to a secondary node.
[14:38:43] <blaubarschbube> ok. but i need to know which ip is primary and wich is secondary since this can swap. or do i missunderstand something?
[14:51:52] <ben2_> I've been running a 3 member replica set for months v2.4.6 with no issues. I recently upgraded both of the secondary members to 2.6 and last night at about 6:40 am one of them quit. The only log output was this: 2014-04-25T06:48:00.029-0400 [conn4219] command admin.$cmd command: replSetHeartbeat { replSetHeartbeat: "cgProd0", v: 20, pv: 1, checkEmpty: false, from: "cgmo0:270
[14:51:53] <ben2_> 2014-04-25T06:48:00.718-0400 [conn4207] command admin.$cmd command: serverStatus { serverStatus: 1 } keyUpdates:0 numYields:0 locks(micros) r:3880 reslen:3523 183ms
[14:52:48] <ben2_> The strange thing is that I have a 3 member TokuMx set running elsewhere and that's based on v2.4.9 and this happens with that to and prints a similar final log message and shuts down at around 6:40 am each morning.
[14:53:32] <ben2_> Does anyone know of an issue introduced in 2.4.9+ that may have caused this or if there's any way I can get more information about what's happening? I'm on Ubuntu 13.10 x64
[14:57:07] <thearchitectnlog> anyone can help me i am inserted some special character from node to mongo db now when i try to insert an object that i define in the scope of the document i get segmentation fault
[15:03:18] <boo1ean> Hi guys, I'm trying to insert doc with geometry to 2dshpere-indexed collection but get "malformed geometry" error
[15:08:50] <Acatalepsy> Is anyone here on the mongodb-user group? Does it require some sort of confirmation before you can post?
[15:10:02] <Derick> boo1ean: also, no two nodes can be the same in a row (although that I believe is getting fixed)
[15:11:55] <Acatalepsy> I've got documents with a field and (sometimes!) an array, and I want to find all documents with a field that matches a given value, or whose array contains that value.
[15:12:41] <d0x> hi, i have a json containing a full url and i like to extract the domain in an aggregation framework stage. Is that possible?? It seems that Regexgroups are not supported
[15:13:14] <Acatalepsy> Is there anything I can do with indexes ( or map-reduce? ) that will help me ensure very fast matching.
[15:14:26] <Derick> Acatalepsy: just put an index on foo, it will multi-index in case its value is an array
[15:16:08] <Acatalepsy> For example, to *just* have the array, and then when I extract the doc from mongo go document.field = document.array[0] ?
[15:22:35] <crombar> can anyone provide some information on the charges that apply for using the mms backup service? I just had an almost empty database estimated to be 150$. If that estimate is accurate, it would be a lot cheaper to run my own replica sets
[15:24:52] <thearchitectnlog> { native_parser: false } when i use this it works when i remove it it give segmentation error
[15:25:34] <crombar> still, I can set up quite a lot of stuff for 150$ these days and I don't understand the purpose of the free usage tier if an empty db will cost me 150$ a month
[15:38:07] <Derick> boo1ean: they can be so close together that they make the polygon microscopically intersecting
[15:45:42] <dgray> Hello, was just wondering if someone could direct me to a reference of the mongodb wire protocol for the new 2.5.5+ features? (specifically, bulk write operations)
[15:47:39] <Derick> dgray: https://github.com/10gen/specifications/blob/master/source/write-commands.rst
[15:50:41] <kali> mmm... it does not extend the wire protocol, but adds command, as far as i can tell
[15:51:14] <dgray> the wire protocol for update doesn't seem to allow for multiple update statements as far as I can tell...
[15:51:42] <thearchitectnlog> after i inserted special character in mongodb from nodejs native driver i an inserted anymore object from node that i define in the scope of the document it s giving me segmentation error cn anyone plz help
[15:53:49] <Derick> commands are implemented as a findOne query )OP_QUERY) against the $cmd collection
[15:55:09] <Derick> dgray: the php extension should have a very simple implementation in C for you to look at
[16:15:46] <Acatalepsy> Just thought of something - what if a field contains either, say, a string, or an array of strings?
[16:24:18] <martinbetz_> Hello, I have a question: I defined a model for Mongo with class RSSPost(db.Document): link = db.URLField(required=True). When I try to display this link in my jinja2 template, I get (<mongoengine.fields.URLField object at 0x1052a29d0>,) instead of the link itself. What do I do wrong?
[16:56:56] <ben2_> MMS is giving me a startup warning the collection lacks a unique index on _id. This index is needed for replication to function properly. When I do db.mycollection.ensureIndex({_id:1}, {unique:true}) it does nothing however. Doesn't the _id column have a unique index by default?
[17:54:51] <qx79> Hello. How would I change an admin user’s password? When I try to do “use admin” then db.addUser(“admin”, “newpassword”); I get "$err" : "unauthorized db:admin lock type:-1 client:127.0.0.1", "code" : 10057
[19:12:49] <arussel> the doc says: The upsert creates a document with data from both <query> and <update>
[19:26:37] <skot> Which makes sense since you don't want any array-like stuff showing up in the new inserted doc
[21:12:19] <ChALkeR> Using nodejs and mongodb-native, I get speed at 1 MB / second for 50 queries / second, average object size: 20k.
[21:22:58] <kali> ChALkeR: i suggest you set the profiler at 5 or 10ms and look at the log: http://docs.mongodb.org/manual/tutorial/manage-the-database-profiler/
[21:23:27] <kali> ChALkeR: trying to decide if "explain" in the console and your production queries have a different behaviour
[21:30:37] <ChALkeR> Strange, I made a separate nodejs-based testcase, and it became 5ms instead of 20ms. Need to debug my code more =).
[22:00:27] <chris_> i've been looking at some nodejs/mongodb tutorials and noticed that they are using collection names with the first letter capitalized, is this a normal convention for mongo?
[22:55:02] <unholycrab> i tried adding 2 agents in different regions to the same mongomms account, both are showing up on the mms panel, but only one is getting data
[22:58:22] <unholycrab> do i need to spin up a server for the agent that can talk to all 4 clusters?
[23:37:28] <unholycrab> if i add the primary, it will add the secondary hostsnames based on what is entered into the rs.conf... but they are local IPs and the mms agent can't connect on those addresses