Log file Viewer
#mongodb logs for Tuesday the 25th of March, 2014
[00:27:34] <chdog44> I have a document and one of the field values is a document. This document also has documents in its fields and the field names are SimpleDateFormat strings. How can I access the fields in the deepest sub document during a MapReduce? My current code is:
[00:27:36] <chdog44> if (this.night[this.date] && this.night[this.date] !== null && this.night[this.date] !== "undefined")
[00:29:14] <chdog44> I'm pretty much a beginner at MongoDB and I'm accessing it through Jasper which I'm equally new to so I'm not sure why that's giving me a exception: TypeError: Cannot read property '2014-02-20' of null near 'this.night[this.date] && this.night'
[03:50:12] <BadHorsie> Hey, I have a collection like [{sName:"something",logs:[{startepoch:1,name:"thing1"},{startepoch:2,name:"thing2"},{startepoch:3,name:"thing3"}]}],I'm wondering if there is a way to use $splice so that I retain the 2 latest records from collection.logs...
[06:32:23] <Soothsayer> Anyone here with first-hand experience of using Cube for Times Serries Data in MongoDB ? http://square.github.io/cube/
[06:43:42] <kPb_in> Hey Guys.. The command db.printSlaveReplicationInfo() give me negative values like "-10 seconds ago" on slaves and positive values on masters.. What does this mean?
[07:02:30] <mylord> i heard querying by mongodb indexes is slow. is that generally true? and if so, how should i insert, say, an auto-incrementing id?
[07:46:58] <mylord> how to prevent adding item to array with same value for eg name. eg, if name: “bot1” exists, don’t update?
[08:19:33] <bin> guys i know thatprobably it's java specific but if i have returned embedded object how do i deserialize it to normal object
[08:46:06] <BogdanR> I have some databases in a replica set and a completely separate database on a different server. I would like to migrate this separate database to be part of the replica set so I can migrate it without downtime. How can I do this?
[09:41:44] <dandre> I am trying to use mongo shell in a script and issuing a javascript command file using HERDOC. I don't want to put my file in a temporary file so how can I do?
[09:42:25] <dandre> I have tried --eval with no success and mongo command expect a file name ending with .js
[10:16:16] <kAworu> I do $pull several items from many documents from a single collection. update gives the number of document updated but what I really want is the number of pulled items.
[10:25:29] <dandre> if there is at least one user, and I issue db.system.users.find().count() without being authenticated I get an error message
[11:17:32] <jakob_> Hi all.. quick question.. Where can I find information about how to debug conflicting mods errors? I'm doing the simplest update queries but still getting it
[11:55:08] <mylord> is mongo a relative poor or good choice for site with lots of concurrent writes (eg 1-10k/s inserts)? ie, is this scenario mongodb’s relative strength or weakness? why?
[12:00:15] <mylord> say i’ll have 10k concurrent game users, and i might want to insert what button the clicked ever 1 second. that db/collection is mostly used to for later security logging, so it won’t be read often.
[12:00:44] <mylord> but another db will save level scores even ~1 minute, and will be both written and read equally, for the ~10k concurrent
[12:03:41] <mylord> I’ve recently started on node.js for the api to recv such updates from my game (as opposed to using php which i’m more familiar with). afaik, node.js works more naturally with mongodb as opposed to some other dbs.. in this case, if i keep node.js, what db might you recommend?
[12:04:02] <mylord> i also just started dev’ing with node.js for this little api, so i can change that also
[12:05:40] <mylord> if i use monggo, i’ll want to shard a lot to support concurrent writes, right? iwhat do i shard on? how much HDD overhead is there etc, generally?
[12:09:00] <mylord> iirc, mongodb usually uses a large amount of disk space per db. is that right? how does that factor in?
[12:11:50] <mylord> i’ll want to shard my db’s to provide more writers, right? when i shard, does that setup another db, with whatever large(?) overhead that implies?
[12:12:08] <mylord> is it common/useful to create multiple VMs just to shard, just to hve multple writers?
[12:14:38] <kali> mylord: sharding on the same physical host has a limited interest, but i would not use VMs to do it. just start multiple processes
[12:16:29] <mylord> kali: how limited? for example on this SP-64 machine I use, in the case I outlined roughly above, how many shards on same physical machine might i try? http://www.ovh.com/us/dedicated-servers/enterprise/
[12:18:04] <mylord> if i grow some, and have 100k concurrent users, with inserts every 1 minute (for game-end scores), how many such physical machines would i need, roughly? 0.1? 1? how about 1 insert per 1 second (100k inserts/second)?
[12:21:06] <kali> but anyway, you need to bench. my point was... there is usually little to be gained by sharding on the same physical host. it might be the case if you hit specific issues, but they tend to be less and less frequent as mongodb progresses
[12:38:22] <mylord> Nodex: why wouldn’t you use mongodb for the every-second movement/score-update insert in my example?
[12:39:10] <mylord> BTW, all, this is for a games platform where I’ll have scores/behavior updating for many games, and i’m hoping to allow for some growth
[12:39:24] <mylord> elastic search here mentions leaderboards and scores: http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis
[12:39:48] <mylord> in what ways might it be better for me, as i’ll have leaderboards etc, typical game turnys stuff
[12:40:33] <mylord> i’ll have users paying and receiving real money, so security and data-integrity is pretty important (more in terms of showing a would-be hacker didn’t really earn all that money)
[12:41:32] <mylord> i’ve read some things about mongodb data integrity, and also some debuning.. but if you guys have some general suggestions for me regarding use of mongodb or other suggestions in my scenarios, please tell..
[12:48:29] <Nodex> mylord : I wouldn't use it because I would use somehting with higher throughput. I do not see the need for the overhead in terms of diskspace / ram useage (for indexes) and ergo would use something else. Again this is ONLY my opinion
[12:51:24] <Nodex> http://www.phoronix.com/scan.php?page=news_item&px=MTY0MTU ... haha, these guys that claim these sorts of things "what that project mean for PostgreSQL - 99.9% percent of projects don't need MongoDB and could be nicely implemented in relational model."..... really don't understand it's not just about modeling data
[13:35:26] <lorgio> I have copied a database from our remote staging machine, but it is now out of date. I really only want a subsection of a collection to add to my local collection. Is there a way to do this?
[13:36:35] <cheeser> lorgio: http://docs.mongodb.org/manual/reference/method/db.cloneCollection/#db.cloneCollection
[13:39:00] <lorgio> This looks like it would work on a single database...but i'm trying to copy data from a collection on the the remote MongoDB to my local instance in the same collection.
[14:00:12] <Diplomat> i was wondering.. can you suggest me any tools that could help me import my mysql database to mongo ?
[14:02:38] <balboah> Diplomat: probably doesn't take more time to make one yourself as you are the one who knows how your data should be represented in mongo, you would still have to specify the mappings in a "tool". You might have several tables in mysql for example w hich y ou want to have one object of in mongo
[14:03:55] <Diplomat> im thinking right now what should be the best way to get those rows from my database or from a sql file
[14:04:02] <Nodex> as balboah says, only you know your data, you're going from someone else's "best practices" to the reverse in MOngodb where you build your data around your app
[14:04:45] <Nodex> I would go from a database and record the PK in the document so if it gets interrupted you can start from there again. Then remove the PK if you need to later
[14:08:48] <zymogens> Anyone know what might be going on here. I have a VPS with UFW enabled - It blocks incoming by default. ‘ufw status’ says it is active and only allows connections on 22 from anywhere. It is blocking nginx without a problem. However my mongo shell on my home pc can connect remotely to it (the VPS) no probs - Not sure how that is possible? It seems like the firewall is letting some things through?
[14:10:10] <zymogens> oh maybe not.. so it is actually connecting itself to my home mongod instance?
[14:11:24] <zymogens> Just had a look and I am not running mongod… also when I reboot my VPS it is no longer able to connect.
[14:21:53] <kees_> aye, i don't have 5B+ rows in mysql, but my server with ~1.5B rows runs nice on quite slow hardware
[14:22:15] <Diplomat> well the process works like.. i enter the name of the item.. and then it will take id from table A and then then takes that id and looks for matches from table B
[14:25:37] <Diplomat> that's why i didnt do it i'd waste tons of space that way but obviously in mongo i'd have to do that
[14:28:48] <kees_> i would still do it in one collection like { key:item-name-1, matches [1,2,3,4] }
[14:30:00] <Diplomat> that's a brilliant idea because then i can do CATEGORY [ product1, product2, product3 ], CATEGORY2 [product 1, product5, product7]
[14:30:43] <kees_> depends on what is the biggest i'd say, categories can have tons of products, but products usually don't have a lot of categories
[14:34:46] <Nodex> [14:25:33] <Nodex> then my document might look liek this... names:['A','B','C','D']
[14:38:22] <Nodex> categories:[{"label":"Foo", "description":"Bar baz"},{"label":"ANother cat","description":......}]
[14:42:08] <Nodex> I think you're trying to store product data inside the products and not have it in another collection also?
[14:44:00] <Nodex> do you need to display the "plucked" data (description or whatever) on a product details page or something?
[14:46:05] <Diplomat> http://pastebin.com/12eTp1X5 if you look this we can play it like.. i need my data in JSON format like I have shown there
[14:48:23] <Diplomat> product1: { "category1": {data:blabla, data2:blabla}, "category2": {data:blabla, data2:blabla}
[14:49:58] <mylord> how much faster will mongo run on a typical physical server than on the fastest option VMs, in Fedora, for example? any very rough idea helps.. ie, is it 10-30% slower, or much slower than that?
[14:59:27] <Diplomat> using the same product and same category (if they are set as indexes) does it write over the row or updates the values ?
[14:59:58] <Diplomat> like.. do i have to do update query or writing it over would be possible? i think it works like that in cassandra
[15:00:16] <Nodex> there is no joins or related data so if you update a document's values don't expect it to update every other document value with the same
[15:06:49] <Nodex> I need to clarify. You can update ALL documents that match a query with multi:true, you cannot update one document and expect it to propogate those values around
[15:25:07] <saml> db.pages.find().sort({'body': {$size: -1}}).limit(10) 10 largest pages.. for example
[15:46:23] <Ramjet> Q: Are there any "Sets of databases used for functional testing.", i.e., *public* MongoDB databases of varying structure/complexity used for functional/regression testing of MongoDB itself and the MongoDB command line utilities. I have a cmd line utility I need to functional test...
[15:51:53] <palominoz> hello there. i implemented localization through documents like { locales: { en: { name:"car" }, it:{name: "automobile" }}}. i need to fulltext search those localized string. had a look at mongodb docs but i didnt understand if i am able to index the dinamically addedd / removed locale objects that represent available locales of the parent document
[16:01:01] <tkeith> Is findOne({_id: SOME_ID, otherField: SOME_VALUE}) as fast as findOne({_id: SOME_ID}) ?
[16:05:57] <Ramjet> <palominoz> I thing you can index any field, sparse or not. You will need different indexes for each of [ 'locales.en', 'locales.it', 'locales.XY', ... ]
[16:10:19] <Ramjet> Q: Are there any "Sets of databases used for functional testing.", i.e., *public* MongoDB databases of varying structure/complexity used for functional/regression testing of MongoDB itself and the MongoDB command line utilities. I have a cmd line utility I need to functional test, preferably by downloading file(s) from mongodump.
[16:13:11] <cheddar> Is there a bulk insert operator that will update objects that duplicate an ID in the DB instead of throwing an error?
[16:24:16] <Ramjet> cheddar will the multi keyword work with an update() doing an upsert? See http://docs.mongodb.org/manual/reference/method/db.collection.update/#multi-parameter
[16:32:45] <Ramjet> @cheddar will the multi keyword work with an update() doing an upsert? See http://docs.mongodb.org/manual/reference/method/db.collection.update/#multi-parameter
[16:52:57] <cheddar> Ramjet, I don't think so. It looks like that is telling mongo that it is ok to change multiple documents. I have batches of documents that I am inserting (say 1k at a time). I am specifying an _id and if I have _id collisions, I want mongo to just update the document instead of error
[17:18:07] <Alexei_987> Hello! What do you think about the theoretical possibility of adding python as a server side language for doing map reduce?
[17:34:25] <Alexei_987> cheeser: thanks that could be an option. However I'm worried that serializing/deserializing data would create a big overhead for something more complex than wordcount.
[17:40:51] <arunbabu> I have a collection with object which has two attribtutes className, students[]
[17:45:26] <alnewkirk> I get the following error on all machines: { "errmsg" : "couldn't initiate : can't find self in the replset config", "ok" : 0 }
[17:46:53] <cheeser> arunbabu: http://docs.mongodb.org/manual/reference/method/db.collection.find/ look at the projection document
[19:02:40] <quantalrabbit> I want to replace the contents of a field in a collection using a regex backreferences. Is this possible with db.colleciton.update? say i want to cat1, cat2, cat3 to become dog1, dog2, dog3. does something like db.animals.update({'name': s/(cat)(\d)/dog\2/ }) work?
[19:14:48] <LoneSoldier728> anyone know why when I call populate I get another user's collection? (mongoose)
[19:23:31] <BadHorsie> Is it me or Perl's MongoDB driver does not handle update with $each and $slice?
[19:32:52] <qutwala> i have few question's, actualy i'm new in mongo but already have few tasks with it from one are done, but left second. In first task i crawl raw data and push to raw collection. Later i should go over all docs in that raw data collection proccess it (it can be just new item, but some times i have increates some keys, for example `count` = 2 if item already exists) and push to another collection
[19:32:52] <qutwala> which stores clean data and also delete already proccessed data. I think the hardest part will be update'int and still have no idea how to realize, maybe there's any similar situation described somewhere? Sorry for "article". :)
[19:52:05] <LoneSoldier728> the issue is, when I call the route I get a, but when I switch to the other user the call still shows A, not sure how, the id changes to the correct user for the query so it makes no sense unless the populate is done incorrectly... and if I create user c, everything comes back empty as if c takes the new place
[20:09:20] <korishev> Hello all, I’m looking into tag sets with the ruby driver, but even though I set read_preference to :nearest, I get errors about ‘cannot combine tag_sets with PRIMARY’
[20:09:55] <korishev> it looks like the ruby driver is overwriting the read preference when I issue a comman (not a query). Anybody see this before?
[20:20:32] <tscanausa> is there a way to add a removed node of 1 replica set to another replica set?
[20:56:12] <BadHorsie> ranman: The library is $MongoDB::VERSION = '0.701.4'; It's very interesting that the '$each' is taken literally, if I comment the '$sort' and '$slice' lines it inserts the subdoc correctly and the '$each' is interpreted correctly
[20:57:01] <cheeser> the shell and db versions are almost certainly the same unless you're doing snapshot testing
[21:07:28] <BadHorsie> ranman: Thanks for taking your time, did you copy the exact script I had or did you modify anything?
[21:07:50] <ranman> BadHorsie: I modified the indentation, that was all, do you have an example initial document?
[21:10:54] <BadHorsie> I did this: db.serverStats.insert({serverName:"this",logs:[]});and then ran the script repeatedly... I think I will end running this on a cron or something: db.serverStats.update({serverName:'this'},{$push:{logs:{$sort:{lastPoll:1},$slice:-3}}}); ... I do not see any references to $slice or $sort on the .pm.... Only see refs for 'multiple' and 'safe'
[21:45:55] <dexteruk> Hi, im new to mongodb and im looking to move over from mysql. What i dont know is what is best options. If you have a document that is variable in the number elements is it best to link via ID or embed the elements
[21:57:32] <ranman> if you feel super generous you can be all like "@jrhunt from @mongodb is a giant nerd who helps people on IRC"
[22:23:02] <BadCodSmell> I'm doing this: update({id: 1}, {id:"1", value:123}, {upsert: true}) where id is a unique index. The type mismatch is an accident and I'm getting an error about duplicate keys...
[22:25:04] <BadCodSmell> dup key: { : "5000" } this is the weird thing and I swear empty keys is off
[22:28:54] <BadCodSmell> My guess is that here mongo just isn't so great at producing sane/relevant error messages
[22:35:31] <bob_123> hello, have a short question that I haven't had much luck with google finding the answer: with the aggregate function, is there a way for me to calculate the average of a list of values, but only taking into consideration the non-zero values?
[22:36:16] <bob_123> so something like: { $group: { averageValue: { $avg: "$value.count" } } } if not zero
[22:36:32] <joannac> BadCodSmell: not clear to me what you're actually doing. More information please?
[22:39:36] <bob_123> @joannac, I also want to sum up another value in the document for the collections, I don't want to filter out zeros of one value because the other value might be non-zero for that document
[22:40:46] <bob_123> in the aggregate function, I want the sum of one value, average of another value, but average only for non-zero values for that field
[23:16:48] <lorgio> I'm calculating my data by hand, and have noticed that they work out fine, but in the map-reduce i'm not getting the same output
[23:28:32] <LoneSoldier728> or if I wanted to do a pushAll without allowing duplicates do I do something else if that does not do it... because I tried addToSet with an array and I get an array within an array