pmxbot IRC Log Viewer

[01:39:19] <EmmEight> Hello

[02:18:09] <EmmEight> I have a mongo question

[03:01:32] <brandon-zacharie> has anyone ever had problem with Node.js driver methods not firing callbacks?

[03:01:37] <brandon-zacharie> http://pastebin.com/YPJ1FXcz

[03:04:42] <ranman> brandon-zacharie: I can't repro your error

[03:04:50] <ranman> is mongodb started?

[03:04:55] <brandon-zacharie> yep

[03:05:08] <ranman> version of node driver?

[03:05:15] <brandon-zacharie> the first console.log prints

[03:05:28] <brandon-zacharie> http://stackoverflow.com/questions/20183933/why-do-the-node-js-mongodb-driver-methods-not-invoke-callbacks

[03:05:52] <brandon-zacharie> I'm using mongodb (2.4.8_1) and Node.js (0.10.22) from MacPorts with the official Node.js mongodb driver (1.3.20) from NPM (1.3.15)

[03:06:09] <brandon-zacharie> this is freaking me out

[03:06:21] <ranman> and you can run operations fine through the shell?

[03:06:29] <brandon-zacharie> i've even tried rebuilding everything

[03:06:47] <ranman> if you startup the node shell and run that snippet you get no output besides connect ?

[03:06:48] <brandon-zacharie> terminal ? node shell? or ?

[03:06:56] <ranman> first try node shell

[03:06:58] <ranman> then the mongo shell

[03:07:09] <ranman> just trying to hone in on where the error is

[03:07:17] <brandon-zacharie> i have the snippet in a file and i run that

[03:07:49] <brandon-zacharie> i just tried it from the node shell to no avail

[03:08:08] <ranman> and if you fire up the mongo shell are you able to perform queries and operations?

[03:08:34] <brandon-zacharie> what's the mongo shell? mongos?

[03:08:40] <ranman> no, just mongo

[03:08:50] <ranman> mongos is the sharded router

[03:08:58] <ranman> I'm predicting mongodb is not started

[03:09:46] <brandon-zacharie> its started

[03:10:04] <brandon-zacharie> mongodb -dbpath .

[03:10:10] <ranman> and when you start the mongo shell and use test and do db.test.findOne()

[03:10:15] <ranman> should be mongod not mongodb

[03:10:25] <brandon-zacharie> typo

[03:10:30] <ranman> kk

[03:10:44] <ranman> and FWIW, the brew version of mongodb is better maintained than the macports version

[03:10:59] <brandon-zacharie> its blocking so i can see output in the terminal

[03:12:18] <ranman> any luck with the mongo shell?

[03:12:34] <brandon-zacharie> i've never used it before so i'm stumbling

[03:12:49] <ranman> no problem, there's a quick tutorial here: try.mongodb.com

[03:12:55] <ranman> takes about 5 mins to go through

[03:13:05] <ranman> ping me on PM if you need more help, I'll be around.

[03:13:35] <brandon-zacharie> that redirects

[03:13:47] <ranman> pardon me, try.mongodb.org

[03:17:48] <ranman> is it possible mongodb isn't able to allocate files in the directory in which you created it? anything strange in it's output?

[03:19:31] <joannac> ranman: PM?

[03:19:43] <ranman> @joannac how do you have ops :P?

[03:19:53] <ranman> pm == private message

[03:20:08] <joannac> ranman: correct. can I PM you?

[03:20:20] <ranman> joannac: /msg ranman ?

[03:21:29] <brandon-zacharie> nothing strange from what i can see

[03:21:44] <brandon-zacharie> var db = connect('localhost:27017/test'); //nothing abnormal

[03:22:03] <brandon-zacharie> var cl = db.test; //nothing abnormal

[03:22:21] <brandon-zacharie> cl.findOne(function(){ console.log('asdf') }); //no output

[03:22:35] <brandon-zacharie> which is abnormal

[03:22:43] <ranman> this is in the mongo shell?

[03:22:47] <brandon-zacharie> yes

[03:23:12] <ranman> use do "db.test.save({'test': 'test'})"

[03:23:14] <ranman> and then

[03:23:20] <ranman> db.test.findOne()

[03:23:36] <ranman> s/use/just

[03:24:20] <brandon-zacharie> that works

[03:24:29] <brandon-zacharie> and i confirmed it with genghis

[03:24:33] <ranman> ok, so it is node then :/

[03:24:39] <brandon-zacharie> fuuuuuuuu

[03:24:49] <ranman> nah, we got this, we know the database is working

[03:24:52] <brandon-zacharie> or the node driver

[03:25:04] <ranman> lets just figure out what's going on with node driver / node

[03:25:32] <ranman> What's weird is that I'm not able to repro under all the same conditions

[03:25:32] <brandon-zacharie> node seems fine from the shell

[03:25:45] <brandon-zacharie> but the driver isn't firing callbacks

[03:25:54] <ranman> brandon-zacharie: there is this bug: https://jira.mongodb.org/browse/NODE-92

[03:25:57] <brandon-zacharie> no errors... nothing

[03:26:17] <ranman> but again -- I'm not able to repro :/ maybe something on my end

[03:26:43] <brandon-zacharie> uhhh that would do it

[03:27:00] <brandon-zacharie> let me try rolling back

[03:28:28] <brandon-zacharie> omfg

[03:28:32] <brandon-zacharie> that was it

[03:28:46] <brandon-zacharie> 1.3.20 is busssted

[03:28:52] <brandon-zacharie> 1.3.19 is fine

[03:29:16] <ranman> ok

[03:29:17] <ranman> weird

[03:29:21] <brandon-zacharie> time to freeze package.json

[03:29:26] <ranman> what's really odd is that I don't have the same issue on 1.3.20

[03:29:28] <joannac> i wonder why we can't repro

[03:29:29] <brandon-zacharie> that report was spot on

[03:30:23] <brandon-zacharie> the reporter is right

[03:31:29] <brandon-zacharie> that looks like an attempt at a proxy pattern but the methods arn't being invoked

[03:31:39] <brandon-zacharie> just returned

[03:31:57] <brandon-zacharie> i'll see if i can confirm that

[03:32:36] <ranman> if you fix it before christkv wakes up and submit a PR, I'll personally mail you a cookie.

[03:33:49] <brandon-zacharie> i might be full of it

[03:34:02] <brandon-zacharie> i rolled forward to 1.3.20 and now it works

[03:34:34] <ranman> rofl

[03:34:38] <ranman> I am so confused right now.

[03:34:48] <brandon-zacharie> what weird is that 3 hours ago i uninstalled (npm uninstall mongodb) and reinstalled it to no avail

[03:35:13] <ranman> does node cache anything it JITs? I sincerely doubt it.

[03:35:31] <brandon-zacharie> i thought it was my code... i did a full rollback to the previous version... even in production (knee jerk)

[03:35:54] <brandon-zacharie> i don't know enough about NPM to say

[03:36:04] <brandon-zacharie> nothing to do with Node though

[03:36:37] <brandon-zacharie> V8 does everything in memory

[03:36:42] <brandon-zacharie> unlike python

[03:37:11] <brandon-zacharie> correction... CPython

[03:38:06] <brandon-zacharie> i'll leave a comment in that report stating i had the same problem and that it might be an NPM issue

[03:38:26] <ranman> kk, can you include the literal NPM commands you ran?

[03:39:42] <brandon-zacharie> sure

[03:40:01] <brandon-zacharie> npm install mongodb@1.3.19

[03:53:45] <brandon-zacharie> thanks for the help ranman

[03:55:16] <ranman> np

[03:55:17] <ranman> m

[03:55:21] <ranman> :P

[09:26:51] <Garito> hi!

[09:26:58] <Garito> I have this on my server:

[09:27:00] <Garito> mr failed, removing collection :: caused by :: 16052

[09:27:12] <Garito> could someone point me where the problem is, please?

[09:27:39] <Garito> it's a mapreduce op

[09:29:03] <Garito> and the collection that refuses to drop is a temporary inc

[09:29:21] <ranman> Garito: version of mongodb ?

[09:29:42] <Garito> 2.4.8

[09:29:46] <Garito> ubuntu 13.10 server

[09:29:51] <Garito> 32 bits

[09:30:09] <Garito> in a custom folder

[09:30:16] <ranman> I would check the common map reduce troubleshooting first

[09:30:17] <Garito> chown to mongodb:mongodb

[09:30:30] <Garito> could you point me it?

[09:30:32] <ranman> but with 32 bits it's possible you don't have enough memory available to create the cursor

[09:30:33] <ranman> sure

[09:30:34] <ranman> one sec

[09:30:35] <Nodex> 32 bits - it might be a size error

[09:30:45] <Garito> mmm

[09:30:54] <Garito> but the database has 500 documents

[09:30:57] <Garito> so tiny

[09:31:17] <ranman> http://docs.mongodb.org/manual/core/map-reduce/

[09:31:25] <Garito> thanks ranman

[09:31:38] <ranman> Garito: this may be more helpful

[09:31:39] <ranman> http://docs.mongodb.org/manual/tutorial/troubleshoot-map-function/

[09:32:10] <Garito> could be a memory problem

[09:32:19] <Garito> cat /proc/meminfo

[09:32:26] <Garito> MemFree: 35672 kB

[09:32:38] <ranman> that's 3gigs no?

[09:32:44] <ranman> or am I blind

[09:32:49] <Zelest> 35mb?

[09:32:50] <Garito> 35 mgs

[09:32:53] <Garito> yes

[09:33:31] <ranman> missed the k there, yeah I'm thinking you're OOM to create the cursor -- 32 bit systems do not tend to run databases very well

[09:33:31] <Garito> confirmed

[09:33:35] <Garito> it's a memory problem

[09:33:52] <Garito> the collections is 33554432

[09:34:06] <Garito> so it's pretty easy I face a memory problem

[09:34:12] <Garito> thank you very much

[09:34:19] <Garito> I almost fall mad

[09:35:05] <Garito> See ya soon

[09:45:45] <qsd_> Can _id be any type (long, ...) ?

[09:45:50] <Derick> yes

[09:45:57] <qsd_> o cool thanks

[09:53:05] <qsd_> I'd also really like a $elemMatch (projection) without 1 matching element restriction

[09:53:19] <qsd_> but according to the doc it's not possible

[09:53:54] <qsd_> I'll change my schema..

[09:55:55] <Nodex> you might be able to get around not using $elemMatch

[12:48:13] <ncls> on debian, should I use the .deb for installation ? or just download binaries and the directory to the PATH ?

[12:53:24] <Derick> ncls: use the .deb - makes it much easier to upgrade later

[12:53:39] <ncls> Derick ok thanks

[13:11:18] <ts33kr_> Hey everyone. Where can I find some sort of guidelines on building effective cluster topology? I need numbers like: recommended max number of connections per node (or per computational core), and how much memory is consumed by one connection, etc.

[13:23:49] <liquid-silence> Hi guys, a schema design question

[13:23:54] <liquid-silence> I want to group users

[13:23:57] <liquid-silence> users and groups

[13:24:19] <liquid-silence> should I have two different collections, users and groups and than just have a suer like { name:

[13:24:41] <liquid-silence> groups: [{ objectid }, { objectid }]

[13:28:56] <cheeser> i would store the users' group IDs as part of the users.

[13:29:49] <liquid-silence> \yeah the groups is an array for a user

[13:37:28] <Nodex> liquid-silence : it really depends on your access patterns and what you want to do with the data

[13:38:02] <liquid-silence> nah my schema will work this way

[13:47:50] <Nodex> only you can answer that as only you know what you're trying to achieve with it

[14:01:56] <synesp> I'm having a small issue.. it seems about a few times a day, mongo stops reading new docs out of a collection of mine. new docs are definitely being written, but I have a process that "monitors" a collection (grabs the last three records using sort and limit) every 2 seconds and sometimes this process just stops reading the last few docas

[14:01:58] <synesp> docs *

[14:02:23] <synesp> so right now, I inserted three new docs, but this query isn't showing them

[14:02:54] <synesp> its almost as if mongo is caching docs and simply not querying the new ones

[14:03:10] <cheeser> but it isn't.

[14:04:36] <synesp> ah just fixed it

[14:04:54] <synesp> I was using sort $natural -1 .. just changed it to _id -1 and it seems to sort correctly now

[14:05:28] <synesp> I guess thats what I get for using a non-official example for sorting

[14:20:29] <Nodex> $natural -1 is disk order iirc

[14:20:48] <Nodex> _id -1 (as long as it's an object id) will be insert order because ObjectId's are timestamp based

[15:06:51] <theRoUS> is this a good place to ask about ruby mongoid ?

[15:10:27] <Nodex> best to just ask, if someone can help they probably will

[16:11:45] <asker> hey guys, i was wondering: is it worth to replace simle mongodb calls by memcache?

[16:12:09] <ron> wat

[16:12:36] <ron> every now and then, you encounter a question to which you can only reply with blinking

[16:13:18] <asker> lets say i have 30 mongodb find-calls. is it worth to store them all into the memcache and fetch results from the memcache?

[16:13:38] <cheeser> maybe

[16:13:47] <ron> you're asking whether it would be useful for your application to have a caching layer?

[16:14:08] <asker> yap caching layer for mongo, like 1 call vs 1 cache call?

[16:14:15] <asker> mongo seems to be pretty fast alrdy

[16:14:45] <ron> do you.... have any performance issues at the moment?

[16:16:21] <asker> yap my request takes ~2s, never cached stuff so, is it worth to cache the mongodb calls or should i cache more big section into 1 cache key?

[16:17:20] <ron> well, that's a complicated question. caching has its costs too. have you tried figuring out what's taking it 2s?

[16:18:23] <asker> ofc the mongo queries, but memcache seems to be as fast as mongo to call

[16:18:41] <ron> huh?

[16:23:55] <asker> well, i benchmarked wrong: memcache 20-30 times faster then mongodb in my case

[16:52:13] <ambushsabre> hey guys, I have a quick question about an update I'm trying to run with the ruby driver. It performs the update perfectly, but still throws an error about type conversion. It's weird, because it isn't crashing the server or anything, just throwing the error and then moving on

[16:52:14] <ambushsabre> pages.update({"name" => "#{params[:name]}"}, {"$set" => {"markdown" => markdown}})

[16:53:54] <ambushsabre> I'm not sure if I should ask in here, #ruby, or #sinatra

[17:22:57] <Lope> I'd like to view my collection data in a formatted way rather than one massive serialized string of data

[17:23:04] <Lope> is ti possible?

[17:25:02] <Nodex> eh?

[17:26:00] <Nodex> care to shed some contextual light on that question?

[17:27:12] <Lope> .pretty() does the job fairly well, thanks.

[17:27:32] <ron> you're welcome. please come again.

[17:35:32] <Nodex> find()'s output is not one long string of serialized data though

[17:35:41] <Nodex> it's just not "pretty printed"

[18:07:37] <Lope> .count is apparently very slow.

[18:07:52] <Lope> so what is a faster way to see if I have any documents in my collection?

[18:08:12] <Lope> I've heard that findOne() is fast. but I don't need the whole document. just need to know it exists.

[18:12:59] <kali> Lope: make it retunr the document _id

[18:13:13] <Lope> how can I return just the _id with findOne?

[18:14:12] <kali> findOne is a find() just a limit(1)

[18:14:24] <kali> s/just/with/

[18:14:36] <Lope> sure. how can I ask mongo to only return the _id?

[18:15:00] <kali> Lope: http://docs.mongodb.org/manual/reference/method/db.collection.find/

[18:15:08] <kali> Lope: it's called "projection"

[18:15:24] <Lope> I've been reading it but I don't properly understand it

[18:15:50] <Lope> when I click on projection the 3 items there all talk about elements of an array.

[18:15:57] <Lope> but a document is not an array... ?

[18:16:40] <Lope> oh cool

[18:16:59] <Lope> db.foo.findOne({}, {_id: 1});

[18:17:04] <Lope> just experimented.

[18:49:46] <meagain> Hello, can anyone please advise me on the best way to import or convert a .sql dump file to json

[18:50:41] <kali> meagain: it's often easier to export the data in json or csv from your sql engine

[18:53:13] <meagain> Oh okay, but for this particular case, I was just sent the .sql dump file. have little knowledge about the contents except I know it contains four tables, and I have the sql queries for two of them which I am supposed to be using

[18:55:55] <kali> well, you have two possible road: load the data back into the sql engine (safe, easy, but potentially slow), or parse the dump yourself (certainly faster to run, but much messier and much longer to code / debug / test)

[18:57:07] <meagain> Okay thanks, Number one wont be possible at this time, would have to do number 2. can you kindly if possible tell me how to do this

[18:58:18] <kali> open your text editor or ide, and code.

[19:19:17] <darius93> question: why use mongodb over mysql, and why is it so hard to understand how it query data or obtain it compared to mysql?

[19:24:24] <harttho> darius93: In regards to the second part of your question… because you haven't taken the time to actually learn it

[19:25:09] <darius93> well i have tried but it all looks to complex and confusing because of how everything is stored is so different from mysql

[20:31:19] <astropirate> I want to run two queries and get the UNION of the two queries

[20:31:22] <astropirate> how do i do this?

[20:33:52] <kali> if it's on the same collection, you can use $or. if it's in two, you can not do it in mongodb, you'll have to perform the two queries and merge them yourself

[20:37:30] <astropirate> kali, how is the performance of using $or vs multiple (2+) separate queries and merging them in application?

[20:37:46] <astropirate> kali, all results being in the same collection

[20:38:45] <astropirate> there are likely to be multiple parameters to find against

[20:41:34] <kali> astropirate: it's usually better as you only pay once the network round trip, and the optimizer should be able to cope with it

[20:42:01] <kali> astropirate: if you've put in the situation where you need a huge number of queries, though, you may have to rething your schema

[20:42:32] <tastycode> There is no way to sort by the object id's getTimestamp() is there?

[20:42:51] <kali> tastycode: just sort by the _id, the timestamp is at the beginning

[20:43:35] <tastycode> YAY… kali++

[20:43:46] <kali> tastycode: note that timestamp is only at the second precision, so different timestamp generated in the same second may not be in order

[20:44:03] <tastycode> that's fine.. only needs to be accurate down to the minute

[20:44:22] <tastycode> Thanks for getting back quickly…. i miss seeing you guys at epicenter!

[20:45:13] <tastycode> (that is if you are part of mongodb's team… they used to have "office hours" at epicenter cafe )

[20:45:57] <kali> i'm not :)

[20:46:05] <tastycode> well thx anyway!

Log file Viewer

Help | Karma | Search:

#mongodb logs for Monday the 25th of November, 2013