PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Saturday the 25th of August, 2012

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[01:59:20] <Xabster> Hi, I've tried this http://pastebin.com/TbieRMBN - I expected to get a resultset limited to 15 sorted results... and they kind of are that, but everytime i run the query I *can* get different results even though the data is not changed... any ideas?
[02:51:09] <a|3x> hi
[02:57:32] <meghan> hi
[03:15:06] <MikeFair> meghan: Hi, kind of amazing that's there's over 300 users, and everyone stays quiet :)
[03:16:40] <meghan> MikeFair: mostly lurkers i suppose!
[03:49:46] <Xabster> Hi, I've tried this http://pastebin.com/TbieRMBN - I expected to get a resultset limited to 15 sorted results... and they kind of are that, but everytime i run the query I *can* get different results even though the data is not changed... any ideas?
[05:23:54] <Xabster> anyone awake now? i asked some hours ago but everyone idled :)
[05:24:03] <Xabster> it's about getting the newest 15 entries in a collection
[05:24:17] <Xabster> i do a sort and then a limit but the results change even though the collection has the same data
[05:28:01] <IAD> Xabster: add "date_created" field into documents and fill it manually, when document be saved
[08:26:43] <JoeyJoeJo> What is the mongo command that will do the same thing as SELECT columnname FROM table?
[08:28:07] <IAD> JoeyJoeJo: you need the list of field in documents of collection?
[08:28:41] <IAD> *fields
[08:29:25] <JoeyJoeJo> No, I know the fields. I just want to return one specific field from every document
[08:30:06] <wereHamster> find(query, { field: 1 })
[08:30:24] <IAD> JoeyJoeJo: http://www.mongodb.org/display/DOCS/Advanced+Queries
[08:31:00] <Xabster> is it correct to find(), sort(), and limit() in that order when you want to latest 15 entries? - the sorting is on a timestamp field
[08:31:10] <Xabster> to get*
[08:35:03] <IAD> Xabster: the mongo driver will make query to database when you will try get first value from cursor
[08:36:06] <IAD> find(), sort(), and limit() - is parameters to a Active Record object
[08:47:54] <Xabster> IAD, yeah, not sure what you're trying to tell me
[08:48:10] <Xabster> http://pastebin.com/MnkM0exH
[08:48:45] <Xabster> i do those 3 lines to find a single entry ("youngest" entry in the 'datetime' field)
[08:49:07] <Xabster> i *can* get different results each time i do it, but the data doesn't change
[08:49:14] <Xabster> so is it not sorting properly?
[08:51:45] <IAD> you should: $status = $cursor->getNext(); instead $status = $coll->findOne();
[08:52:21] <IAD> $status = $coll->findOne(); is a new query
[08:53:09] <Xabster> thanks!
[08:53:13] <Xabster> it works
[08:54:56] <IAD> and use $cursor = $cursor->limit(1); before fetch value from cursor. if you want only one value
[08:59:10] <Xabster> is that more effective?
[09:00:59] <IAD> documentation: limit() is analogous to the LIMIT statement in MySQL: it specifies a maximum number of results to return. For best performance, use limit() whenever possible. Otherwise, the database may return more objects than are required for processing.
[09:01:01] <IAD> http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-RetrievingaSubsetofFields
[09:03:15] <cubud> Hi all
[09:04:24] <cubud> If my db is going to have billions of objects in a single collection would it be worthwhile me using the immutable ID (which is a string) and creating a hash of it, and then using multiple collections MyCollection_{hash}
[09:05:13] <NodeX> sharding the collection would be better
[09:07:51] <cubud> At first I will be using a single server (before there are billions obviously)
[09:08:14] <cubud> Yeah, I think sharding will be better all round, thanks :)
[09:08:44] <cubud> Makes no sense to write a software solution to a problem I currently do not have and that is already solved by the DB :)
[09:09:14] <cubud> Thanks
[09:09:56] <NodeX> ;)
[09:11:44] <boll> How do I figure out why my non-mapped virtual memory is around 4GB?
[09:26:35] <NodeX> off the top of their head does anyone know how to get a facebook page id (a fan page I suppose you would call it)
[09:26:58] <kali> facewhat ?
[09:27:36] <NodeX> lol
[09:28:08] <Xabster> kali: it's instagram for lesser hippies
[09:28:46] <NodeX> nvm, got it
[09:29:49] <NodeX> instagram is a photo sharing site no?
[09:30:08] <NodeX> how on earth that can be compared to a full blown social platform is beyond me
[09:32:54] <Xabster> well, it's about as relevant and proper as you asking for facebook IDs in a #mongodb channel on IRC
[09:33:19] <IAD> =)
[10:26:15] <NodeX> Yeh dude that's awesome
[10:26:24] <NodeX> hence the "off the top of their head" part
[12:00:27] <lizzin> anyone familiar with scala and mongodb?
[12:01:08] <lizzin> trying to figure out the most direct way to insert a JObject
[14:54:25] <emocakes> howdy
[14:54:30] <emocakes> I have a game in development
[14:54:43] <emocakes> and I am thinking about moving over from postgresql to mongodb
[14:55:09] <emocakes> I was only using SQL due to familiarity and I want to push myself to learn something new
[14:55:31] <emocakes> my models aren't very complicated, I would need a standard user profile document with some statistics on it
[14:55:49] <emocakes> I know if I ask if MongoDB is suitable for this, the answer will probably be yes
[14:55:52] <emocakes> so Im going to ask
[14:56:11] <emocakes> is MongoDB suitable for this?
[14:56:12] <emocakes> :p
[15:20:35] <kali> emocakes: yes
[15:20:56] <emocakes> thanks kali, one more question
[15:20:57] <kali> glad to, help, that's 100€, next.
[15:21:08] <emocakes> I only have $'s?
[15:21:19] <emocakes> one more question, kinda off topic
[15:21:30] <emocakes> I am using PHP at the moment, CakePHP to be exact
[15:21:35] <emocakes> I also wanted to try something new
[15:21:45] <emocakes> have you got any opinions on node.js?
[15:22:30] <kali> I hate javascript, and i think node.js is generally not used for what it is intended
[15:24:47] <emocakes> fair enough
[15:24:55] <emocakes> you a php or RoR or python guy?
[15:25:21] <kali> RoR but also java and scala
[15:25:34] <emocakes> mmm scala
[15:46:32] <cubud> Is scala any good, I know nothing about it
[16:10:29] <kali> cubud: exotic, ambitious, elitist
[16:12:22] <cubud> I took at look at some source, didn't immediately strike me as brilliant
[16:30:51] <emocakes> kali its like
[16:30:55] <emocakes> erm
[16:30:57] <emocakes> cubud
[16:31:05] <emocakes> 'high performance'
[16:35:37] <mikeywaites> hey hoping someone can help, mongo has completely died on my server, its not running and it just falls over everytime i try to restart it. can anyone suggest anything i can try to figure this out?
[16:38:14] <mikeywaites> in the logs im seeing an error about an old lock file]
[16:38:31] <mikeywaites> i have run mongod --repair which i thought would clean that up
[16:57:19] <aptx49> time
[17:15:54] <emocakes> do you use scala with mongo kali
[17:20:15] <kali> emocakes: yeah, sure
[19:08:31] <ppetermann> o/
[19:17:31] <ppetermann> in an explain() what exactly is indexBounds telling me?
[19:23:29] <ppetermann> i seem todo something horribly wrong, i got a collection with 709068 documents, but an explain on a find shows me 2316051 nscanned, even with an index that should cover all fields, so i seem to be missing something
[19:49:04] <ppetermann> it seems the sort i'm doing is totally killing me, without the sort its all like i expect it, as soon as i add the sort, it goes downhill
[19:49:13] <ppetermann> from 54ms to 85k ms
[19:50:45] <ppetermann> it seems to be about what indexes it uses
[20:01:59] <kali> ppetermann: show us a getIndexes() on the collection, the query, and the explain()
[20:02:58] <ppetermann> kali: sure, one moment
[20:04:51] <ppetermann> http://nopaste.info/749b3f57fe.html document, http://nopaste.info/39c85c240f.html explain, http://nopaste.info/97027261f9.html the query
[20:06:08] <ppetermann> oh, the getIndexes aswell, i assume its ok to post only the index used?
[20:07:06] <kali> mmmm better to show me what you have
[20:07:47] <kali> but, from this already, this query will be hard to optimized
[20:08:10] <ppetermann> http://nopaste.info/31e0091f02.html
[20:08:12] <ppetermann> thats the one used
[20:08:29] <ppetermann> if i dont hint() it, its using one that has three more fields in it, which aren't used for this query
[20:09:02] <ppetermann> which is even sloer
[20:09:03] <ppetermann> +w
[20:09:11] <ppetermann> http://nopaste.info/10117d7bd7.html this one (reverse)
[20:09:14] <kali> ok. this query is not optimizable
[20:09:49] <ppetermann> as i said it seems to be the sort that kills it. withou the sort the or can use two single field indexes and is fast
[20:09:58] <kali> you can either split it in two and have two indexes (attackers.corpId, killTime) and (victim.corpId), one for each query
[20:10:38] <ppetermann> thats not really helping me much, because then i have to resort in the application
[20:10:43] <ppetermann> which makes pagination hell
[20:11:08] <kali> or you need to add an hybrid field (inConflict.corpId) contaning both ids of attackers and victim, then add an index on (inConflict.corpId, killTime)
[20:11:57] <ppetermann> it might be more than one attackers id, but yeah thats the worst-case solution i've come up with aswell, but i was hoping to find a better way
[20:12:24] <kali> if you don't understand why, i strongly advie to loog at the presentation there: http://www.mongodb.org/display/DOCS/Indexes#Indexes-Presentations
[20:12:41] <kali> (and to correct my typos)
[20:13:16] <kali> and... if you still in the design phase, you may want to shorten your labels
[20:13:43] <ppetermann> i wish there was a way that i could run the sort on the result of the find()
[20:13:49] <ppetermann> what labels?
[20:13:55] <kali> the field names
[20:14:16] <ppetermann> ah, those are reflecting the names of the API its pulling the data from and therefor out of my hand
[20:14:51] <ppetermann> but let me ask: why?
[20:15:20] <kali> because, contrary to what you may be used too if you come from the SQL world, the field names are stored for each document
[20:15:28] <ppetermann> i'm aware of that
[20:15:48] <kali> so you will consume more RAM, more DISK
[20:15:59] <kali> it will not change anything to your indexing issue :)
[20:16:24] <ppetermann> i'm aware of that aswell, but thats ok, since it improves the maintainability alot
[20:16:48] <kali> ppetermann: ok :)
[20:17:04] <ppetermann> especially having the names the same as from the api they are comming from enables guys who are shy about touching it (because of mongo) to at least semi-understand whats going on
[20:17:22] <kali> ppetermann: yeah, yeah, i'm aware of the non-technical aspects
[20:17:40] <ppetermann> and since its opensource the understanding of those touching it is quite important
[20:18:46] <ppetermann> i haven't really got why people are so scared of mongo, in most cases its quite simple in use - except if you run into indexing stuff like in this case
[20:18:51] <kali> but anyway, if you want to get this results efficiently, you need a involved: [ corp1, corp2, ... ]
[20:19:01] <ppetermann> yeah, i'm afraid so
[20:19:04] <kali> because it's new
[20:19:12] <kali> that's enough for many people
[20:19:13] <ppetermann> also i have to do that for character, alliance and faction ids =)
[20:19:26] <ppetermann> it isn't that new, im using mongo for like 3 or 4 years now
[20:19:48] <kali> compared to SQL, it's new.
[20:19:55] <ppetermann> fair enough
[20:20:36] <kali> and SQL/oracle marketting and lobbying as made nearly synonyms the words "oracle" and "database"
[20:20:53] <kali> it's not rational
[20:21:23] <ppetermann> well the famous "webscale" video where they make fun of it might be one of the reasons too
[20:21:30] <kali> i'm not sur sure
[20:23:01] <kali> i think the various blog entries explainging how mongodb hurt their platform, their data or their business is more harmful
[20:24:56] <ppetermann> not reading too many blogs lately, what problems do people have?
[20:25:34] <kali> fuck ! amstrong is dead
[20:25:56] <kali> (not lance, the real one)
[20:26:39] <ppetermann> yah, a few hours ago
[20:27:52] <ppetermann> honestly with 82 years thats ok
[21:12:01] <Yiq> mongodb is not relational but is it suited for a task where you store blog posts by title and text? or basically if you need mysql or postgres you dont need mongo?
[21:23:00] <MikeFair_> Yiq: It's hard to say
[21:23:42] <MikeFair_> SQL is really cumbersome to work with hierarchical data or data where the records don't have the same types of attributes
[21:24:41] <MikeFair_> But that doesn't mean that some aspects of your appliation wouldn't find a relational DB useful
[21:25:48] <MikeFair_> Synchronizing with other applications like mobile apps is another area where the NoSQL stuff seems to be doing really well
[21:27:28] <MikeFair_> Yiq: Though you might be sourcing the data to send over from a relational DB
[23:58:36] <Yiq> ERROR: listen(): bind() failed errno:98 Address already in use for socket: 0.0.0.0:27017
[23:58:50] <Yiq> I have done the remove mongo.lock and mongo --repair
[23:59:10] <Yiq> but then it is the same problem, or after repair i shopudltn run mongod again?