PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Saturday the 18th of January, 2014

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[00:16:57] <mrpoundsign> yo dawg, I heard you liked documents
[00:18:24] <mrpoundsign> mongodb + go makes me very happy :)
[00:35:34] <b_oneal> much document, so json
[00:37:22] <ruphos> very mongo
[00:39:26] <mrpoundsign> haha
[04:22:56] <meltingwax> does db.collection.count() run in O(1) time?
[04:25:37] <TehShrike|Work> I'm saving documents with an array/set in it - how would I go about finding the document with the largest number of items in that list, or even just find the largest count of items in a set?
[06:23:00] <lgp171188> How do I get run the db.serverStatus() command from pymongo?
[06:23:52] <lgp171188> *get to run
[07:21:35] <lgp171188> I am trying to insert a python dictionary into a mongodb collection. The python dictionary is obtained by running the serverStatus command from pymongo. But when I try to insert the data, I get "bson.errors.InvalidDocument: key '.' must not contain '.'" What could be the issue here? I don't see any keys having a . in their names.
[07:36:53] <regreddit> why does this query work: users.findOne({ user_id: {$regex: userid, $options:'i'}}, but this does not: users.findOne({ user_id: {$regex: userid, $options:'i'}, field2:'bar'}?
[07:37:18] <regreddit> it seems that mongo $regex omly allows a single field in the query
[07:38:25] <regreddit> i think this may be specific to the node native driver
[07:39:35] <mrpoundsign> are you sure field2: 'bar' exists?
[07:40:03] <regreddit> yes, the exact query works flawlessly in mongo shell
[07:40:30] <regreddit> copy/paste to node app, no results, but if i remove field2, it works
[07:41:53] <mrpoundsign> db.pixels.findOne({color: {$regex: /#/}, x: 1}).x resulted in NumberLong(1)
[07:42:17] <mrpoundsign> perhaps userid isn't a valid regex?
[07:42:33] <regreddit> mrpoundsign, can you try something for me? make the value for regex a variable, vs a literal
[07:42:42] <regreddit> thats the issue i think
[07:42:54] <mrpoundsign> in the shell? I don't run node.js
[07:43:04] <regreddit> userid is a variable, but the 'value' appears to be a valid regex
[07:43:21] <regreddit> mrpoundsign, it works fine using the string literal
[07:43:41] <regreddit> i think it may be trying to put regex syntax in a variable
[07:43:42] <mrpoundsign> a = /#/; db.pixels.findOne({color: {$regex: a}, x: 1}).x -> NumberLong(1)
[07:44:08] <regreddit> let me try it in the shell, hadnt thought of that
[07:44:24] <regreddit> try this: /^#/
[07:44:43] <regreddit> as in a=/^#/
[07:45:32] <mrpoundsign> db.pixels.findOne({color: {$regex: "#", $options: 'i'}, x: 1}).x -- same result
[07:45:40] <regreddit> yeah me too
[07:45:47] <regreddit> maybe a node driver issue
[07:45:50] <regreddit> thanks
[07:45:53] <mrpoundsign> sure
[07:46:01] <mrpoundsign> double check all your inputs? :)
[07:46:42] <regreddit> i actually hardcoded them to the values that work in the shell, no dice
[07:47:16] <regreddit> i know expressing regex syntax as strings, in variables can be rticky in javascript
[07:48:33] <mrpoundsign> just add \'s and []'s and *s till it works.
[07:49:26] <mrpoundsign> http://stackoverflow.com/questions/10728043/mongo-query-with-regex-in-node-js-operating-on-a-variable ?
[07:49:55] <mrpoundsign> new RegExp(userid)
[07:50:15] <regreddit> just figured it out - field2 is a number in the db, so when i build the variable, I need to *1 it
[07:50:20] <regreddit> to cast it to a number
[07:50:45] <regreddit> like : var agencyid = username.split('@')[1] * 1;
[07:50:46] <mrpoundsign> parseInt()
[07:50:51] <regreddit> or that
[07:51:03] <regreddit> i like *1, google says it's faster
[07:51:15] <mrpoundsign> I just spent all day doing JavaSctipt. Always makes me bipolar.
[07:51:26] <regreddit> not that performance of parsing a user id is that critical
[07:51:47] <regreddit> it has grown on me
[07:52:35] <mrpoundsign> let me just close down some of my umpteen-bajillion prowser windows. haha
[07:52:41] <mrpoundsign> browser*
[07:52:47] <regreddit> ive done vb, vb.net, C#, C, a bit of python, and js isn't horrible
[07:52:57] <mrpoundsign> I'm in love with Go
[07:53:17] <regreddit> i am entranced with it, but wish it had more support
[07:53:31] <regreddit> fro some reason, it SHOULD be what node.js uses
[07:53:34] <mrpoundsign> yeah
[07:53:46] <mrpoundsign> a lot of things I look for a library then realize I don't need it.
[07:53:49] <regreddit> but it just hasn't caught traction
[07:54:00] <mrpoundsign> Other things, there needs to be a library haha
[07:54:01] <regreddit> it is really good in most every way
[07:56:28] <mrpoundsign> https://twitter.com/hipsterhacker/status/378257131222216704
[07:57:17] <regreddit> heh
[07:57:35] <regreddit> go is supposedly heavily used inside google, so it may hang around a while
[07:58:05] <mrpoundsign> If I'm wrong, and I never am... it will be. :p
[07:58:46] <mrpoundsign> wrote http://www.pixelparty.us/ -- every pixel is in Mongodb. It runs in < 40mb RAM, all over websockets. I'm having a blast :)
[07:59:42] <mrpoundsign> like, I am generating the export images on-the-fly haha. It's not optimized at all.
[08:23:48] <lgp171188> I ran db.serverStatus() command on mongodb and in the data returned, there is a key '.' in the value of 'locks' key. Does mongodb allow . in the key name?
[10:11:32] <brucelee> is it a bad idea to run mongodb off an nfs export?
[10:11:41] <brucelee> mongodb shards
[13:10:58] <strangewildthing> is mongodb the right choice to use for processing 'limit orders' in a 'central limit order book'?
[13:13:14] <kali> what's that ?
[13:14:03] <strangewildthing> kali: its a method of matching buy/sell orders in a trading engine
[13:14:22] <strangewildthing> eg: if buyPrice > sellPrice, continue with trade
[13:14:43] <kali> it sounds highly transactional. it can be done, but it might get awkward
[13:14:56] <strangewildthing> please elaborate
[13:15:19] <kali> mongodb is not transactional
[13:15:27] <strangewildthing> will 2 phase commits work?
[13:15:38] <kali> yeah
[13:15:45] <kali> but it's awkward :)
[13:15:58] <strangewildthing> hmm
[13:16:07] <strangewildthing> am i better off not using a database?
[13:16:20] <kali> what you descrive seems just right the only thing in the industry i would stick to a sql database
[13:18:07] <strangewildthing> agh my platform only uses mongo
[13:19:49] <kali> well, good luck with that :/
[13:20:37] <kali> are you expecting a heavy trafic ?
[13:20:55] <strangewildthing> not for a while
[13:20:59] <strangewildthing> probably not for a year
[13:38:21] <Chepra> Hi. how do i force flushing?
[13:51:35] <xaep> считаете, что нужна переговорка - можем забронить что-нибудь.
[13:54:07] <xaep> oops, sorry
[14:37:24] <d0x> Hi, i've wrote a aggregation query (with unwind and one projection) to get a flat json that i need to export as CSV for my boss. But the result is larger then 16MB. Is that an identicator that mongo db is not the best database for our usecase or should i use mapreduce or smth. like that?
[14:44:19] <d0x> hm, i think map reduce is not the beneficial for me. Because i like to transform the whole collection
[14:44:27] <d0x> there is nothing to reduce
[14:47:22] <ppetermann> i'm not sure what you are doing there, neither map/reduce nor the aggregation framework are really for transformation..
[14:49:29] <kali> mongodb is not for tranformation at all
[14:49:36] <d0x> hm, sry.I dont like to transform the current data
[14:49:50] <d0x> look, thats my aggregation query http://pastebin.com/bcaZVygw
[14:50:00] <d0x> and the result i need to deliver as CSV
[14:51:11] <kali> well, you can map/reduce to get the result you want in a new collection (who cares if there is no reduce) then dump the resulting collection with mongodump
[14:52:10] <kali> but you may want to check out tools like talend for the next time
[14:52:31] <kali> or just read the file, and rewrite it with any language you're comfortable with
[14:53:18] <d0x> i'm not sure whether mongodb is the right technologie for our usecase
[14:54:01] <d0x> We store data from different plattforms to analyze them later
[17:52:17] <lampe2> i have a collection items every object in that collectoin has a entry "category" this category is a Array of strings ["cat1","cat2"] . What is the best way to find all strings and return a object/array with only that strings? Example: item1.category = ["cat1","cat2"] and item2.category = ["cat2","cat3"] and i want catgeory = ["cat1","cat2","cat3"]
[17:53:54] <kali> lampe2: look for the "distint" command
[17:54:17] <lampe2> thx kali
[17:55:04] <kali> distinct :)
[17:55:10] <kali> not distint :)
[17:55:48] <lampe2> yeah thought so :)
[17:55:56] <lampe2> Oh this is great!
[17:56:21] <lampe2> it works like a charm thx kali
[18:11:27] <kali> of course it does, if you can spell it :)
[20:53:57] <roboto> How can I find entries in collection created after [somedate]?
[20:53:58] <roboto> ah found it
[21:00:54] <cm0s> Hi there
[21:01:09] <cm0s> How do you usually translate a many to many relation in Mongodb? Do your create two documents with a field containing an array of ObjectId to keep the references? Or do you keep the reference on only one document?
[21:01:54] <cm0s> For example a Bookmark document and a Category document
[21:16:05] <kali> cm0s: whatever makes your application data access pattern efficient
[21:16:07] <cm0s> kali: yeah, it seems there is no generally accepted pattern for this case.
[21:16:07] <cm0s> I have a "relational db" background and changing the way to think about it is sometime difficult ;)
[21:17:37] <kali> you need to hink about the data access patterns
[21:17:39] <noizex> well, with m2m its not really much different
[21:18:09] <noizex> you wont embed one inside another anyway
[21:19:21] <kali> instead of thinking of the data in a static way, you need to think about the access
[21:19:22] <kali> try to picture what your application will require, and what data layout can provide the data in an efficient way
[21:19:22] <cm0s> So depending if I mainly need to read the data or modify it?
[21:19:22] <kali> i hope you get the idea, because i can't rephrase this idea anymore :)
[21:19:22] <cm0s> lol
[21:19:22] <cm0s> It makes sense ;)
[21:19:40] <kali> yeah, and what will you need together to build a page (or a ui view)
[21:21:07] <kali> shingara would say "one http hit, one single mongo document read". (i think it is extreme)
[21:24:23] <cm0s> When you need to modify one element you will need to modify it everywhere it has been embedded.
[21:25:25] <cm0s> But it seems to be a good idea if you rarely modify the document. Accessing the document will be extremely fast!
[21:25:47] <cm0s> no need to resolve the referenced..
[21:26:06] <cm0s> typo : references not referenced
[21:26:28] <kali> yeah, i was following somehow
[21:26:43] <kali> anyway, you get the idea. there is no rule. it's a compromise
[21:27:10] <cm0s> Yeap I got it. Thanks for your help !
[21:27:54] <kali> cm0s: just one thing: getting data out of mongodb tends to be way faster than a sql engine
[21:28:42] <kali> cm0s: so making two queries (one to get bookmarks, then on to get category) with no denormalisation could get you quite a long way
[21:29:35] <kali> also, categories are usually a quite small dataset, that will fit in mongodb ram (making access fast), or even in your application ram, making them negligible
[21:34:44] <cm0s> Do you think if we create a totaly normalized schema on mongodb the perf for getting data should be pretty similar (or even worse) than on sql?
[21:36:53] <kali> no, i actually think if you craft the queries sensibly (like i said, read a page of bookmarks, then read all categories, so two queries), you can get better performance than relational databases
[21:37:41] <kali> but you can probably get one order of magnitude faster by denormalizing
[21:50:53] <cm0s> But if you make two separate queries you'll then have to "reassable" the to documents. If, for example, you need a json document with an array of categories which all have their associated bookmarks embedded in the same document.
[23:14:43] <swak> http://pastebin.com/eZFnp6YT Any ideas on having the "target.update({b" use the key parameter instead of something set like 'b'? This is with javascript.