PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Saturday the 10th of January, 2015

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[00:40:56] <cheeser> anybroad: http://docs.mongodb.org/manual/reference/bson-types/
[00:49:06] <scottbessler> with morphia i'm seeing performance issues where i'm spending a ton of time in MappedField.discoverConstructor. is that expected?
[02:17:45] <iClunk> Given than Mongo supports GeoJSON points is there anyway to use the GeoJSON Feature object with Mongo?
[03:43:12] <anybroad> scottbessler: what is morphia?
[04:50:42] <FunnyLookinHat> Can anyone recommend a foolproof way to secure a localhost only mongodb to a specific user / password? Similiar to a create user and grant all to db w/ their name in MySQL?
[05:14:40] <FunnyLookinHat> Ah - this is good ( except for 2.4 ) - but now I get it. :) http://blog.amussey.com/post/62809509642/mongodb-installing-and-configuring-authentication
[06:24:22] <jayjo> Does a dynamic document have to allocate more memory for itself than a standard document not knowing its schema in advance, or does it play no role?
[10:03:55] <erkin_> hi there
[10:05:51] <erkin_> I couldn't fix this problem http://stackoverflow.com/questions/27874469/mongodb-push-in-nested-array
[10:19:30] <amitprakash> Hi, I have a document D with two keys, status E (live, completed, billed) and a date key d.. I want to create shards with the following conditions, status == live OR d less than 3 days old on shard A, status == completed or d less than 6 but more than 3 days old on shard B, and finally, status in [billed] and d older than 6 days old in shard C
[10:19:33] <amitprakash> is this achievable
[10:21:02] <amitprakash> s/OR/else if
[11:35:49] <drager> How do you guys handle large collections?
[11:35:51] <drager> I would like to sort that collection
[11:35:53] <drager> but that is damn slow
[11:36:06] <drager> Without a limit it makes my site lag
[11:36:07] <drager> …
[11:37:46] <kali> you need an index
[11:38:16] <kali> and if it's not good enough, you need to consider doing that out of mongodb somehow
[11:38:48] <drager> Yeah, I have an index
[11:39:16] <drager> but I added that afterwards
[11:39:23] <kali> should not be an issue
[11:39:29] <kali> is it used ?
[11:39:41] <kali> how big is the collection ?
[11:39:43] <kali> how slow is it ?
[11:40:01] <drager> 65084 items
[11:40:08] <drager> My page responds very slow
[11:40:16] <drager> but this is on my local computer running everything as well
[11:40:42] <drager> Or I cant navigate in it
[11:41:21] <kali> ok, that's not big
[11:41:30] <drager> Maybe it's my computer then
[11:41:37] <drager> I'll try on another server
[11:41:40] <kali> show us the query which is slow
[11:41:56] <kali> an explain() for it
[11:42:06] <drager> Im running; return AdItems.find({}, {sort: {createdAt: -1}}, {limit: limit});
[11:42:08] <kali> and a getIndexes() on the collection
[11:43:24] <drager> kali: https://gist.github.com/drager/b420c4b0e4a57b8ee42f
[11:44:23] <kali> ok, the explain ?
[11:46:36] <drager> kali: https://gist.github.com/drager/b420c4b0e4a57b8ee42f
[11:56:17] <kali> ok. for some reason, it does not pick the index.
[11:56:33] <kali> can you try the query from the shell ?
[11:58:01] <drager> Thats what I did
[11:58:54] <drager> Ah
[11:58:59] <drager> It skips the limit
[11:59:02] <drager> thats why its so slow
[11:59:08] <drager> the client gets all the 60k items
[11:59:32] <kali> :)
[11:59:41] <drager> How can I solve that? :<
[12:00:33] <kali> drager: you need to look at your driver/odm documentation
[12:04:18] <drager> For what?
[12:04:45] <kali> for how to call limit() on a cursor :)
[12:06:51] <drager> so ,{limit: 10}); wont work?
[12:07:16] <kali> well you say it doesn't
[12:13:07] <drager> Well
[12:13:38] <joeyjones> alexi5: what's your usecase?
[12:15:58] <alexi5> i want to build a sms polling app where a user can create a poll, which comprises of keywords to text to and that is comprised of numbers of mobile users who text to those number and the total vote count
[12:16:26] <alexi5> another service woul capture sms and add them to the appropriate keyword and increment number of votes
[12:32:33] <drager> kali: Solved it by putting the limit in the sort object like this; return AdItems.find({}, {sort: {createdAt: -1}, limit: limit});
[12:32:36] <drager> Thanks alot
[12:34:57] <kali> you're welcome
[13:15:28] <tommy_the_dragon> can I sort a collection by a value in a subdocument?
[13:15:57] <tommy_the_dragon> if all documents have said subdocument of course...
[16:47:02] <anybroad> hi again
[16:47:16] <anybroad> So I got a collection categories and a collection books (this is an example).
[16:47:26] <anybroad> So in books I simply add the id of category as field category_id?
[16:47:30] <anybroad> Or how to link them now?
[16:47:41] <anybroad> Will mongodb automatically referenciate to it?
[16:52:33] <dimon222> you cant
[16:52:45] <dimon222> mongodb is NoSQL, its not a relational database
[16:52:53] <anybroad> dimon222: so how would one do this?
[16:53:00] <anybroad> dimon222: I got categories and want to re-use them
[16:53:17] <hagb4rd> hi. i'm trying to mongorestore my dump on a remote machine mongodb installation.. and now i get the following error:
[16:53:24] <dimon222> you just add it manually, but you cant use references, because there are no references term in MongoDB
[16:53:31] <hagb4rd> ERROR: Error creating index test.logs_irc.german-elite.net: 16730 err: "bad textIndexVersion: 2"
[16:53:42] <hagb4rd> how to help myself now?
[16:54:03] <anybroad> dimon222: hm, ok
[16:54:25] <dimon222> anybroad, but basically, if you need references - you dont use MongoDB at all, and move to something like MariaDB/MySQL/Postgresql, Mongodb is supposed to be used as basic data warehouse
[16:55:32] <dimon222> or some document storage where files/json needs to be retrieves fast, its not fullfilling features of relations that are used for actual website CMS or anything that works with actual database of users
[16:55:39] <dimon222> *retrieved
[16:57:02] <dimon222> hagb4rd, you're restoring backup to the wrong version of MongoDB
[16:57:45] <dimon222> backups from different revisions of MongoDB are not compatible, 2.4.x - version 1, 2.6.x - version 2
[16:57:50] <anybroad> dimon222: thank you for clearing this up
[17:02:40] <hagb4rd> what? are you serious? :o
[17:02:54] <hagb4rd> what kind of backup is that?
[17:03:11] <hagb4rd> can i do sth about it now?
[17:03:53] <cheeser> i've not heard anything along those lines before.
[17:04:16] <cheeser> and what does "backup" mean in this case to you, dimon222 ?
[17:05:10] <hagb4rd> can i somehow move my data from a running older version of mongodb to a newer version? .. i mean.. it must be^^
[17:06:48] <anybroad> dimon222: ok, one thing: What is about 'references' in mongodb, can I use those to link?
[17:06:49] <hagb4rd> i thight every schoolchild would code a backup in a way it can be restored in future versions..
[17:08:42] <hagb4rd> can i just connect to both mongodbs and use sth like node.js to tranfer objects from one db to the other? that sounds reasonable though it seems still ridicuous to me there is no "native" and safe way to do this
[17:09:24] <cheeser> i do a mongodump from 2.4 and mongorestore to 2.6 on a regular basis with 0 problems.
[17:09:44] <cheeser> so you should probably curb your moral outrage until you do some investigation.
[17:10:22] <hagb4rd> well i thought coming here would be a goof start
[17:10:49] <cheeser> a goof start indeed
[17:11:12] <hagb4rd> because that's like a elementary school db usecase
[17:11:26] <hagb4rd> u guys should throw options at me
[17:11:33] <cheeser> again. investigate before you whip out the condescension.
[17:11:54] <cheeser> "we guys" owe you nothing. we offer input at our leisure based on our experience.
[17:12:25] <cheeser> if you want formal, official feedback, mongodb offers support options.
[17:12:33] <hagb4rd> is this x-files or sth? thanks. i know how to handle this. i don't need irc to be told to use google.
[17:12:37] <hagb4rd> consider this.
[17:13:52] <anybroad> So one should rather use *sql instead mongodb when dealing with 'usual' data? mongodb is intended for document, arbitrary data?
[17:14:16] <cheeser> i use mongodb for "usual" data.
[17:14:42] <cheeser> highly relational data might not be a good fit but it depends on how you access that data via relations
[17:15:20] <anybroad> aha
[17:27:35] <dimon222> hagb4rd, just create indexes manually, actual backuping of data should be already done when it reaches index creation process
[17:29:08] <dimon222> according to docs such error happens when you're moving data from newer version with new index type to old version that doesn't support it
[17:29:56] <dimon222> so correcting myself -> from old to new is okay, from new to old is not okay
[17:39:37] <cheeser> and that's reasonable.
[18:06:11] <anybroad> OK, so I got a book and I want to assign a category to it, books and categories coming from two different collections in the database.
[18:06:20] <anybroad> So how can I do this? Can I link them somehow?
[18:06:25] <anybroad> mongodb documentation tells something about relations.
[18:06:33] <anybroad> But how does this work?
[18:06:45] <anybroad> book.category_id - I want to use book.category.whatever
[18:06:46] <anybroad> so?
[18:10:07] <hagb4rd> dimon222: actually i found out i tried to restore the backup of a newer version into an older, which failure at last is reasonable. in consequence i could upgrade the empty db to a newer version! which in the end worked flawlessly. so .. i must excuse dear mongodb fans. also thanks for help dimon222
[18:13:05] <dimon222> anybroad, well, its kinda done manually, so you take id and you add it manually
[18:13:27] <dimon222> as soon as something gets removed, there's no real relationship, its kinda fake
[18:20:50] <anybroad> dimon222: so I need a orm-odm thing?
[18:20:52] <anybroad> or a rdbs?
[18:20:55] <anybroad> *rdbms
[18:21:27] <dimon222> yes
[18:21:50] <dimon222> if you want real reference aka "dependency" tree, you need rdbms
[18:22:21] <dimon222> otherwise in nosql you can only store copy of data as part of reference, but there's no realtime relationship
[18:26:33] <anybroad> dimon222: but how are larger enterprises do this? they got relational stuff, do
[18:26:34] <anybroad> *too
[18:33:51] <anybroad> So I am a bit mislead here, please give me some help finding back the right direction.
[18:34:02] <anybroad> For what are ODMs used? For blog posts?
[19:31:33] <mdev> does mongodb scale well
[19:31:47] <mdev> as in millions of records and still fast
[19:33:12] <kali> https://www.youtube.com/watch?v=b2F-DItXtZs
[19:37:33] <alexi5> lol
[19:38:20] <mdev> heh
[19:38:38] <mdev> is what he's saying accurate though, no indication the data you're trying to write gets written?
[19:44:50] <cheeser> no, it wasn't accurate then and isn't now.
[19:45:22] <cheeser> the default write way back then was unacknowledged writes (fire and forget though you could *ask* for confirmation)
[19:45:35] <cheeser> the default has been acknowledged writes for a few years now.
[20:30:32] <mdev> ahh yeah I just read that
[20:30:58] <mdev> what channel do you own cheeser, #java?
[20:31:29] <mdev> whatever, i'll tip you anyway, as I try and tip people that help
[20:55:41] <dimon222> <mdev> as in millions of records and still fast
[20:55:58] <dimon222> sharding
[22:19:40] <enigmeow> is there a specific scons ctarget to get the client library built and installed?
[22:39:24] <anybroad> hi again
[22:39:32] <anybroad> http://mongoosejs.com/docs/populate.html
[22:39:37] <anybroad> Will this do what I want (n:1) relation?
[23:00:53] <alexi5> hello ladies and gentlemen
[23:06:14] <sententiousness> What's the best place to ask general database related questions?