pmxbot IRC Log Viewer

[03:51:54] <macUzer> how do I reach into the JSON structure to retrieve other fileds in arrays when using mongoexport's -f option?

[04:25:57] <Jeffrey_> Hey. I am currently loading up 2.7gb of CSV files to mongo and I am having some weird write degradation. At first I was inserting at around 100ms for each insert, and now after loading about 1.5gb I am at 26 seconds. Anyone see anything like this?

[04:26:08] <Jeffrey_> Looking at mongostat the lock % is at 0 most of the time.

[04:26:15] <Jeffrey_> And I am running a OCZ vertex 4 SSD.

[04:27:29] <Jeffrey_> And mongod is pegged at 100%

[10:28:04] <_johnny> hi, i have set up a collection with documents like: {prop1: .., prop2: ..., chapters:[{chapter_id: .., start: 0, end: 1}]}. with a named index of chapeters.chapter_id, and for chapters.start and .end. so appending is easy with $push. i'm wondering, can i use $pull to removed based solely on chapters.chapter_id, or do i need to recreate the entire object (with id, start and end)?

[10:37:02] <mids> _johnny: $pull should work, tried it?

[10:46:21] <_johnny> mids: yes, i'm trying with update({prop1:..{$pull{ chapters { chapters.chapter_id: '..'}..

[10:46:49] <_johnny> but bad timing, i have to go. i see that it should work, like how ... find.. works lol :) i will give it a go. thanks

[11:50:36] <multiHYP> hi

[16:55:00] <gnoirzox> hi

[18:06:54] <_johnny> mids: sorry to bug you directly, but do you have a sec? :)

[18:11:31] <mids> _johnny: shoot

[18:15:44] <_johnny> mids: thanks. i've written up an example of a doc here http://pastebin.com/mubuuYmW

[18:16:38] <_johnny> my pull command runs and "matches" 1 row (as it should, but that's just the find criteria hehe), but nothing is changed. the document is identical before and after

[18:17:09] <mids> let me digest that

[18:17:18] <_johnny> there are no errors from syntax or the cursor, so i must be screwing up the notation

[18:19:23] <mids> chapters isnt an array

[18:19:40] <mids> that could be it

[18:20:55] <_johnny> hmm. i mean, yes i can see that it isn't a array in the regular js sense, but that format might be the tool i use to view it (rockmongo). the actual query which adds to it (and produced that output) uses $push

[18:22:37] <mids> ok, let me mess with this

[18:23:27] <_johnny> http://www.mongodb.org/display/DOCS/Updating#Updating-%24pull hmm, is this suppose to be read as that i need to provide the "chapter" as a value here? that would mean that i had to supply not just chapters.chapter_id, but also the start, end, and name property

[18:25:54] <kali> _johnny: $pull : { chapters: { chapter_id : xxx } }

[18:26:29] <mids> https://gist.github.com/07139b9c096f8ea3f24b

[18:26:32] <dstorrs> I'm setting up a new mongo and want to make sure I config it properly. Is there a recommended guide anywhere, aside from docs?

[18:26:35] <mids> yeah that works just fine

[18:26:52] <dstorrs> I don't want to miss some key option that would have made a big difference

[18:27:50] <_johnny> kali mids: thank you guys so much! :)

[18:38:45] <mids> dstorrs: what kind of setup are you planning? replica sets? sharding?

[20:12:39] <__neilg> i have a primary-secondary-secondary repl set configured. I rs.initiate()ed the primary, and the two secondaries are no in RECOVERING. All machined are on a gigabit LAN, but are syncing at about 5k/s. What am I doing wrong?

[20:13:04] <__neilg> (2.0.1 on all)

[20:13:13] <Derick> does the primary has lots of load?

[20:13:25] <__neilg> 0.41, 0.34, 0.42

[20:13:41] <Derick> no, how many writes f.e. (see mongostat output)

[20:13:49] <Derick> and check vmstat too

[20:13:52] <__neilg> one sec

[20:14:04] <Derick> (run "vmstat 2" for about a minute on the primary)

[20:15:12] <__neilg> http://www.pastie.org/4144487

[20:16:41] <Derick> hmm, looks okay

[20:16:59] <__neilg> yeah it's weird

[20:17:07] <Derick> that's on the primary, right?

[20:17:30] <__neilg> correct

[20:17:40] <Derick> how does it look like on the secondary

[20:18:42] <__neilg> http://www.pastie.org/4144501

[20:19:07] <__neilg> that's the one secondary

[20:19:13] <__neilg> should be similar on the other

[20:19:42] <Derick> the primary has quite a bit of traffic though, 1mb/second?

[20:20:05] <__neilg> it's in production, so at the moment our web servers are pointing at the primary

[20:20:50] <Derick> hmh

[20:21:15] <Derick> is this an initial sync?

[20:21:21] <__neilg> yeah full initial sync

[20:21:33] <Derick> weird- no idea right now, sorry :S

[20:21:37] <__neilg> it's taken about 4 hours to do 3 million small records

[20:21:48] <__neilg> and we have about 20 to go :/

[20:21:50] <__neilg> cool thanks

[20:25:38] <__neilg> is this normal during an initial sync? "optimeDate" : ISODate("1970-01-01T00:00:00Z"),

[20:25:45] <__neilg> (on the secondary)

[21:16:28] <spillere> my db is { "_id" : ObjectId("4fe7742b99883608e2000000"), "photos" : [ { "file_folder" : "zc2gjq0vfi" }, { "file_folder" : "3zi17o4r6l" } ] }

[21:16:50] <spillere> i wanna run a query and check if a file_folder exists

[21:16:58] <spillere> z = db.dataz.find({"username": cookie_usr}, {"photos.file_folder": picname}), but it's wrong

[21:17:45] <spillere> and check if z == Non

[21:17:46] <spillere> none

[21:18:30] <wereHamster> why is it wrong?

[21:20:05] <spillere> I run the query db.dataz.find({"username": 'dansku'}, {"photos.file_folder": "zc2gjq0vfi"}) and the result is ->> { "_id" : ObjectId("4fe7742b99883608e2000000"), "photos" : [ { "file_folder" : "zc2gjq0vfi" }, { "file_folder" : "3zi17o4r6l" } ] }

[21:20:22] <wereHamster> spillere: in the console?

[21:20:28] <wereHamster> mongo shell I mean..

[21:20:45] <spillere> i'm doing in python

[21:20:50] <spillere> let me show how the console iis

[21:20:59] <wereHamster> what is the semantics of the find() method?

[21:21:07] <spillere> whe I query db.dataz.find({"username": 'dansku'}, {"photos.file_folder": "zc2gjq0vfi"})

[21:21:14] <wereHamster> well, that is clearly wrong

[21:21:23] <wereHamster> look up which arguments the find() method takes

[21:21:25] <spillere> { "_id" : ObjectId("4fe7742b99883608e2000000"), "photos" : [ { "file_folder" : "zc2gjq0vfi" }, { "file_folder" : "3zi17o4r6l" } ] }

[21:21:33] <spillere> well, what I have is

[21:22:04] <spillere> {username: username, photos [ {photoname: photoname}, {photoname:photoname} ]}

[21:22:16] <spillere> if you can understand

[21:22:32] <spillere> I wanna run a query to check in a photoname existis in the database

[21:22:32] <wereHamster> yes I do. But you are ignoring my advise

[21:22:42] <wereHamster> you are passing two arguments to the find() function

[21:22:45] <spillere> ill look now

[21:22:59] <wereHamster> does that function take two arguments? What is the first argument and what is the second argument?

[21:22:59] <spillere> I saw that somewhere in the internet :)

[21:23:04] <wereHamster> http://www.mongodb.org/display/DOCS/DBA+Operations+from+the+Shell

[21:23:09] <wereHamster> yes, you saw it on google

[21:24:10] <spillere> how do I look for a embedded data?

[21:24:51] <wereHamster> did you figure out what you did wrong in the find() call?

[21:29:53] <spillere> sry was watching the penalties

[21:31:22] <spillere> i

[21:31:30] <spillere> idk

[21:31:47] <spillere> ah

[21:31:51] <spillere> i'm doing the other way around?

[21:32:05] <wereHamster> what other way

[21:32:13] <spillere> nope

[21:32:16] <spillere> what was i doing wrong?

[21:32:39] <wereHamster> spillere: explain me which arguments teh find() call takes

[21:32:51] <spillere> db.foo.find( [query] , [fields])

[21:33:06] <wereHamster> spillere: what does query and fields represent?

[21:33:51] <spillere> query is the filter, and the second only the field I want to return :)

[21:34:31] <wereHamster> you are calling the find() function with query=? and fields=? (fill in the quesiton marks)

[21:35:27] <spillere> {"username": 'dansku'} and {"photos.file_folder": "zc2gjq0vfi"}

[21:35:43] <wereHamster> is {"username": 'dansku'} a valid value for the query argument?

[21:35:50] <spillere> yeah

[21:35:58] <spillere> nope?

[21:36:02] <wereHamster> is {"photos.file_folder": "zc2gjq0vfi"} a valid value for the fields argument?

[21:36:23] <spillere> yeah

[21:36:26] <spillere> i guess

[21:36:29] <wereHamster> no. now you explain why

[21:37:08] <spillere> when I run a find() I get this http://pastie.org/4144850

[21:37:34] <spillere> let me think

[21:37:35] <wereHamster> spillere: explain in english words what find({"username": 'dansku'}, {"photos.file_folder": "zc2gjq0vfi"}) returns.

[21:38:29] <spillere> from username = dansku, return the information from the embedded photos containing file_folder = zc2gjq0vfi

[21:40:08] <spillere> nope?

[21:40:28] <wereHamster> no

[21:40:59] <spillere> i'm new with mongodb, those find's are not 100% on my head :)

[21:41:10] <wereHamster> it means: find all documents where username=dansku (and in the matching documents only include fields which have "photos.file_folder"="zc2gjq0vfi".

[21:41:47] <wereHamster> for documents which don't have that field, mongo will return the whole document

[21:42:11] <wereHamster> or an empty document.

[21:42:25] <wereHamster> you can trivially test it with: db.foo.insert({a:1,b:2}); db.foo.find({a:1},{c:1})

[21:42:53] <wereHamster> what you want is: find me a document where username=dansku AND photos.file_folder=zc2gjq0vfi

[21:44:17] <spillere> exactly! :)

[21:45:10] <spillere> and return false if there is none

[21:45:45] <wereHamster> what you can do is check if count(...) == 1

[21:45:59] <wereHamster> because you don't actually want the document, you only want to see if it exists

[21:47:23] <spillere> on another part of the code, i was using like z = db.dataz.find_one({'photos.file_folder':fname})

[21:47:31] <spillere> find_one instead of find count=1

[21:47:55] <spillere> but how would be the query be for that?

[21:48:06] <wereHamster> username=dansku AND photos.file_folder=zc2gjq0vfi

[21:48:27] <spillere> db.dataz.find(username=dansku AND photos.file_folder=zc2gjq0vfi)?

[21:48:50] <wereHamster> that is not even valid python, nor javascript

[21:49:07] <spillere> i mean

[21:49:15] <spillere> the mongodb query

[21:49:51] <wereHamster> the first argument to find() is an object. How would you express an AND condition?

[21:50:17] <wereHamster> you used {"username": 'dansku'}. It's not rocket science to figure out how to do an AND

[21:51:56] <spillere> db.dataz.find({username:dansku,photos.file_folder": "zc2gjq0vfi})?

[21:51:59] <spillere> something like this?

[21:52:20] <wereHamster> that is not valid javascript nor python

[21:53:15] <spillere> i'm confuse

[21:53:25] <spillere> i know how to do it python

[21:53:29] <wereHamster> well, try and run it in python or teh mongo shell

[21:53:53] <spillere> does it work?

[21:53:57] <wereHamster> you tell me

[21:54:01] <spillere> db.dataz.find({'username':'dansku' and 'photos.file_folder': "zc2gjq0vfi"})

[21:54:05] <wereHamster> I don't have access to your database.

[21:54:33] <spillere> i mean, would that AND work there?

[21:54:52] <wereHamster> no, because that is NOT VALID PYTHON NOR JAVASCRIPT

[21:54:57] <wereHamster> just make it valid code and try.

[21:55:29] <spillere> valid python you mean what's inside find?

[21:55:50] <wereHamster> open a python shell and type: x = {'username':'dansku' and 'photos.file_folder': "zc2gjq0vfi"}

[21:56:28] <spillere> ok

[21:56:48] <spillere> db.dataz.find('username'=='dansku and 'photos.file_folder'== 'zc2gjq0vfi')

[21:56:59] <wereHamster> is *that* valid python?

[21:57:35] <wereHamster> do you even know python?

[21:57:43] <spillere> i know

[21:57:50] <wereHamster> it seems that you fail even at basic python understanding

[21:58:00] <spillere> but I don't know how to use mongodb and python toguether

[21:58:31] <wereHamster> what data type is {"username": 'dansku'}

[21:59:06] <spillere> it's a dictionary

[21:59:16] <wereHamster> so it can have multiple key/value pairs, right?

[21:59:32] <spillere> db.dataz.find({'username':'dansku', 'photos.file_folder': "zc2gjq0vfi"})

[21:59:46] <wereHamster> no go to your mongo shell and try that

[21:59:49] <wereHamster> now go ..

[22:00:01] <spillere> duh

[22:00:04] <spillere> taht woudlnt work

[22:00:11] <wereHamster> no? why not

[22:00:14] <spillere> it return the whole user

[22:00:41] <wereHamster> so do a count() instead of a find()

[22:01:27] <spillere> retured 1

[22:01:32] <spillere> seems right then

[22:01:35] <spillere> db.dataz.count({'username':'dansku', 'photos.file_folder': "zc2gjq0vfi"})

[22:01:49] <spillere> yay

[23:45:59] <jribeiro> Hi all!! I'm starting with mongo and was wondering if anyone could help me out on this or point me to some good documentation... http://stackoverflow.com/questions/11181810/mongo-db-mapreduce-in-php

Log file Viewer

Help | Karma | Search:

#mongodb logs for Sunday the 24th of June, 2012