pmxbot IRC Log Viewer

[00:29:35] <pawn> joannac: idk

[00:33:18] <joannac> How did you install it?

[00:33:23] <joannac> see what's in ls /Library/LaunchDaemons/

[03:15:03] <michela> hi, should mongodump save capped attribute of a collection? restored collection was missing this

[04:56:39] <Botanic> hi i am trying to get the info out of a collection in a database, im familer with mysql so im assuming a collection in similer to a table when i run db.admin.find({}) i get nothing as output however from the console

[05:40:46] <terriblesarcasm> Botanic: admin being the collecting?

[05:40:56] <Botanic> yes

[05:41:00] <Botanic> i get no output

[05:41:09] <terriblesarcasm> I don't think you need the {} try find()

[05:41:18] <Botanic> on some of the other collectioning i get data

[05:41:35] <Botanic> is there something other then just find()?

[05:42:03] <terriblesarcasm> theres also findone which would find a single record, i don't use the console much let me see

[05:42:57] <Botanic> tryin to figure out if something happened to my database and data went poof, or if im just selecting it wrong

[05:44:03] <terriblesarcasm> hm interesting db.collection.find() doesn't return anything for me either

[05:44:15] <terriblesarcasm> i think it's because it's looking at the wrong DB

[05:44:21] <Botanic> it seems to work if there isnt a subobject

[05:44:28] <Botanic> from what i can guess

[05:45:10] <Botanic> > db.event.find()

[05:45:10] <Botanic> { "_id" : ObjectId("532b92523607f20c1d6e7855"), "t" : "event", "time" : NumberLong("1395364434345"), "datetime" : ISODate("2014-03-21T01:13:54.345Z"), "key" : "EvtNvrDiscoveredPending", "nvr" : "67ee9258-cd80-40e5-a270-d76bc17626a5" }

[05:45:13] <Botanic> for example works

[05:45:27] <terriblesarcasm> Botanic: if you do the command db what is the result?

[05:46:12] <Botanic> it shows the correct database

[05:47:08] <terriblesarcasm> hm, that's strange

[05:47:26] <Botanic> its returning data on some collections

[05:47:28] <Botanic> but not others

[05:47:33] <terriblesarcasm> it's case sensitive if that makes a different here

[05:48:29] <Botanic> i coppied it from "show collections"

[05:48:35] <Botanic> is there another tool i can use?

[05:48:45] <terriblesarcasm> I use the site mongolab

[05:49:22] <terriblesarcasm> mongolab.com I have an account, it gives you a free cloud mongo db, however I mostly use it as a visual representation of my server's db that you can add there

[05:56:25] <Botanic> i trid robomongo but same thing

[05:56:27] <Botanic> i see index's

[05:56:31] <Botanic> but no idea how to get at the data

[05:56:46] <terriblesarcasm> How was the data created?

[05:57:30] <joannac> what does "it doesn't have a subobject" mean?

[05:58:31] <Botanic> no idea how it was created this is a piece of software that we use

[05:58:47] <Botanic> joannac, if i do

[05:58:48] <Botanic> > db.admin.find({1})

[05:58:48] <Botanic> Thu Mar 20 22:57:29 SyntaxError: missing : after property id (shell):1

[05:58:58] <Botanic> it seems to see somethig there

[05:59:28] <Botanic> tho i have no idea what im doin so :)

[06:00:23] <joannac> Botanic: I don't think the {1} makes a difference

[06:00:27] <Botanic> http://codepad.org/ezWpsGWG

[06:00:29] <Botanic> for example

[06:00:50] <joannac> okay

[06:01:14] <joannac> not sure what that's meant to prove

[06:01:20] <joannac> let's start again

[06:01:37] <Botanic> like i said i have no clue what im really doin :P

[06:01:49] <joannac> type the following:

[06:01:55] <joannac> use ubnt-video

[06:02:07] <joannac> db.admin.find()

[06:02:10] <Botanic> got that far :)

[06:02:14] <Botanic> no output

[06:02:53] <Botanic> db.nvr.find() has output

[06:02:57] <joannac> show collections

[06:02:59] <Botanic> so im not totally off base :)

[06:03:02] <Botanic> its there

[06:03:18] <joannac> pastebin it

[06:03:19] <Botanic> http://codepad.org/asMzyItM

[06:04:10] <joannac> db.admin.stats()

[06:04:30] <Botanic> http://codepad.org/1olI7sJV

[06:04:48] <joannac> there you go. 0 documents

[06:05:04] <joannac> the collection is empty

[06:05:08] <Botanic> ok so it is empty

[06:05:11] <Botanic> how the heck...

[06:05:13] <Botanic> :(

[06:05:43] <Botanic> ok so im not just bein dumb

[06:05:45] <Botanic> thanks

[06:05:49] <joannac> np

[06:06:02] <Botanic> figured 50/50 i was just screwin up :D

[07:30:29] <fleetfox> Hello, how do i scroll the buffer in shell?

[07:35:01] <joannac> the mongo shell doesn't keep a buffer, it would be whatever terminal program you're using

[07:35:44] <fleetfox> oh, it was in screen so i was confused

[07:35:44] <fleetfox> :P

[07:36:54] <fleetfox> thanks

[07:46:31] <Dinoop> hi

[07:47:56] <Dinoop> In mongodb document

[07:47:57] <Dinoop> http://docs.mongodb.org/manual/core/index-sparse/#sparse-index-with-unique-constraint

[07:48:34] <Dinoop> the code example given "However, this index would not permit adding the following documents:"

[07:48:43] <Dinoop> is seems to be wrong

[07:48:56] <Dinoop> can anyone check it

[07:49:07] <Dinoop> ?

[08:07:40] <joannac> why do you think it's wrong?

[08:13:25] <Dinoop> db.scores.insert( { "userid": "PWWfO8lFs1", "score": 82 } )

[08:13:27] <Dinoop> and

[08:13:36] <Dinoop> db.scores.insert( { "userid": "XlSOX66gEy", "score": 90 } )

[08:13:42] <Dinoop> are given as examples

[08:14:03] <joannac> yes, I can see that

[08:14:10] <joannac> I'm asking *why* you think it's incorrect

[08:14:22] <Dinoop> ok I will tell

[08:15:03] <Dinoop> As per documentation "However, this index would not permit adding the following documents:"

[08:15:14] <Dinoop> why it is not permitting ?

[08:15:14] <sqlnoob> Hi I'm trying mongo just for fun. I'm trying to select documents based on _id. How's it possible to pass multiple _id in the mongo like IN query in SQL?

[08:15:57] <joannac> Dinoop: unique index won't let you add duplicates

[08:16:03] <Dinoop> score is indexed as sparse

[08:16:05] <joannac> sqlnoob: $in

[08:16:17] <joannac> sqlnoob: http://docs.mongodb.org/manual/reference/operator/query/in/

[08:16:22] <Dinoop> but there is no duplicate for score field

[08:16:28] <joannac> Dinoop: yes there is

[08:16:29] <sqlnoob> thanks joannat and Dinoop

[08:17:05] <Dinoop> joannac ,

[08:17:07] <Dinoop> "score": 82

[08:17:16] <Dinoop> and "score": 90

[08:17:23] <Dinoop> is not duplicate ryte?

[08:17:58] <joannac> look earlier in the article where it says "consider a collection scores that contains the following documents

[08:18:23] <joannac> there's already a document with score: 82

[08:18:23] <Dinoop> oh... sorry joannac now I understood

[08:18:53] <Dinoop> I only saw that last two examples

[08:19:24] <Dinoop> Thank you, and sorry if I disturbed you

[08:19:37] <Dinoop> joannac

[09:44:11] <jinmatt> which user permission is required to perform db.copDatabase() ?

[09:45:22] <balboah> the sheriff permission

[09:47:27] <jinmatt> how do everyone usually manage users in mongodb production? I'm totally new to this. I have a admin user with userAdminAnyDatabase and another user with readWrite and dbAdmin permission, which I use within the application to connect to mongodb

[09:52:38] <jinmatt> so many people here and no one every answers anything here?

[09:53:12] <balboah> that's usually the case

[09:53:16] <balboah> I don't use multiple users

[09:58:23] <jinmatt> balboah: then how do you do it? and which all permission do you assign to your mongodb user?

[09:58:49] <balboah> I don't assign any, I use the default

[09:58:56] <balboah> which is no particular user

[09:59:51] <jinmatt> in production?

[10:00:29] <jinmatt> then the mongodb will be open to the world, nah? anyone can access it from anywhere if they know the ip address?

[10:00:33] <balboah> mongodb didn't even have this functionality from the start

[10:00:54] <balboah> well you would probably not allow internets to connect to your production db's

[10:01:09] <balboah> I wouldn't trust mongodb user privileges to block that

[10:02:51] <noqqe> jinmatt: i run my mongocluster in a secure environment and use separate users for each db and keyfile on. because it works.

[10:02:54] <jinmatt> I use mongodb currently like that without no users, but still I feel its not enough, thats why I was asking how everyone does it in their production

[10:04:09] <jinmatt> so you only access to mongodb is allowed from specific ips, opened on your firewall?

[10:04:51] <balboah> no way of connecting to it from outside world

[10:06:27] <jinmatt> my current problem is that I have dynamic auto scaling app servers, so I can't specify the range of IPs from whihc my app servers connect to mongodb

[10:09:06] <Nodex> surely your app servers can notify a service that they're up and running

[10:09:11] <Nodex> + their IP

[10:26:35] <balboah> jinmatt: depends on your service provider as well, for example in Amazon you could specify a security group that allows its own group to talk to itself, not depending on specific ip's

[10:27:07] <balboah> or as Nodex says you could listen for events and change things

[11:19:08] <LDA_> Hi all. Could someone lend me a hand with a mongoDB query from the perl api?

[11:19:20] <LDA_> I can't seem to get it to work for the life of me

[11:22:00] <retran> don't ask to ask

[11:22:07] <retran> just state the details

[11:27:34] <LDA_> @retran making pastie

[11:40:19] <LDA_> http://pastie.org/8956237

[11:40:32] <LDA_> those are the relevant snippets to my problem

[11:40:37] <LDA_> mongodb document

[11:40:46] <LDA_> tested query in mongoshell which works

[11:41:01] <LDA_> perl code i am trying to use which does not seem to execute the query

[11:53:04] <Nodex> LDA_ : mongodb is an implied AND, I think you can lose some complexity (if I'm reading your code properly)

[11:53:56] <LDA_> @Nodex How could I loose some of the complexity?

[11:54:47] <Nodex> are your opts always key/value pairs?

[11:56:07] <LDA_> some of the arrays in opts may repeat themselves

[11:56:26] <LDA_> as in "reference","url1"

[11:56:29] <LDA_> "reference","url2"

[11:56:41] <Nodex> yeh

[11:56:49] <LDA_> from what i can tell it is just a array of arrays

[11:56:51] <LDA_> no key /value

[11:57:01] <Nodex> it's adding complexity for no reason

[11:57:42] <LDA_> the structure come from http://search.cpan.org/~rharman/Parse-Snort-0.6/lib/Parse/Snort.pm

[11:58:32] <LDA_> i am basically reading a complex line from a file and parsing it with that object which i just dump into mongo as is

[12:00:07] <Nodex> it would be far better to dump it in like this... opts : [{"gid":1},{"url":"/bar"}] .. then you can query like this .. {"opts.gid":1, "opts.url":"/bar"}

[12:01:22] <J43> my boss wants to use elasticsearch as our main data store instead of mongodb, its an illogical idea but i cant think of a reason not to ._.

[12:01:30] <LDA_> it would make my life much less painful that is for sure

[12:02:28] <LDA_> @J43 cant logstash use both?

[12:03:16] <Nodex> J43 : elastic search is a search daemon not a data store, your boss is an idiot

[12:03:29] <Nodex> Tell him "Lets use the right tool for the job"

[12:03:35] <J43> hahaha thats what i said Nodex

[12:03:45] <Nodex> (which might not even be mongodb

[12:04:22] <J43> it could be used as a data store and thats why they are considering it but i cant find a good reason to make them change their mind

[12:04:28] <J43> other than... its not a data store

[12:04:42] <J43> ¬_¬

[12:06:09] <Nodex> I don;t know ES all that well but I can only imagine that the insert / update throughput will not be as good as something more suited to it

[12:49:38] <kali> Derick: are you here ?

[13:01:00] <Derick> kali: yeah

[13:16:32] <kali> Derick: great. maybe you can help me a bit with this: i'm hacking a proxy for mongodb wire protocol, and i was hoping that i could somewhow hijack some driver or metadrivers tests to check i'm not breaking more than i intend to

[13:17:18] <cheeser> if it's a proxy, there's no need to hijack. just point the tests at your proxy.

[13:18:19] <kali> cheeser: yeah. but which set of test should i use ? jstest ?

[13:18:44] <kali> cheeser: i was hoping the metadriver has some authoritative test suite covering the protocol

[13:19:18] <cheeser> dunno much about that one.

[13:19:39] <cheeser> i would expect any driver's tests would cover most of the protocol facets that matter to most users.

[13:20:56] <kali> yeah, I just need to pick a convenient one

[13:30:05] <katspaugh> Hi! I'm opening a collection, then insert record by record upon reading lines from a CSV-file. Sometimes it hangs. Sometimes it won't unhang until I drop the collection. How do I approach this problem?

[13:30:55] <cheeser> how are you importing?

[13:30:58] <katspaugh> Database connection hangs, that is.

[13:31:32] <katspaugh> cheeser: through a Node.js driver. I'm reading lines of CSV, transform them into JavaScript objects and then insert.

[13:31:48] <katspaugh> I set write concern to 0 (unacknowledged).

[13:32:43] <cheeser> 1. probably shouldn't use unacknowledged. 2. put in debug to verify it's not your app that's hanging. 3. connect to mongo via the shell and see if the db is still responsive.

[13:34:00] <katspaugh> cheeser: thanks! What would change if I use w: 1?

[13:34:51] <cheeser> your throughput would drop because you'd be waiting for the ack. one possibility is that you're saturating your connection.

[13:37:05] <katspaugh> cheeser: hmm, interesting! Thanks, I'll try your suggestions.

[13:39:15] <katspaugh> Oh, one more thing. Are there any gotchas when tunneling a MongoDB connection through ssh?

[13:39:41] <cheeser> i can't think of any

[13:39:47] <katspaugh> OK :)

[13:41:44] <pithagora> hi all. i do a dump of db from one mongo instance and restore it into an other instance. the db in first instance has about 34 G, as per mongo client, while in second it has only 2. how its possible?

[13:41:53] <pithagora> i do show dbs in mongo client

[15:24:19] <doxavore> I recently started getting errors in 2.4.9 that my user, that i've been using for ages, isn't authorized to create the gridfs fs.chunks index. the user has both dbAdmin and readWrite privs to this DB... is there something else they need?

[15:43:30] <SolarSailor> Hi guys - I'm considering using Mongo for a project in which I need to receive, store and parse JSON documents that may be nearly identical - but not totally identical. One of the challenges I face in my PoC is how to search for an element in a document. I can only be sure that I know the name of the element, not its place (path) in the document. How do I do that?

[15:43:58] <SolarSailor> It should be said that the document may be of arbitrary depth.

[15:45:39] <Nodex> you can't search key names

[15:45:56] <Nodex> and you must specify the depth .. i/e "foo.bar.baz.fred" : "/abc"

[15:45:58] <SolarSailor> I'm used to querying XML with XPath and CSS expressions, in which you can make these kinds of searches, but I can't seem to find out how to perform such searches in Mongo.

[15:46:03] <SolarSailor> Oh, I see

[15:47:03] <SolarSailor> So if the document isn't following a well-known schema, I can't lookup an element just by element names? I still have to know the structure of the document in order to find it?

[15:49:54] <Nodex> your app does yes

[15:50:08] <SolarSailor> Another thing I've considered is using Hadoop+MapReduce, but it seemed overkill at the time. Hmm, OK, thanks for the effort.

[15:58:13] <BlakeRG> hey all, i have a collection of documents and i want to run a query against all documents and pull out a unique, counted list of unique values for a single field - analogous to a GROUP BY in SQL. Any tips?

[16:00:40] <cheeser> db.collection.distinct("value").length

[16:57:20] <NaN> hi there

[17:06:14] <daveys110> hi. on M101J: MongoDB for Java Developers ; HW 5.2, I'm not sure I understand the question - can anyone help with this?

[17:07:06] <daveys110> I've written what I believe is the correct aggregation function but I must be mis-understanding the question because the check-value doesn't match.

[17:08:08] <daveys110> db.zips.aggregate([ {$match:{pop:{$gte:25000}}}, {$match:{state:{$in:["CA","NY"]}}}, {$group:{ _id:null, total:{$avg:"$pop"} ]);

[17:15:41] <jchamberlain> hey all

[17:16:07] <jchamberlain> dumb question, but in a schema-less world, where in your code do you define things like indexes and stuff?

[17:16:18] <jchamberlain> I'm coming from a PHP+MySQL background

[17:16:42] <jchamberlain> and am used to having a migration script setup the schema of the database and its indexes

[17:17:01] <NaN> do I need mapreduce to get this? > https://ghostbin.com/paste/pwsgu

[17:17:21] <jchamberlain> but in mongo I don't have a place that's setting up the collection's schema, so I'm not sure what the best practice is for where to do that kind of thing

[17:24:03] <jchamberlain> anybody?

[17:24:52] <NaN> jchamberlain: http://docs.mongodb.org/manual/core/indexes-introduction/

[17:25:43] <jchamberlain> NaN, I've actually read that, and it doesn't answer my question, but thanks. :)

[17:26:07] <NaN> oups, sorry, then I don't know, I'm new in the nosql world :)

[17:28:16] <jchamberlain> np

[17:28:29] <jchamberlain> I guess this gets close to answering my question: http://stackoverflow.com/questions/7000777/mongodb-when-to-call-ensureindex

[17:30:43] <NaN> looks good

[17:51:28] <Jadenn> is there a way to refer to other objects, so when i query for one object, and there is a field that refers to another object, it returns that referred objects data?

[17:56:51] <Joeskyyy> you just used a lot of words for "joins" haha

[17:57:35] <Jadenn> i know, i also know there are no joins

[17:57:38] <Jadenn> *sigh*

[17:57:44] <Joeskyyy> which mongodb doesn't support, but there ways around it most times

[17:58:20] <Joeskyyy> http://docs.mongodb.org/manual/reference/database-references/

[17:58:31] <Jadenn> mongo has really improved my apps performance, i could get away with recursively querying

[18:04:20] <Nodex> Jadenn : perhaps think about duplicating your data to further improve performance

[18:04:46] <Jadenn> i will probably just kaibosh the dynamic nickname thing

[18:04:56] <Jadenn> users can live with usernames lol

[18:04:57] <Nodex> haha

[18:05:17] <Nodex> or put them in memcache/redis ;)

[18:06:29] <Jadenn> well when you have queries like (f.friendUid = m.id && (uid = :uid && confirmed = 1)) || ((f.uid = m.id && f.uid != :uid) && (f.friendUid = :uid && confirmed = 1)) you really have to simplify

[18:12:25] <Jadenn> forgive me almighty Ledorf, for today i shall be recursively recursing

[18:13:16] <cheeser> i'm always recursing. stupid bugs...

[18:13:34] <Jadenn> i meant i have to foreach inside of a foreach

[18:20:52] <Jadenn> what would be the best way to store the friends? having a friends object with field named the friends id, and the value his username, or with each friend having their own object and appropriately named fields

[18:24:24] <Nodex> this si the 64k question

[18:24:26] <Nodex> is*

[18:25:03] <Nodex> I did it in mongo and it didn't scale as well as I needed so I ended up storing in redis

[18:26:07] <Jadenn> individual objects it is then <_>

[18:46:00] <talbott_> hello all

[18:46:02] <talbott_> quick q

[18:46:08] <NaN> hi talbott_

[18:46:28] <talbott_> my arbiter, can it be a lot lower spec than my primary and secondary nodes in a replicaset?

[18:46:38] <talbott_> trying to save a bit of £$£

[18:46:52] <talbott_> as i originally built primary, secondary AND arbiter as high spec boxes

[18:46:59] <talbott_> and i'm thinking the arbitur can be low spec

[18:51:17] <NaN> is there a way to execute js files in shell? something like $mongo --execute=myscript.js

[18:52:26] <talbott_> no

[18:54:02] <rkgarcia> talbott_: exists mongo < script.js

[18:54:21] <talbott_> no

[18:55:00] <NaN> it worked, like a normal console command

[18:56:54] <talbott_> sorry got really confused

[18:57:10] <talbott_> i read that as is my ARBITER required to execute js commands in its shell :)

[18:57:25] <talbott_> then i thought rkhatcia was asking cryptically, again!

[18:57:36] <talbott_> i knew i shouldn't have had that red stripe!

[19:43:51] <NaN> is there an $out operator por pipeline aggregation?

[19:44:30] <NaN> nevermind

[19:48:22] <cheeser> yes, there is. :)

[20:01:43] <MANCHUCK> can i use $slice and $sort operators with $addToSet?

[20:08:00] <NaN> cheeser: is it for 2.4.9?

[20:09:37] <cheeser> no. 2.6

[20:10:20] <NaN> :c any workaround for 2.4.9?

[20:18:24] <cheeser> for $out? no.

[20:18:50] <cheeser> well, you could run js in the shell to execute the pipeline, iterate the docs that come back and insert them in to some collection

[20:25:18] <NaN> cheeser: that's what I'm doing but what kind of object does aggregate returns?

[20:25:49] <cheeser> an array of docs, iirc

[20:27:31] <NaN> I doub, it has no forEach method

[20:28:54] <cheeser> just print it. what do you see?

[20:30:35] <NaN> the docs

[20:30:36] <NaN> mmm

[20:30:54] <NaN> insert(docs);

[20:30:58] <NaN> u_u'

[20:31:03] <NaN> got it

[20:31:05] <NaN> thanks cheeser

[20:31:11] <cheeser> np

[20:31:54] <cheeser> in 2.6, you can get a cursor back as well so you're not limited to 16MB

[20:33:09] <NaN> looks nice, but seems there isn't any update for fedora jet

[20:45:20] <alexw> I have a list of business activities (i.e. fashion wholesaler), and I want to be able search this list (I assume using mongodb). But I need it to match similar keywords, i.e. if I search "car" it will return automotive manufacturer

[20:45:28] <alexw> Is this mongodb or spinx?

[20:45:34] <alexw> sphinx*

[21:01:09] <NaN> is there a way to format a mapreduce output or do I need to aggregate over the new collection?

[21:06:56] <LoneSoldier728> hey so I have a user and a list, a list contains two arrays of users good and bad... and the user has two arrays good and bad... I want to ask for lists, but then need to check whether or not the user is in the good or bad list, to avoid showing it... what is the most efficient way? Option a: grab the user and all the lists, and check each user's good and bad array if they have the list id if so ignore it? Or is there a more efficient way?

[22:34:19] <redsand> hey i've ran into something

[22:34:36] <redsand> has anyone ever experience aggregate function truncating float values ? I'm not getting the full value stored

[23:29:01] <__j0k3r__> hallo gents

[23:29:18] <__j0k3r__> i have a little question about mongodb mapreduce

[23:30:14] <__j0k3r__> i have a mapreduce job (that is working), but i need it to be run on an *ordered* sset, so i need to specify the sort condition on the query

[23:30:20] <__j0k3r__> but i am not able to achieve that

[23:30:27] <__j0k3r__> i am using pymongo

[23:30:50] <__j0k3r__> i am trying to order the values collection in the reduce step

[23:30:57] <__j0k3r__> but apparently does not work

[23:31:17] <__j0k3r__> so i think i should specify an order criterium on the general query

Log file Viewer

Help | Karma | Search:

#mongodb logs for Friday the 21st of March, 2014