PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Monday the 18th of March, 2013

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[01:29:39] <fjay> what events force a primary election in a replica set?
[01:29:45] <fjay> s/force/cause/
[01:45:34] <oinkon> how did one do simple summing operations before the aggregation framework? Map reduce?
[03:33:55] <michaeltwofish> Do users replicate across replica set members?
[03:35:33] <michaeltwofish> I could have sworn that they did last time I added users, but I've just added a user that hasn't replicated. I can't find anything in the documentation.
[05:12:26] <michaeltwofish> It helps when you create the user on the right database. Yes, users do replicate.
[05:46:59] <pilgo> What's wrong with this query? http://pastie.org/6599019
[05:47:32] <pilgo> I'm getting this error: Modifier $pushAll/pullAll allowed for arrays only
[06:09:07] <pilgo> What's wrong with this query? http://pastie.org/6599019
[06:09:13] <pilgo> Sorry, I got disconnected
[08:06:41] <maasan> I have first name, middle name and last name. I want to search for names. I used 'or' operator among each names. But it does not work when user gave both first name and lastname in the string.
[08:46:11] <[AD]Turbo> hola
[10:29:27] <phretor> Hi. Is there any way to improve the speed of aggregated queries with a $match clause on a grouped field? E.g., db.tags.aggregate([{"$group": {"_id": "$subject", "count": {"$sum": 1}}}, {'$match': {'count': {'$gt': 2}}}])
[10:49:39] <darnte> hi guys, my app have a filter for specific fields in my document ( name, provider, email, company ). What is the best approach, create a compound index for each field combination or use map reduce ?
[10:50:18] <Nodex> dont use map/reduce
[10:52:02] <darnte> thanks Nodex !
[10:52:11] <darnte> nnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnn
[10:52:21] <darnte> sorry guys !
[10:52:46] <darnte> Nodex: how could I handle the filter
[10:53:24] <Nodex> it depends how you use the filter...
[10:53:32] <darnte> Nodex: compound index , the user could search for company name is "foo" and provider startwith "go"
[10:53:34] <Nodex> is it a combination of any of the above
[10:54:57] <Nodex> are they "AND" searches
[10:55:05] <darnte> for instance: company is "foo" and provider startwith "go" and company is "whatever"
[10:55:07] <darnte> yeah
[10:55:23] <Nodex> how big is your collection
[10:55:46] <darnte> Nodex: not so big, now the collection has 100k documents
[10:56:08] <Nodex> for me personaly i would not handle this in Mongo
[10:56:40] <darnte> Nodex: nice, where do you handle that ?
[10:56:50] <Nodex> but if you must then I would have an "idx" array with name,provider,email,company and regex an $and
[10:56:59] <DinMamma> Hello, im doing a mongorestore into a sharded cluster, unfortunately one shard have most of the shards(the key is a hash). I saw this on a small collection(4gb) were most of the shards ended up on one shard and only when the restore of the collection was finished the chunks distributed evenly.
[10:57:06] <Nodex> solr or elastic search or somehting like this
[10:57:39] <DinMamma> The collection im restoring now is 500gb big and yeah, one shards holds 1600 chunks while the two other shards only holds 900 chunks.
[10:58:06] <DinMamma> Can I safely ctrl-z my mongorestore and wait for the cluster to rebalance itself and the foreground the process again?
[10:58:14] <darnte> Nodex: interesting, create a new field "idx" as array and combine all fields, right ?
[10:58:36] <Nodex> combine and normalise
[10:59:11] <Nodex> idx:['a',123,'c','d'] .....
[10:59:16] <darnte> Nodex: what means normalize ? parse the fields, stemming ...
[10:59:47] <Nodex> put them to lowercase or w/e
[10:59:59] <darnte> Nodex: yeah yeah, what I thought
[11:00:10] <darnte> Nodex: and if I need OR searches ?
[11:00:17] <Nodex> db.foo.find({$and:[idx:/^a/,idx:/^b/]})
[11:00:37] <Nodex> or searches are just db.foo.find({idx:/^a/});
[11:00:54] <Nodex> ^^ that will select anything in the "idx" array starting with "a"
[11:01:26] <Nodex> the former will select anything with a AND b in any of the array fields - not really ideal when you need to serach exact field names
[11:01:29] <darnte> but if I have company "foo" or name is "dar"
[11:01:51] <Nodex> you can do an "or" in regex
[11:02:01] <Nodex> ... /^foo|bar/
[11:02:21] <Nodex> Mongo is really not designed for this kind of thing
[11:02:32] <darnte> Nodex: yeah, as u told, I better create idx as an object to search for field name
[11:02:42] <darnte> better use elasticsearch
[11:03:17] <Nodex> I prefer solr personaly but yes elastic search would work fine
[11:03:58] <darnte> thanks for help
[11:04:08] <Nodex> yw
[11:04:10] <darnte> Nodex: why not use mapreduce
[11:04:34] <Nodex> map/reduce cannot run in parralell and will lock your DB when you're doing a query
[11:04:51] <Nodex> so you can only ever really have one client with a garunteed query at any one time
[11:04:55] <darnte> Nodex: wow :(
[11:04:59] <Nodex> not to mention performance
[11:05:15] <Nodex> map/reduce is not meant to be used for live env;s
[11:05:35] <darnte> Nodex: interesting, better use mr in slaves, right ?
[11:05:40] <darnte> not in the live db
[11:06:28] <Nodex> yehj
[11:06:37] <Nodex> more for offline batch processing / data warehouse
[11:06:51] <Nodex> you can use it but it has drawbacks
[11:07:11] <Nodex> if you need aggregations then try the Aggregation Framework
[11:38:04] <master_op> hi, how can i convert bson.objectid.ObjectId to dict after insert with pymongo ? (python)
[11:56:29] <maasan> I have first name, middle name and last name. I want to search for names. I used 'or' operator among each names. But it does not work when user gave both first name and lastname in the string.
[11:59:19] <Nodex> can you pastebin the query you used?
[12:31:54] <maasan> Nodex: http://pastebin.com/exLcqbxq
[12:47:33] <sivang> hi all
[12:47:53] <sivang> in my 10gen mongodb pkg installation on Ubuntu, I have two 'config' files:
[12:47:58] <sivang> - /etc/init/mongodb.conf
[12:48:27] <sivang> - /etc/mongodb.conf
[12:48:45] <sivang> My guess is the first is an upstrea config file, so one cannot specify mongodb server config params into it
[12:48:49] <sivang> am I right?
[12:48:53] <sivang> *upstart
[12:50:30] <maasan> Nodex: Did you check the pastebin?
[12:51:29] <sivang> stupid qestion, sorry- I just verified that.
[12:52:54] <sivang> It would be wise to use different names I guess, in the mongodb puppet class
[12:54:44] <Nodex> yeh, I was waiting half an hour to respond
[12:56:28] <Nodex> you're going to have to define "Doesn't work" maasan
[12:58:14] <maasan> Nodex: sorry, could not get it
[12:58:29] <GeertJohan> What mongo gui is used a lot?
[12:58:58] <GeertJohan> For development (watching what happens on the server, if documents are stored the way you're expecting them to be)
[13:07:48] <DinMamma> Hiya, I did a mongoimport to a sharded cluster, I had to ctrl-c it since one of the shards had twice as many shards as the other two. The cluster works for inserts and reads but its not redistributing the shards :o
[13:13:33] <DinMamma> Ok, its slowly rebalancing.
[13:39:16] <Darxval> Hey everyone
[13:41:17] <Darxval> I am getting a compiler error when trying to use Scons and the C++ Driver (Windows) any ideas?
[13:41:19] <Darxval> cons --extrapath=c:\boost\boost_1_49_0
[13:41:19] <Darxval> scons: Reading SConscript files ...
[13:41:19] <Darxval> Checking for C++ library boost_thread-mt... (cached) no
[13:41:19] <Darxval> Checking for C++ library boost_thread... (cached) no
[13:41:19] <Darxval> Checking for C++ library boost_filesystem-mt... (cached) no
[13:41:19] <Darxval> Checking for C++ library boost_filesystem... (cached) no
[13:41:20] <Darxval> Checking for C++ library boost_system-mt... (cached) no
[13:41:20] <Darxval> Checking for C++ library boost_system... (cached) no
[13:41:21] <Darxval> scons: done reading SConscript files.
[13:41:21] <Darxval> scons: Building targets ...
[13:41:22] <Darxval> cl /Fobuild\mongo\bson\oid.obj /c src\mongo\bson\oid.cpp /TP /nologo /EHsc /O2 /
[13:41:22] <Darxval> D_SCONS /DMONGO_EXPOSE_MACROS /Ibuild /Isrc /Ibuild\mongo /Isrc\mongo /IC:\boost
[13:41:23] <Darxval> \boost_1_49_0\include /IC:\boost\boost_1_49_0
[13:41:23] <Darxval> oid.cpp
[13:49:19] <Nodex> use a pastebin please
[13:52:02] <Darxval> sry bout that, http://pastebin.com/WmPKQ7Wh
[14:04:22] <Puma1337> Hi -- I have to apologize in advanced because I know very little about mongo or how it works or what it does, but we are using it and it crashed over the weekend. The log file says: [journal] MapViewOfFileEx for C:/path/to/mongodb/data/db.1 failed with error errno:487 Attemt to access invalid address. (file size is 134217728) in MemoryMappedFile::remapPrivateView. Any help would be much appreciated. Sorry again in adv
[14:04:22] <Puma1337> ance for my lack of knowledge.
[14:14:14] <Darxval> @Puma1337 you may want to check that status of your hdd, as i see this issue has occured before with the hdd being the cause.
[14:14:41] <Darxval> https://jira.mongodb.org/browse/SERVER-3403
[14:15:23] <Puma1337> Darxval, thanks for the info, I'll take a look
[14:20:07] <Darxval> is anyone aware of issues compiling with VS2012? via Scons? I just attempted the git repo instructions for the C++ driver and that failed as well, noticed it was referencing Visual studio 11.0\ for "found Visual Studio"
[14:21:23] <Darxval> http://pastebin.com/MRsmc9eY
[14:21:25] <Puma1337> Darxval, the SMART status reads at 98%
[14:22:04] <Puma1337> I'm trying some other tools now too
[14:40:39] <Puma1337> Darxval, yeah, everything I am seeing with disk tools shows a healthy disk
[14:50:14] <MANCHUCK> in order for mongod to ping replica sets for the heartbeat, do I need to allow ICMP?
[15:03:36] <skot> no, it uses an application tcp/ip protocol for heart beats and "pings"
[15:03:51] <skot> Nothing uses ICMP.
[15:04:02] <skot> (at least related to mongodb)
[15:09:41] <grahamhar> there is a built in { ping: 1 } to check latency and response times but is nothing to do with ICMP
[15:13:42] <dezel> Anyone have any idea as to why this is returning a result, searching a collection for a users email and an active flag set to true
[15:13:43] <dezel> http://pastie.org/6604347
[15:14:05] <dezel> I should not be getting a result, but I am getting a result strangely enough
[16:20:12] <Darxval> @Puma1337 looking at your log that you outputted, does this "C:/path/to/mongodb/data/" directory even exist? or did you rename that path for putting in chat?
[16:20:47] <Puma1337> Darxval, I renamed it before putting in the chat, the path does exist.
[16:43:03] <bean> no point in replacing the path.... really just makes troubleshooting harder.
[16:50:41] <joshua> Has anyone seen any pre-written scripts for doing stuff like gathering all the indexes for all databases?
[16:52:06] <Darxval> woot fixed my issue, i had the 6.0a windows sdk installed and it kept messing up the c++ driver compilation.
[16:57:04] <kali> joshua: look for something called "variety" on github
[16:58:22] <joshua> That looks pretty handy. Its fun being the guy managing most of the DB servers but not having direct access to the developer
[17:45:29] <joshua> 10gen has some kind of thing they run on the logs to generate info about things like slow queries and make pretty output. Is that publicly available, does anyone know?
[18:48:51] <sirious_> has anyone used document_class from pymongo's MongoClient() before?
[19:23:48] <wakawaka> Hello. I have a collection that is 500gb free, and my server only have 8gb free. So im thinking of deleteing 100gb worth of old documents.
[19:24:11] <wakawaka> I know that mongo wont actually free up the space on disk for those 100gb that is deleted from the database.
[19:24:29] <wakawaka> My question is, will I be able to write to those memory mapped files?
[19:24:40] <wakawaka> That are now empty since the data is gone from them.
[19:24:53] <wakawaka> All the data Im about to delete is sequential
[19:44:36] <bjori> joshua: yes.. let me find it
[19:44:58] <bjori> joshua: https://github.com/rueckstiess/mtools/blob/master/README.md#mlogfilter
[19:57:30] <Darxval> ugh, has anyone successfully compiled the test sample using VS2012? I built the mongoclient.lib using the git repo and scons --mongoclient. and have boost libs built, but appear to be getting linker errors for boost... even though the project is configured
[19:59:13] <Darxval> http://pastebin.com/qgQBBiKP
[21:33:39] <danielbeilinson> Hi! Could answer a little question about dynamic variables in MapReduce?
[21:35:15] <danielbeilinson> I want to use variables in map, but can't do this...
[22:49:28] <pilgo> Hi all. I have an array with objects and I want to change it to an array of strings. How do I do that?
[22:50:48] <pilgo> So, right now it's like this: {name: "Test 1", users: [{_id: 'blah blah', name: 'username'}]}
[22:51:09] <pilgo> I want "users" to just have the _id string
[23:25:49] <Bluefoxicy> so in mongodb
[23:26:07] <Bluefoxicy> let's say you have something like a problem solving system, and you list a problem document as something that describes a deviation among other things
[23:27:55] <Bluefoxicy> would it be appropriate to have a document with an embedded document for the deviation { "Name": "Web software corrupts session info", "Description": "The Web application stores incorrect session info, so people can log in and then the second they take action they are treated as not logged in." }
[23:28:18] <Bluefoxicy> and then treat the embedded document essentially as another document, analagous to another table in SQL
[23:28:28] <Bluefoxicy> in this case it's really ridiculous to anachronize modifications
[23:28:44] <Bluefoxicy> but it's HIGHLY relevant to re-use the same problem if something similar/identical happens in the future
[23:29:17] <Bluefoxicy> so i.e. I may want to search for something like "corrupt session info" and find deviations that were like that, and say, "Yes, that's what's happening", and apply it to the current problem
[23:29:46] <Bluefoxicy> in an SQL database I figured on having deviations be their own table... in MongoDB I'm thinking instead of referencing, I'd make it an embedded document, which would make more sense i think...
[23:41:40] <Bluefoxicy> oh. Map-Reduce looks neat... that'll apply heavily to this.