PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Friday the 15th of March, 2013

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[00:16:48] <AAA_awright> Where would I get information on the MongoDB wire protocol?
[01:47:21] <bjori> any specific parts you are looking for AAA_awright ?
[01:47:30] <bjori> AAA_awright: http://docs.mongodb.org/meta-driver/latest/legacy/mongodb-wire-protocol/ should get you started..
[01:47:45] <AAA_awright> I'm looking at that right now but it says "legacy" for some reason
[01:47:53] <bjori> its not
[01:47:57] <AAA_awright> meh
[01:48:04] <bjori> the documentation infrastructure is just being moved and improved
[01:48:06] <AAA_awright> Also stuff like serialization of the query
[01:48:13] <bjori> and that specific page is part of the old doc standard :)
[01:48:21] <AAA_awright> At one point a driver I was using was allowing . and $ characters in the query
[01:48:46] <AAA_awright> There doesn't seem to be any standard on what's appropriate and what's not, and where that's handled
[01:48:47] <bjori> thats not cool
[01:48:51] <AAA_awright> Also, replication protocol
[01:49:13] <AAA_awright> How is replication handled? I'm having no end of problems from that from my client library
[01:49:14] <bjori> I don't think there is a public doc for that
[01:49:25] <bjori> but its just essentially reading the oplog and reply it
[01:50:07] <bjori> rreplay
[01:50:22] <bjori> one more time. *replay it* :)
[01:50:45] <AAA_awright> I don't even know how that works from the user end... what do I have to do to ensure that my library can fall back to the necessary servers?
[01:51:35] <bjori> uh.. what?
[01:51:39] <bjori> what sort of library are you creating?
[01:52:32] <bjori> are you talking about picking a secondary to execute commands/queries on with depending on its read preferences?
[01:58:09] <AAA_awright> I'm trying to fix up an existing one
[01:58:28] <AAA_awright> It's great but it's lacking some features and has certain bugs
[01:59:14] <AAA_awright> bjori: Yeah, I'm looking to specify read preferences, write requirements (guarentee that it's committed), and automatic failover
[01:59:32] <AAA_awright> And I'm not even very sure how to do that as a user, never mind a developer
[03:39:32] <bjori> AAA_awright: its called write concern :)
[03:40:04] <bjori> AAA_awright: the failover is done with periodic ismaster/ping commands to all of the servers
[03:40:18] <bjori> or when a query/command fails, or if the socket is closed
[03:40:45] <bjori> write concerns are an additional query option
[03:40:55] <bjori> read preferences are on the driver level
[03:41:17] <bjori> none of this is in the wire protocol itself
[03:41:36] <bjori> 10gen drivers do this all in the same way, community drivers may vary
[03:42:08] <bjori> http://docs.mongodb.org/meta-driver/latest/ this page is aimed to document how things should behave.. but currently its lacking boatload of info
[03:42:14] <AAA_awright> But some of this is up to the driver... for instance, I can seem to store {"http://example.com/": 10} just fine over the wire protocol
[03:44:35] <bjori> I see no reason why it shouldn't work either
[03:44:52] <bjori> is the driver you are using preventing it?
[03:45:24] <AAA_awright> Yeah
[03:45:45] <AAA_awright> I'm having to encode % $ and .
[03:45:58] <bjori> ouh, wait, right
[03:46:10] <AAA_awright> (% because I'm using URL encoding)
[03:46:28] <AAA_awright> But there's no technical limitation for that, which is puzzling
[03:46:40] <AAA_awright> And the fact I can't store arbritrary values is somewhat of a hurdle
[03:47:02] <AAA_awright> Databases are so I /don't/ have to worry about data >:(
[03:47:14] <bjori> using a . for a key is a very bad idea, because it breaks querying with the dot notation
[03:47:43] <bjori> the db support querying on a subdocument using dot as a seperator
[03:47:48] <bjori> which is why its blocked at the driver level
[03:47:54] <bjori> although technically strictly valid bson
[03:48:30] <bjori> and similar thing with $
[03:48:43] <bjori> I don't think % is reserved for anything
[03:49:08] <bjori> http://docs.mongodb.org/manual/reference/glossary/#term-dot-notation
[03:49:39] <bjori> http://docs.mongodb.org/manual/reference/operators/
[03:50:26] <bjori> I've no idea why your driver doesn't allow you to use %, all the 10gen drivers allow it just fine afaik
[03:58:17] <AAA_awright> bjori: It doesn't, I said for encoding
[03:58:27] <bjori> right right
[03:58:31] <AAA_awright> You need to escape your escape character, of course
[04:03:04] <abhinav> Hi guys. Can you please help me with this http://stackoverflow.com/questions/15404850/return-alias-field-via-json-using-mongoid
[04:32:36] <AAA_awright> Why doesn't the wire protocol use network order >:(
[04:36:26] <ctorp> Can anyone see something wrong with this mapReduce? http://www.hastebin.com/xedadiwuga.js
[04:36:36] <ctorp> I'm getting values much smaller than they seem like they should be
[04:37:02] <ctorp> 13000+ records, but that MR shows about 200 results. It should be more like 12000+
[06:53:59] <AAA_awright> The primary handles write conserns, correct? Who handles read preferences?
[07:48:44] <Garo_> It seems that the $or : [] -query does each or clause in parallel, but how it chooses which it will return? The first one which finishes?
[08:38:49] <[AD]Turbo> ciao all
[09:48:01] <raffer__> Hi. For our mongodb we use a primary/secondary setup. If the primary goes down, 2 becomes 1. If the first is back up, he is 2. Now HAProxy routes every request to the 2 ... (unless we rewrite the HAProxy of course...)
[09:48:21] <raffer__> How would one solve this?
[09:48:44] <fredix> hi
[09:48:51] <raffer__> Can I enforce this in the client? I use a Node.js client.
[09:49:03] <raffer__> thanks
[09:49:17] <fredix> with mongofiles command is it possible to delete files with a jocker like *.txt ?
[09:49:34] <fredix> I try this but it doesn't works mongofiles -h 9.0.0.3 -d nodecast_prod delete *.cfu
[11:11:58] <marcqualie> Does anyone know if atom processor boxes are okay to use for sharded config servers?
[12:22:05] <Vill> Greetings Folks, is #mongodb an active IRC channel, or just a dead-end on the path to NoSQL world? Is someone willing to enlighten a newbie like me with their piece of knowledge?
[12:23:04] <kali> Vill: if you ask a specific question, you may get an answer
[12:25:37] <FrenkyNet|lunch> Vill: and there are quite a lot of resources you could first go through before asking questions :)
[12:25:50] <Vill> FrenkyNet, believe me, i've been there
[12:27:33] <Vill> I have a data modeling question, I know that MongoDB doesn't support ACID transactions, I'm currently trying to transfer from RDBMS to MongoDB with database holding Customer-Contract-Process-Product. But at some point I realized that the ACID operations can be implemented inside my server scripts. What my problem is, in the current MS Access database, I've found a lot of data inconsistency...
[12:27:35] <Vill> ...and even some circular references. I just love, how easy it is to organize all Customer details in one Document. But now I need to step into "process" which requires advanced technique...
[12:29:01] <Vill> The process, is not a simple "in-out" chain, where you get material, and "do operation" on it, and return product, it's manufacturing chain, where each operation, can require multiple parts, and different processes, and different tools. My question is: Is NoSQL a way to go with this?
[12:49:31] <BadCodSmell> In mongo nosql webscale 2.0 schemaless document object storage engine, if I have a struct as such: {y:1,x:[{a:1,b:2},{a:2,b:2}]} is there a way to update the array child that has a:2 so that b:3?
[12:52:21] <BadCodSmell> also i have been using mongo for 6 months why arnt i facebook yet?
[12:57:44] <BadCodSmell> An ha
[12:57:47] <BadCodSmell> $ operator
[12:57:48] <BadCodSmell> thanks
[13:01:06] <Derick> you're welcome ;-)
[13:04:00] <BadCodSmell> I have a question
[13:04:18] <BadCodSmell> What if I have an array of docs with an array of docs with an array of docs?
[13:04:26] <BadCodSmell> Is it multidepth?
[13:04:32] <BadCodSmell> Because the manual gives no hint
[13:04:47] <Derick> sure, you can do that
[13:05:05] <BadCodSmell> nice, now that I know I can I will bother to figure out how
[13:05:06] <BadCodSmell> thanks
[13:05:07] <Derick> but searching on or updating that can become difficult
[13:05:14] <Derick> so mostly useful for read-only properties
[13:18:11] <BadCodSmell> db.sq1.update({x:2, 'y.b.i':1},{$set:{'y.$.b.$.h':0}},{ multi: true }); can't append to array using string field name [$]
[13:18:13] <BadCodSmell> why?
[14:24:30] <baegle> Trying to run a find query from the command line without entering into the mongo shell, being told db.collection_name is undefined. Any idea why?
[14:33:03] <gazarsgo> is it a php driver thing that i can't insert json directly to mongo, or a mongo thing ?
[14:33:11] <Derick> a mongo thing
[14:33:27] <gazarsgo> because mongo won't convert json to bson or ?
[14:33:31] <Derick> mongo doesn't actually store json, but a binary variant of it
[14:33:47] <Derick> the drivers convert their language's structures into bson
[14:34:00] <Derick> in php, you can use json_decode to get an array structure to save into MongoDB
[14:35:48] <baegle> So I have the following that works in the mongo shell: db.products.find({drsrcId:/.{24}/},{vcode:1});
[14:36:14] <baegle> I put this exact same line in a file and ran: mongo localhost:27017/baws_uat script.js and I get nothing
[14:36:42] <baegle> I don't understand the difference between running it as a script file and running it directly in the cli, anyone know?
[14:50:47] <baegle> anybody?
[14:52:04] <kali> yeah
[14:52:12] <kali> there used to be a oage about that
[14:52:16] <kali> in the old wiki
[14:52:21] <kali> can't find it in the new
[14:52:26] <baegle> Well, I got it to stop failing
[14:52:30] <baegle> but it won't output anything now
[14:53:11] <kali> baegle: you can't use the "use" keyword, and there are a few other traps
[14:53:37] <baegle> kali: it's a simple find
[14:53:40] <baegle> with a regex
[14:53:47] <baegle> db.products.find({drsrcId:/.{24}/},{vcode:1});
[14:54:18] <kali> ha ! add .forEach(printjson) at the end
[14:55:09] <baegle> wow, default behavior is to just discard output
[15:17:37] <failshell> hello. im running sh.stopBalancer() on a test cluster, and it seems to be taking a very long time. how long is it supposed to take to stop it?
[15:18:02] <failshell> its been running for over 10 mins now
[15:53:39] <Scrat> in a findAndModify upsert, is it possible to skip updating a field if it already exists?
[15:54:17] <veesahni> how does the aggregation framework play with the interpreter lock? Is the collection it's running over lock while a query is running?
[16:08:36] <Scrat> looks like $setOnInsert in 2.4 does what I need. does it work on findAndModify?
[16:10:08] <Scrat> only update() is mentioned in the 2.4 manual
[16:41:24] <sinisa> hi all.. anyone from business insider here?
[16:43:25] <apetresc> LOL
[16:43:35] <sinisa> LOL what :)
[16:44:54] <sinisa> there was few of them here, long time ago hmmm :)
[16:46:15] <sinisa> i wanted to tell somebody from BI that their "views" on BI site looks like they have mind of their own :)
[17:34:18] <gazarsgo> woohoo got my 3 node replica set up!
[17:43:52] <apetresc> :)
[17:49:10] <gazarsgo> had to convert the example cloudformation template to use vpc … that was the biggest pain. all the mongo stuff was super easy and straightforward :)
[17:59:45] <gazarsgo> so if i have a mongodb node that's run out of space, how can i get it to replicate to my new replicaset ?
[18:04:49] <JoeyJoeJo> How can I count the results from a distinct() query? I tried db.collection.distinct('field').count() but that didn't work
[18:07:35] <anewkirk> JoeyJoeJo, http://docs.mongodb.org/manual/reference/command/distinct/
[18:08:16] <sirious_> has anyone worked with MongoEngine before?
[18:08:29] <anewkirk> what's that?
[18:08:47] <anewkirk> JoeyJoeJo, (db.collection.distinct('field').count()).count() might work
[18:09:04] <sirious_> anewkirk: python object-document wrapper for mongo
[18:10:11] <anewkirk> oh
[18:46:48] <OliverJAsh> I have an array of documents, returned from save. I now want to do a query that finds any documents added since (newer than the newest in the current collection). Can you think of a query for doing that? I have a field on the schema called date which is a JavaScript date object, such as "Tue Mar 12 2013 17:46:07 GMT+0000 (GMT)"
[18:59:56] <cmacmurray> After noon, running version 1.8 did an rs.stepDown() on a node, and now they are all secondary
[19:00:01] <cmacmurray> not sure how to fix that
[19:29:31] <cmacmurray> Hi, I have another interesting issue going on with by local db gettting really big
[19:35:32] <JoeyJoeJo> I want to find one field in each document and return the number of times each data point occurs. For example, if the field has data between 1-10 and in 5 documents field =2 and 3 documents field = 7, how can I return 5 and 3?
[19:38:42] <JoeyJoeJo> Or maybe a better example, in 5 documents the field field == foo and in 3 documents field == bar. How do I return foo:5, bar:3?
[20:34:33] <failshell> can anyone explain to me how the balancer works in a cluster? i was under the impression it was running constantly
[20:34:36] <failshell> and yet, it seems it doesnt
[20:43:31] <saml> do you use casbah?
[20:43:38] <saml> o.get("foo.bar")
[20:44:00] <saml> do i need to manually split the key to access sub document
[22:19:21] <drockna> I am using mongohub. When I look at the monitoring it says locked %. does anyone know what that could be refering to?
[22:25:24] <broerd> Can you do a query with $lt or $gt where the value being compared is another field in the same record?
[22:26:04] <broerd> Context: We have items in our database that contain "old_price" and "price" (current price). I want to see where there has been a price reduction by comparing the two.
[22:26:06] <broerd> Is that possible?
[22:30:24] <broerd> Figured it out. I'm using a where.
[22:46:13] <defunctzombie> if I have documents where a field is a string (of a number) or a number value
[22:46:30] <defunctzombie> and I want to do a .find({ field: number_value });
[22:46:39] <defunctzombie> is there a nice way to make it work with any fields which are string
[22:47:53] <rweeks> Hi, I'm trying to do keyword searching with mongodb. I've already extracted the keywords I want from my documents. The idea is to search as you type, so my plan is to tokenize the query string, do a full match on all but the last token and a regex match on the last token (because the user is maybe still typing it).
[22:48:47] <rweeks> I've tried several different queries but I'm not getting the results I expect. Both the "full match on tokens 1 to n-1" and "regex match on token n" work separately, but together I get 0 results.
[22:49:01] <rweeks> pastie is here: http://pastie.org/6554147