Log file Viewer
#mongodb logs for Saturday the 23rd of May, 2015
[00:40:53] <Antiarc> In both the python and ruby drivers, if you connect to an RS with an invalid RS name, the connection succeeds, but leaves you with an empty cluster that fails all requests. Is there any scenario under which such a connection could become available? AFAIK, nodes can't join an RS that don't share the RS's name, so it seems like it would make sense to fast-fail and throw an exception if you connect to an RS with the wrong RS name, no?
[00:42:02] <Antiarc> It makes sense for sharded setups, since a new shard with that RS name could join the cluster, but when connecting directly to an RS, I don't think that's a possibility?
[00:44:48] <Boomtime> Antiarc: it doesn't make sense for sharded clusters either, you must not connect to shards
[00:45:33] <Antiarc> You could connect to a mongos that doesn't have any nodes for a given RS, but which could have new nodes come online during the connection's lifetime though, couldn't you?
[00:46:12] <Boomtime> you do not need, or every know, the names of the replica-sets forming the shards
[00:46:34] <Boomtime> what you describe sounds odd, a replica-set name isn't particularly important except for verification of what you've connected to
[00:46:58] <Boomtime> all the members must have the same name, but you can omit the name entirely from a connection-string and it will still work
[00:47:36] <Boomtime> i would certainly expect a driver to either validate the replica-set is what you expect (throw an error if wrong) or ignore it entirely
[00:47:42] <Antiarc> Yup! I'm just trying various iterations of connection params, and when you specify a replicaset name that doesn't match the RS you connect to, it permits the connection but leaves you with an unusable client
[00:48:07] <Antiarc> It ignores it, since none of the nodes in the RS match the provided RS name, so all server selection operations return an empty set and eventually time out
[00:48:58] <Antiarc> Yeah, the client attempts to find a valid server for a given read preference, and gives up after some time if it can't
[00:49:18] <Antiarc> But if the RS name is wrong there will *never* be a valid server, so I think it should probably fast-fail at connect time
[00:50:00] <Antiarc> Yeah, it's "can't find a suitable server" - pymongo.errors.ServerSelectionTimeoutError: No replica set members available for replica set name "foobar"
[00:50:04] <Boomtime> you can't fail at conect time, because no operation is in flight - that is, the driver doesn't know your intention yet - but it can fail quickly the moment you give it an operation to perform
[00:50:41] <Antiarc> It seems like server selection should complain if the list of servers is empty prior to read preference/tag set filtering
[00:51:14] <Boomtime> well, that is probably defined behavior, it's hunting.. but i tend to agree that it should be able to detect something is amiss the moment a seed server responds with a different replname than the expected one
[00:51:29] <Antiarc> When you connect, it polls the RS for the list of nodes, and then filters them by RS name - if the result of that filtering is empty, it could throw an exception
[00:51:42] <Antiarc> connect-time is wrong, more like "client instantiation and topology-discovery time"
[00:52:11] <Boomtime> the connect line rarely does anything in most drivers, it's just object instantiation
[00:52:39] <Antiarc> Okay. I'll open an improvement ticket on the Ruby driver. Happy to open one on the Python driver as well, but I'm just using it to consistency-check the Ruby driver's behavior :)
[00:54:44] <Boomtime> yeah, i'd think improvement is the way to go, clearly the driver can fail fast the moment it gets a response on the seed list whose replname is different to expected - that condition minimally means the connection-string contains a mistake, plowing on from that point is silly
[00:56:51] <Antiarc> Also, I'm not sure if it's specced behavior or not, but the Ruby driver isn't resolving node names to IPs or anything - if I connect via localhost, I end up with 2 nodes in my cluster, one named localhost and one named luna (my machine's name), which seems like it might cause issues at some point.
[00:57:15] <Antiarc> Mongo::Client.new(["localhost:27019"], replica_set: "kerrigan", read: {mode: :secondary_preferred}).cluster => #<Mongo::Cluster:0x2386 servers=[#<Mongo::Server:0x2388 address=luna:27019>, #<Mongo::Server:0x2390 address=localhost:27019>] topology=Replica Set>
[00:57:40] <Antiarc> Both those servers in the servers list are the same node, but the RS names it "luna", and the seed name given was "localhost"
[00:58:00] <Antiarc> Still going through the selection code to see if it might cause a problem, but it seems like it might be unintended
[01:01:09] <Antiarc> It's the parameter that determines the durability required for the write to succeed
[01:02:16] <Boomtime> @Antiarc: generally speaking, drivers use the seed list only temporarily, once they get some responses from the servers the seed list should effectively be thrown away
[01:02:43] <Boomtime> there is nothing stopping a driver from retaining the seed list for it's own purposes
[01:03:10] <Boomtime> with that said, using different names in the seed list than what is in the replica-set configuration will lead to trouble sooner or later
[01:07:03] <Antiarc> It looks like it's adding the seeds unconditionally here: https://github.com/mongodb/mongo-ruby-driver/blob/master/lib/mongo/cluster.rb#L102
[01:09:24] <Antiarc> (I'm working on tracking down where the cluster node list is actually managed during discovery)
[01:09:55] <Boomtime> i don't believe that is illegal in the driver spec, but it's probably not a good idea - i know most drivers don't mix the lists like that
[01:11:43] <Boomtime> indeed, C# driver appears to have addressed this specific condition: https://jira.mongodb.org/browse/CSHARP-1283
[01:13:21] <Boomtime> not sure what you mean - but seeds are not directly resolved to be "added" to the topology - connections are made using the seed list, but members are added to the topology based on what the servers at the end of those connections actually report
[01:14:32] <Antiarc> Yup! I'm saying that the deduplication is handled by resolving each discovered node and keeping a list of IP addresses to know which nodes are known, but no lookup is done for seeds and seeds are unconditionally added to the servers list
[01:15:32] <Boomtime> the only real reason to ever try to match between these two lists is to utilize the existing connections made using the seed names
[01:16:35] <Boomtime> the safe way of doing it would be close all connections made from seed list names and open new ones to the names given in the topology as discovered - but that's wasting connections then
[01:17:29] <Boomtime> so drivers do attempt name match up, using address resolution when available, and de-dup'ing, associating, etc as you say
[01:33:42] <aniasis> Boomtime, I want to take a large set of data and shrink it to views. Those views would be represented as XML and stored as documents in some sort of content repository. Is that possible with MongoDb?
[01:35:07] <Boomtime> i would suggest storing as JSON instead since you'll have a much more powerful query language then, but otherwise yes, mongodb is a document store, the rest is just formatting
[01:38:08] <aniasis> I don't want to use mongoDB's query language to traverse the documents only access them.
[04:17:54] <Streemo> for example collection.update({name:"bob"},{$inc: {field: 5}, $unset: {field: ""}}) will Bob have a field with value 5?
[04:23:24] <Boomtime> can you paste the exact update you tested with in the shell, and the reponse you got?
[04:40:46] <Boomtime> in a replica-set you can only write to the primary instance (whichever member that is at the time)
[04:41:13] <Boomtime> your data will be replicated, for the purposes of durability, to the other data bearing members of the set
[04:41:33] <Boomtime> an arbiter is not data-bearing, it exists solely to cast a vote in elections, so you will never need to connect to it
[19:30:03] <loadh> i'm trying to get the value of a 'virtual' function to output in the JSON my Web App returns as a result of a query to my MongoDB instance
[19:31:21] <loadh> I've defined the virtual function 'isOpen' and would like the value as a key / value in the JSON, but it doesn't appear in the JSON..I tried setting: Schema.set('toJSON', { virtuals: true })
[19:59:05] <StephenLynx> and from what I noticed, I am the only node/io dev here giving advice. I suggest trying mongoose's channel.
[20:06:10] <loadh> i want to add a value to the JSON i return from my Web Service, that isn;t persisted
[20:18:59] <loadh> the 'Business' has an opening start and end time (integers) and i want to provide an isOpen field in the JSON for that business...all dates are in UTC and I have a timezone field with the business's timezone
[20:22:22] <StephenLynx> and protip: any ODM won't do it magically, it will just perform additional operations on the driver.
[20:39:17] <rOOb> Anyone here using mongoose? I create a schema using a custom get and set function to format the field, but the get never seems to be called. I'm using this:
[20:40:00] <rOOb> getRealPrice is: return (num/100).toFixed(2); and setRealPrice is: return num*100;
[20:41:07] <rOOb> If I enter "1.99" it's store in the db as 199. When I get it from the database it's still 199. It's never formated according to getRealPrice
[20:46:55] <StephenLynx> it causes loss performances of about 600% and doesn't quite follow mongo's specifications on some aspects, like _id
[20:58:52] <rOOb> StephenLynx Hmm. I'm new to the node/express/mongo world. Is there a "better" alternative to mongoose that you know of?
[21:19:16] <Streemo> I've got two options: return .find()with fields: {field1: 1, field2: 1, field3: 1,...} OR I can restructure the document to have field_i as a subfield of a parentField, so I only have to do: .find() with fields: {parentField:1}. from your experience, does this offer performance boost
[21:21:40] <Streemo> if i wanted to sort, is it moreefficient to sort by field1: -1 or by parentField.field1: -1? does the extra nest level help projection speed more than it hurts sort speed?