Log file Viewer

[00:49:58] <peachblossom> Hi. Has anyone installed MongoDB on a shared filesystem ?

[04:01:01] <jason__> If I have files in MongoDB GridFS such as images that I wish to display on a web page, how should I serve them? Create a route in my web applications which retrieves them?

[04:48:50] <GothAlice> jason__: https://github.com/mdirolf/nginx-gridfs

[04:49:29] <GothAlice> Nginx-gridfs is a method of using Nginx as a front-end server for GridFS stored assets.

[10:24:40] <nicvw> Hi, I have a question about pymongo and how its behaviour differs when using coroutines (gevent) instead of simple threads (threading).

[10:27:09] <nicvw> Basically I have built to small programs to simulate the behavior of the application I am working on. Both connect to a mongo replicaset (mongo 2.6.10), the only difference is one is using gevent to spawn a pool of workers and the other one using the builtin Python (2.7.9) threading module

[10:27:57] <nicvw> Both perform mostly finds with a find_and_modify thrown in every 10 seconds. Read preference is Primary.

[10:29:04] <nicvw> In the threaded example, when the primary node is disconnected from the network no reads or writes occur for a about 50 seconds, then the program resumes. This is the desired behavior.

[10:30:10] <nicvw> The program using gevent does NOT resume after this reasonable amount of time. It stalls for 15 minutes before any further transactions are able to be performed against the replicaset.

[10:32:01] <Rumbles> I'm trying to query mongo using pymongo to find the IP addresses of the secondaries, I hoped that I could find that information with MongoClient().nodes but that returns frozenset.... I guess I'm doing something wrong... https://paste.fedoraproject.org/477739/

[10:35:50] <nicvw> have your tried .secondaries

[10:36:05] <nicvw> it returns a set of secondary members

[10:36:50] <Rumbles> oh no i haven't

[10:37:06] <Rumbles> I didn't see that in the docs: http://api.mongodb.com/python/current/api/pymongo/mongo_client.html#pymongo.mongo_client.MongoClient.nodes

[10:37:50] <Rumbles> just returns: set([])

[10:41:23] <qswz> what's the diff between $elemMatch https://docs.mongodb.com/v3.2/reference/operator/query/elemMatch/ and $or?

[10:41:56] <nicvw> given the info you pasted, you are connecting to a standalone mongo. you probably want to specify the replicaset in you instantiate MongoClient

[10:43:10] <Rumbles> it's not a standalone...

[10:43:22] <Rumbles> if I connect to mongo and run rs.status() I can see the otherss

[10:46:41] <Rumbles> according to the config it's using a replsetname of dev, so I woudl expect MongoClient(replicaset='dev').secondaries to work, but I get set([]) still :/

[10:49:07] <Rumbles> https://paste.fedoraproject.org/477759/60666147/

[10:53:28] <nicvw> my only other thought is that you are connecting to the arbiter instead of one of the actual database nodes

[10:54:14] <Rumbles> nope I'm on the primary

[10:54:30] <Rumbles> that pastebin was from the same machine as the one I am running a python console on :/

[10:54:54] <nicvw> I only see the behavior you are experience when I connect to my arbiter, did you establish what version of pymongo you are running?

[10:55:09] <Rumbles> 3.0.3

[10:55:15] <Rumbles> and mongodb 3.2.8

[11:01:29] <nicvw> I suppose you could just use .admin.command('replSetGetStatus') and parse the output of that do get your secondaries... I have no idea why you can't get them using .secondaries

[11:03:18] <Rumbles> yeah that returns some useful info

[11:03:19] <Rumbles> thanks

[13:57:31] <kryl> hi, please how to try/catch an exception when I try to connect if the database is not up : I don't want to just get the error message and leave my program like that ! I use php 7

[13:57:58] <kryl> $mdb = new \MongoDB\Client; < for example deliver the message : Default exception : No suitable servers found (`serverselectiontryonce` set): [Failed connecting to 'localhost:27017': Connection refused] and immediately exit !

[13:58:13] <kryl> I want to catch the error and do some actions if you can help ?

[13:59:01] <kryl> I use "mongodb/mongodb": "^1.0.0" from composer on a high level layer to the php new driver with php 7 for information

[14:03:07] <StephenLynx> Derick, is it you that knows the PHP driver?

[14:19:52] <Derick> StephenLynx: yes

[14:21:40] <StephenLynx> yeah, someone asking stuff about it

[14:21:55] <StephenLynx> but he left

[14:21:56] <StephenLynx> RIP

[15:27:35] <qswz> does it change something if I call MongoClient.connect once only and pass around db, or call MongoClient.connect multiple times in several files

[15:34:24] <cheeser> which language?

[16:12:09] <teprrr> hmm, upserting some 60k documents with just <timestamp, string> has taken now already 20 minutes.. I must be doing something wrong here? I'm using bulk operation mode of pymongo (though my earlier bulk_write with bunch of updateones was the problem, but alas no.)

[16:12:20] <teprrr> any ideas how to debug that bottleneck?

[16:14:29] <teprrr> it must be the use of upsert though..

[16:14:42] <cheeser> yeah. every upsert does a query first.

[16:16:31] <teprrr> oh yeah. and I'm also logging all the queries it seems

[16:25:38] <qswz_> cheeser: NodeJayEss

[16:48:42] <teprrr> uhuh, it took 54min to do that upsert :P

[16:57:27] <teprrr> (caused by missing indexes, duh)

[16:57:37] <teprrr> now its flying, whee

[17:04:54] <qswz_> woa really?

[17:05:05] <qswz_> how fat is a the db?

[17:05:26] <qswz_> that's worrying

[17:06:40] <teprrr> it's not really fat, had some 300k rows in total before

[17:07:14] <teprrr> the db has now 700k rows and is of size 50MB

[17:07:40] <teprrr> someone should smack me to the head for not creating indexes in the first place

[17:08:04] <qswz_> oh ok, you upserted many

[17:08:04] <teprrr> but still, inserting 60k tuples in an hour is kinda .. :)

[17:08:11] <teprrr> yeah

[17:08:19] <qswz_> with upsert

[17:10:02] <teprrr> yes. I wanted to make it foolproof, if someone erroneously runs the same script again.

[17:10:28] <teprrr> I was battling with myself when thinking when would it be better to handle duplicates, when writing or when reading

[17:10:38] <teprrr> or in-between. but decided to go with upsert

[17:11:32] <qswz_> did you use buldWrite?

[17:11:37] <qswz_> bilkWrite*

[17:14:16] <teprrr> yes, through pymongo though

[17:14:43] <qswz_> probly not the fastest thing on the earth

[17:14:59] <teprrr> so I initialized an ordered bulk op, added all my upserts there and executed that

[17:15:10] <teprrr> but it's now fast and smooth, after both fields have indexes

[17:15:27] <qswz_> you ran a createIndex too?

[17:15:37] <teprrr> now handling the data will be ready this year :)

[17:15:43] <teprrr> er, handling the import

[17:15:51] <teprrr> yes, I did createIndexes manually

[17:15:56] <teprrr> that made it fast

[19:49:25] <alexbevi> i'm not sure if anyone can help, but i'm on mongod 3.0.12 and am having an issue with db.eval

[19:49:32] <alexbevi> http://pastie.org/10960674

[19:49:56] <alexbevi> this was working yesterday, this morning it's not working. I updated to 3.0.14 to test but still having the same issue

[19:50:22] <alexbevi> we're using the C# driver to call these js functions, so the Eval is necessary

[19:50:33] <alexbevi> any help would be appreciated

[20:43:26] <TehGrub> hi

[20:46:55] <TehGrub> so i have a bunch of points in a mongo collection and i'm using the $box to get back the points but i'm not getting anything back. does the $box support earth like coordinates at all?

[21:57:01] <blizzow> I have a sharded cluster with ~600million records across a couple collections within three DBs. We have I'm running mongo-connector to get all the data into a replica set with no sharding. I've let the system run for a couple days now and the record count isn't exactly the same between for databases between the two clusters.

[21:57:12] <blizzow> Is there a way to tell when the mongo-connector thinks it's close to being in sync?

[21:57:31] <teprrr> TehGrub: are your query values exact?

[21:57:41] <TehGrub> teprrr, heh. i got it working

[21:57:48] <TehGrub> i didnt need to use $box

[21:58:00] <teprrr> but yea, it looks like $box should accept earth-like coordinates, latitude & longitude

[21:58:00] <TehGrub> wait i did

[21:58:12] <TehGrub> i used something like this http://geojsonlint.com/

[21:58:13] <teprrr> no idea in which format though. I've never tested out any geospatial queries

[21:58:31] <TehGrub> yeah in $box i had to put in the 2 corners that covered the gps coordinates

[21:58:33] <TehGrub> and it worked

[21:58:46] <TehGrub> but im not sure how scalable this is or whether this would work on all earth queries

[21:58:56] <TehGrub> like maybe some north pole or some edge case nonsense :p

[22:03:58] <teprrr> ah, okay.. no idea really :) nice that you got it working though