pmxbot IRC Log Viewer

[00:45:51] <AlgorithmWizard> test

[00:45:52] <AlgorithmWizard> can you guys hear me

[00:46:43] <AlgorithmWizard> does mongodb have clustering of some sort in order to scale? I want to prepare for a hit of about 30k users. currently my site and mongodb run in the same docker container/server and I know this should be separated. aside from this, what else can I do?

[00:49:48] <GothAlice> AlgorithmWizard: Yes.

[00:50:09] <GothAlice> AlgorithmWizard: https://docs.mongodb.com/manual/sharding/

[00:51:12] <GothAlice> AlgorithmWizard: This will allow you to divide your data among several servers to distribute read and write load (as well as storage load) using an index to control where records are distributed and a "mongos" proxy/middleware that your applications connect to to distribute queries.

[00:52:18] <GothAlice> Combine this technique with https://docs.mongodb.com/manual/replication/ to ensure your data is redundant and highly available, meaning loss of a single server won't result in overall instability, service outage, or data loss.

[01:00:00] <niftylettuce> 99% of startups fail because they don't have cool fonts in their emails https://github.com/crocodilejs/custom-fonts-in-emails#custom-fonts-in-emails

[09:16:50] <ihre> Hi, I'd like to enable the profiler (mongodb-server 2.4.10-5), so I'm trying `db.setProfilingLevel(1,50)`, but running `show profile` returns `db.system.profile is empty Use db.setProfilingLevel(2) will enable profiling`. Is there anything I might have missed while enabling the profiler?

[09:37:59] <chris|> when using the Java API, what is the propper way of creating a find() cursor to read from secondaries without creating a new client?

[12:37:55] <ngperfex> how can i get the count before applying match and after applying match

[12:38:04] <ngperfex> in a aggregate query

[13:16:39] <deever> hi

[13:17:36] <deever> i can't find instruction on how to update a mongodb installation (from 2.6.x to 3.2.x). are there any resources?

[13:52:54] <chris|> when using the Java API, what is the propper way of creating a find() cursor to read from secondaries without creating a new client or collection?

[13:54:35] <cheeser> set the ReadPreference

[15:00:43] <chris|> cheeser: well, yes, but there seems to be a gap in the API

[15:06:34] <cheeser> how so?

[15:06:41] <cheeser> you have a DBCursor?

[15:11:36] <chris|> I believe that is only part of the old synchronous API, the async API has FindIterable

[15:14:38] <cheeser> ok. just making sure. :) one sec.

[15:18:38] <chris|> i can build the FindOperation myself and run it using a custom executor with read preference, but that seems likely to break easily

[15:19:25] <cheeser> so you have your MongoCollection, just do: collection.withReadPreference(ReadPreference.secondaryPreferred()).find(...)

[15:27:23] <chris|> hm, yeah, not sure how I missed that, thanks

[15:34:26] <deever> i can't find instruction on how to update a mongodb installation (from 2.6.x to 3.2.x). are there any resources?

[15:36:45] <cheeser> https://docs.mongodb.com/v3.2/release-notes/3.0-upgrade/

[15:36:54] <cheeser> https://docs.mongodb.com/v3.2/release-notes/3.2-upgrade/

[17:30:12] <cpama> hi all. can anyone help me answer this question? http://stackoverflow.com/questions/40361269/mongoclient-in-php-trying-to-create-and-test-a-full-text-search-index

[17:30:21] <cpama> been stuck on that for a few days..

[17:31:55] <tsturzl> I have never used PHP and mongo together, but I could probably help you out

[17:32:02] <cpama> tsturzl, cool

[17:32:07] <cpama> i'd appreciate any tips

[17:32:10] <cpama> just new to mongo

[17:34:15] <tsturzl> oh man, I forgot PHP syntax. Give me a minute

[17:34:23] <tsturzl> I haven't worked with it in nearly 8 years here

[17:34:26] <cpama> tsturzl, yeah no worries

[17:34:45] <cpama> take your time. I'm feverishly working on other features for the same app... so I'm keeping busy anyways!

[17:37:21] <tsturzl> Do you have an example document?

[17:39:04] <tsturzl> nvm, I don't actually need it

[17:39:18] <cpama> k

[17:40:32] <tsturzl> What field are you trying to create a text index for?

[17:40:36] <tsturzl> 'name'?

[17:40:59] <cpama> all the fields

[17:41:02] <cpama> they're all text

[17:41:12] <cpama> so the user wants to be able to do a google-like search

[17:41:19] <cpama> where it'll check all fields.

[17:43:48] <tsturzl> I think you're using the createIndex method wrong

[17:43:53] <cpama> k

[17:43:57] <cpama> <listening>

[17:44:17] <tsturzl> You're providing 2 arrays, whereas createIndex only takes 1 param which is an array

[17:45:10] <cpama> k... checking documentation again.

[17:45:32] <tsturzl> You're passing the second array where it would expect options

[17:45:46] <tsturzl> createIndexes is probably what you're looking for

[17:46:04] <cpama> ah ok

[17:46:05] <tsturzl> but then you should embed your arrays into one array

[17:46:17] <cpama> are you looking at a specific doc? can you share the link?

[17:46:18] <blizzow> Is 3.4 current, or 3.2? The docs say 3.2, but I got an email saying 3.4 is released? Running apt-get update/dist-upgrade isn't picking up a changed version yet. Do I need a different package name?

[17:46:33] <tsturzl> I think 3.4 is enterprise release

[17:46:44] <tsturzl> and then it will trickle down to us peasants

[17:47:43] <tsturzl> Actually I think 3.4 is an unstable release

[17:48:04] <cpama> tsturzl, http://php.net/manual/en/mongocollection.createindex.php

[17:48:19] <tsturzl> You don't see it because mongodb repos are minor versions, so 3.4 is a different repo than 3.2

[17:48:29] <cpama> sounds like for version 2.6+ mongo the above method will actually trigger the createIndexes method

[17:48:40] <tsturzl> cpama: I'm looking at a different source

[17:49:00] <tsturzl> https://docs.mongodb.com/php-library/master/reference/method/MongoDBCollection-createIndexes/

[17:49:34] <tsturzl> yeah these might be outdated

[17:49:49] <tsturzl> this one is 1.1(current)

[17:49:56] <tsturzl> can you verify your local version?

[17:51:10] <cpama> 1.6.13

[17:51:45] <cpama> that's what i see when I do a phpinfo on my server under "mongo"

[17:51:49] <cpama> <brb>

[17:52:05] <tsturzl> try just creating the text index once

[17:55:12] <cpama> tsturzl, ok i'm back. sorry had to step away for a minute

[17:55:27] <cpama> ok so just create it manually once and then use my script maybe to just test it?

[17:55:33] <cpama> is that what you mean?

[17:55:34] <blizzow> cpama: https://docs.mongodb.com/manual/ says 3.4 is upcoming.

[17:56:35] <cpama> blizzow, i "have to" use mongo 3.2.10 for now...

[17:57:46] <tsturzl> blizzow: You can still add the repo for it

[18:01:36] <tsturzl> You know what I'm completely confused, the php mongo driver is just different than what I'm used to

[18:02:03] <tsturzl> array( 'name' => "TextSearchIndex") IS the option

[18:02:20] <tsturzl> but I've never seen it before where you have to provide that when creating an index

[18:02:29] <tsturzl> not even when you do it directly in mongo shell

[18:02:46] <cpama> tsturzl, i just created the index manually

[18:03:06] <cpama> > db.mycollection.createIndex( { "$**": "text" } )

[18:03:06] <cpama> {

[18:03:07] <cpama> "createdCollectionAutomatically" : false,

[18:03:07] <cpama> "numIndexesBefore" : 1,

[18:03:07] <cpama> "numIndexesAfter" : 2,

[18:03:07] <cpama> "ok" : 1

[18:03:09] <cpama> }

[18:03:21] <cpama> and now I'm going to try using this an example to do a full text search:

[18:03:25] <cpama> http://stackoverflow.com/questions/23084243/php-mongodb-full-text-search-and-sort

[18:03:34] <tsturzl> Wait drop that index and run your code again

[18:03:52] <tsturzl> I see the problem

[18:03:55] <cpama> oh

[18:03:57] <cpama> ok

[18:04:01] <cpama> one sec... dropping index.

[18:04:13] <tsturzl> I just want to make sure before I spout off

[18:04:21] <tsturzl> Run the code, and see if the code created the index

[18:04:50] <tsturzl> Also you don't have to pass the `name` option

[18:05:01] <tsturzl> You're getting an error because the index already exists

[18:05:13] <cpama> it creates the index

[18:05:24] <cpama> i commented out the code that first attempts to delete

[18:05:27] <tsturzl> You should have some bootstrap logic to run everytime you deploy your app to create indexes

[18:05:42] <cpama> that was the idea with doing things in the following order in my script:

[18:05:45] <cpama> 1. delete the index.

[18:05:46] <tsturzl> And when you do that, you should first check to see if they exist

[18:05:47] <cpama> 2. recreate index

[18:05:50] <cpama> ok

[18:05:54] <tsturzl> Don't delete them

[18:06:09] <tsturzl> because on a large collection indexing can take several minutes

[18:06:12] <cpama> ah .

[18:06:18] <cpama> ok

[18:06:29] <tsturzl> sorry, I'm just really unfamiliar with PHP

[18:06:33] <cpama> ok... getting back to my test.

[18:06:34] <tsturzl> secondaly

[18:06:45] <tsturzl> 'text' is not a command

[18:06:52] <tsturzl> you're looking for `text`

[18:07:10] <tsturzl> and is db->command taking a query?

[18:07:17] <cpama> yeah

[18:07:35] <tsturzl> sorry you're looking for `$text`

[18:08:13] <tsturzl> but limit and project are not query operators either

[18:08:17] <cpama> which instance of "text" are you referring to?

[18:08:23] <cpama> when i create the index

[18:08:28] <cpama> or when i run db->command

[18:08:34] <tsturzl> db->command

[18:09:01] <tsturzl> also shouldn't you be using the $collection object to run the query against a specific collection?

[18:09:07] <tsturzl> http://php.net/manual/en/mongocollection.find.php

[18:09:27] <tsturzl> db->command sounds like `db.runCommand()` in mongo shell which is used to run administrative commands

[18:09:49] <tsturzl> What you want is $collection->find()

[18:10:12] <tsturzl> and that will return a cursor, and you can apply your limit to that cursor

[18:10:49] <cpama> hm. ok. and so I just assume that the find() command will use the index?

[18:10:56] <cpama> any way to prove that it's being used?

[18:11:04] <cpama> like maybe a timing test?

[18:11:21] <cpama> find() with and then find() without my index and then compare times?

[18:11:26] <cpama> *times = response times?

[18:12:10] <cpama> ah i found the answer

[18:12:24] <cpama> $string = "search string";

[18:12:24] <cpama> $cursor = $collection->find(

[18:12:24] <cpama> array('$text' => array('$search' => $string)),

[18:12:24] <cpama> array('score' => array('$meta' => 'textScore'))

[18:12:24] <cpama> );

[18:12:57] <cpama> in the first array i pass to find()... the $text must be what's telling to it do a full text search/

[18:14:10] <cpama> bingo

[18:14:11] <tsturzl> Are you at all concerned with the score?

[18:14:14] <cpama> that's what it is

[18:14:24] <tsturzl> You can just omit array('score' => array('$meta' => 'textScore')

[18:14:43] <cpama> i don't understand yet what scoring is all about / how it works exactly... so I can't comment.

[18:15:27] <cpama> tsturzl, thanks so much for taking the time to chat with me.

[18:15:28] <cpama> appreciate it

[18:15:33] <cpama> looks like I found my answer!

[18:15:34] <cpama> :D

[18:16:00] <tsturzl> the $meta: textScore

[18:16:05] <tsturzl> is for sorting by text score

[18:16:24] <tsturzl> in a nutshell the higher the score the more relevant the information

[18:16:33] <cpama> but how do you create the score? do you create per document?

[18:16:53] <tsturzl> its created per document by the text search

[18:16:58] <cpama> i see...

[18:17:31] <tsturzl> Essentially you create an instance and break the work into "tokens". Tokens are just stemmed words, like digging becomes dig.

[18:17:39] <tsturzl> and tokens populate the index

[18:17:57] <tsturzl> so when you text search, it tokenizes your search text the same way

[18:18:02] <tsturzl> and then compares them

[18:18:43] <tsturzl> So lets say a document contains your tokens a total of 30 times, and another 20 times. The one with 30 matches is going to score higher

[18:18:49] <tsturzl> and this is done automatically

[18:18:52] <cpama> oh cool

[18:19:02] <tsturzl> scoring is part of text search since it isn't an exact match

[18:20:05] <tsturzl> It gets a lot more complicated but that's the gist of it

[18:20:20] <tsturzl> so the $meta: textScore field is really just for sorting on

[18:20:30] <tsturzl> Like if for some reason you wanted the least relevant documents

[18:20:32] <cpama> ok. appreciate it

[18:20:38] <cpama> <listening>

[18:20:56] <tsturzl> Or if you wanted to filter a minimum score out, so only documents with a score X or higher would come out

[18:21:27] <tsturzl> but really you can probably disregard that, because mongodb will just assume you want to sort highest score descending.

[18:21:51] <cpama> ok

[18:21:59] <tsturzl> Also you can pass an array as your second param to `find` to do what you were trying to accomplish with `project`

[18:22:27] <tsturzl> `$project` is an aggregation stage, it doesn't work in regular queries

[18:23:39] <cpama> i don't think i understand your last sentence. but I think I have enough information I need for now. If we need to really look into scoring I guess I will do some research first and then post back here if I have more problems / questions.

[18:31:35] <tsturzl> You were trying to project the fields description, and pidnum

[18:32:14] <tsturzl> you can do that, just pass an array like ["description" => 1, "pidnum" =>1] as your second param to $collection->find()

[21:21:08] <arunbabu> Is it possible that I get different results when querying from pymongo and mongodb console? I'm seeing this behaviour now!

[21:50:35] <StephenLynx> arunbabu, you are querying a different db.

[21:50:47] <StephenLynx> or you don`t get pymongo`s syntax.

[21:50:56] <StephenLynx> or you are querying a different collection.

[21:51:07] <StephenLynx> I find it very hard for pymongo to be bugging out.

Log file Viewer

Help | Karma | Search:

#mongodb logs for Friday the 4th of November, 2016