pmxbot IRC Log Viewer

[02:39:18] <Gavilan2> v

[02:43:42] <Gavilan2> Hi! How bad an idea is to use a single "collection" for all my apps documents? When will that stop performing well?

[03:49:46] <timeturner> qq, is it commonplace to give confirmation links that look like: http://mydomain.com/confirm/238#*asfdDF OR http://mydomain.com/confirm/?user=test@example.com&code=238#*asfdDF ?

[03:50:10] <timeturner> not sure where else to ask this question :/

[03:55:04] <Gavilan2> Hi! How bad an idea is to use a single "collection" for all my apps documents? When will that stop performing well?

[03:56:11] <timeturner> single collection? not good

[03:56:17] <timeturner> very bad in fact

[03:56:24] <timeturner> but it all depends on your schema

[03:56:35] <timeturner> I mean how simple or complex your app is

[03:56:50] <Gavilan2> timeturner: No schema at all xD

[03:57:46] <Gavilan2> I'm building a kind of schemaless ORM, and I either need to store all docs in a single collection

[03:57:48] <timeturner> give me an example of what it might look like

[03:58:03] <Gavilan2> or reference through _id + collectionName

[03:58:23] <timeturner> basically, you split up stuff into collections when you know it has to grow unbounded

[03:58:36] <timeturner> and you need to separate it somehow

[03:58:39] <timeturner> and query for it every time

[03:58:42] <Gavilan2> {..., someMember, ...} gets mapped to {..., _id_someMember, _collection_someMember, ...}

[03:58:59] <Gavilan2> grow unbounded?

[03:59:08] <timeturner> that's just a deferred dbref

[03:59:16] <timeturner> deferred multi-query thing

[03:59:28] <Gavilan2> i split each object into a different document, because each object has an independent lifetime...

[03:59:31] <timeturner> unbounded as in you can't fit all of the stuff in a subdocument array or something

[03:59:52] <timeturner> it's a big organizational issue if you don't have some order

[03:59:58] <Gavilan2> some order?

[04:00:03] <timeturner> being entirely schemaless isn't the best way to go

[04:00:13] <timeturner> you should have some general schema per collection

[04:00:16] <timeturner> and stick to it

[04:00:22] <timeturner> for uniformity's sake

[04:00:29] <timeturner> the rest is up to you though

[04:00:39] <Gavilan2> what's so good about uniformity? :)

[04:01:01] <timeturner> you can query more easily

[04:01:07] <timeturner> less to keep track of

[04:01:20] <timeturner> easier, scratch that, much easier to maintain

[04:01:21] <Gavilan2> i don't want to query... I want my ORM thing to be 100% transparent

[04:06:26] <IAD> Gavilan2: look at http://en.wikipedia.org/wiki/Neo4j

[04:19:44] <Gavilan2> IAD: Yes, mine will be a graph thingy too....

[04:21:37] <Gavilan2> IAD: Coming back to my question, what problems will I face if I just use a single collection?

[04:24:06] <IAD> Gavilan2: no problems, only overhead in future

[04:25:01] <Gavilan2> what kind of overhead? and how much data are we talking?

[04:25:16] <Gavilan2> I don't think i have a lot of data... let's say 1 GB at the worst....

[04:25:41] <IAD> thats ok

[04:27:38] <Gavilan2> so, 1 collection is fine then? :)

[07:06:20] <cmex> good morning

[07:20:03] <Dr{Wh0}> Can i exclude a database from replication?

[08:19:56] <dotblank3> I have a question about queries and Date indexes

[08:20:14] <dotblank3> How do I query for the first closest date

[08:26:30] <kali> dotblank3: if you want the first after, db.foo.find({ timestamp: { $gte : .... } }).sort({timestamp: 1}).limit(1)

[08:26:56] <kali> dotblank3: if you want the first before, same with $lte and reverse order

[08:27:27] <kali> dotblank3: and if you want the closest whatever the direction is, you need to do both request and find out which one is better application-side

[08:27:54] <dotblank3> Thanks, that is helpfull

[08:28:54] <dotblank3> Why do I need to use sort? If I have an index on Date can I use the natural order to quickly return entries that have been added sequentially

[08:30:21] <kali> dotblank3: if you never delete or update records in the collection, that could work. but as you have an index on the date, the sort will just do the right thing

[08:30:55] <kali> dotblank3: i prefer not to rely on physical layout when i'm expressing a "logical" thing

[08:31:29] <dotblank3> well, this is more or less a log file and a capped collection

[08:31:50] <dotblank3> soI will always now the natural order will be in sequential ascending date order

[08:32:16] <dotblank3> always know*

[08:32:53] <kali> in an ideal world, yes :)

[08:33:40] <kali> i have a similar situation in my infrastructure, and the insertion order is often slightly off

[08:33:48] <dotblank3> Is the penalty large moving to sorted indexed result from a natural order sort

[08:33:50] <kali> because various soure will take more or less time to write

[08:34:39] <kali> dotblank3: nope, it will realy be marginal in this case, so i recommend strongly putting the sort on timestamp :)

[08:35:03] <dotblank3> Awesome, thanks for your help!

[12:54:51] <ron> say we finally have a semi-production environment, and we notice a load that's... a bit more than we expected, how would we go about debugging the situation?

[13:43:26] <gigo1980> does have anyone tried the mongodb hadoop connector ?

[13:44:10] <gigo1980> because i have an bootlneck in my map reduce in mongodb / so only one core for the v8 engine is available

[14:36:41] <defaultro> good morning folks, what tcp port does MongoDB use? And how do I add username/password to access the collection?

[14:37:34] <Derick> 27017

[14:37:47] <IAD> defaultro: http://www.mongodb.org/display/DOCS/Starting+and+Stopping+Mongo and http://www.mongodb.org/display/DOCS/Security+and+Authentication

[14:37:55] <defaultro> excellent, thanks!!!

[14:38:00] <Derick> and http://es.wiki.mongodb.org/display/DOCS/Security+and+Authentication#SecurityandAuthentication-RunninginSecureMode%28with\author\keyFile%29

[14:38:04] <Derick> right, what IAD said

[14:38:14] <defaultro> the reason why I am doing this is because dreamhost doesn't allow installation of mongodb on their machine

[14:38:33] <defaultro> i'll have my app there connect to my home mongodb :D

[14:39:43] <Derick> get a better host perhaps? :)

[14:39:58] <defaultro> yup :D

[14:40:16] <defaultro> a friend has offered to me free hosting :D

[14:40:24] <defaultro> i'll do it once I get a chance

[14:40:34] <defaultro> my dreamhost expires on Feb

[14:43:47] <defaultro> so everytime I run "use somestring", it creates that collection?

[14:44:45] <IAD> defaultro: database and collection will be created, when you write a first document into it

[14:45:35] <defaultro> how come when I run use sdgsgsdg, it took it

[14:45:40] <defaultro> it didn't complain

[14:52:31] <defaultro> I created a user. How come, when I run mongo localhost/mydb, it doesn't ask for username/password?

[14:52:53] <Derick> try "show tables"

[14:53:00] <Derick> it will tell you that you're not auth-ed

[14:53:06] <defaultro> ok

[14:53:08] <Derick> and you will need to use db.auth('user', 'pass');

[14:53:15] <defaultro> also, I think i need to turn on auth in mongodb.conf

[14:53:39] <defaultro> what does that do?

[14:53:51] <defaultro> i did db.addUser earlier

[14:55:04] <defaultro> i was still able to access the data from my collection

[14:55:24] <defaultro> show tables just shows the db name, system.indexes and system.users

[14:57:46] <defaultro> Derick, it still didn't work

[14:57:52] <defaultro> here is what I did

[14:58:45] <defaultro> use mydb then db.auth('someuser','somepass') then db.addUser('someuser','somepass') then i type exit to go back to console. I ran mongo localhost/mydb

[14:59:15] <defaultro> i ran show tables and I didn't see "you're not authed"

[15:18:40] <hillct> Good afternoon all. I'm trying to understand ranged request pagination where the initial request can be more or less arbitrary, as opposed to a simple timeline type ranged pagination

[15:19:41] <hillct> To accomplish this, I'm looking at the Twitter style of pagination which seems to be to create a hash representing the initial search query plus the ranged offset value

[15:19:56] <hillct> for next and previous pages

[15:20:47] <hillct> It seems to me I should be able to utilize the timestamp encoded in the default mongoDB object ID formatted string

[15:21:16] <hillct> I was hoping someone might have an example of this type of implementation

[15:33:00] <jpfarias> hey guys

[15:33:24] <jpfarias> I upgraded my shard server to 2.2.0 and now I am having some problems, is it ok if I downgrade to 2.0.7?

[15:41:39] <jpfarias> this is the error I am getting: http://pastie.org/4690770

[15:41:46] <jpfarias> I have no idea why that is happening

[15:55:09] <defaultro> so happy, I am now able to save data to mongodb from my leaflet and openstreetmaps :)

[16:45:25] <hillct> When you sort by _id you get an order by _id creation time which is handy, but is there a way to test that timestamp in a query? I tested the query {$id.generation_time:{$lt: mongoIdTimestamp } } but found it failed to survive the Node.js module compilation

[16:47:05] <hillct> Is there a way to implement a query that evaluates the timestamp embedded within an ObjectID?

[16:51:07] <hillct> BurtyB: the default ID objects have a timestamp embedded in the first 4 bits

[17:38:03] <hillct> There's actually docs describing most of what I need to be able to do: http://www.mongodb.org/display/DOCS/Optimizing+Object+IDs but I don't see how to utilize _id or $id timestamp values in queries, for example $id.getTimeStamp(): {$lt: mongoID.getTimestamp()}

[17:38:32] <hillct> however that syntax isn't getting it done

[17:51:13] <hackeron> hey, if I have a recording collection that references a camera and a camera collection that references zone - is there anyway to query something like Recording.where(:camera.zone => my_zone)?

[18:20:35] <hillct> In reviewing these patches relating to timestamp extraction from objectIDs, I can't discern how to perform comparison operations on the timestamp portion of an _id from within a query, but it seems like that functionality was added two years back...

[18:24:16] <defaultro> hey folks, is it possible to remove _id?

[18:25:22] <kali> hillct: the timestamp are at the beginning if the _id, so you can just compare them

[18:26:02] <kali> hillct: be aware that it is only at the granularity of a second

[18:26:23] <kali> defaultro: you can't

[18:26:37] <defaultro> ok

[18:26:55] <defaultro> i'm trying to parse the data from javascript but the random id is giving me a hard time

[18:28:48] <hillct> kali: I've been looking at this all afternoon and can't work out how to use the _id timestamp component in a query constraint. Can you give me an idea how that would work? Comparing the first 4 characters, or parsing the first 4 characters as a timestamp doesn't do it. What do they need? Hex decoding?

[18:29:38] <kali> hillct: you need to generate fake objectid

[18:29:52] <kali> hillct: what driver are you using ?

[18:29:59] <hillct> kali: node

[18:30:32] <kali> can't help there, but look at the ObjectId object... there should be a constructor that take a datetime

[18:31:15] <kali> hillct: http://api.mongodb.org/java/current/org/bson/types/ObjectId.html#ObjectId(java.util.Date)

[18:31:20] <hillct> kali: according to the docs, the constructor with no args generates a string that encodes the datetime

[18:31:37] <kali> hillct: you're looking for a constructor with an argument.

[18:31:48] <kali> hillct: equivalent to the java one i just linked

[18:32:38] <hillct> kali: why would I want to modify the default ID generation, when the default generated IDs include the encoded date time?

[18:32:49] <hillct> what am I missing here?

[18:34:01] <hillct> kali: or do you mean to generate a bogus ObjectID, then perform the comparison of ObjectID to ObjectID, rather than extract the date time from the existing objects against which the comparison is being performed

[18:34:03] <kali> hillct: you want to create an objectid with the date you want to use as a comparison

[18:34:22] <kali> hillct: and compare your collection objectid to this one

[18:34:48] <hillct> kali: ok, I guess I was thinking about comparing the raw (exported) date time strings, which is obviously a wrong approach, now that I look at it

[18:35:25] <kali> hillct: you need to compare the actual type stored by mongodb if you want the index to work... hence the "fake" objectid

[18:35:35] <hillct> kali: thanks. Now that it's been pointed out, it of course seems obvious

[18:36:13] <hillct> kali: I'll code it up as you suggest, and I'm pretty sure it'll work...

[18:36:16] <hillct> thanks

[18:36:53] <kali> i'm prettu sure too :)

[20:05:02] <gigo1980> hi, is there a way to scale map reduse without sharding ? any eperinces with other MR frameworks ?

[20:24:26] <dominik-zogg> jmikola: Are you here?

[20:26:37] <jmikola> yes

[20:26:57] <dominik-zogg> i use the MongoDBBundle, and got a problem with the form type document

[20:27:28] <dominik-zogg> have you an idea why no set add method get called in document?

[20:28:35] <jmikola> not offhand; that's all common functionality of the DoctrineType in Symfony -- likely not specific to the bundle

[20:28:44] <dominik-zogg> other data i form get stored form -> doctrine -> mongdb without any problem

[20:29:37] <dominik-zogg> i mean this here: https://github.com/doctrine/DoctrineMongoDBBundle/blob/master/Form/Type/DocumentType.php

[20:30:54] <jmikola> the actual logic is in https://github.com/symfony/symfony/blob/master/src/Symfony/Bridge/Doctrine/Form/Type/DoctrineType.php -- the class you're pasting is just as wrapper for some ODM binding

[20:31:12] <dominik-zogg> ah k, thx

[20:31:30] <jmikola> i don't see anything specific in the DoctrineType about calling add/set methods, so this is likely not even a DoctrineType issue -- probably a generic Symfony forms question

[20:31:41] <dominik-zogg> i worked with the entity version of this last week, on a other project, without a problem

[20:32:08] <dominik-zogg> symfony/form bug ;-)

[20:32:24] <dominik-zogg> well or i have done something wrong, thx for help

[20:32:48] <jmikola> if you're able to create a functional test that fails, and submit a PR against the odm bundle, i could look into it

[20:32:54] <jmikola> otherwise there's not much i can do

[20:34:04] <dominik-zogg> jmikola: well i go sleep, and will see, if tomorrow is a better day to find the problem

[20:58:19] <macBigRig> mongoImport assertion error with date before 1970: git://gist.github.com/3687245.git

[21:15:57] <defaultro> hey guys, I wrote a simple map application and it saves data to mongodb :) http://192.168.2.194/map_experimentv2.html

[23:21:43] <Guest28705> is there no way to colorize the prompt from .mongorc.js?

Log file Viewer

Help | Karma | Search:

#mongodb logs for Sunday the 9th of September, 2012